[GitHub] [spark] AmplabJenkins removed a comment on issue #26772: [WIP][SPARK-28210][CORE] Introducing Shuffle Reader API

2019-12-05 Thread GitBox
AmplabJenkins removed a comment on issue #26772: [WIP][SPARK-28210][CORE] Introducing Shuffle Reader API URL: https://github.com/apache/spark/pull/26772#issuecomment-562396887 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.be

[GitHub] [spark] SparkQA commented on issue #26772: [WIP][SPARK-28210][CORE] Introducing Shuffle Reader API

2019-12-05 Thread GitBox
SparkQA commented on issue #26772: [WIP][SPARK-28210][CORE] Introducing Shuffle Reader API URL: https://github.com/apache/spark/pull/26772#issuecomment-562396954 **[Test build #114922 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/114922/testReport)** for

[GitHub] [spark] beliefer commented on a change in pull request #25001: [SPARK-28083][SQL] Support LIKE ... ESCAPE syntax

2019-12-05 Thread GitBox
beliefer commented on a change in pull request #25001: [SPARK-28083][SQL] Support LIKE ... ESCAPE syntax URL: https://github.com/apache/spark/pull/25001#discussion_r35465 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/regexpExpressions.scala

[GitHub] [spark] SparkQA commented on issue #26772: [WIP][SPARK-28210][CORE] Introducing Shuffle Reader API

2019-12-05 Thread GitBox
SparkQA commented on issue #26772: [WIP][SPARK-28210][CORE] Introducing Shuffle Reader API URL: https://github.com/apache/spark/pull/26772#issuecomment-562396551 **[Test build #114922 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/114922/testReport)** for

[GitHub] [spark] ifilonenko opened a new pull request #26772: [WIP][SPARK-28210][CORE] Introducing Shuffle Reader API

2019-12-05 Thread GitBox
ifilonenko opened a new pull request #26772: [WIP][SPARK-28210][CORE] Introducing Shuffle Reader API URL: https://github.com/apache/spark/pull/26772 WIP (still requires unit tests to pass) ### What changes were proposed in this pull request? Extending the ExecutorComponents to inc

[GitHub] [spark] AmplabJenkins removed a comment on issue #26752: [SPARK-30112][SQL] Allow insert overwrite same table if using dynamic partition overwrite

2019-12-05 Thread GitBox
AmplabJenkins removed a comment on issue #26752: [SPARK-30112][SQL] Allow insert overwrite same table if using dynamic partition overwrite URL: https://github.com/apache/spark/pull/26752#issuecomment-562393395 Test PASSed. Refer to this link for build results (access rights to CI server

[GitHub] [spark] AmplabJenkins removed a comment on issue #26752: [SPARK-30112][SQL] Allow insert overwrite same table if using dynamic partition overwrite

2019-12-05 Thread GitBox
AmplabJenkins removed a comment on issue #26752: [SPARK-30112][SQL] Allow insert overwrite same table if using dynamic partition overwrite URL: https://github.com/apache/spark/pull/26752#issuecomment-562393388 Merged build finished. Test PASSed.

[GitHub] [spark] AmplabJenkins commented on issue #26752: [SPARK-30112][SQL] Allow insert overwrite same table if using dynamic partition overwrite

2019-12-05 Thread GitBox
AmplabJenkins commented on issue #26752: [SPARK-30112][SQL] Allow insert overwrite same table if using dynamic partition overwrite URL: https://github.com/apache/spark/pull/26752#issuecomment-562393388 Merged build finished. Test PASSed.

[GitHub] [spark] AmplabJenkins commented on issue #26752: [SPARK-30112][SQL] Allow insert overwrite same table if using dynamic partition overwrite

2019-12-05 Thread GitBox
AmplabJenkins commented on issue #26752: [SPARK-30112][SQL] Allow insert overwrite same table if using dynamic partition overwrite URL: https://github.com/apache/spark/pull/26752#issuecomment-562393395 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] SparkQA commented on issue #26752: [SPARK-30112][SQL] Allow insert overwrite same table if using dynamic partition overwrite

2019-12-05 Thread GitBox
SparkQA commented on issue #26752: [SPARK-30112][SQL] Allow insert overwrite same table if using dynamic partition overwrite URL: https://github.com/apache/spark/pull/26752#issuecomment-562393076 **[Test build #114921 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequest

[GitHub] [spark] rdblue commented on a change in pull request #26684: [SPARK-30001][SQL] ResolveRelations should handle both V1 and V2 tables.

2019-12-05 Thread GitBox
rdblue commented on a change in pull request #26684: [SPARK-30001][SQL] ResolveRelations should handle both V1 and V2 tables. URL: https://github.com/apache/spark/pull/26684#discussion_r354627788 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analy

[GitHub] [spark] bersprockets commented on a change in pull request #26016: [SPARK-24914][SQL] New statistic to improve data size estimate for columnar storage formats

2019-12-05 Thread GitBox
bersprockets commented on a change in pull request #26016: [SPARK-24914][SQL] New statistic to improve data size estimate for columnar storage formats URL: https://github.com/apache/spark/pull/26016#discussion_r354627478 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sq

[GitHub] [spark] imback82 commented on a change in pull request #26684: [SPARK-30001][SQL] ResolveRelations should handle both V1 and V2 tables.

2019-12-05 Thread GitBox
imback82 commented on a change in pull request #26684: [SPARK-30001][SQL] ResolveRelations should handle both V1 and V2 tables. URL: https://github.com/apache/spark/pull/26684#discussion_r354626929 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Ana

[GitHub] [spark] bersprockets commented on a change in pull request #26016: [SPARK-24914][SQL] New statistic to improve data size estimate for columnar storage formats

2019-12-05 Thread GitBox
bersprockets commented on a change in pull request #26016: [SPARK-24914][SQL] New statistic to improve data size estimate for columnar storage formats URL: https://github.com/apache/spark/pull/26016#discussion_r354623854 ## File path: sql/core/src/main/scala/org/apache/spark/sql/ex

[GitHub] [spark] viirya commented on a change in pull request #26752: [SPARK-30112][SQL] Allow insert overwrite same table if using dynamic partition overwrite

2019-12-05 Thread GitBox
viirya commented on a change in pull request #26752: [SPARK-30112][SQL] Allow insert overwrite same table if using dynamic partition overwrite URL: https://github.com/apache/spark/pull/26752#discussion_r354624788 ## File path: sql/core/src/test/scala/org/apache/spark/sql/sources/In

[GitHub] [spark] rdblue commented on a change in pull request #26684: [SPARK-30001][SQL] ResolveRelations should handle both V1 and V2 tables.

2019-12-05 Thread GitBox
rdblue commented on a change in pull request #26684: [SPARK-30001][SQL] ResolveRelations should handle both V1 and V2 tables. URL: https://github.com/apache/spark/pull/26684#discussion_r354624656 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analy

[GitHub] [spark] vanzin commented on a change in pull request #26416: [SPARK-29779][CORE] Compact old event log files and cleanup

2019-12-05 Thread GitBox
vanzin commented on a change in pull request #26416: [SPARK-29779][CORE] Compact old event log files and cleanup URL: https://github.com/apache/spark/pull/26416#discussion_r354621867 ## File path: core/src/main/scala/org/apache/spark/deploy/history/EventFilter.scala ## @@

[GitHub] [spark] vanzin commented on a change in pull request #26416: [SPARK-29779][CORE] Compact old event log files and cleanup

2019-12-05 Thread GitBox
vanzin commented on a change in pull request #26416: [SPARK-29779][CORE] Compact old event log files and cleanup URL: https://github.com/apache/spark/pull/26416#discussion_r354622853 ## File path: core/src/main/scala/org/apache/spark/deploy/history/EventLogFilterRateCalculator.scal

[GitHub] [spark] vanzin commented on a change in pull request #26416: [SPARK-29779][CORE] Compact old event log files and cleanup

2019-12-05 Thread GitBox
vanzin commented on a change in pull request #26416: [SPARK-29779][CORE] Compact old event log files and cleanup URL: https://github.com/apache/spark/pull/26416#discussion_r354622258 ## File path: core/src/main/scala/org/apache/spark/deploy/history/EventFilter.scala ## @@

[GitHub] [spark] vanzin commented on a change in pull request #26416: [SPARK-29779][CORE] Compact old event log files and cleanup

2019-12-05 Thread GitBox
vanzin commented on a change in pull request #26416: [SPARK-29779][CORE] Compact old event log files and cleanup URL: https://github.com/apache/spark/pull/26416#discussion_r354619053 ## File path: core/src/main/scala/org/apache/spark/deploy/history/EventFilter.scala ## @@

[GitHub] [spark] vanzin commented on a change in pull request #26416: [SPARK-29779][CORE] Compact old event log files and cleanup

2019-12-05 Thread GitBox
vanzin commented on a change in pull request #26416: [SPARK-29779][CORE] Compact old event log files and cleanup URL: https://github.com/apache/spark/pull/26416#discussion_r354622717 ## File path: core/src/main/scala/org/apache/spark/deploy/history/EventLogFileCompactor.scala

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #26752: [SPARK-30112][SQL] Allow insert overwrite same table if using dynamic partition overwrite

2019-12-05 Thread GitBox
dongjoon-hyun commented on a change in pull request #26752: [SPARK-30112][SQL] Allow insert overwrite same table if using dynamic partition overwrite URL: https://github.com/apache/spark/pull/26752#discussion_r354624321 ## File path: sql/core/src/test/scala/org/apache/spark/sql/sou

[GitHub] [spark] imback82 commented on a change in pull request #26684: [SPARK-30001][SQL] ResolveRelations should handle both V1 and V2 tables.

2019-12-05 Thread GitBox
imback82 commented on a change in pull request #26684: [SPARK-30001][SQL] ResolveRelations should handle both V1 and V2 tables. URL: https://github.com/apache/spark/pull/26684#discussion_r354623705 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Ana

[GitHub] [spark] srowen commented on issue #26722: [SPARK-24666][ML] Fix infinity vectors produced by Word2Vec when numIterations are large

2019-12-05 Thread GitBox
srowen commented on issue #26722: [SPARK-24666][ML] Fix infinity vectors produced by Word2Vec when numIterations are large URL: https://github.com/apache/spark/pull/26722#issuecomment-562385030 I'm OK with putting it in 2.4, I think. It's a minor behavior change, but, also appears to be mo

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #26752: [SPARK-30112][SQL] Allow insert overwrite same table if using dynamic partition overwrite

2019-12-05 Thread GitBox
dongjoon-hyun commented on a change in pull request #26752: [SPARK-30112][SQL] Allow insert overwrite same table if using dynamic partition overwrite URL: https://github.com/apache/spark/pull/26752#discussion_r354621458 ## File path: sql/core/src/test/scala/org/apache/spark/sql/sou

[GitHub] [spark] viirya commented on a change in pull request #26752: [SPARK-30112][SQL] Allow insert overwrite same table if using dynamic partition overwrite

2019-12-05 Thread GitBox
viirya commented on a change in pull request #26752: [SPARK-30112][SQL] Allow insert overwrite same table if using dynamic partition overwrite URL: https://github.com/apache/spark/pull/26752#discussion_r354621480 ## File path: sql/core/src/test/scala/org/apache/spark/sql/sources/In

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #26752: [SPARK-30112][SQL] Allow insert overwrite same table if using dynamic partition overwrite

2019-12-05 Thread GitBox
dongjoon-hyun commented on a change in pull request #26752: [SPARK-30112][SQL] Allow insert overwrite same table if using dynamic partition overwrite URL: https://github.com/apache/spark/pull/26752#discussion_r354621289 ## File path: sql/core/src/test/scala/org/apache/spark/sql/sou

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #26752: [SPARK-30112][SQL] Allow insert overwrite same table if using dynamic partition overwrite

2019-12-05 Thread GitBox
dongjoon-hyun commented on a change in pull request #26752: [SPARK-30112][SQL] Allow insert overwrite same table if using dynamic partition overwrite URL: https://github.com/apache/spark/pull/26752#discussion_r354621366 ## File path: sql/core/src/test/scala/org/apache/spark/sql/sou

[GitHub] [spark] AmplabJenkins commented on issue #26752: [SPARK-30112][SQL] Allow insert overwrite same table if using dynamic partition overwrite

2019-12-05 Thread GitBox
AmplabJenkins commented on issue #26752: [SPARK-30112][SQL] Allow insert overwrite same table if using dynamic partition overwrite URL: https://github.com/apache/spark/pull/26752#issuecomment-562384370 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] AmplabJenkins removed a comment on issue #26752: [SPARK-30112][SQL] Allow insert overwrite same table if using dynamic partition overwrite

2019-12-05 Thread GitBox
AmplabJenkins removed a comment on issue #26752: [SPARK-30112][SQL] Allow insert overwrite same table if using dynamic partition overwrite URL: https://github.com/apache/spark/pull/26752#issuecomment-562384364 Merged build finished. Test PASSed.

[GitHub] [spark] AmplabJenkins removed a comment on issue #26752: [SPARK-30112][SQL] Allow insert overwrite same table if using dynamic partition overwrite

2019-12-05 Thread GitBox
AmplabJenkins removed a comment on issue #26752: [SPARK-30112][SQL] Allow insert overwrite same table if using dynamic partition overwrite URL: https://github.com/apache/spark/pull/26752#issuecomment-562384370 Test PASSed. Refer to this link for build results (access rights to CI server

[GitHub] [spark] AmplabJenkins commented on issue #26752: [SPARK-30112][SQL] Allow insert overwrite same table if using dynamic partition overwrite

2019-12-05 Thread GitBox
AmplabJenkins commented on issue #26752: [SPARK-30112][SQL] Allow insert overwrite same table if using dynamic partition overwrite URL: https://github.com/apache/spark/pull/26752#issuecomment-562384364 Merged build finished. Test PASSed.

[GitHub] [spark] SparkQA commented on issue #26752: [SPARK-30112][SQL] Allow insert overwrite same table if using dynamic partition overwrite

2019-12-05 Thread GitBox
SparkQA commented on issue #26752: [SPARK-30112][SQL] Allow insert overwrite same table if using dynamic partition overwrite URL: https://github.com/apache/spark/pull/26752#issuecomment-562384008 **[Test build #114920 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequest

[GitHub] [spark] dongjoon-hyun commented on issue #26752: [SPARK-30112][SQL] Allow insert overwrite same table if using dynamic partition overwrite

2019-12-05 Thread GitBox
dongjoon-hyun commented on issue #26752: [SPARK-30112][SQL] Allow insert overwrite same table if using dynamic partition overwrite URL: https://github.com/apache/spark/pull/26752#issuecomment-562383812 Thank you for making a PR for this issue. --

[GitHub] [spark] dongjoon-hyun commented on issue #26752: [SPARK-30112][SQL] Allow insert overwrite same table if using dynamic partition overwrite

2019-12-05 Thread GitBox
dongjoon-hyun commented on issue #26752: [SPARK-30112][SQL] Allow insert overwrite same table if using dynamic partition overwrite URL: https://github.com/apache/spark/pull/26752#issuecomment-562383398 Retest this please. Thi

[GitHub] [spark] AmplabJenkins removed a comment on issue #26740: [SPARK-30053][SQL] Add the ability for v2 datasource so specify a vacuum action on the table

2019-12-05 Thread GitBox
AmplabJenkins removed a comment on issue #26740: [SPARK-30053][SQL] Add the ability for v2 datasource so specify a vacuum action on the table URL: https://github.com/apache/spark/pull/26740#issuecomment-562382628 Test FAILed. Refer to this link for build results (access rights to CI serv

[GitHub] [spark] AmplabJenkins removed a comment on issue #26740: [SPARK-30053][SQL] Add the ability for v2 datasource so specify a vacuum action on the table

2019-12-05 Thread GitBox
AmplabJenkins removed a comment on issue #26740: [SPARK-30053][SQL] Add the ability for v2 datasource so specify a vacuum action on the table URL: https://github.com/apache/spark/pull/26740#issuecomment-562382554 Merged build finished. Test PASSed. -

[GitHub] [spark] AmplabJenkins commented on issue #26740: [SPARK-30053][SQL] Add the ability for v2 datasource so specify a vacuum action on the table

2019-12-05 Thread GitBox
AmplabJenkins commented on issue #26740: [SPARK-30053][SQL] Add the ability for v2 datasource so specify a vacuum action on the table URL: https://github.com/apache/spark/pull/26740#issuecomment-562382628 Test FAILed. Refer to this link for build results (access rights to CI server neede

[GitHub] [spark] AmplabJenkins removed a comment on issue #26740: [SPARK-30053][SQL] Add the ability for v2 datasource so specify a vacuum action on the table

2019-12-05 Thread GitBox
AmplabJenkins removed a comment on issue #26740: [SPARK-30053][SQL] Add the ability for v2 datasource so specify a vacuum action on the table URL: https://github.com/apache/spark/pull/26740#issuecomment-562382623 Merged build finished. Test FAILed. -

[GitHub] [spark] SparkQA removed a comment on issue #26740: [SPARK-30053][SQL] Add the ability for v2 datasource so specify a vacuum action on the table

2019-12-05 Thread GitBox
SparkQA removed a comment on issue #26740: [SPARK-30053][SQL] Add the ability for v2 datasource so specify a vacuum action on the table URL: https://github.com/apache/spark/pull/26740#issuecomment-562382260 **[Test build #114919 has started](https://amplab.cs.berkeley.edu/jenkins/job/Spark

[GitHub] [spark] AmplabJenkins commented on issue #26740: [SPARK-30053][SQL] Add the ability for v2 datasource so specify a vacuum action on the table

2019-12-05 Thread GitBox
AmplabJenkins commented on issue #26740: [SPARK-30053][SQL] Add the ability for v2 datasource so specify a vacuum action on the table URL: https://github.com/apache/spark/pull/26740#issuecomment-562382623 Merged build finished. Test FAILed. -

[GitHub] [spark] SparkQA commented on issue #26740: [SPARK-30053][SQL] Add the ability for v2 datasource so specify a vacuum action on the table

2019-12-05 Thread GitBox
SparkQA commented on issue #26740: [SPARK-30053][SQL] Add the ability for v2 datasource so specify a vacuum action on the table URL: https://github.com/apache/spark/pull/26740#issuecomment-562382617 **[Test build #114919 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullReq

[GitHub] [spark] AmplabJenkins removed a comment on issue #26740: [SPARK-30053][SQL] Add the ability for v2 datasource so specify a vacuum action on the table

2019-12-05 Thread GitBox
AmplabJenkins removed a comment on issue #26740: [SPARK-30053][SQL] Add the ability for v2 datasource so specify a vacuum action on the table URL: https://github.com/apache/spark/pull/26740#issuecomment-562382561 Test PASSed. Refer to this link for build results (access rights to CI serv

[GitHub] [spark] AmplabJenkins commented on issue #26740: [SPARK-30053][SQL] Add the ability for v2 datasource so specify a vacuum action on the table

2019-12-05 Thread GitBox
AmplabJenkins commented on issue #26740: [SPARK-30053][SQL] Add the ability for v2 datasource so specify a vacuum action on the table URL: https://github.com/apache/spark/pull/26740#issuecomment-562382554 Merged build finished. Test PASSed. -

[GitHub] [spark] AmplabJenkins commented on issue #26740: [SPARK-30053][SQL] Add the ability for v2 datasource so specify a vacuum action on the table

2019-12-05 Thread GitBox
AmplabJenkins commented on issue #26740: [SPARK-30053][SQL] Add the ability for v2 datasource so specify a vacuum action on the table URL: https://github.com/apache/spark/pull/26740#issuecomment-562382561 Test PASSed. Refer to this link for build results (access rights to CI server neede

[GitHub] [spark] SparkQA commented on issue #26740: [SPARK-30053][SQL] Add the ability for v2 datasource so specify a vacuum action on the table

2019-12-05 Thread GitBox
SparkQA commented on issue #26740: [SPARK-30053][SQL] Add the ability for v2 datasource so specify a vacuum action on the table URL: https://github.com/apache/spark/pull/26740#issuecomment-562382260 **[Test build #114919 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequ

[GitHub] [spark] AmplabJenkins removed a comment on issue #26740: [SPARK-30053][SQL] Add the ability for v2 datasource so specify a vacuum action on the table

2019-12-05 Thread GitBox
AmplabJenkins removed a comment on issue #26740: [SPARK-30053][SQL] Add the ability for v2 datasource so specify a vacuum action on the table URL: https://github.com/apache/spark/pull/26740#issuecomment-560554339 Can one of the admins verify this patch?

[GitHub] [spark] dongjoon-hyun commented on issue #26740: [SPARK-30053][SQL] Add the ability for v2 datasource so specify a vacuum action on the table

2019-12-05 Thread GitBox
dongjoon-hyun commented on issue #26740: [SPARK-30053][SQL] Add the ability for v2 datasource so specify a vacuum action on the table URL: https://github.com/apache/spark/pull/26740#issuecomment-562381786 Thank you for contribution, @AndrewKL . Please run `./dev/lint-scala` and fix the s

[GitHub] [spark] dongjoon-hyun commented on issue #26740: [SPARK-30053][SQL] Add the ability for v2 datasource so specify a vacuum action on the table

2019-12-05 Thread GitBox
dongjoon-hyun commented on issue #26740: [SPARK-30053][SQL] Add the ability for v2 datasource so specify a vacuum action on the table URL: https://github.com/apache/spark/pull/26740#issuecomment-562381599 ok to test This is a

[GitHub] [spark] dongjoon-hyun commented on issue #26493: [SPARK-29574][K8S] Add SPARK_DIST_CLASSPATH to the executor class path

2019-12-05 Thread GitBox
dongjoon-hyun commented on issue #26493: [SPARK-29574][K8S] Add SPARK_DIST_CLASSPATH to the executor class path URL: https://github.com/apache/spark/pull/26493#issuecomment-562380703 Gentle ping, @sshakeri . This is an automa

[GitHub] [spark] viirya commented on issue #26722: [SPARK-24666][ML] Fix infinity vectors produced by Word2Vec when numIterations are large

2019-12-05 Thread GitBox
viirya commented on issue #26722: [SPARK-24666][ML] Fix infinity vectors produced by Word2Vec when numIterations are large URL: https://github.com/apache/spark/pull/26722#issuecomment-562380369 Thanks! Merging to master and 2.4. -

[GitHub] [spark] viirya closed pull request #26722: [SPARK-24666][ML] Fix infinity vectors produced by Word2Vec when numIterations are large

2019-12-05 Thread GitBox
viirya closed pull request #26722: [SPARK-24666][ML] Fix infinity vectors produced by Word2Vec when numIterations are large URL: https://github.com/apache/spark/pull/26722 This is an automated message from the Apache Git Ser

[GitHub] [spark] dongjoon-hyun commented on issue #23059: [SPARK-26091][SQL] Upgrade to 2.3.4 for Hive Metastore Client 2.3

2019-12-05 Thread GitBox
dongjoon-hyun commented on issue #23059: [SPARK-26091][SQL] Upgrade to 2.3.4 for Hive Metastore Client 2.3 URL: https://github.com/apache/spark/pull/23059#issuecomment-562377726 Since SPARK-26091 was an improvement issue, we didn't backport this. Is there any problem with Apache Spark 2.

[GitHub] [spark] AmplabJenkins removed a comment on issue #26770: [SPARK-30142][BUILD][test-maven] Upgrade Maven to 3.6.3

2019-12-05 Thread GitBox
AmplabJenkins removed a comment on issue #26770: [SPARK-30142][BUILD][test-maven] Upgrade Maven to 3.6.3 URL: https://github.com/apache/spark/pull/26770#issuecomment-562376882 Merged build finished. Test PASSed. This is an au

[GitHub] [spark] AmplabJenkins removed a comment on issue #26770: [SPARK-30142][BUILD][test-maven] Upgrade Maven to 3.6.3

2019-12-05 Thread GitBox
AmplabJenkins removed a comment on issue #26770: [SPARK-30142][BUILD][test-maven] Upgrade Maven to 3.6.3 URL: https://github.com/apache/spark/pull/26770#issuecomment-562376889 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.ber

[GitHub] [spark] AmplabJenkins commented on issue #26770: [SPARK-30142][BUILD][test-maven] Upgrade Maven to 3.6.3

2019-12-05 Thread GitBox
AmplabJenkins commented on issue #26770: [SPARK-30142][BUILD][test-maven] Upgrade Maven to 3.6.3 URL: https://github.com/apache/spark/pull/26770#issuecomment-562376889 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.ed

[GitHub] [spark] AmplabJenkins commented on issue #26770: [SPARK-30142][BUILD][test-maven] Upgrade Maven to 3.6.3

2019-12-05 Thread GitBox
AmplabJenkins commented on issue #26770: [SPARK-30142][BUILD][test-maven] Upgrade Maven to 3.6.3 URL: https://github.com/apache/spark/pull/26770#issuecomment-562376882 Merged build finished. Test PASSed. This is an automated

[GitHub] [spark] SparkQA removed a comment on issue #26770: [SPARK-30142][BUILD][test-maven] Upgrade Maven to 3.6.3

2019-12-05 Thread GitBox
SparkQA removed a comment on issue #26770: [SPARK-30142][BUILD][test-maven] Upgrade Maven to 3.6.3 URL: https://github.com/apache/spark/pull/26770#issuecomment-562243238 **[Test build #114915 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/114915/testReport)

[GitHub] [spark] SparkQA commented on issue #26770: [SPARK-30142][BUILD][test-maven] Upgrade Maven to 3.6.3

2019-12-05 Thread GitBox
SparkQA commented on issue #26770: [SPARK-30142][BUILD][test-maven] Upgrade Maven to 3.6.3 URL: https://github.com/apache/spark/pull/26770#issuecomment-562376435 **[Test build #114915 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/114915/testReport)** for

[GitHub] [spark] yuchenhuo commented on issue #26614: [SPARK-29976][CORE] Trigger speculation for stages with too few tasks

2019-12-05 Thread GitBox
yuchenhuo commented on issue #26614: [SPARK-29976][CORE] Trigger speculation for stages with too few tasks URL: https://github.com/apache/spark/pull/26614#issuecomment-562374386 @tgravescs @jiangxb1987 Thanks for the feedbacks! I've updated the PR corresponding to the discussion. May I get

[GitHub] [spark] viirya commented on issue #26722: [SPARK-24666][ML] Fix infinity vectors produced by Word2Vec when numIterations are large

2019-12-05 Thread GitBox
viirya commented on issue #26722: [SPARK-24666][ML] Fix infinity vectors produced by Word2Vec when numIterations are large URL: https://github.com/apache/spark/pull/26722#issuecomment-562371645 Ok. I tried to increase number of partitions and iterations of existing test, but can not reprod

[GitHub] [spark] vanzin commented on issue #23820: [SPARK-22860][CORE][YARN] Redact command line arguments for running Driver and Executor before logging (standalone and YARN)

2019-12-05 Thread GitBox
vanzin commented on issue #23820: [SPARK-22860][CORE][YARN] Redact command line arguments for running Driver and Executor before logging (standalone and YARN) URL: https://github.com/apache/spark/pull/23820#issuecomment-562370025 Feel free to open a PR.

[GitHub] [spark] viirya edited a comment on issue #26722: [SPARK-24666][ML] Fix infinity vectors produced by Word2Vec when numIterations are large

2019-12-05 Thread GitBox
viirya edited a comment on issue #26722: [SPARK-24666][ML] Fix infinity vectors produced by Word2Vec when numIterations are large URL: https://github.com/apache/spark/pull/26722#issuecomment-562363790 Let me check if it is also reproducible with big partition number + iteration on existing

[GitHub] [spark] AmplabJenkins commented on issue #25514: [SPARK-28784][SS] Use CheckpointFileManager in StreamExecution/StreamingQueryManager for checkpoint dirs

2019-12-05 Thread GitBox
AmplabJenkins commented on issue #25514: [SPARK-28784][SS] Use CheckpointFileManager in StreamExecution/StreamingQueryManager for checkpoint dirs URL: https://github.com/apache/spark/pull/25514#issuecomment-562368721 Merged build finished. Test PASSed.

[GitHub] [spark] AmplabJenkins removed a comment on issue #25514: [SPARK-28784][SS] Use CheckpointFileManager in StreamExecution/StreamingQueryManager for checkpoint dirs

2019-12-05 Thread GitBox
AmplabJenkins removed a comment on issue #25514: [SPARK-28784][SS] Use CheckpointFileManager in StreamExecution/StreamingQueryManager for checkpoint dirs URL: https://github.com/apache/spark/pull/25514#issuecomment-562368726 Test PASSed. Refer to this link for build results (access righ

[GitHub] [spark] AmplabJenkins removed a comment on issue #25514: [SPARK-28784][SS] Use CheckpointFileManager in StreamExecution/StreamingQueryManager for checkpoint dirs

2019-12-05 Thread GitBox
AmplabJenkins removed a comment on issue #25514: [SPARK-28784][SS] Use CheckpointFileManager in StreamExecution/StreamingQueryManager for checkpoint dirs URL: https://github.com/apache/spark/pull/25514#issuecomment-562368721 Merged build finished. Test PASSed.

[GitHub] [spark] AmplabJenkins commented on issue #25514: [SPARK-28784][SS] Use CheckpointFileManager in StreamExecution/StreamingQueryManager for checkpoint dirs

2019-12-05 Thread GitBox
AmplabJenkins commented on issue #25514: [SPARK-28784][SS] Use CheckpointFileManager in StreamExecution/StreamingQueryManager for checkpoint dirs URL: https://github.com/apache/spark/pull/25514#issuecomment-562368726 Test PASSed. Refer to this link for build results (access rights to CI

[GitHub] [spark] tooptoop4 commented on issue #23059: [SPARK-26091][SQL] Upgrade to 2.3.4 for Hive Metastore Client 2.3

2019-12-05 Thread GitBox
tooptoop4 commented on issue #23059: [SPARK-26091][SQL] Upgrade to 2.3.4 for Hive Metastore Client 2.3 URL: https://github.com/apache/spark/pull/23059#issuecomment-562368501 can this go in spark 2.4.5? This is an automated me

[GitHub] [spark] SparkQA commented on issue #25514: [SPARK-28784][SS] Use CheckpointFileManager in StreamExecution/StreamingQueryManager for checkpoint dirs

2019-12-05 Thread GitBox
SparkQA commented on issue #25514: [SPARK-28784][SS] Use CheckpointFileManager in StreamExecution/StreamingQueryManager for checkpoint dirs URL: https://github.com/apache/spark/pull/25514#issuecomment-562368329 **[Test build #114916 has finished](https://amplab.cs.berkeley.edu/jenkins/job/

[GitHub] [spark] SparkQA removed a comment on issue #25514: [SPARK-28784][SS] Use CheckpointFileManager in StreamExecution/StreamingQueryManager for checkpoint dirs

2019-12-05 Thread GitBox
SparkQA removed a comment on issue #25514: [SPARK-28784][SS] Use CheckpointFileManager in StreamExecution/StreamingQueryManager for checkpoint dirs URL: https://github.com/apache/spark/pull/25514#issuecomment-562290562 **[Test build #114916 has started](https://amplab.cs.berkeley.edu/jenk

[GitHub] [spark] tooptoop4 commented on issue #23820: [SPARK-22860][CORE][YARN] Redact command line arguments for running Driver and Executor before logging (standalone and YARN)

2019-12-05 Thread GitBox
tooptoop4 commented on issue #23820: [SPARK-22860][CORE][YARN] Redact command line arguments for running Driver and Executor before logging (standalone and YARN) URL: https://github.com/apache/spark/pull/23820#issuecomment-562366993 @vanzin can this go in 2.4.5? --

[GitHub] [spark] viirya commented on issue #26722: [SPARK-24666][ML] Fix infinity vectors produced by Word2Vec when numIterations are large

2019-12-05 Thread GitBox
viirya commented on issue #26722: [SPARK-24666][ML] Fix infinity vectors produced by Word2Vec when numIterations are large URL: https://github.com/apache/spark/pull/26722#issuecomment-562363790 Let me check if it is also reproducible with big partition number + iteration on existing mllib

[GitHub] [spark] maropu commented on a change in pull request #26656: [SPARK-27986][SQL] Support ANSI SQL filter clause for aggregate expression

2019-12-05 Thread GitBox
maropu commented on a change in pull request #26656: [SPARK-27986][SQL] Support ANSI SQL filter clause for aggregate expression URL: https://github.com/apache/spark/pull/26656#discussion_r354595927 ## File path: sql/core/src/test/resources/sql-tests/inputs/postgreSQL/window_part3.s

[GitHub] [spark] mukulmurthy commented on a change in pull request #26771: [SPARK-30143][SS] Add a timeout on stopping a streaming query

2019-12-05 Thread GitBox
mukulmurthy commented on a change in pull request #26771: [SPARK-30143][SS] Add a timeout on stopping a streaming query URL: https://github.com/apache/spark/pull/26771#discussion_r354588985 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/StreamExecuti

[GitHub] [spark] mukulmurthy commented on a change in pull request #26771: [SPARK-30143][SS] Add a timeout on stopping a streaming query

2019-12-05 Thread GitBox
mukulmurthy commented on a change in pull request #26771: [SPARK-30143][SS] Add a timeout on stopping a streaming query URL: https://github.com/apache/spark/pull/26771#discussion_r354587841 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala #

[GitHub] [spark] mukulmurthy commented on a change in pull request #26771: [SPARK-30143][SS] Add a timeout on stopping a streaming query

2019-12-05 Thread GitBox
mukulmurthy commented on a change in pull request #26771: [SPARK-30143][SS] Add a timeout on stopping a streaming query URL: https://github.com/apache/spark/pull/26771#discussion_r354591734 ## File path: sql/core/src/test/scala/org/apache/spark/sql/execution/streaming/MicroBatchExe

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #26769: [SPARK-30011][SQL] Inline 2.12 "AsIfIntegral" classes, not present in 2.13

2019-12-05 Thread GitBox
dongjoon-hyun commented on a change in pull request #26769: [SPARK-30011][SQL] Inline 2.12 "AsIfIntegral" classes, not present in 2.13 URL: https://github.com/apache/spark/pull/26769#discussion_r35458 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/types/Decimal.s

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #26769: [SPARK-30011][SQL] Inline 2.12 "AsIfIntegral" classes, not present in 2.13

2019-12-05 Thread GitBox
dongjoon-hyun commented on a change in pull request #26769: [SPARK-30011][SQL] Inline 2.12 "AsIfIntegral" classes, not present in 2.13 URL: https://github.com/apache/spark/pull/26769#discussion_r35458 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/types/Decimal.s

[GitHub] [spark] AmplabJenkins removed a comment on issue #26771: [SPARK-30143][SS] Add a timeout on stopping a streaming query

2019-12-05 Thread GitBox
AmplabJenkins removed a comment on issue #26771: [SPARK-30143][SS] Add a timeout on stopping a streaming query URL: https://github.com/apache/spark/pull/26771#issuecomment-562350881 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.

[GitHub] [spark] SparkQA removed a comment on issue #26771: [SPARK-30143][SS] Add a timeout on stopping a streaming query

2019-12-05 Thread GitBox
SparkQA removed a comment on issue #26771: [SPARK-30143][SS] Add a timeout on stopping a streaming query URL: https://github.com/apache/spark/pull/26771#issuecomment-562350455 **[Test build #114917 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/114917/testR

[GitHub] [spark] AmplabJenkins removed a comment on issue #26771: [SPARK-30143][SS] Add a timeout on stopping a streaming query

2019-12-05 Thread GitBox
AmplabJenkins removed a comment on issue #26771: [SPARK-30143][SS] Add a timeout on stopping a streaming query URL: https://github.com/apache/spark/pull/26771#issuecomment-562350968 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab

[GitHub] [spark] AmplabJenkins removed a comment on issue #26771: [SPARK-30143][SS] Add a timeout on stopping a streaming query

2019-12-05 Thread GitBox
AmplabJenkins removed a comment on issue #26771: [SPARK-30143][SS] Add a timeout on stopping a streaming query URL: https://github.com/apache/spark/pull/26771#issuecomment-562350957 Merged build finished. Test PASSed. This is

[GitHub] [spark] AmplabJenkins removed a comment on issue #26771: [SPARK-30143][SS] Add a timeout on stopping a streaming query

2019-12-05 Thread GitBox
AmplabJenkins removed a comment on issue #26771: [SPARK-30143][SS] Add a timeout on stopping a streaming query URL: https://github.com/apache/spark/pull/26771#issuecomment-562350873 Merged build finished. Test FAILed. This is

[GitHub] [spark] AmplabJenkins commented on issue #26771: [SPARK-30143][SS] Add a timeout on stopping a streaming query

2019-12-05 Thread GitBox
AmplabJenkins commented on issue #26771: [SPARK-30143][SS] Add a timeout on stopping a streaming query URL: https://github.com/apache/spark/pull/26771#issuecomment-562350873 Merged build finished. Test FAILed. This is an auto

[GitHub] [spark] AmplabJenkins commented on issue #26771: [SPARK-30143][SS] Add a timeout on stopping a streaming query

2019-12-05 Thread GitBox
AmplabJenkins commented on issue #26771: [SPARK-30143][SS] Add a timeout on stopping a streaming query URL: https://github.com/apache/spark/pull/26771#issuecomment-562350968 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berk

[GitHub] [spark] AmplabJenkins commented on issue #26771: [SPARK-30143][SS] Add a timeout on stopping a streaming query

2019-12-05 Thread GitBox
AmplabJenkins commented on issue #26771: [SPARK-30143][SS] Add a timeout on stopping a streaming query URL: https://github.com/apache/spark/pull/26771#issuecomment-562350881 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berke

[GitHub] [spark] AmplabJenkins commented on issue #26771: [SPARK-30143][SS] Add a timeout on stopping a streaming query

2019-12-05 Thread GitBox
AmplabJenkins commented on issue #26771: [SPARK-30143][SS] Add a timeout on stopping a streaming query URL: https://github.com/apache/spark/pull/26771#issuecomment-562350957 Merged build finished. Test PASSed. This is an auto

[GitHub] [spark] SparkQA commented on issue #26771: [SPARK-30143][SS] Add a timeout on stopping a streaming query

2019-12-05 Thread GitBox
SparkQA commented on issue #26771: [SPARK-30143][SS] Add a timeout on stopping a streaming query URL: https://github.com/apache/spark/pull/26771#issuecomment-562350861 **[Test build #114917 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/114917/testReport)*

[GitHub] [spark] SparkQA commented on issue #26771: [SPARK-30143][SS] Add a timeout on stopping a streaming query

2019-12-05 Thread GitBox
SparkQA commented on issue #26771: [SPARK-30143][SS] Add a timeout on stopping a streaming query URL: https://github.com/apache/spark/pull/26771#issuecomment-562350455 **[Test build #114917 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/114917/testReport)**

[GitHub] [spark] SparkQA commented on issue #26590: [SPARK-29953][SS] Don't clean up source files for FileStreamSource if the files belong to the output of FileStreamSink

2019-12-05 Thread GitBox
SparkQA commented on issue #26590: [SPARK-29953][SS] Don't clean up source files for FileStreamSource if the files belong to the output of FileStreamSink URL: https://github.com/apache/spark/pull/26590#issuecomment-562350454 **[Test build #114918 has started](https://amplab.cs.berkeley.edu

[GitHub] [spark] HeartSaVioR commented on issue #26590: [SPARK-29953][SS] Don't clean up source files for FileStreamSource if the files belong to the output of FileStreamSink

2019-12-05 Thread GitBox
HeartSaVioR commented on issue #26590: [SPARK-29953][SS] Don't clean up source files for FileStreamSource if the files belong to the output of FileStreamSink URL: https://github.com/apache/spark/pull/26590#issuecomment-562350043 retest this, please -

[GitHub] [spark] brkyvz commented on issue #26771: [SPARK-30143][SS] Add a timeout on stopping a streaming query

2019-12-05 Thread GitBox
brkyvz commented on issue #26771: [SPARK-30143][SS] Add a timeout on stopping a streaming query URL: https://github.com/apache/spark/pull/26771#issuecomment-562349837 cc @zsxwing @tdas This is an automated message from the A

[GitHub] [spark] brkyvz opened a new pull request #26771: [SPARK-30143][SS] Add a timeout on stopping a streaming query

2019-12-05 Thread GitBox
brkyvz opened a new pull request #26771: [SPARK-30143][SS] Add a timeout on stopping a streaming query URL: https://github.com/apache/spark/pull/26771 ### What changes were proposed in this pull request? Add a timeout configuration for StreamingQuery.stop() ### Why are the cha

[GitHub] [spark] squito commented on issue #26633: [SPARK-29994][CORE] Add WILDCARD task location

2019-12-05 Thread GitBox
squito commented on issue #26633: [SPARK-29994][CORE] Add WILDCARD task location URL: https://github.com/apache/spark/pull/26633#issuecomment-562346943 are you sure that in your case the slowdown is even caused by SPARK-18886? Even when that is solved, you could still get end up with *one*

[GitHub] [spark] zsxwing commented on issue #26590: [SPARK-29953][SS] Don't clean up source files for FileStreamSource if the files belong to the output of FileStreamSink

2019-12-05 Thread GitBox
zsxwing commented on issue #26590: [SPARK-29953][SS] Don't clean up source files for FileStreamSource if the files belong to the output of FileStreamSink URL: https://github.com/apache/spark/pull/26590#issuecomment-562346871 LGTM. retest this please. Triggering another test since the

[GitHub] [spark] srowen commented on a change in pull request #26769: [SPARK-30011][SQL] Inline 2.12 "AsIfIntegral" classes, not present in 2.13

2019-12-05 Thread GitBox
srowen commented on a change in pull request #26769: [SPARK-30011][SQL] Inline 2.12 "AsIfIntegral" classes, not present in 2.13 URL: https://github.com/apache/spark/pull/26769#discussion_r354579360 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/types/Decimal.scala #

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #26769: [SPARK-30011][SQL] Inline 2.12 "AsIfIntegral" classes, not present in 2.13

2019-12-05 Thread GitBox
dongjoon-hyun commented on a change in pull request #26769: [SPARK-30011][SQL] Inline 2.12 "AsIfIntegral" classes, not present in 2.13 URL: https://github.com/apache/spark/pull/26769#discussion_r354578040 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/types/Decimal.s

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #26769: [SPARK-30011][SQL] Inline 2.12 "AsIfIntegral" classes, not present in 2.13

2019-12-05 Thread GitBox
dongjoon-hyun commented on a change in pull request #26769: [SPARK-30011][SQL] Inline 2.12 "AsIfIntegral" classes, not present in 2.13 URL: https://github.com/apache/spark/pull/26769#discussion_r354578268 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/types/DoubleTy

[GitHub] [spark] ueshin commented on a change in pull request #26512: [WIP][SPARK-29493][SQL] Arrow MapType support

2019-12-05 Thread GitBox
ueshin commented on a change in pull request #26512: [WIP][SPARK-29493][SQL] Arrow MapType support URL: https://github.com/apache/spark/pull/26512#discussion_r354558089 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/util/ArrowUtils.scala ## @@ -88,6 +90,16

[GitHub] [spark] ueshin commented on a change in pull request #26512: [WIP][SPARK-29493][SQL] Arrow MapType support

2019-12-05 Thread GitBox
ueshin commented on a change in pull request #26512: [WIP][SPARK-29493][SQL] Arrow MapType support URL: https://github.com/apache/spark/pull/26512#discussion_r354564602 ## File path: sql/core/src/test/scala/org/apache/spark/sql/execution/arrow/ArrowWriterSuite.scala ## @@

<    1   2   3   4   5   6   7   >