[GitHub] [spark] SparkQA commented on pull request #34575: [SPARK-37273][SQL] Support hidden file metadata columns in Spark SQL

2021-11-12 Thread GitBox
SparkQA commented on pull request #34575: URL: https://github.com/apache/spark/pull/34575#issuecomment-967796938 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49652/ -- This is an automated message from the Apache

[GitHub] [spark] SparkQA commented on pull request #34579: [SPARK-37314][K8S][BUILD] Upgrade kubernetes-client to 5.10.1

2021-11-12 Thread GitBox
SparkQA commented on pull request #34579: URL: https://github.com/apache/spark/pull/34579#issuecomment-967796776 **[Test build #145184 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145184/testReport)** for PR 34579 at commit [`0b683a8`](https://github.com

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33588: [SPARK-36346][SQL] Support TimestampNTZ type in Orc file source

2021-11-12 Thread GitBox
AmplabJenkins removed a comment on pull request #33588: URL: https://github.com/apache/spark/pull/33588#issuecomment-967796597 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145180/ -

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34579: [SPARK-37314][K8S][BUILD] Upgrade kubernetes-client to 5.10.1

2021-11-12 Thread GitBox
AmplabJenkins removed a comment on pull request #34579: URL: https://github.com/apache/spark/pull/34579#issuecomment-967796596 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145182/ -

[GitHub] [spark] AmplabJenkins commented on pull request #33588: [SPARK-36346][SQL] Support TimestampNTZ type in Orc file source

2021-11-12 Thread GitBox
AmplabJenkins commented on pull request #33588: URL: https://github.com/apache/spark/pull/33588#issuecomment-967796597 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145180/ -- This

[GitHub] [spark] AmplabJenkins commented on pull request #34579: [SPARK-37314][K8S][BUILD] Upgrade kubernetes-client to 5.10.1

2021-11-12 Thread GitBox
AmplabJenkins commented on pull request #34579: URL: https://github.com/apache/spark/pull/34579#issuecomment-967796596 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145182/ -- This

[GitHub] [spark] hgs19921112 opened a new pull request #34492: [SPARK-37216][SQL] Add the Hive macro functionality to SparkSQL

2021-11-12 Thread GitBox
hgs19921112 opened a new pull request #34492: URL: https://github.com/apache/spark/pull/34492 ### What changes were proposed in this pull request? Add the Hive macro functionality to SparkSQL ### Why are the changes needed? Some Hive sql can move to SparkSQL Smoothly

[GitHub] [spark] hgs19921112 closed pull request #34492: [SPARK-37216][SQL] Add the Hive macro functionality to SparkSQL

2021-11-12 Thread GitBox
hgs19921112 closed pull request #34492: URL: https://github.com/apache/spark/pull/34492 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-un

[GitHub] [spark] SparkQA removed a comment on pull request #34579: [SPARK-37314][K8S][BUILD] Upgrade kubernetes-client to 5.10.1

2021-11-12 Thread GitBox
SparkQA removed a comment on pull request #34579: URL: https://github.com/apache/spark/pull/34579#issuecomment-96255 **[Test build #145182 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145182/testReport)** for PR 34579 at commit [`0b683a8`](https://gi

[GitHub] [spark] SparkQA removed a comment on pull request #33588: [SPARK-36346][SQL] Support TimestampNTZ type in Orc file source

2021-11-12 Thread GitBox
SparkQA removed a comment on pull request #33588: URL: https://github.com/apache/spark/pull/33588#issuecomment-967763140 **[Test build #145180 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145180/testReport)** for PR 33588 at commit [`91331c7`](https://gi

[GitHub] [spark] SparkQA commented on pull request #34579: [SPARK-37314][K8S][BUILD] Upgrade kubernetes-client to 5.10.1

2021-11-12 Thread GitBox
SparkQA commented on pull request #34579: URL: https://github.com/apache/spark/pull/34579#issuecomment-967794722 **[Test build #145182 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145182/testReport)** for PR 34579 at commit [`0b683a8`](https://github.co

[GitHub] [spark] SparkQA commented on pull request #33588: [SPARK-36346][SQL] Support TimestampNTZ type in Orc file source

2021-11-12 Thread GitBox
SparkQA commented on pull request #33588: URL: https://github.com/apache/spark/pull/33588#issuecomment-967794238 **[Test build #145180 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145180/testReport)** for PR 33588 at commit [`91331c7`](https://github.co

[GitHub] [spark] srowen commented on pull request #34572: Don't obtain JDBC connection for empty partitions

2021-11-12 Thread GitBox
srowen commented on pull request #34572: URL: https://github.com/apache/spark/pull/34572#issuecomment-967794243 Oops I just forgot to link it -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [spark] dongjoon-hyun commented on pull request #34579: [SPARK-37314][K8S][BUILD] Upgrade kubernetes-client to 5.10.1

2021-11-12 Thread GitBox
dongjoon-hyun commented on pull request #34579: URL: https://github.com/apache/spark/pull/34579#issuecomment-967792488 There are two known failures, but this PR has another failure. Although it looks like a flaky one, I re-trigger tests to make it sure. ``` - Run SparkRemoteFileTest u

[GitHub] [spark] dongjoon-hyun commented on pull request #34579: [SPARK-37314][K8S][BUILD] Upgrade kubernetes-client to 5.10.1

2021-11-12 Thread GitBox
dongjoon-hyun commented on pull request #34579: URL: https://github.com/apache/spark/pull/34579#issuecomment-967792347 Retest this please -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the spec

[GitHub] [spark] dongjoon-hyun commented on pull request #34576: [SPARK-37282][TESTS][FOLLOWUP] Mark `YarnShuffleServiceSuite` as ExtendedLevelDBTest

2021-11-12 Thread GitBox
dongjoon-hyun commented on pull request #34576: URL: https://github.com/apache/spark/pull/34576#issuecomment-967792227 Thank you so much! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the spec

[GitHub] [spark] HyukjinKwon closed pull request #34574: [SPARK-37139][PYTHON][FOLLOWUP] Fix class variable type hints in taskcontext.py

2021-11-12 Thread GitBox
HyukjinKwon closed pull request #34574: URL: https://github.com/apache/spark/pull/34574 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-un

[GitHub] [spark] HyukjinKwon commented on pull request #34574: [SPARK-37139][PYTHON][FOLLOWUP] Fix class variable type hints in taskcontext.py

2021-11-12 Thread GitBox
HyukjinKwon commented on pull request #34574: URL: https://github.com/apache/spark/pull/34574#issuecomment-967792124 Merged to master. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specifi

[GitHub] [spark] HyukjinKwon commented on pull request #34572: Don't obtain JDBC connection for empty partitions

2021-11-12 Thread GitBox
HyukjinKwon commented on pull request #34572: URL: https://github.com/apache/spark/pull/34572#issuecomment-967792085 @srowen, it's sort of minor but I think it might better to create a JIRA (since it's a perf improvement technically). -- This is an automated message from the Apache Git S

[GitHub] [spark] SparkQA commented on pull request #34575: [SPARK-37273][SQL] Support hidden file metadata columns in Spark SQL

2021-11-12 Thread GitBox
SparkQA commented on pull request #34575: URL: https://github.com/apache/spark/pull/34575#issuecomment-967792027 **[Test build #145183 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145183/testReport)** for PR 34575 at commit [`fc043fd`](https://github.com

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34575: [SPARK-37273][SQL] Support hidden file metadata columns in Spark SQL

2021-11-12 Thread GitBox
AmplabJenkins removed a comment on pull request #34575: URL: https://github.com/apache/spark/pull/34575#issuecomment-967680792 Can one of the admins verify this patch? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use t

[GitHub] [spark] HyukjinKwon commented on pull request #34575: [SPARK-37273][SQL] Support hidden file metadata columns in Spark SQL

2021-11-12 Thread GitBox
HyukjinKwon commented on pull request #34575: URL: https://github.com/apache/spark/pull/34575#issuecomment-967791845 ok to test -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comme

[GitHub] [spark] sarutak commented on pull request #34576: [SPARK-37282][TESTS][FOLLOWUP] Mark `YarnShuffleServiceSuite` as ExtendedLevelDBTest

2021-11-12 Thread GitBox
sarutak commented on pull request #34576: URL: https://github.com/apache/spark/pull/34576#issuecomment-967790839 Merged to `master`. Thank you @dongjoon-hyun ! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL a

[GitHub] [spark] sarutak closed pull request #34576: [SPARK-37282][TESTS][FOLLOWUP] Mark `YarnShuffleServiceSuite` as ExtendedLevelDBTest

2021-11-12 Thread GitBox
sarutak closed pull request #34576: URL: https://github.com/apache/spark/pull/34576 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubs

[GitHub] [spark] HyukjinKwon commented on pull request #34555: [SPARK-37288][PYTHON][3.2] Backport since annotation update

2021-11-12 Thread GitBox
HyukjinKwon commented on pull request #34555: URL: https://github.com/apache/spark/pull/34555#issuecomment-967790800 I switched the JIRA to a bug, and described which bug this PR fixes. It wouldn't introduce any user-facing behaviour. cc @dongjoon-hyun. -- This is an automated message fr

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34579: [SPARK-37314][K8S][BUILD] Upgrade kubernetes-client to 5.10.1

2021-11-12 Thread GitBox
AmplabJenkins removed a comment on pull request #34579: URL: https://github.com/apache/spark/pull/34579#issuecomment-967790143 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/49651/

[GitHub] [spark] AmplabJenkins commented on pull request #34543: [SPARK-37266][SQL] View text can only be SELECT queries

2021-11-12 Thread GitBox
AmplabJenkins commented on pull request #34543: URL: https://github.com/apache/spark/pull/34543#issuecomment-967790208 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145178/ -- This

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34543: [SPARK-37266][SQL] View text can only be SELECT queries

2021-11-12 Thread GitBox
AmplabJenkins removed a comment on pull request #34543: URL: https://github.com/apache/spark/pull/34543#issuecomment-967790208 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145178/ -

[GitHub] [spark] AmplabJenkins commented on pull request #34579: [SPARK-37314][K8S][BUILD] Upgrade kubernetes-client to 5.10.1

2021-11-12 Thread GitBox
AmplabJenkins commented on pull request #34579: URL: https://github.com/apache/spark/pull/34579#issuecomment-967790143 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/49651/ -- T

[GitHub] [spark] SparkQA removed a comment on pull request #34543: [SPARK-37266][SQL] View text can only be SELECT queries

2021-11-12 Thread GitBox
SparkQA removed a comment on pull request #34543: URL: https://github.com/apache/spark/pull/34543#issuecomment-967752729 **[Test build #145178 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145178/testReport)** for PR 34543 at commit [`0f46831`](https://gi

[GitHub] [spark] SparkQA commented on pull request #34543: [SPARK-37266][SQL] View text can only be SELECT queries

2021-11-12 Thread GitBox
SparkQA commented on pull request #34543: URL: https://github.com/apache/spark/pull/34543#issuecomment-967789997 **[Test build #145178 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145178/testReport)** for PR 34543 at commit [`0f46831`](https://github.co

[GitHub] [spark] dongjoon-hyun commented on pull request #34576: [SPARK-37282][TESTS][FOLLOWUP] Mark `YarnShuffleServiceSuite` as ExtendedLevelDBTest

2021-11-12 Thread GitBox
dongjoon-hyun commented on pull request #34576: URL: https://github.com/apache/spark/pull/34576#issuecomment-967788291 Could you review this, @sarutak ? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [spark] dongjoon-hyun edited a comment on pull request #34577: [SPARK-37312][TESTS] Add `.java-version` to `.gitignore` and `.rat-excludes`

2021-11-12 Thread GitBox
dongjoon-hyun edited a comment on pull request #34577: URL: https://github.com/apache/spark/pull/34577#issuecomment-967788067 Thank you, @viirya and @sarutak ! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL a

[GitHub] [spark] dongjoon-hyun commented on pull request #34577: [SPARK-37312][TESTS] Add `.java-version` to `.gitignore` and `.rat-excludes`

2021-11-12 Thread GitBox
dongjoon-hyun commented on pull request #34577: URL: https://github.com/apache/spark/pull/34577#issuecomment-967788067 Thank you, @sarutak ! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the s

[GitHub] [spark] SparkQA commented on pull request #34579: [SPARK-37314][K8S][BUILD] Upgrade kubernetes-client to 5.10.1

2021-11-12 Thread GitBox
SparkQA commented on pull request #34579: URL: https://github.com/apache/spark/pull/34579#issuecomment-967787532 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49651/ -- This is an automated message from the A

[GitHub] [spark] sarutak closed pull request #34577: [SPARK-37312][TESTS] Add `.java-version` to `.gitignore` and `.rat-excludes`

2021-11-12 Thread GitBox
sarutak closed pull request #34577: URL: https://github.com/apache/spark/pull/34577 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubs

[GitHub] [spark] sarutak commented on pull request #34577: [SPARK-37312][TESTS] Add `.java-version` to `.gitignore` and `.rat-excludes`

2021-11-12 Thread GitBox
sarutak commented on pull request #34577: URL: https://github.com/apache/spark/pull/34577#issuecomment-967786943 LGTM. Merging to `master`. Thank you @dongjoon-hyun ! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use t

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34504: [SPARK-37226][SQL] Filter push down through window

2021-11-12 Thread GitBox
AmplabJenkins removed a comment on pull request #34504: URL: https://github.com/apache/spark/pull/34504#issuecomment-967784061 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/49650/

[GitHub] [spark] AmplabJenkins commented on pull request #34504: [SPARK-37226][SQL] Filter push down through window

2021-11-12 Thread GitBox
AmplabJenkins commented on pull request #34504: URL: https://github.com/apache/spark/pull/34504#issuecomment-967784061 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/49650/ -- T

[GitHub] [spark] SparkQA commented on pull request #34579: [SPARK-37314][K8S][BUILD] Upgrade kubernetes-client to 5.10.1

2021-11-12 Thread GitBox
SparkQA commented on pull request #34579: URL: https://github.com/apache/spark/pull/34579#issuecomment-967782600 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49651/ -- This is an automated message from the Apache

[GitHub] [spark] SparkQA commented on pull request #34504: [SPARK-37226][SQL] Filter push down through window

2021-11-12 Thread GitBox
SparkQA commented on pull request #34504: URL: https://github.com/apache/spark/pull/34504#issuecomment-967781922 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49650/ -- This is an automated message from the A

[GitHub] [spark] LuciferYang commented on pull request #34532: [SPARK-37256][SQL] Replace `ScalaObjectMapper` with `ClassTagExtensions` to fix compilation warning

2021-11-12 Thread GitBox
LuciferYang commented on pull request #34532: URL: https://github.com/apache/spark/pull/34532#issuecomment-967778794 Should we merge this ? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the sp

[GitHub] [spark] LuciferYang commented on pull request #34454: [SPARK-37013][CORE][SQL][FOLLOWUP] Use the new error framework to throw error in `FormatString`

2021-11-12 Thread GitBox
LuciferYang commented on pull request #34454: URL: https://github.com/apache/spark/pull/34454#issuecomment-967778711 Anything else needs to be changed? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL a

[GitHub] [spark] SparkQA commented on pull request #34579: [SPARK-37314][K8S][BUILD] Upgrade kubernetes-client to 5.10.1

2021-11-12 Thread GitBox
SparkQA commented on pull request #34579: URL: https://github.com/apache/spark/pull/34579#issuecomment-96255 **[Test build #145182 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145182/testReport)** for PR 34579 at commit [`0b683a8`](https://github.com

[GitHub] [spark] sarutak opened a new pull request #34579: [SPARK-37314][K8S][BUILD] Upgrade kubernetes-client to 5.10.1

2021-11-12 Thread GitBox
sarutak opened a new pull request #34579: URL: https://github.com/apache/spark/pull/34579 ### What changes were proposed in this pull request? This PR upgrades kubernetes-client from `5.9.0` to `5.10.1`. ### Why are the changes needed? kubernetes-client 5.10.0 and 5.10.1

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34577: [SPARK-37312][TESTS] Add `.java-version` to `.gitignore` and `.rat-excludes`

2021-11-12 Thread GitBox
AmplabJenkins removed a comment on pull request #34577: URL: https://github.com/apache/spark/pull/34577#issuecomment-967776695 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145179/ -

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33008: [WIP][SPARK-35801][SQL] Support DELETE operations that require rewriting data

2021-11-12 Thread GitBox
AmplabJenkins removed a comment on pull request #33008: URL: https://github.com/apache/spark/pull/33008#issuecomment-967776577 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145176/ -

[GitHub] [spark] AmplabJenkins commented on pull request #34577: [SPARK-37312][TESTS] Add `.java-version` to `.gitignore` and `.rat-excludes`

2021-11-12 Thread GitBox
AmplabJenkins commented on pull request #34577: URL: https://github.com/apache/spark/pull/34577#issuecomment-967776695 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145179/ -- This

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33588: [SPARK-36346][SQL] Support TimestampNTZ type in Orc file source

2021-11-12 Thread GitBox
AmplabJenkins removed a comment on pull request #33588: URL: https://github.com/apache/spark/pull/33588#issuecomment-967776576 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/49649/

[GitHub] [spark] AmplabJenkins commented on pull request #33588: [SPARK-36346][SQL] Support TimestampNTZ type in Orc file source

2021-11-12 Thread GitBox
AmplabJenkins commented on pull request #33588: URL: https://github.com/apache/spark/pull/33588#issuecomment-967776576 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/49649/ -- T

[GitHub] [spark] AmplabJenkins commented on pull request #33008: [WIP][SPARK-35801][SQL] Support DELETE operations that require rewriting data

2021-11-12 Thread GitBox
AmplabJenkins commented on pull request #33008: URL: https://github.com/apache/spark/pull/33008#issuecomment-967776577 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145176/ -- This

[GitHub] [spark] SparkQA removed a comment on pull request #34577: [SPARK-37312][TESTS] Add `.java-version` to `.gitignore` and `.rat-excludes`

2021-11-12 Thread GitBox
SparkQA removed a comment on pull request #34577: URL: https://github.com/apache/spark/pull/34577#issuecomment-967755244 **[Test build #145179 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145179/testReport)** for PR 34577 at commit [`6d0a9a4`](https://gi

[GitHub] [spark] SparkQA commented on pull request #34577: [SPARK-37312][TESTS] Add `.java-version` to `.gitignore` and `.rat-excludes`

2021-11-12 Thread GitBox
SparkQA commented on pull request #34577: URL: https://github.com/apache/spark/pull/34577#issuecomment-967776464 **[Test build #145179 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145179/testReport)** for PR 34577 at commit [`6d0a9a4`](https://github.co

[GitHub] [spark] SparkQA removed a comment on pull request #33008: [WIP][SPARK-35801][SQL] Support DELETE operations that require rewriting data

2021-11-12 Thread GitBox
SparkQA removed a comment on pull request #33008: URL: https://github.com/apache/spark/pull/33008#issuecomment-967722938 **[Test build #145176 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145176/testReport)** for PR 33008 at commit [`5e95a57`](https://gi

[GitHub] [spark] SparkQA commented on pull request #33008: [WIP][SPARK-35801][SQL] Support DELETE operations that require rewriting data

2021-11-12 Thread GitBox
SparkQA commented on pull request #33008: URL: https://github.com/apache/spark/pull/33008#issuecomment-967775164 **[Test build #145176 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145176/testReport)** for PR 33008 at commit [`5e95a57`](https://github.co

[GitHub] [spark] SparkQA commented on pull request #34504: [SPARK-37226][SQL] Filter push down through window

2021-11-12 Thread GitBox
SparkQA commented on pull request #34504: URL: https://github.com/apache/spark/pull/34504#issuecomment-967774937 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49650/ -- This is an automated message from the Apache

[GitHub] [spark] SparkQA commented on pull request #33588: [SPARK-36346][SQL] Support TimestampNTZ type in Orc file source

2021-11-12 Thread GitBox
SparkQA commented on pull request #33588: URL: https://github.com/apache/spark/pull/33588#issuecomment-967773338 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49649/ -- This is an automated message from the A

[GitHub] [spark] SparkQA commented on pull request #34504: [SPARK-37226][SQL] Filter push down through window

2021-11-12 Thread GitBox
SparkQA commented on pull request #34504: URL: https://github.com/apache/spark/pull/34504#issuecomment-967770239 **[Test build #145181 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145181/testReport)** for PR 34504 at commit [`a3a7648`](https://github.com

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34577: [SPARK-37312][TESTS] Add `.java-version` to `.gitignore` and `.rat-excludes`

2021-11-12 Thread GitBox
AmplabJenkins removed a comment on pull request #34577: URL: https://github.com/apache/spark/pull/34577#issuecomment-967770093 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/49648/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34576: [SPARK-37282][TESTS][FOLLOWUP] Mark `YarnShuffleServiceSuite` as ExtendedLevelDBTest

2021-11-12 Thread GitBox
AmplabJenkins removed a comment on pull request #34576: URL: https://github.com/apache/spark/pull/34576#issuecomment-967770094 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/49646/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34543: [SPARK-37266][SQL] View text can only be SELECT queries

2021-11-12 Thread GitBox
AmplabJenkins removed a comment on pull request #34543: URL: https://github.com/apache/spark/pull/34543#issuecomment-967770095 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/49647/

[GitHub] [spark] AmplabJenkins commented on pull request #34577: [SPARK-37312][TESTS] Add `.java-version` to `.gitignore` and `.rat-excludes`

2021-11-12 Thread GitBox
AmplabJenkins commented on pull request #34577: URL: https://github.com/apache/spark/pull/34577#issuecomment-967770093 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/49648/ -- T

[GitHub] [spark] AmplabJenkins commented on pull request #34543: [SPARK-37266][SQL] View text can only be SELECT queries

2021-11-12 Thread GitBox
AmplabJenkins commented on pull request #34543: URL: https://github.com/apache/spark/pull/34543#issuecomment-967770095 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/49647/ -- T

[GitHub] [spark] AmplabJenkins commented on pull request #34576: [SPARK-37282][TESTS][FOLLOWUP] Mark `YarnShuffleServiceSuite` as ExtendedLevelDBTest

2021-11-12 Thread GitBox
AmplabJenkins commented on pull request #34576: URL: https://github.com/apache/spark/pull/34576#issuecomment-967770094 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/49646/ -- T

[GitHub] [spark] SparkQA commented on pull request #34576: [SPARK-37282][TESTS][FOLLOWUP] Mark `YarnShuffleServiceSuite` as ExtendedLevelDBTest

2021-11-12 Thread GitBox
SparkQA commented on pull request #34576: URL: https://github.com/apache/spark/pull/34576#issuecomment-967769116 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49646/ -- This is an automated message from the A

[GitHub] [spark] SparkQA commented on pull request #33588: [SPARK-36346][SQL] Support TimestampNTZ type in Orc file source

2021-11-12 Thread GitBox
SparkQA commented on pull request #33588: URL: https://github.com/apache/spark/pull/33588#issuecomment-967768287 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49649/ -- This is an automated message from the Apache

[GitHub] [spark] rmcyang commented on pull request #34461: [SPARK-37023][CORE] Avoid fetching merge status when shuffleMergeEnabled is false for a shuffleDependency during retry

2021-11-12 Thread GitBox
rmcyang commented on pull request #34461: URL: https://github.com/apache/spark/pull/34461#issuecomment-967767497 Filed SPARK-37313 for the issue that @Ngone51 raised. Thanks for the reviews! @mridulm @dongjoon-hyun @Ngone51 -- This is an automated message from the Apache Git Service.

[GitHub] [spark] SparkQA commented on pull request #34577: [SPARK-37312][TESTS] Add `.java-version` to `.gitignore` and `.rat-excludes`

2021-11-12 Thread GitBox
SparkQA commented on pull request #34577: URL: https://github.com/apache/spark/pull/34577#issuecomment-967767066 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49648/ -- This is an automated message from the A

[GitHub] [spark] SparkQA commented on pull request #34543: [SPARK-37266][SQL] View text can only be SELECT queries

2021-11-12 Thread GitBox
SparkQA commented on pull request #34543: URL: https://github.com/apache/spark/pull/34543#issuecomment-967766557 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49647/ -- This is an automated message from the A

[GitHub] [spark] sleep1661 commented on pull request #33872: [SPARK-36575][CORE] Should ignore task finished event if its task set is gone in TaskSchedulerImpl.handleSuccessfulTask

2021-11-12 Thread GitBox
sleep1661 commented on pull request #33872: URL: https://github.com/apache/spark/pull/33872#issuecomment-967763573 @Ngone51 @mridulm I had created new PR (https://github.com/apache/spark/pull/34578) for this. BTW sorry about the unit test problems caused by this PR. -- This is an au

[GitHub] [spark] SparkQA commented on pull request #33588: [SPARK-36346][SQL] Support TimestampNTZ type in Orc file source

2021-11-12 Thread GitBox
SparkQA commented on pull request #33588: URL: https://github.com/apache/spark/pull/33588#issuecomment-967763140 **[Test build #145180 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145180/testReport)** for PR 33588 at commit [`91331c7`](https://github.com

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34576: [SPARK-37282][TESTS][FOLLOWUP] Mark `YarnShuffleServiceSuite` as ExtendedLevelDBTest

2021-11-12 Thread GitBox
AmplabJenkins removed a comment on pull request #34576: URL: https://github.com/apache/spark/pull/34576#issuecomment-967762581 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145177/ -

[GitHub] [spark] AmplabJenkins commented on pull request #34578: [SPARK-37300][CORE] TaskSchedulerImpl should ignore task finished eve…

2021-11-12 Thread GitBox
AmplabJenkins commented on pull request #34578: URL: https://github.com/apache/spark/pull/34578#issuecomment-967762640 Can one of the admins verify this patch? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL a

[GitHub] [spark] AmplabJenkins commented on pull request #34576: [SPARK-37282][TESTS][FOLLOWUP] Mark `YarnShuffleServiceSuite` as ExtendedLevelDBTest

2021-11-12 Thread GitBox
AmplabJenkins commented on pull request #34576: URL: https://github.com/apache/spark/pull/34576#issuecomment-967762581 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145177/ -- This

[GitHub] [spark] SparkQA commented on pull request #34577: [SPARK-37312][TESTS] Add `.java-version` to `.gitignore` and `.rat-excludes`

2021-11-12 Thread GitBox
SparkQA commented on pull request #34577: URL: https://github.com/apache/spark/pull/34577#issuecomment-967761980 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49648/ -- This is an automated message from the Apache

[GitHub] [spark] SparkQA commented on pull request #34576: [SPARK-37282][TESTS][FOLLOWUP] Mark `YarnShuffleServiceSuite` as ExtendedLevelDBTest

2021-11-12 Thread GitBox
SparkQA commented on pull request #34576: URL: https://github.com/apache/spark/pull/34576#issuecomment-967761226 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49646/ -- This is an automated message from the Apache

[GitHub] [spark] SparkQA removed a comment on pull request #34576: [SPARK-37282][TESTS][FOLLOWUP] Mark `YarnShuffleServiceSuite` as ExtendedLevelDBTest

2021-11-12 Thread GitBox
SparkQA removed a comment on pull request #34576: URL: https://github.com/apache/spark/pull/34576#issuecomment-967752700 **[Test build #145177 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145177/testReport)** for PR 34576 at commit [`3c7494f`](https://gi

[GitHub] [spark] SparkQA commented on pull request #34543: [SPARK-37266][SQL] View text can only be SELECT queries

2021-11-12 Thread GitBox
SparkQA commented on pull request #34543: URL: https://github.com/apache/spark/pull/34543#issuecomment-967760254 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49647/ -- This is an automated message from the Apache

[GitHub] [spark] wangyum commented on pull request #34504: [SPARK-37226][SQL] Filter push down through window

2021-11-12 Thread GitBox
wangyum commented on pull request #34504: URL: https://github.com/apache/spark/pull/34504#issuecomment-967757751 @tanelk I did a benchmark and this implementation is faster than SPARK-37099. ```scala import org.apache.spark.benchmark.Benchmark val numRows = 1024 * 1024 * 20 spark

[GitHub] [spark] sleep1661 opened a new pull request #34578: [SPARK-37300][CORE] TaskSchedulerImpl should ignore task finished eve…

2021-11-12 Thread GitBox
sleep1661 opened a new pull request #34578: URL: https://github.com/apache/spark/pull/34578 ### What changes were proposed in this pull request? `TaskSchedulerImpl` handle task finished event at `handleSuccessfulTask` and `handleFailedTask` , but in some case the task was alr

[GitHub] [spark] SparkQA commented on pull request #34576: [SPARK-37282][TESTS][FOLLOWUP] Mark `YarnShuffleServiceSuite` as ExtendedLevelDBTest

2021-11-12 Thread GitBox
SparkQA commented on pull request #34576: URL: https://github.com/apache/spark/pull/34576#issuecomment-967756785 **[Test build #145177 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145177/testReport)** for PR 34576 at commit [`3c7494f`](https://github.co

[GitHub] [spark] SparkQA commented on pull request #34577: [SPARK-37312][TESTS] Add `.java-version` to `.gitignore` and `.rat-excludes`

2021-11-12 Thread GitBox
SparkQA commented on pull request #34577: URL: https://github.com/apache/spark/pull/34577#issuecomment-967755244 **[Test build #145179 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145179/testReport)** for PR 34577 at commit [`6d0a9a4`](https://github.com

[GitHub] [spark] dongjoon-hyun opened a new pull request #34577: [SPARK-37312][TESTS] Add `.java-version` to `.gitignore` and `.rat-excludes`

2021-11-12 Thread GitBox
dongjoon-hyun opened a new pull request #34577: URL: https://github.com/apache/spark/pull/34577 ### What changes were proposed in this pull request? To support Java 8/11/17 test more easily, this PR aims to add `.java-version` to `.gitignore` and `.rat-excludes`. ### Why are t

[GitHub] [spark] SparkQA commented on pull request #34543: [SPARK-37266][SQL] View text can only be SELECT queries

2021-11-12 Thread GitBox
SparkQA commented on pull request #34543: URL: https://github.com/apache/spark/pull/34543#issuecomment-967752729 **[Test build #145178 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145178/testReport)** for PR 34543 at commit [`0f46831`](https://github.com

[GitHub] [spark] SparkQA commented on pull request #34576: [SPARK-37282][TESTS][FOLLOWUP] Mark `YarnShuffleServiceSuite` as ExtendedLevelDBTest

2021-11-12 Thread GitBox
SparkQA commented on pull request #34576: URL: https://github.com/apache/spark/pull/34576#issuecomment-967752700 **[Test build #145177 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145177/testReport)** for PR 34576 at commit [`3c7494f`](https://github.com

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33008: [WIP][SPARK-35801][SQL] Support DELETE operations that require rewriting data

2021-11-12 Thread GitBox
AmplabJenkins removed a comment on pull request #33008: URL: https://github.com/apache/spark/pull/33008#issuecomment-967752566 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/49645/

[GitHub] [spark] AmplabJenkins commented on pull request #33008: [WIP][SPARK-35801][SQL] Support DELETE operations that require rewriting data

2021-11-12 Thread GitBox
AmplabJenkins commented on pull request #33008: URL: https://github.com/apache/spark/pull/33008#issuecomment-967752566 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/49645/ -- T

[GitHub] [spark] dongjoon-hyun opened a new pull request #34576: [SPARK-37282][TESTS][FOLLOWUP] Mark `YarnShuffleServiceSuite` as ExtendedLevelDBTest

2021-11-12 Thread GitBox
dongjoon-hyun opened a new pull request #34576: URL: https://github.com/apache/spark/pull/34576 ### What changes were proposed in this pull request? This PR is a follow-up of #34548. This is missed due to `-Pyarn` profile. ### Why are the changes needed? This is required

[GitHub] [spark] SparkQA commented on pull request #33008: [WIP][SPARK-35801][SQL] Support DELETE operations that require rewriting data

2021-11-12 Thread GitBox
SparkQA commented on pull request #33008: URL: https://github.com/apache/spark/pull/33008#issuecomment-967748148 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49645/ -- This is an automated message from the A

[GitHub] [spark] HyukjinKwon commented on a change in pull request #34396: [SPARK-37124][SQL] Support RowToColumnarExec with Arrow format

2021-11-12 Thread GitBox
HyukjinKwon commented on a change in pull request #34396: URL: https://github.com/apache/spark/pull/34396#discussion_r748657399 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/Columnar.scala ## @@ -458,6 +462,34 @@ case class RowToColumnarExec(child: SparkP

[GitHub] [spark] HyukjinKwon commented on a change in pull request #34396: [SPARK-37124][SQL] Support RowToColumnarExec with Arrow format

2021-11-12 Thread GitBox
HyukjinKwon commented on a change in pull request #34396: URL: https://github.com/apache/spark/pull/34396#discussion_r748657399 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/Columnar.scala ## @@ -458,6 +462,34 @@ case class RowToColumnarExec(child: SparkP

[GitHub] [spark] github-actions[bot] closed pull request #33114: [SPARK-35913][SQL] Create hive permanent function with owner name

2021-11-12 Thread GitBox
github-actions[bot] closed pull request #33114: URL: https://github.com/apache/spark/pull/33114 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: re

[GitHub] [spark] github-actions[bot] commented on pull request #27432: [SPARK-28325][SQL] Support ANSI SQL: SIMILAR TO ... ESCAPE syntax

2021-11-12 Thread GitBox
github-actions[bot] commented on pull request #27432: URL: https://github.com/apache/spark/pull/27432#issuecomment-967738019 We're closing this PR because it hasn't been updated in a while. This isn't a judgement on the merit of the PR in any way. It's just a way of keeping the PR queue ma

[GitHub] [spark] github-actions[bot] closed pull request #33520: [SPARK-36289][SQL] Rewrite distinct count case when expressions without Expand node

2021-11-12 Thread GitBox
github-actions[bot] closed pull request #33520: URL: https://github.com/apache/spark/pull/33520 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: re

[GitHub] [spark] github-actions[bot] commented on pull request #32979: [SPARK-35828][K8S] Skip retrieving the non-exist driver pod for client mode

2021-11-12 Thread GitBox
github-actions[bot] commented on pull request #32979: URL: https://github.com/apache/spark/pull/32979#issuecomment-967738007 We're closing this PR because it hasn't been updated in a while. This isn't a judgement on the merit of the PR in any way. It's just a way of keeping the PR queue ma

[GitHub] [spark] github-actions[bot] closed pull request #33361: [SPARK-36155][SQL] Eliminate outer join base uniqueness

2021-11-12 Thread GitBox
github-actions[bot] closed pull request #33361: URL: https://github.com/apache/spark/pull/33361 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: re

[GitHub] [spark] BryanCutler commented on a change in pull request #34396: [SPARK-37124][SQL] Support RowToColumnarExec with Arrow format

2021-11-12 Thread GitBox
BryanCutler commented on a change in pull request #34396: URL: https://github.com/apache/spark/pull/34396#discussion_r748648880 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/Columnar.scala ## @@ -458,6 +462,34 @@ case class RowToColumnarExec(child: SparkP

[GitHub] [spark] brkyvz commented on a change in pull request #34575: [SPARK-37273][SQL] Support hidden file metadata columns in Spark SQL

2021-11-12 Thread GitBox
brkyvz commented on a change in pull request #34575: URL: https://github.com/apache/spark/pull/34575#discussion_r748647686 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/LogicalPlan.scala ## @@ -276,3 +276,10 @@ object LogicalPlanIntegrit

[GitHub] [spark] brkyvz commented on a change in pull request #34575: [SPARK-37273][SQL] Support hidden file metadata columns in Spark SQL

2021-11-12 Thread GitBox
brkyvz commented on a change in pull request #34575: URL: https://github.com/apache/spark/pull/34575#discussion_r748647591 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/namedExpressions.scala ## @@ -438,3 +438,70 @@ object VirtualColumn {

[GitHub] [spark] brkyvz commented on pull request #34575: [SPARK-37273][SQL] Support hidden file metadata columns in Spark SQL

2021-11-12 Thread GitBox
brkyvz commented on pull request #34575: URL: https://github.com/apache/spark/pull/34575#issuecomment-967735222 ok to test -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

  1   2   3   4   5   >