[GitHub] [spark] dongjoon-hyun commented on pull request #33903: [SPARK-36656][SQL] CollapseProject should not collapse correlated scalar subqueries

dongjoon-hyun commented on pull request #33903: URL: https://github.com/apache/spark/pull/33903#issuecomment-912922559 cc @sunchao -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific c

[GitHub] [spark] dongjoon-hyun commented on pull request #33903: [SPARK-36656][SQL] CollapseProject should not collapse correlated scalar subqueries

dongjoon-hyun commented on pull request #33903: URL: https://github.com/apache/spark/pull/33903#issuecomment-912922530 Hi, @allisonwang-db . I have two more questions. 1. According to the PR description, do we have another test case? > Both test cases throw the following error before t

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #33903: [SPARK-36656][SQL] CollapseProject should not collapse correlated scalar subqueries

dongjoon-hyun commented on a change in pull request #33903: URL: https://github.com/apache/spark/pull/33903#discussion_r702246883 ## File path: sql/core/src/test/scala/org/apache/spark/sql/SubquerySuite.scala ## @@ -1902,4 +1902,28 @@ class SubquerySuite extends QueryTest with

[GitHub] [spark] dongjoon-hyun closed pull request #33894: [SPARK-36643][SQL] Add more information in ERROR log while SparkConf is modified when spark.sql.legacy.setCommandRejectsSparkCoreConfs is set

dongjoon-hyun closed pull request #33894: URL: https://github.com/apache/spark/pull/33894 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-

[GitHub] [spark] dongjoon-hyun commented on pull request #33894: [SPARK-36643][SQL] Add more information in ERROR log while SparkConf is modified when spark.sql.legacy.setCommandRejectsSparkCoreConfs

dongjoon-hyun commented on pull request #33894: URL: https://github.com/apache/spark/pull/33894#issuecomment-912921397 Yep. Tests passed. Thank you, @senthh . Merged to master for Apache Spark 3.3.0. -- This is an automated message from the Apache Git Service. To respond to the message

[GitHub] [spark] dongjoon-hyun commented on pull request #33910: [SPARK-36666][SQL] Fix regression in AQEShuffleReadExec

dongjoon-hyun commented on pull request #33910: URL: https://github.com/apache/spark/pull/33910#issuecomment-912918618 Also, cc @gengliangwang -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to th

[GitHub] [spark] dongjoon-hyun edited a comment on pull request #33910: [SPARK-36666][SQL] Fix regression in AQEShuffleReadExec

dongjoon-hyun edited a comment on pull request #33910: URL: https://github.com/apache/spark/pull/33910#issuecomment-912918410 cc @cloud-fan , @viirya, @maryannxue , @ulysses-you , @sunchao -- This is an automated message from the Apache Git Service. To respond to the message, please log

[GitHub] [spark] dongjoon-hyun commented on pull request #33910: [SPARK-36666][SQL] Fix regression in AQEShuffleReadExec

dongjoon-hyun commented on pull request #33910: URL: https://github.com/apache/spark/pull/33910#issuecomment-912918410 cc @cloud-fan , @viirya @maryannxue , @ulysses-you -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and us

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33803: [SPARK-36556][SQL] Add DSV2 filters

AmplabJenkins removed a comment on pull request #33803: URL: https://github.com/apache/spark/pull/33803#issuecomment-912917780 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/142990/ -

[GitHub] [spark] AmplabJenkins commented on pull request #33803: [SPARK-36556][SQL] Add DSV2 filters

AmplabJenkins commented on pull request #33803: URL: https://github.com/apache/spark/pull/33803#issuecomment-912917780 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/142990/ -- This

[GitHub] [spark] AmplabJenkins commented on pull request #33911: [SPARK-36653][PYTHON] implement Series.__xor__

AmplabJenkins commented on pull request #33911: URL: https://github.com/apache/spark/pull/33911#issuecomment-912917644 Can one of the admins verify this patch? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL a

[GitHub] [spark] SparkQA removed a comment on pull request #33803: [SPARK-36556][SQL] Add DSV2 filters

SparkQA removed a comment on pull request #33803: URL: https://github.com/apache/spark/pull/33803#issuecomment-912883989 **[Test build #142990 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/142990/testReport)** for PR 33803 at commit [`d1b4ec1`](https://gi

[GitHub] [spark] SparkQA commented on pull request #33803: [SPARK-36556][SQL] Add DSV2 filters

SparkQA commented on pull request #33803: URL: https://github.com/apache/spark/pull/33803#issuecomment-912917564 **[Test build #142990 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/142990/testReport)** for PR 33803 at commit [`d1b4ec1`](https://github.co

[GitHub] [spark] dgd-contributor opened a new pull request #33911: [SPARK-36653][PYTHON] implement Series.__xor__

dgd-contributor opened a new pull request #33911: URL: https://github.com/apache/spark/pull/33911 ### What changes were proposed in this pull request? Implement Series.\_\_xor__ ### Why are the changes needed? Follow pandas ### Does this PR introduce _any_ use

[GitHub] [spark] senthh commented on pull request #33894: [SPARK-36643][SQL] Add more information in ERROR log while SparkConf is modified when spark.sql.legacy.setCommandRejectsSparkCoreConfs is set

senthh commented on pull request #33894: URL: https://github.com/apache/spark/pull/33894#issuecomment-912910915 @HyukjinKwon @dongjoon-hyun Tests are passed now. Thank you for your kind assistance and review comments. -- This is an automated message from the Apache Git Service.

[GitHub] [spark] senthh removed a comment on pull request #33894: [SPARK-36643][SQL] Add more information in ERROR log while SparkConf is modified when spark.sql.legacy.setCommandRejectsSparkCoreConfs

senthh removed a comment on pull request #33894: URL: https://github.com/apache/spark/pull/33894#issuecomment-912910081 @dongjoon-hyun @HyukjinKwon I see tests are successful in https://github.com/senthh/spark-1/actions/runs/1197237025, but in PR it still shows failed status. Do

[GitHub] [spark] AmplabJenkins commented on pull request #33906: [SPARK-36660][SQL] Add cot as Scala and Python functions

AmplabJenkins commented on pull request #33906: URL: https://github.com/apache/spark/pull/33906#issuecomment-912910076 Can one of the admins verify this patch? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL a

[GitHub] [spark] senthh commented on pull request #33894: [SPARK-36643][SQL] Add more information in ERROR log while SparkConf is modified when spark.sql.legacy.setCommandRejectsSparkCoreConfs is set

senthh commented on pull request #33894: URL: https://github.com/apache/spark/pull/33894#issuecomment-912910081 @dongjoon-hyun @HyukjinKwon I see tests are successful in https://github.com/senthh/spark-1/actions/runs/1197237025, but in PR it still shows failed status. Do you hav

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33873: [SPARK-36624][YARN] In yarn client mode, when ApplicationMaster failed with KILLED/FAILED, driver should exit with code not 0

AmplabJenkins removed a comment on pull request #33873: URL: https://github.com/apache/spark/pull/33873#issuecomment-912904455 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/47494/

[GitHub] [spark] AmplabJenkins commented on pull request #33873: [SPARK-36624][YARN] In yarn client mode, when ApplicationMaster failed with KILLED/FAILED, driver should exit with code not 0

AmplabJenkins commented on pull request #33873: URL: https://github.com/apache/spark/pull/33873#issuecomment-912904455 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/47494/ -- T

[GitHub] [spark] SparkQA commented on pull request #33873: [SPARK-36624][YARN] In yarn client mode, when ApplicationMaster failed with KILLED/FAILED, driver should exit with code not 0

SparkQA commented on pull request #33873: URL: https://github.com/apache/spark/pull/33873#issuecomment-912904452 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/47494/ -- This is an automated message from the A

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33873: [SPARK-36624][YARN] In yarn client mode, when ApplicationMaster failed with KILLED/FAILED, driver should exit with code not 0

AmplabJenkins removed a comment on pull request #33873: URL: https://github.com/apache/spark/pull/33873#issuecomment-912903946 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/142992/ -

[GitHub] [spark] AmplabJenkins commented on pull request #33873: [SPARK-36624][YARN] In yarn client mode, when ApplicationMaster failed with KILLED/FAILED, driver should exit with code not 0

AmplabJenkins commented on pull request #33873: URL: https://github.com/apache/spark/pull/33873#issuecomment-912903946 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/142992/ -- This

[GitHub] [spark] SparkQA commented on pull request #33873: [SPARK-36624][YARN] In yarn client mode, when ApplicationMaster failed with KILLED/FAILED, driver should exit with code not 0

SparkQA commented on pull request #33873: URL: https://github.com/apache/spark/pull/33873#issuecomment-912903640 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/47494/ -- This is an automated message from the Apache

[GitHub] [spark] SparkQA removed a comment on pull request #33873: [SPARK-36624][YARN] In yarn client mode, when ApplicationMaster failed with KILLED/FAILED, driver should exit with code not 0

SparkQA removed a comment on pull request #33873: URL: https://github.com/apache/spark/pull/33873#issuecomment-912899600 **[Test build #142992 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/142992/testReport)** for PR 33873 at commit [`94021f5`](https://gi

[GitHub] [spark] SparkQA commented on pull request #33873: [SPARK-36624][YARN] In yarn client mode, when ApplicationMaster failed with KILLED/FAILED, driver should exit with code not 0

SparkQA commented on pull request #33873: URL: https://github.com/apache/spark/pull/33873#issuecomment-912901726 **[Test build #142992 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/142992/testReport)** for PR 33873 at commit [`94021f5`](https://github.co

[GitHub] [spark] dgd-contributor commented on pull request #33814: [SPARK-36396][PYTHON] implement data frame.cov

dgd-contributor commented on pull request #33814: URL: https://github.com/apache/spark/pull/33814#issuecomment-912901001 FYI @ueshin @xinrong-databricks @itholic -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the U

[GitHub] [spark] SparkQA commented on pull request #33873: [SPARK-36624][YARN] In yarn client mode, when ApplicationMaster failed with KILLED/FAILED, driver should exit with code not 0

SparkQA commented on pull request #33873: URL: https://github.com/apache/spark/pull/33873#issuecomment-912899600 **[Test build #142992 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/142992/testReport)** for PR 33873 at commit [`94021f5`](https://github.com

[GitHub] [spark] AngersZhuuuu removed a comment on pull request #33873: [SPARK-36624][YARN] In yarn client mode, when ApplicationMaster failed with KILLED/FAILED, driver should exit with code not 0

AngersZh removed a comment on pull request #33873: URL: https://github.com/apache/spark/pull/33873#issuecomment-912895370 > A callback sounds find to me. > @AngersZh doesn't a callback require user code changes, which he has said is hard for him? Not so sure about what you

[GitHub] [spark] SparkQA removed a comment on pull request #33873: [SPARK-36624][YARN] In yarn client mode, when ApplicationMaster failed with KILLED/FAILED, driver should exit with code not 0

SparkQA removed a comment on pull request #33873: URL: https://github.com/apache/spark/pull/33873#issuecomment-912898560 **[Test build #142991 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/142991/testReport)** for PR 33873 at commit [`752d359`](https://gi

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33873: [SPARK-36624][YARN] In yarn client mode, when ApplicationMaster failed with KILLED/FAILED, driver should exit with code not 0

AmplabJenkins removed a comment on pull request #33873: URL: https://github.com/apache/spark/pull/33873#issuecomment-912899304 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/142991/ -

[GitHub] [spark] AmplabJenkins commented on pull request #33873: [SPARK-36624][YARN] In yarn client mode, when ApplicationMaster failed with KILLED/FAILED, driver should exit with code not 0

AmplabJenkins commented on pull request #33873: URL: https://github.com/apache/spark/pull/33873#issuecomment-912899304 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/142991/ -- This

[GitHub] [spark] SparkQA commented on pull request #33873: [SPARK-36624][YARN] In yarn client mode, when ApplicationMaster failed with KILLED/FAILED, driver should exit with code not 0

SparkQA commented on pull request #33873: URL: https://github.com/apache/spark/pull/33873#issuecomment-912899291 **[Test build #142991 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/142991/testReport)** for PR 33873 at commit [`752d359`](https://github.co

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33873: [SPARK-36624][YARN] In yarn client mode, when ApplicationMaster failed with KILLED/FAILED, driver should exit with code not 0

AmplabJenkins removed a comment on pull request #33873: URL: https://github.com/apache/spark/pull/33873#issuecomment-912899199 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/47493/

[GitHub] [spark] AmplabJenkins commented on pull request #33873: [SPARK-36624][YARN] In yarn client mode, when ApplicationMaster failed with KILLED/FAILED, driver should exit with code not 0

AmplabJenkins commented on pull request #33873: URL: https://github.com/apache/spark/pull/33873#issuecomment-912899199 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/47493/ -- T

[GitHub] [spark] SparkQA commented on pull request #33873: [SPARK-36624][YARN] In yarn client mode, when ApplicationMaster failed with KILLED/FAILED, driver should exit with code not 0

SparkQA commented on pull request #33873: URL: https://github.com/apache/spark/pull/33873#issuecomment-912899197 Kubernetes integration test unable to build dist. exiting with code: 1 URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/47493/ -- This

[GitHub] [spark] SparkQA commented on pull request #33873: [SPARK-36624][YARN] In yarn client mode, when ApplicationMaster failed with KILLED/FAILED, driver should exit with code not 0

SparkQA commented on pull request #33873: URL: https://github.com/apache/spark/pull/33873#issuecomment-912898560 **[Test build #142991 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/142991/testReport)** for PR 33873 at commit [`752d359`](https://github.com

[GitHub] [spark] AngersZhuuuu commented on pull request #33873: [SPARK-36624][YARN] In yarn client mode, when ApplicationMaster failed with KILLED/FAILED, driver should exit with code not 0

AngersZh commented on pull request #33873: URL: https://github.com/apache/spark/pull/33873#issuecomment-912895370 > A callback sounds find to me. > @AngersZh doesn't a callback require user code changes, which he has said is hard for him? Not so sure about what you mean, y

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33803: [SPARK-36556][SQL] Add DSV2 filters

AmplabJenkins removed a comment on pull request #33803: URL: https://github.com/apache/spark/pull/33803#issuecomment-912891317 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/47492/

[GitHub] [spark] AmplabJenkins commented on pull request #33803: [SPARK-36556][SQL] Add DSV2 filters

AmplabJenkins commented on pull request #33803: URL: https://github.com/apache/spark/pull/33803#issuecomment-912891317 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/47492/ -- T

[GitHub] [spark] SparkQA commented on pull request #33803: [SPARK-36556][SQL] Add DSV2 filters

SparkQA commented on pull request #33803: URL: https://github.com/apache/spark/pull/33803#issuecomment-912888971 Kubernetes integration test unable to build dist. exiting with code: 1 URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/47492/ -- This

[GitHub] [spark] AngersZhuuuu commented on pull request #33869: [SPARK-36615][CORE] Register shutdown hook earlier when start SC

AngersZh commented on pull request #33869: URL: https://github.com/apache/spark/pull/33869#issuecomment-912886208 > @mridulm I would assume the ctrl-c would normally kill the init thread to run the shutdown hook, if it doesn't for some reason, I would think the worst case is we don't h

[GitHub] [spark] SparkQA commented on pull request #33803: [SPARK-36556][SQL] Add DSV2 filters

SparkQA commented on pull request #33803: URL: https://github.com/apache/spark/pull/33803#issuecomment-912883989 **[Test build #142990 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/142990/testReport)** for PR 33803 at commit [`d1b4ec1`](https://github.com

[GitHub] [spark] huaxingao commented on a change in pull request #33803: [SPARK-36556][SQL] Add DSV2 filters

huaxingao commented on a change in pull request #33803: URL: https://github.com/apache/spark/pull/33803#discussion_r702214588 ## File path: sql/catalyst/src/main/java/org/apache/spark/sql/connector/expressions/filter/And.java ## @@ -0,0 +1,55 @@ +/* + * Licensed to the Apache

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33877: [SPARK-36625][SPARK-36661][PYTHON] Support TimestampNTZ in pandas API on Spark

AmplabJenkins removed a comment on pull request #33877: URL: https://github.com/apache/spark/pull/33877#issuecomment-912847871 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/142985/ -

[GitHub] [spark] ueshin commented on a change in pull request #33877: [SPARK-36625][SPARK-36661][PYTHON] Support TimestampNTZ in pandas API on Spark

ueshin commented on a change in pull request #33877: URL: https://github.com/apache/spark/pull/33877#discussion_r702168125 ## File path: python/pyspark/pandas/data_type_ops/base.py ## @@ -184,10 +186,21 @@ def _as_other_type( ) assert not need_pre_process, "Pre-proces

[GitHub] [spark] AmplabJenkins commented on pull request #33910: [SPARK-36666][SQL] Fix regression in AQEShuffleReadExec

AmplabJenkins commented on pull request #33910: URL: https://github.com/apache/spark/pull/33910#issuecomment-912872942 Can one of the admins verify this patch? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL a

[GitHub] [spark] HyukjinKwon commented on a change in pull request #33906: [SPARK-36660][SQL] Add cot as Scala and Python functions

HyukjinKwon commented on a change in pull request #33906: URL: https://github.com/apache/spark/pull/33906#discussion_r702206167 ## File path: sql/core/src/main/scala/org/apache/spark/sql/functions.scala ## @@ -1800,6 +1800,24 @@ object functions { */ def cosh(columnName:

[GitHub] [spark] andygrove opened a new pull request #33910: [SPARK-36666][SQL] Fix regression in AQEShuffleReadExec

andygrove opened a new pull request #33910: URL: https://github.com/apache/spark/pull/33910 Fix regression in AQEShuffleReadExec when used in conjunction with Spark plugins with custom partitioning. Signed-off-by: Andy Grove ### What changes were proposed in this pull

[GitHub] [spark] github-actions[bot] commented on pull request #32364: [SPARK-35242][SQL] Support change catalog default database for spark

github-actions[bot] commented on pull request #32364: URL: https://github.com/apache/spark/pull/32364#issuecomment-912871339 We're closing this PR because it hasn't been updated in a while. This isn't a judgement on the merit of the PR in any way. It's just a way of keeping the PR queue ma

[GitHub] [spark] github-actions[bot] closed pull request #31257: [SPARK-33630][SQL] Support SHOW TABLES command as table valued function

github-actions[bot] closed pull request #31257: URL: https://github.com/apache/spark/pull/31257 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: re

[GitHub] [spark] SparkQA removed a comment on pull request #33877: [SPARK-36625][SPARK-36661][PYTHON] Support TimestampNTZ in pandas API on Spark

SparkQA removed a comment on pull request #33877: URL: https://github.com/apache/spark/pull/33877#issuecomment-912575453 **[Test build #142985 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/142985/testReport)** for PR 33877 at commit [`83afb0b`](https://gi

[GitHub] [spark] AmplabJenkins commented on pull request #33877: [SPARK-36625][SPARK-36661][PYTHON] Support TimestampNTZ in pandas API on Spark

AmplabJenkins commented on pull request #33877: URL: https://github.com/apache/spark/pull/33877#issuecomment-912847871 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/142985/ -- This

[GitHub] [spark] SparkQA commented on pull request #33877: [SPARK-36625][SPARK-36661][PYTHON] Support TimestampNTZ in pandas API on Spark

SparkQA commented on pull request #33877: URL: https://github.com/apache/spark/pull/33877#issuecomment-912847527 **[Test build #142985 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/142985/testReport)** for PR 33877 at commit [`83afb0b`](https://github.co

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33888: [SPARK-36634][SQL] Support access and read parquet file by column ordinal

AmplabJenkins removed a comment on pull request #33888: URL: https://github.com/apache/spark/pull/33888#issuecomment-912839798 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/142989/ -

[GitHub] [spark] AmplabJenkins commented on pull request #33888: [SPARK-36634][SQL] Support access and read parquet file by column ordinal

AmplabJenkins commented on pull request #33888: URL: https://github.com/apache/spark/pull/33888#issuecomment-912839798 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/142989/ -- This

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33908: [SPARK-36662][SQL] Special timestamps support for path filters - modifiedBefore/modifiedAfter

AmplabJenkins removed a comment on pull request #33908: URL: https://github.com/apache/spark/pull/33908#issuecomment-912839065 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/142987/ -

[GitHub] [spark] AmplabJenkins commented on pull request #33908: [SPARK-36662][SQL] Special timestamps support for path filters - modifiedBefore/modifiedAfter

AmplabJenkins commented on pull request #33908: URL: https://github.com/apache/spark/pull/33908#issuecomment-912839065 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/142987/ -- This

[GitHub] [spark] SparkQA removed a comment on pull request #33888: [SPARK-36634][SQL] Support access and read parquet file by column ordinal

SparkQA removed a comment on pull request #33888: URL: https://github.com/apache/spark/pull/33888#issuecomment-912693505 **[Test build #142989 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/142989/testReport)** for PR 33888 at commit [`b989aa3`](https://gi

[GitHub] [spark] SparkQA commented on pull request #33888: [SPARK-36634][SQL] Support access and read parquet file by column ordinal

SparkQA commented on pull request #33888: URL: https://github.com/apache/spark/pull/33888#issuecomment-912837879 **[Test build #142989 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/142989/testReport)** for PR 33888 at commit [`b989aa3`](https://github.co

[GitHub] [spark] SparkQA removed a comment on pull request #33908: [SPARK-36662][SQL] Special timestamps support for path filters - modifiedBefore/modifiedAfter

SparkQA removed a comment on pull request #33908: URL: https://github.com/apache/spark/pull/33908#issuecomment-912690104 **[Test build #142987 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/142987/testReport)** for PR 33908 at commit [`74c4aad`](https://gi

[GitHub] [spark] SparkQA commented on pull request #33908: [SPARK-36662][SQL] Special timestamps support for path filters - modifiedBefore/modifiedAfter

SparkQA commented on pull request #33908: URL: https://github.com/apache/spark/pull/33908#issuecomment-912831286 **[Test build #142987 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/142987/testReport)** for PR 33908 at commit [`74c4aad`](https://github.co

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33909: [SPARK-36639][SQL][3.0] Fix an issue that sequence builtin function causes ArrayIndexOutOfBoundsException if the arguments are

AmplabJenkins removed a comment on pull request #33909: URL: https://github.com/apache/spark/pull/33909#issuecomment-912820054 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/142986/ -

[GitHub] [spark] AmplabJenkins commented on pull request #33909: [SPARK-36639][SQL][3.0] Fix an issue that sequence builtin function causes ArrayIndexOutOfBoundsException if the arguments are under th

AmplabJenkins commented on pull request #33909: URL: https://github.com/apache/spark/pull/33909#issuecomment-912820054 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/142986/ -- This

[GitHub] [spark] SparkQA removed a comment on pull request #33909: [SPARK-36639][SQL][3.0] Fix an issue that sequence builtin function causes ArrayIndexOutOfBoundsException if the arguments are under

SparkQA removed a comment on pull request #33909: URL: https://github.com/apache/spark/pull/33909#issuecomment-912690015 **[Test build #142986 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/142986/testReport)** for PR 33909 at commit [`11925c1`](https://gi

[GitHub] [spark] SparkQA commented on pull request #33909: [SPARK-36639][SQL][3.0] Fix an issue that sequence builtin function causes ArrayIndexOutOfBoundsException if the arguments are under the cond

SparkQA commented on pull request #33909: URL: https://github.com/apache/spark/pull/33909#issuecomment-912819082 **[Test build #142986 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/142986/testReport)** for PR 33909 at commit [`11925c1`](https://github.co

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33877: [SPARK-36625][SPARK-36661][PYTHON] Support TimestampNTZ in pandas API on Spark

AmplabJenkins removed a comment on pull request #33877: URL: https://github.com/apache/spark/pull/33877#issuecomment-912768521 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/142982/ -

[GitHub] [spark] SparkQA removed a comment on pull request #33877: [SPARK-36625][SPARK-36661][PYTHON] Support TimestampNTZ in pandas API on Spark

SparkQA removed a comment on pull request #33877: URL: https://github.com/apache/spark/pull/33877#issuecomment-912462593 **[Test build #142982 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/142982/testReport)** for PR 33877 at commit [`6bdfad2`](https://gi

[GitHub] [spark] AmplabJenkins commented on pull request #33877: [SPARK-36625][SPARK-36661][PYTHON] Support TimestampNTZ in pandas API on Spark

AmplabJenkins commented on pull request #33877: URL: https://github.com/apache/spark/pull/33877#issuecomment-912768521 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/142982/ -- This

[GitHub] [spark] SparkQA commented on pull request #33877: [SPARK-36625][SPARK-36661][PYTHON] Support TimestampNTZ in pandas API on Spark

SparkQA commented on pull request #33877: URL: https://github.com/apache/spark/pull/33877#issuecomment-912767698 **[Test build #142982 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/142982/testReport)** for PR 33877 at commit [`6bdfad2`](https://github.co

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33909: [SPARK-36639][SQL][3.0] Fix an issue that sequence builtin function causes ArrayIndexOutOfBoundsException if the arguments are

AmplabJenkins removed a comment on pull request #33909: URL: https://github.com/apache/spark/pull/33909#issuecomment-912728295 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/47490/

[GitHub] [spark] AmplabJenkins commented on pull request #33909: [SPARK-36639][SQL][3.0] Fix an issue that sequence builtin function causes ArrayIndexOutOfBoundsException if the arguments are under th

AmplabJenkins commented on pull request #33909: URL: https://github.com/apache/spark/pull/33909#issuecomment-912728295 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/47490/ -- T

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33888: [SPARK-36634][SQL] Support access and read parquet file by column ordinal

AmplabJenkins removed a comment on pull request #33888: URL: https://github.com/apache/spark/pull/33888#issuecomment-912727679 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/47491/

[GitHub] [spark] SparkQA commented on pull request #33909: [SPARK-36639][SQL][3.0] Fix an issue that sequence builtin function causes ArrayIndexOutOfBoundsException if the arguments are under the cond

SparkQA commented on pull request #33909: URL: https://github.com/apache/spark/pull/33909#issuecomment-912728264 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/47490/ -- This is an automated message from the A

[GitHub] [spark] SparkQA commented on pull request #33888: [SPARK-36634][SQL] Support access and read parquet file by column ordinal

SparkQA commented on pull request #33888: URL: https://github.com/apache/spark/pull/33888#issuecomment-912727631 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/47491/ -- This is an automated message from the A

[GitHub] [spark] AmplabJenkins commented on pull request #33888: [SPARK-36634][SQL] Support access and read parquet file by column ordinal

AmplabJenkins commented on pull request #33888: URL: https://github.com/apache/spark/pull/33888#issuecomment-912727679 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/47491/ -- T

[GitHub] [spark] ueshin closed pull request #33901: [SPARK-36655][PYTHON] Add `versionadded` for API added in Spark 3.3.0

ueshin closed pull request #33901: URL: https://github.com/apache/spark/pull/33901 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubsc

[GitHub] [spark] ueshin commented on pull request #33901: [SPARK-36655][PYTHON] Add `versionadded` for API added in Spark 3.3.0

ueshin commented on pull request #33901: URL: https://github.com/apache/spark/pull/33901#issuecomment-912726106 Thanks! merging to master. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the spe

[GitHub] [spark] SparkQA commented on pull request #33888: [SPARK-36634][SQL] Support access and read parquet file by column ordinal

SparkQA commented on pull request #33888: URL: https://github.com/apache/spark/pull/33888#issuecomment-912722307 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/47491/ -- This is an automated message from the Apache

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33908: [SPARK-36662][SQL] Special timestamps support for path filters - modifiedBefore/modifiedAfter

AmplabJenkins removed a comment on pull request #33908: URL: https://github.com/apache/spark/pull/33908#issuecomment-912721690 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/47488/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33901: [SPARK-36655][PYTHON] Add `versionadded` for API added in Spark 3.3.0

AmplabJenkins removed a comment on pull request #33901: URL: https://github.com/apache/spark/pull/33901#issuecomment-912721688 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment

[GitHub] [spark] AmplabJenkins commented on pull request #33901: [SPARK-36655][PYTHON] Add `versionadded` for API added in Spark 3.3.0

AmplabJenkins commented on pull request #33901: URL: https://github.com/apache/spark/pull/33901#issuecomment-912721691 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To un

[GitHub] [spark] AmplabJenkins commented on pull request #33908: [SPARK-36662][SQL] Special timestamps support for path filters - modifiedBefore/modifiedAfter

AmplabJenkins commented on pull request #33908: URL: https://github.com/apache/spark/pull/33908#issuecomment-912721690 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/47488/ -- T

[GitHub] [spark] SparkQA removed a comment on pull request #33901: [SPARK-36655][PYTHON] Add `versionadded` for API added in Spark 3.3.0

SparkQA removed a comment on pull request #33901: URL: https://github.com/apache/spark/pull/33901#issuecomment-912690181 **[Test build #142988 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/142988/testReport)** for PR 33901 at commit [`3928cc5`](https://gi

[GitHub] [spark] SparkQA commented on pull request #33908: [SPARK-36662][SQL] Special timestamps support for path filters - modifiedBefore/modifiedAfter

SparkQA commented on pull request #33908: URL: https://github.com/apache/spark/pull/33908#issuecomment-912717209 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/47488/ -- This is an automated message from the A

[GitHub] [spark] SparkQA commented on pull request #33909: [SPARK-36639][SQL][3.0] Fix an issue that sequence builtin function causes ArrayIndexOutOfBoundsException if the arguments are under the cond

SparkQA commented on pull request #33909: URL: https://github.com/apache/spark/pull/33909#issuecomment-912714979 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/47490/ -- This is an automated message from the Apache

[GitHub] [spark] SparkQA commented on pull request #33908: [SPARK-36662][SQL] Special timestamps support for path filters - modifiedBefore/modifiedAfter

SparkQA commented on pull request #33908: URL: https://github.com/apache/spark/pull/33908#issuecomment-912711978 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/47488/ -- This is an automated message from the Apache

[GitHub] [spark] ueshin commented on a change in pull request #33850: [SPARK-36531][SPARK-36515][PYTHON] Improve test coverage for data_type_ops/* and groupby.

ueshin commented on a change in pull request #33850: URL: https://github.com/apache/spark/pull/33850#discussion_r702075761 ## File path: python/pyspark/pandas/data_type_ops/binary_ops.py ## @@ -100,7 +99,9 @@ def astype(self, index_ops: IndexOpsLike, dtype: Union[str, type, Dt

[GitHub] [spark] SparkQA commented on pull request #33901: [SPARK-36655][PYTHON] Add `versionadded` for API added in Spark 3.3.0

SparkQA commented on pull request #33901: URL: https://github.com/apache/spark/pull/33901#issuecomment-912710278 Kubernetes integration test unable to build dist. exiting with code: 1 URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/47489/ -- This

[GitHub] [spark] SparkQA commented on pull request #33901: [SPARK-36655][PYTHON] Add `versionadded` for API added in Spark 3.3.0

SparkQA commented on pull request #33901: URL: https://github.com/apache/spark/pull/33901#issuecomment-912706007 **[Test build #142988 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/142988/testReport)** for PR 33901 at commit [`3928cc5`](https://github.co

[GitHub] [spark] senthh commented on pull request #33894: [SPARK-36643][SQL] Add more information in ERROR log while SparkConf is modified when spark.sql.legacy.setCommandRejectsSparkCoreConfs is set

senthh commented on pull request #33894: URL: https://github.com/apache/spark/pull/33894#issuecomment-912705626 > can you retrigger this? https://github.com/senthh/spark-1/actions/runs/1197237025 @HyukjinKwon I am getting the same error . Tests Passed but after that it failed with

[GitHub] [spark] ueshin closed pull request #33752: [SPARK-36401][PYTHON] Implement Series.cov

ueshin closed pull request #33752: URL: https://github.com/apache/spark/pull/33752 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubsc

[GitHub] [spark] ueshin commented on pull request #33752: [SPARK-36401][PYTHON] Implement Series.cov

ueshin commented on pull request #33752: URL: https://github.com/apache/spark/pull/33752#issuecomment-912702441 Thanks! merging to master. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the spe

[GitHub] [spark] dongjoon-hyun commented on pull request #33904: [SPARK-36659][SQL] Promote spark.sql.execution.topKSortFallbackThreshold to a user-facing config

dongjoon-hyun commented on pull request #33904: URL: https://github.com/apache/spark/pull/33904#issuecomment-912695574 Thank you, @yaooqinn . I backported to branch-3.2. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

[GitHub] [spark] SparkQA commented on pull request #33888: [SPARK-36634][SQL] Support access and read parquet file by column ordinal

SparkQA commented on pull request #33888: URL: https://github.com/apache/spark/pull/33888#issuecomment-912693505 **[Test build #142989 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/142989/testReport)** for PR 33888 at commit [`b989aa3`](https://github.com

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33901: [SPARK-36655][PYTHON] Add `versionadded` for API added in Spark 3.3.0

AmplabJenkins removed a comment on pull request #33901: URL: https://github.com/apache/spark/pull/33901#issuecomment-912106447 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/47443/

[GitHub] [spark] SparkQA commented on pull request #33901: [SPARK-36655][PYTHON] Add `versionadded` for API added in Spark 3.3.0

SparkQA commented on pull request #33901: URL: https://github.com/apache/spark/pull/33901#issuecomment-912690181 **[Test build #142988 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/142988/testReport)** for PR 33901 at commit [`3928cc5`](https://github.com

[GitHub] [spark] SparkQA commented on pull request #33908: [SPARK-36662][SQL] Special timestamps support for path filters - modifiedBefore/modifiedAfter

SparkQA commented on pull request #33908: URL: https://github.com/apache/spark/pull/33908#issuecomment-912690104 **[Test build #142987 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/142987/testReport)** for PR 33908 at commit [`74c4aad`](https://github.com

[GitHub] [spark] SparkQA commented on pull request #33909: [SPARK-36639][SQL][3.0] Fix an issue that sequence builtin function causes ArrayIndexOutOfBoundsException if the arguments are under the cond

SparkQA commented on pull request #33909: URL: https://github.com/apache/spark/pull/33909#issuecomment-912690015 **[Test build #142986 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/142986/testReport)** for PR 33909 at commit [`11925c1`](https://github.com

[GitHub] [spark] sarutak opened a new pull request #33909: [SPARK-36639][SQL][3.0] Fix an issue that sequence builtin function causes ArrayIndexOutOfBoundsException if the arguments are under the cond

sarutak opened a new pull request #33909: URL: https://github.com/apache/spark/pull/33909 ### What changes were proposed in this pull request? This PR backports the change of SPARK-36639 (#33895) for `branch-3.0`. This PR fixes an issue that `sequence` builtin function causes

  1   2   3   4   >