[GitHub] [spark] SparkQA commented on pull request #34767: [SPARK-37461][YARN][FOLLOWUP] Refactor YARN Client code to avoid add unnecessary parameter of `appId`

SparkQA commented on pull request #34767: URL: https://github.com/apache/spark/pull/34767#issuecomment-983414848 **[Test build #145807 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145807/testReport)** for PR 34767 at commit

[GitHub] [spark] HyukjinKwon commented on pull request #34715: [SPARK-37445][BUILD] Rename the maven hadoop profile to hadoop-3 and hadoop-2

HyukjinKwon commented on pull request #34715: URL: https://github.com/apache/spark/pull/34715#issuecomment-983435047 From a cursory look, seems fine to me too but I think it's best to defer to @sunchao -- This is an automated message from the Apache Git Service. To respond to the

[GitHub] [spark] AmplabJenkins commented on pull request #34765: [WIP][SPARK-37487][SQL][CORE] Avoid performing CollectMetrics twice if the operation is followed by global sort.

AmplabJenkins commented on pull request #34765: URL: https://github.com/apache/spark/pull/34765#issuecomment-983449401 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145798/ -- This

[GitHub] [spark] AmplabJenkins commented on pull request #34767: [SPARK-37461][YARN][FOLLOWUP] Refactor YARN Client code to avoid add unnecessary parameter of `appId`

AmplabJenkins commented on pull request #34767: URL: https://github.com/apache/spark/pull/34767#issuecomment-983449549 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145807/ -- This

[GitHub] [spark] AmplabJenkins commented on pull request #34657: [SPARK-37511][PYTHON] Introduce TimedeltaIndex to pandas API on Spark

AmplabJenkins commented on pull request #34657: URL: https://github.com/apache/spark/pull/34657#issuecomment-983449553 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/50272/ --

[GitHub] [spark] AmplabJenkins commented on pull request #34741: [SPARK-37463][SQL] Read/Write Timestamp ntz from/to Orc uses UTC time zone

AmplabJenkins commented on pull request #34741: URL: https://github.com/apache/spark/pull/34741#issuecomment-983449402 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145792/ -- This

[GitHub] [spark] SparkQA commented on pull request #34657: [SPARK-37511][PYTHON] Introduce TimedeltaIndex to pandas API on Spark

SparkQA commented on pull request #34657: URL: https://github.com/apache/spark/pull/34657#issuecomment-983394520 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/50272/ -- This is an automated message from the Apache

[GitHub] [spark] cloud-fan commented on pull request #34735: [SPARK-37481][Core][WebUI] Fix disappearance of skipped stages after they retry

cloud-fan commented on pull request #34735: URL: https://github.com/apache/spark/pull/34735#issuecomment-983394118 cc @jiangxb1987 @Ngone51 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [spark] SparkQA commented on pull request #34765: [WIP][SPARK-37487][SQL][CORE] Avoid performing CollectMetrics twice if the operation is followed by global sort.

SparkQA commented on pull request #34765: URL: https://github.com/apache/spark/pull/34765#issuecomment-983399136 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/50269/ -- This is an automated message from the

[GitHub] [spark] cloud-fan closed pull request #34766: [SPARK-37513][SQL][DOC] date +/- interval with only day-time fields returns different data type between Spark3.2 and Spark3.1

cloud-fan closed pull request #34766: URL: https://github.com/apache/spark/pull/34766 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [spark] SparkQA commented on pull request #34673: [SPARK-37343][SQL] Implement createIndex, IndexExists and dropIndex in JDBC (Postgres dialect)

SparkQA commented on pull request #34673: URL: https://github.com/apache/spark/pull/34673#issuecomment-983406703 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/50275/ -- This is an automated message from the Apache

[GitHub] [spark] cloud-fan commented on a change in pull request #34673: [SPARK-37343][SQL] Implement createIndex, IndexExists and dropIndex in JDBC (Postgres dialect)

cloud-fan commented on a change in pull request #34673: URL: https://github.com/apache/spark/pull/34673#discussion_r759956337 ## File path: sql/core/src/main/scala/org/apache/spark/sql/jdbc/JdbcDialects.scala ## @@ -358,6 +359,18 @@ abstract class JdbcDialect extends

[GitHub] [spark] SparkQA commented on pull request #34367: [SPARK-37099][SQL] Impl a rank-based filter to optimize top-k computation

SparkQA commented on pull request #34367: URL: https://github.com/apache/spark/pull/34367#issuecomment-983419906 **[Test build #145790 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145790/testReport)** for PR 34367 at commit

[GitHub] [spark] cloud-fan commented on a change in pull request #34753: [SPARK-37494][SQL] Unify v1 and v2 options output of `SHOW CREATE TABLE` command

cloud-fan commented on a change in pull request #34753: URL: https://github.com/apache/spark/pull/34753#discussion_r759970181 ## File path: sql/core/src/test/scala/org/apache/spark/sql/ShowCreateTableSuite.scala ## @@ -200,6 +200,32 @@ abstract class ShowCreateTableSuite

[GitHub] [spark] zhengruifeng commented on pull request #34504: [SPARK-37226][SQL] Filter push down through window if partitionSpec isEmpty

zhengruifeng commented on pull request #34504: URL: https://github.com/apache/spark/pull/34504#issuecomment-983419947 > > @zhengruifeng can you highlight the differences between your PR and this one? > > IMHO, there are two main differences: > > 1, a new node `RankLimit` is

[GitHub] [spark] beliefer commented on a change in pull request #34741: [SPARK-37463][SQL] Read/Write Timestamp ntz from/to Orc uses UTC time zone

beliefer commented on a change in pull request #34741: URL: https://github.com/apache/spark/pull/34741#discussion_r759972787 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/orc/OrcUtils.scala ## @@ -531,13 +533,16 @@ object OrcUtils extends

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34766: [SPARK-37513][SQL][DOC] date +/- interval with only day-time fields returns different data type between Spark3.2 and Spark3.1

AmplabJenkins removed a comment on pull request #34766: URL: https://github.com/apache/spark/pull/34766#issuecomment-983449554 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/50273/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34741: [SPARK-37463][SQL] Read/Write Timestamp ntz from/to Orc uses UTC time zone

AmplabJenkins removed a comment on pull request #34741: URL: https://github.com/apache/spark/pull/34741#issuecomment-983449402 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145792/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34657: [SPARK-37511][PYTHON] Introduce TimedeltaIndex to pandas API on Spark

AmplabJenkins removed a comment on pull request #34657: URL: https://github.com/apache/spark/pull/34657#issuecomment-983449553 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/50272/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34765: [WIP][SPARK-37487][SQL][CORE] Avoid performing CollectMetrics twice if the operation is followed by global sort.

AmplabJenkins removed a comment on pull request #34765: URL: https://github.com/apache/spark/pull/34765#issuecomment-983449401 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145798/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34767: [SPARK-37461][YARN][FOLLOWUP] Refactor YARN Client code to avoid add unnecessary parameter of `appId`

AmplabJenkins removed a comment on pull request #34767: URL: https://github.com/apache/spark/pull/34767#issuecomment-983449549 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145807/

[GitHub] [spark] SparkQA commented on pull request #34741: [SPARK-37463][SQL] Read/Write Timestamp ntz from/to Orc uses UTC time zone

SparkQA commented on pull request #34741: URL: https://github.com/apache/spark/pull/34741#issuecomment-983450419 **[Test build #145810 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145810/testReport)** for PR 34741 at commit

[GitHub] [spark] SparkQA commented on pull request #34753: [SPARK-37494][SQL] Unify v1 and v2 options output of `SHOW CREATE TABLE` command

SparkQA commented on pull request #34753: URL: https://github.com/apache/spark/pull/34753#issuecomment-983450312 **[Test build #145809 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145809/testReport)** for PR 34753 at commit

[GitHub] [spark] AmplabJenkins commented on pull request #34367: [SPARK-37099][SQL] Impl a rank-based filter to optimize top-k computation

AmplabJenkins commented on pull request #34367: URL: https://github.com/apache/spark/pull/34367#issuecomment-983449556 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145790/ -- This

[GitHub] [spark] AmplabJenkins commented on pull request #34766: [SPARK-37513][SQL][DOC] date +/- interval with only day-time fields returns different data type between Spark3.2 and Spark3.1

AmplabJenkins commented on pull request #34766: URL: https://github.com/apache/spark/pull/34766#issuecomment-983449554 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/50273/ --

[GitHub] [spark] SparkQA commented on pull request #34767: [SPARK-37461][YARN][FOLLOWUP] Refactor YARN Client code to avoid add unnecessary parameter of `appId`

SparkQA commented on pull request #34767: URL: https://github.com/apache/spark/pull/34767#issuecomment-983450256 **[Test build #145808 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145808/testReport)** for PR 34767 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34367: [SPARK-37099][SQL] Impl a rank-based filter to optimize top-k computation

AmplabJenkins removed a comment on pull request #34367: URL: https://github.com/apache/spark/pull/34367#issuecomment-983449556 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145790/

[GitHub] [spark] AmplabJenkins commented on pull request #34212: [SPARK-36402][PYTHON] Implement Series.combine

AmplabJenkins commented on pull request #34212: URL: https://github.com/apache/spark/pull/34212#issuecomment-983498759 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/50277/ --

[GitHub] [spark] AmplabJenkins commented on pull request #34767: [SPARK-37461][YARN][FOLLOWUP] Refactor YARN Client code to avoid add unnecessary parameter of `appId`

AmplabJenkins commented on pull request #34767: URL: https://github.com/apache/spark/pull/34767#issuecomment-983498754 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145808/ -- This

[GitHub] [spark] AmplabJenkins commented on pull request #34758: [SPARK-37501][SQL] CREATE/REPLACE TABLE should qualify location for v2 command

AmplabJenkins commented on pull request #34758: URL: https://github.com/apache/spark/pull/34758#issuecomment-983498755 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/50276/ --

[GitHub] [spark] AmplabJenkins commented on pull request #34718: [SPARK-37460][DOCS] Add the description of ALTER DATABASE SET LOCATION

AmplabJenkins commented on pull request #34718: URL: https://github.com/apache/spark/pull/34718#issuecomment-983498756 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145811/ -- This

[GitHub] [spark] SparkQA commented on pull request #34758: [SPARK-37501][SQL] CREATE/REPLACE TABLE should qualify location for v2 command

SparkQA commented on pull request #34758: URL: https://github.com/apache/spark/pull/34758#issuecomment-983504384 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/50278/ -- This is an automated message from the

[GitHub] [spark] AmplabJenkins commented on pull request #34758: [SPARK-37501][SQL] CREATE/REPLACE TABLE should qualify location for v2 command

AmplabJenkins commented on pull request #34758: URL: https://github.com/apache/spark/pull/34758#issuecomment-983504413 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/50278/ --

[GitHub] [spark] SparkQA commented on pull request #34753: [SPARK-37494][SQL] Unify v1 and v2 options output of `SHOW CREATE TABLE` command

SparkQA commented on pull request #34753: URL: https://github.com/apache/spark/pull/34753#issuecomment-983514707 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/50283/ -- This is an automated message from the Apache

[GitHub] [spark] SparkQA removed a comment on pull request #34753: [SPARK-37494][SQL] Unify v1 and v2 options output of `SHOW CREATE TABLE` command

SparkQA removed a comment on pull request #34753: URL: https://github.com/apache/spark/pull/34753#issuecomment-983403459 **[Test build #145805 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145805/testReport)** for PR 34753 at commit

[GitHub] [spark] AmplabJenkins commented on pull request #34753: [SPARK-37494][SQL] Unify v1 and v2 options output of `SHOW CREATE TABLE` command

AmplabJenkins commented on pull request #34753: URL: https://github.com/apache/spark/pull/34753#issuecomment-983514506 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145805/ -- This

[GitHub] [spark] SparkQA removed a comment on pull request #34763: [SPARK-37389][SQL][FOLLOWUP] SET command shuold not parse comments

SparkQA removed a comment on pull request #34763: URL: https://github.com/apache/spark/pull/34763#issuecomment-983329503 **[Test build #145796 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145796/testReport)** for PR 34763 at commit

[GitHub] [spark] SparkQA commented on pull request #34212: [SPARK-36402][PYTHON] Implement Series.combine

SparkQA commented on pull request #34212: URL: https://github.com/apache/spark/pull/34212#issuecomment-983395095 **[Test build #145803 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145803/testReport)** for PR 34212 at commit

[GitHub] [spark] cloud-fan commented on pull request #34766: [SPARK-37513][SQL][DOC] date +/- interval with only day-time fields returns different data type between Spark3.2 and Spark3.1

cloud-fan commented on pull request #34766: URL: https://github.com/apache/spark/pull/34766#issuecomment-983398707 thanks, merging to master/3.2! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [spark] cloud-fan commented on pull request #34763: [SPARK-37389][SQL][FOLLOWUP] SET command shuold not parse comments

cloud-fan commented on pull request #34763: URL: https://github.com/apache/spark/pull/34763#issuecomment-983407688 thanks for the review, merging to master! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

[GitHub] [spark] AmplabJenkins commented on pull request #34657: [SPARK-37511][PYTHON] Introduce TimedeltaIndex to pandas API on Spark

AmplabJenkins commented on pull request #34657: URL: https://github.com/apache/spark/pull/34657#issuecomment-983414387 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/50268/ --

[GitHub] [spark] AngersZhuuuu opened a new pull request #34767: [SPARK-37461][YARN][FOLLOWUP] Refactor YARN Client code to avoid add unnecessary parameter of `appId`

AngersZh opened a new pull request #34767: URL: https://github.com/apache/spark/pull/34767 ### What changes were proposed in this pull request? In https://github.com/apache/spark/pull/34710, we assign ApplicationId to `appId` in client mode too. After this change we can refactor

[GitHub] [spark] SparkQA commented on pull request #34657: [SPARK-37511][PYTHON] Introduce TimedeltaIndex to pandas API on Spark

SparkQA commented on pull request #34657: URL: https://github.com/apache/spark/pull/34657#issuecomment-983414346 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/50268/ -- This is an automated message from the

[GitHub] [spark] SparkQA commented on pull request #34766: [SPARK-37513][SQL][DOC] date +/- interval with only day-time fields returns different data type between Spark3.2 and Spark3.1

SparkQA commented on pull request #34766: URL: https://github.com/apache/spark/pull/34766#issuecomment-983437514 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/50273/ -- This is an automated message from the

[GitHub] [spark] SparkQA removed a comment on pull request #34767: [SPARK-37461][YARN][FOLLOWUP] Refactor YARN Client code to avoid add unnecessary parameter of `appId`

SparkQA removed a comment on pull request #34767: URL: https://github.com/apache/spark/pull/34767#issuecomment-983414848 **[Test build #145807 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145807/testReport)** for PR 34767 at commit

[GitHub] [spark] AngersZhuuuu commented on a change in pull request #34757: [SPARK-37504][PYTHON] Pyspark create SparkSession with existed session should not pass static conf

AngersZh commented on a change in pull request #34757: URL: https://github.com/apache/spark/pull/34757#discussion_r759996260 ## File path: python/pyspark/sql/session.py ## @@ -304,8 +329,13 @@ def __init__( and not

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34212: [SPARK-36402][PYTHON] Implement Series.combine

AmplabJenkins removed a comment on pull request #34212: URL: https://github.com/apache/spark/pull/34212#issuecomment-983498759 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/50277/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34767: [SPARK-37461][YARN][FOLLOWUP] Refactor YARN Client code to avoid add unnecessary parameter of `appId`

AmplabJenkins removed a comment on pull request #34767: URL: https://github.com/apache/spark/pull/34767#issuecomment-983498754 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145808/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34767: [SPARK-37461][YARN][FOLLOWUP] Refactor YARN Client code to avoid add unnecessary parameter of `appId`

AmplabJenkins removed a comment on pull request #34767: URL: https://github.com/apache/spark/pull/34767#issuecomment-983545864 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/50281/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34763: [SPARK-37389][SQL][FOLLOWUP] SET command shuold not parse comments

AmplabJenkins removed a comment on pull request #34763: URL: https://github.com/apache/spark/pull/34763#issuecomment-983545861 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145796/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34753: [SPARK-37494][SQL] Unify v1 and v2 options output of `SHOW CREATE TABLE` command

AmplabJenkins removed a comment on pull request #34753: URL: https://github.com/apache/spark/pull/34753#issuecomment-983514506 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [spark] AmplabJenkins commented on pull request #34712: [SPARK-37463][SQL] Read/Write Timestamp ntz or ltz to Orc uses UTC timestamp

AmplabJenkins commented on pull request #34712: URL: https://github.com/apache/spark/pull/34712#issuecomment-983545862 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

[GitHub] [spark] AmplabJenkins commented on pull request #34767: [SPARK-37461][YARN][FOLLOWUP] Refactor YARN Client code to avoid add unnecessary parameter of `appId`

AmplabJenkins commented on pull request #34767: URL: https://github.com/apache/spark/pull/34767#issuecomment-983545864 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/50281/ --

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34712: [SPARK-37463][SQL] Read/Write Timestamp ntz or ltz to Orc uses UTC timestamp

AmplabJenkins removed a comment on pull request #34712: URL: https://github.com/apache/spark/pull/34712#issuecomment-983545862 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34673: [SPARK-37343][SQL] Implement createIndex, IndexExists and dropIndex in JDBC (Postgres dialect)

AmplabJenkins removed a comment on pull request #34673: URL: https://github.com/apache/spark/pull/34673#issuecomment-983545863 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/50280/

[GitHub] [spark] SparkQA commented on pull request #34764: [SPARK-37330][SQL] Migrate ReplaceTableStatement to v2 command

SparkQA commented on pull request #34764: URL: https://github.com/apache/spark/pull/34764#issuecomment-983384960 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/50270/ -- This is an automated message from the

[GitHub] [spark] dongjoon-hyun commented on pull request #34706: [SPARK-37458][SS] Remove unnecessary SerializeFromObject from the plan of foreachBatch

dongjoon-hyun commented on pull request #34706: URL: https://github.com/apache/spark/pull/34706#issuecomment-983383567 Oops. Sorry, I merged it before seeing your message, @HeartSaVioR . -- This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] HeartSaVioR commented on pull request #34706: [SPARK-37458][SS] Remove unnecessary SerializeFromObject from the plan of foreachBatch

HeartSaVioR commented on pull request #34706: URL: https://github.com/apache/spark/pull/34706#issuecomment-983384170 Never mind. I guess there's no possibility someone else modifies the relevant code, so we're good to go. Thanks for taking care of! -- This is an automated message

[GitHub] [spark] dongjoon-hyun commented on pull request #34706: [SPARK-37458][SS] Remove unnecessary SerializeFromObject from the plan of foreachBatch

dongjoon-hyun commented on pull request #34706: URL: https://github.com/apache/spark/pull/34706#issuecomment-983383823 BTW, thank you so much, @HeartSaVioR and @cloud-fan . -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

[GitHub] [spark] cloud-fan commented on a change in pull request #34684: [SPARK-37442][SQL] InMemoryRelation statistics bug causing broadcast join failures with AQE enabled

cloud-fan commented on a change in pull request #34684: URL: https://github.com/apache/spark/pull/34684#discussion_r759942207 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/columnar/InMemoryRelation.scala ## @@ -237,7 +238,23 @@ case class

[GitHub] [spark] Yikun commented on a change in pull request #34750: [SPARK-37495][PYTHON] Skip identical index checking of Series.compare when config 'compute.eager_check' is disabled

Yikun commented on a change in pull request #34750: URL: https://github.com/apache/spark/pull/34750#discussion_r759833192 ## File path: python/pyspark/pandas/series.py ## @@ -5781,6 +5781,25 @@ def compare( """ Compare to another Series and show the

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34758: [SPARK-37501][SQL] CREATE/REPLACE TABLE should qualify location for v2 command

AmplabJenkins removed a comment on pull request #34758: URL: https://github.com/apache/spark/pull/34758#issuecomment-983412550 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145784/

[GitHub] [spark] AmplabJenkins commented on pull request #34758: [SPARK-37501][SQL] CREATE/REPLACE TABLE should qualify location for v2 command

AmplabJenkins commented on pull request #34758: URL: https://github.com/apache/spark/pull/34758#issuecomment-983412550 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145784/ -- This

[GitHub] [spark] SparkQA commented on pull request #34767: [SPARK-37461][YARN][FOLLOWUP] Refactor YARN Client code to avoid add unnecessary parameter of `appId`

SparkQA commented on pull request #34767: URL: https://github.com/apache/spark/pull/34767#issuecomment-983436474 **[Test build #145807 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145807/testReport)** for PR 34767 at commit

[GitHub] [spark] yutoacts commented on a change in pull request #34718: [SPARK-37460][DOCS] Add the description of ALTER DATABASE SET LOCATION

yutoacts commented on a change in pull request #34718: URL: https://github.com/apache/spark/pull/34718#discussion_r759994496 ## File path: docs/sql-ref-syntax-ddl-alter-database.md ## @@ -21,25 +21,47 @@ license: | ### Description -You can alter metadata associated with a

[GitHub] [spark] SparkQA commented on pull request #34757: [SPARK-37504][PYTHON] Pyspark create SparkSession with existed session should not pass static conf

SparkQA commented on pull request #34757: URL: https://github.com/apache/spark/pull/34757#issuecomment-983451503 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/50274/ -- This is an automated message from the

[GitHub] [spark] SparkQA commented on pull request #34753: [SPARK-37494][SQL] Unify v1 and v2 options output of `SHOW CREATE TABLE` command

SparkQA commented on pull request #34753: URL: https://github.com/apache/spark/pull/34753#issuecomment-983465649 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/50279/ -- This is an automated message from the Apache

[GitHub] [spark] SparkQA commented on pull request #34767: [SPARK-37461][YARN][FOLLOWUP] Refactor YARN Client code to avoid add unnecessary parameter of `appId`

SparkQA commented on pull request #34767: URL: https://github.com/apache/spark/pull/34767#issuecomment-983472097 **[Test build #145808 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145808/testReport)** for PR 34767 at commit

[GitHub] [spark] SparkQA commented on pull request #34712: [SPARK-37463][SQL] Read/Write Timestamp ntz or ltz to Orc uses UTC timestamp

SparkQA commented on pull request #34712: URL: https://github.com/apache/spark/pull/34712#issuecomment-983515417 **[Test build #145812 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145812/testReport)** for PR 34712 at commit

[GitHub] [spark] SparkQA commented on pull request #34763: [SPARK-37389][SQL][FOLLOWUP] SET command shuold not parse comments

SparkQA commented on pull request #34763: URL: https://github.com/apache/spark/pull/34763#issuecomment-983541157 **[Test build #145796 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145796/testReport)** for PR 34763 at commit

[GitHub] [spark] SparkQA commented on pull request #34718: [SPARK-37460][DOCS] Add the description of ALTER DATABASE SET LOCATION

SparkQA commented on pull request #34718: URL: https://github.com/apache/spark/pull/34718#issuecomment-983540899 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/50285/ -- This is an automated message from the Apache

[GitHub] [spark] SparkQA commented on pull request #34712: [SPARK-37463][SQL] Read/Write Timestamp ntz or ltz to Orc uses UTC timestamp

SparkQA commented on pull request #34712: URL: https://github.com/apache/spark/pull/34712#issuecomment-983548980 **[Test build #145813 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145813/testReport)** for PR 34712 at commit

[GitHub] [spark] SparkQA commented on pull request #34771: [SPARK-37326][SQL][FOLLOWUP] Fix the test for Java 11

SparkQA commented on pull request #34771: URL: https://github.com/apache/spark/pull/34771#issuecomment-983811002 **[Test build #145819 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145819/testReport)** for PR 34771 at commit

[GitHub] [spark] SparkQA commented on pull request #34770: [SPARK-37480][K8S][DOC][3.2] Sync Kubernetes configuration to latest in running-on-k8s.md

SparkQA commented on pull request #34770: URL: https://github.com/apache/spark/pull/34770#issuecomment-983834863 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/50293/ -- This is an automated message from the

[GitHub] [spark] risinga commented on a change in pull request #34359: [SPARK-36986][SQL] Improving external schema management flexibility on DataSet and StructType

risinga commented on a change in pull request #34359: URL: https://github.com/apache/spark/pull/34359#discussion_r760404271 ## File path: sql/core/src/main/scala/org/apache/spark/sql/SparkSession.scala ## @@ -511,6 +511,20 @@ class SparkSession private(

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34769: [SPARK-37463][SQL] Read/Write Timestamp ntz to Orc uses UTC timestamp

AmplabJenkins removed a comment on pull request #34769: URL: https://github.com/apache/spark/pull/34769#issuecomment-983864270 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145814/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34741: [SPARK-37463][SQL] Read/Write Timestamp ntz from/to Orc uses UTC time zone

AmplabJenkins removed a comment on pull request #34741: URL: https://github.com/apache/spark/pull/34741#issuecomment-983864272 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145815/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34770: [SPARK-37480][K8S][DOC][3.2] Sync Kubernetes configuration to latest in running-on-k8s.md

AmplabJenkins removed a comment on pull request #34770: URL: https://github.com/apache/spark/pull/34770#issuecomment-983864273 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/50293/

[GitHub] [spark] AmplabJenkins commented on pull request #34770: [SPARK-37480][K8S][DOC][3.2] Sync Kubernetes configuration to latest in running-on-k8s.md

AmplabJenkins commented on pull request #34770: URL: https://github.com/apache/spark/pull/34770#issuecomment-983864273 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/50293/ --

[GitHub] [spark] SparkQA commented on pull request #32298: [SPARK-34079][SQL] Merge non-correlated scalar subqueries

SparkQA commented on pull request #32298: URL: https://github.com/apache/spark/pull/32298#issuecomment-983872401 **[Test build #145822 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145822/testReport)** for PR 32298 at commit

[GitHub] [spark] AmplabJenkins commented on pull request #34754: [SPARK-37496][SQL] Migrate ReplaceTableAsSelectStatement to v2 command

AmplabJenkins commented on pull request #34754: URL: https://github.com/apache/spark/pull/34754#issuecomment-983938196 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/50295/ --

[GitHub] [spark] AmplabJenkins commented on pull request #34771: [SPARK-37326][SQL][FOLLOWUP] Fix the test for Java 11

AmplabJenkins commented on pull request #34771: URL: https://github.com/apache/spark/pull/34771#issuecomment-983938195 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/50294/ --

[GitHub] [spark] mridulm commented on pull request #34715: [SPARK-37445][BUILD] Rename the maven hadoop profile to hadoop-3 and hadoop-2

mridulm commented on pull request #34715: URL: https://github.com/apache/spark/pull/34715#issuecomment-983962059 +CC @xkrogen -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [spark] AmplabJenkins commented on pull request #32298: [SPARK-34079][SQL] Merge non-correlated scalar subqueries

AmplabJenkins commented on pull request #32298: URL: https://github.com/apache/spark/pull/32298#issuecomment-983983324 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/50297/ --

[GitHub] [spark] SparkQA commented on pull request #32298: [SPARK-34079][SQL] Merge non-correlated scalar subqueries

SparkQA commented on pull request #32298: URL: https://github.com/apache/spark/pull/32298#issuecomment-983983293 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/50297/ -- This is an automated message from the

[GitHub] [spark] SparkQA removed a comment on pull request #34738: [SPARK-37483][SQL] Support push down top N to JDBC data source V2

SparkQA removed a comment on pull request #34738: URL: https://github.com/apache/spark/pull/34738#issuecomment-983706656 **[Test build #145817 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145817/testReport)** for PR 34738 at commit

[GitHub] [spark] AmplabJenkins commented on pull request #34607: [SPARK-36038][CORE] Speculation metrics summary at stage level

AmplabJenkins commented on pull request #34607: URL: https://github.com/apache/spark/pull/34607#issuecomment-983983439 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145821/ -- This

[GitHub] [spark] AmplabJenkins commented on pull request #34599: [SPARK-37331][K8S] Add the ability to create resources before driverPod creating

AmplabJenkins commented on pull request #34599: URL: https://github.com/apache/spark/pull/34599#issuecomment-983810370 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/50291/ --

[GitHub] [spark] SparkQA removed a comment on pull request #34769: [SPARK-37463][SQL] Read/Write Timestamp ntz to Orc uses UTC timestamp

SparkQA removed a comment on pull request #34769: URL: https://github.com/apache/spark/pull/34769#issuecomment-983594523 **[Test build #145814 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145814/testReport)** for PR 34769 at commit

[GitHub] [spark] SparkQA commented on pull request #34754: [SPARK-37496][SQL] Migrate ReplaceTableAsSelectStatement to v2 command

SparkQA commented on pull request #34754: URL: https://github.com/apache/spark/pull/34754#issuecomment-983862023 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/50295/ -- This is an automated message from the Apache

[GitHub] [spark] SparkQA commented on pull request #34607: [SPARK-36038][CORE] Speculation metrics summary at stage level

SparkQA commented on pull request #34607: URL: https://github.com/apache/spark/pull/34607#issuecomment-983867464 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/50296/ -- This is an automated message from the Apache

[GitHub] [spark] cloud-fan commented on a change in pull request #34754: [SPARK-37496][SQL] Migrate ReplaceTableAsSelectStatement to v2 command

cloud-fan commented on a change in pull request #34754: URL: https://github.com/apache/spark/pull/34754#discussion_r760421099 ## File path: sql/core/src/main/scala/org/apache/spark/sql/DataFrameWriter.scala ## @@ -586,19 +586,22 @@ final class DataFrameWriter[T]

[GitHub] [spark] SparkQA commented on pull request #34771: [SPARK-37326][SQL][FOLLOWUP] Fix the test for Java 11

SparkQA commented on pull request #34771: URL: https://github.com/apache/spark/pull/34771#issuecomment-983927653 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/50294/ -- This is an automated message from the

[GitHub] [spark] huaxingao commented on a change in pull request #34754: [SPARK-37496][SQL] Migrate ReplaceTableAsSelectStatement to v2 command

huaxingao commented on a change in pull request #34754: URL: https://github.com/apache/spark/pull/34754#discussion_r760456645 ## File path: sql/core/src/main/scala/org/apache/spark/sql/DataFrameWriter.scala ## @@ -586,19 +586,22 @@ final class DataFrameWriter[T]

[GitHub] [spark] sunchao commented on a change in pull request #32875: [SPARK-35703][SQL] Relax constraint for bucket join and remove HashClusteredDistribution

sunchao commented on a change in pull request #32875: URL: https://github.com/apache/spark/pull/32875#discussion_r760476421 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/exchange/EnsureRequirements.scala ## @@ -70,13 +70,63 @@ case class

[GitHub] [spark] sunchao commented on a change in pull request #34659: [SPARK-34863][SQL] Support complex types for Parquet vectorized reader

sunchao commented on a change in pull request #34659: URL: https://github.com/apache/spark/pull/34659#discussion_r760484066 ## File path: sql/core/src/main/java/org/apache/spark/sql/execution/datasources/parquet/ParquetColumnVector.java ## @@ -0,0 +1,321 @@ +/* + * Licensed

[GitHub] [spark] mridulm commented on pull request #34672: [SPARK-37394][CORE] Skip registering with ESS if a customized shuffle manager is configured

mridulm commented on pull request #34672: URL: https://github.com/apache/spark/pull/34672#issuecomment-983972185 Agree with @tgravescs. We are increasingly seeing external use/extensions of traits/classes explicitly marked private to spark, and changes to accommodate this pattern. This

[GitHub] [spark] SparkQA commented on pull request #34738: [SPARK-37483][SQL] Support push down top N to JDBC data source V2

SparkQA commented on pull request #34738: URL: https://github.com/apache/spark/pull/34738#issuecomment-983982638 **[Test build #145817 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145817/testReport)** for PR 34738 at commit

[GitHub] [spark] sunchao commented on a change in pull request #34659: [SPARK-34863][SQL] Support complex types for Parquet vectorized reader

sunchao commented on a change in pull request #34659: URL: https://github.com/apache/spark/pull/34659#discussion_r760523416 ## File path: sql/core/src/main/java/org/apache/spark/sql/execution/datasources/parquet/ParquetReadState.java ## @@ -42,24 +44,52 @@ /** The current

[GitHub] [spark] sathiyapk commented on a change in pull request #34729: [SPARK-37475][SQL] Add scale parameter to floor and ceil functions

sathiyapk commented on a change in pull request #34729: URL: https://github.com/apache/spark/pull/34729#discussion_r760536604 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/mathExpressions.scala ## @@ -243,40 +243,26 @@ case class

<    1   2   3   4   5   6   7   >