[GitHub] [spark] gaborgsomogyi commented on a change in pull request #30389: [SPARK-33143][PYTHON] Add configurable timeout to python server and client

gaborgsomogyi commented on a change in pull request #30389: URL: https://github.com/apache/spark/pull/30389#discussion_r525876645 ## File path: core/src/test/scala/org/apache/spark/api/python/PythonRDDSuite.scala ## @@ -76,12 +79,22 @@ class PythonRDDSuite extends SparkFunSuite

[GitHub] [spark] SparkQA commented on pull request #30406: [SPARK-33473][SQL] Extend interpreted subexpression elimination to other interpreted projections

SparkQA commented on pull request #30406: URL: https://github.com/apache/spark/pull/30406#issuecomment-729502912 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/35861/ -

[GitHub] [spark] gaborgsomogyi commented on pull request #30336: [SPARK-33287][SS][UI]Expose state custom metrics information on SS UI

gaborgsomogyi commented on pull request #30336: URL: https://github.com/apache/spark/pull/30336#issuecomment-729502108 > O.K, it seems to be better to document how to get the list of custom metrics. I'll add it to the doc of the new parameter. --

[GitHub] [spark] gaborgsomogyi commented on a change in pull request #30336: [SPARK-33287][SS][UI]Expose state custom metrics information on SS UI

gaborgsomogyi commented on a change in pull request #30336: URL: https://github.com/apache/spark/pull/30336#discussion_r525874550 ## File path: sql/core/src/main/scala/org/apache/spark/sql/streaming/ui/StreamingQueryStatisticsPage.scala ## @@ -236,12 +239,61 @@ private[ui] cla

[GitHub] [spark] gengliangwang commented on pull request #28841: [SPARK-31962][SQL] Provide modifiedAfter and modifiedBefore options when filtering from a batch-based file data source

gengliangwang commented on pull request #28841: URL: https://github.com/apache/spark/pull/28841#issuecomment-729501113 I'm sorry that I was forcing on other tasks and couldn't follow this thread. Thanks for the great work, @cchighman ! --

[GitHub] [spark] SparkQA commented on pull request #30373: [SPARK-33472][SQL] Adjust RemoveRedundantSorts rule order

SparkQA commented on pull request #30373: URL: https://github.com/apache/spark/pull/30373#issuecomment-729501057 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/35859/ ---

[GitHub] [spark] AmplabJenkins commented on pull request #30373: [SPARK-33472][SQL] Adjust RemoveRedundantSorts rule order

AmplabJenkins commented on pull request #30373: URL: https://github.com/apache/spark/pull/30373#issuecomment-729501074 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] AmplabJenkins commented on pull request #30164: [SPARK-32919][SHUFFLE][test-maven][test-hadoop2.7] Driver side changes for coordinating push based shuffle by selecting external shuffl

AmplabJenkins commented on pull request #30164: URL: https://github.com/apache/spark/pull/30164#issuecomment-729500060 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] SparkQA commented on pull request #30164: [SPARK-32919][SHUFFLE][test-maven][test-hadoop2.7] Driver side changes for coordinating push based shuffle by selecting external shuffle serv

SparkQA commented on pull request #30164: URL: https://github.com/apache/spark/pull/30164#issuecomment-729500038 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/35858/ ---

[GitHub] [spark] SparkQA commented on pull request #30406: [SPARK-33473][SQL] Extend interpreted subexpression elimination to other interpreted projections

SparkQA commented on pull request #30406: URL: https://github.com/apache/spark/pull/30406#issuecomment-729499173 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/35860/ -

[GitHub] [spark] AmplabJenkins commented on pull request #30164: [SPARK-32919][SHUFFLE][test-maven][test-hadoop2.7] Driver side changes for coordinating push based shuffle by selecting external shuffl

AmplabJenkins commented on pull request #30164: URL: https://github.com/apache/spark/pull/30164#issuecomment-729494363 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] SparkQA commented on pull request #30164: [SPARK-32919][SHUFFLE][test-maven][test-hadoop2.7] Driver side changes for coordinating push based shuffle by selecting external shuffle serv

SparkQA commented on pull request #30164: URL: https://github.com/apache/spark/pull/30164#issuecomment-729494347 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/35857/ ---

[GitHub] [spark] xuzikun2003 commented on a change in pull request #29725: [SPARK-32096][SQL] Improve sorting performance of Spark SQL window function by removing window partition key from sort order

xuzikun2003 commented on a change in pull request #29725: URL: https://github.com/apache/spark/pull/29725#discussion_r525862140 ## File path: sql/core/src/main/java/org/apache/spark/sql/execution/UnsafeExternalRowWindowSorter.java ## @@ -0,0 +1,453 @@ +/* + * Licensed to the A

[GitHub] [spark] xuzikun2003 commented on a change in pull request #29725: [SPARK-32096][SQL] Improve sorting performance of Spark SQL window function by removing window partition key from sort order

xuzikun2003 commented on a change in pull request #29725: URL: https://github.com/apache/spark/pull/29725#discussion_r525862140 ## File path: sql/core/src/main/java/org/apache/spark/sql/execution/UnsafeExternalRowWindowSorter.java ## @@ -0,0 +1,453 @@ +/* + * Licensed to the A

[GitHub] [spark] xuzikun2003 commented on a change in pull request #29725: [SPARK-32096][SQL] Improve sorting performance of Spark SQL window function by removing window partition key from sort order

xuzikun2003 commented on a change in pull request #29725: URL: https://github.com/apache/spark/pull/29725#discussion_r525862140 ## File path: sql/core/src/main/java/org/apache/spark/sql/execution/UnsafeExternalRowWindowSorter.java ## @@ -0,0 +1,453 @@ +/* + * Licensed to the A

[GitHub] [spark] dongjoon-hyun commented on pull request #25870: [SPARK-27936][K8S] Support python deps

dongjoon-hyun commented on pull request #25870: URL: https://github.com/apache/spark/pull/25870#issuecomment-729492170 Gentle ping, @skonto . This is an automated message from the Apache Git Service. To respond to the message

[GitHub] [spark] xuzikun2003 commented on a change in pull request #29725: [SPARK-32096][SQL] Improve sorting performance of Spark SQL window function by removing window partition key from sort order

xuzikun2003 commented on a change in pull request #29725: URL: https://github.com/apache/spark/pull/29725#discussion_r525862140 ## File path: sql/core/src/main/java/org/apache/spark/sql/execution/UnsafeExternalRowWindowSorter.java ## @@ -0,0 +1,453 @@ +/* + * Licensed to the A

[GitHub] [spark] gengliangwang commented on a change in pull request #30407: [SPARK-32852][SQL][Doc] Revise the documentation of spark.sql.hive.metastore.jars

gengliangwang commented on a change in pull request #30407: URL: https://github.com/apache/spark/pull/30407#discussion_r525863735 ## File path: sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveUtils.scala ## @@ -96,17 +96,18 @@ private[spark] object HiveUtils extends Loggi

[GitHub] [spark] dongjoon-hyun commented on pull request #30405: [SPARK-33476][CORE] Generalize ExecutorSource to expose user-given file system schemes

dongjoon-hyun commented on pull request #30405: URL: https://github.com/apache/spark/pull/30405#issuecomment-729490935 Thank you, @viirya ! This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #30407: [SPARK-32852][SQL][Doc] Revise the documentation of spark.sql.hive.metastore.jars

dongjoon-hyun commented on a change in pull request #30407: URL: https://github.com/apache/spark/pull/30407#discussion_r525862572 ## File path: sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveUtils.scala ## @@ -96,17 +96,18 @@ private[spark] object HiveUtils extends Loggi

[GitHub] [spark] SparkQA commented on pull request #30373: [SPARK-33472][SQL] Adjust RemoveRedundantSorts rule order

SparkQA commented on pull request #30373: URL: https://github.com/apache/spark/pull/30373#issuecomment-729490026 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/35859/ -

[GitHub] [spark] xuzikun2003 commented on a change in pull request #29725: [SPARK-32096][SQL] Improve sorting performance of Spark SQL window function by removing window partition key from sort order

xuzikun2003 commented on a change in pull request #29725: URL: https://github.com/apache/spark/pull/29725#discussion_r525862140 ## File path: sql/core/src/main/java/org/apache/spark/sql/execution/UnsafeExternalRowWindowSorter.java ## @@ -0,0 +1,453 @@ +/* + * Licensed to the A

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #30407: [SPARK-32852][SQL][Doc] Revise the documentation of spark.sql.hive.metastore.jars

dongjoon-hyun commented on a change in pull request #30407: URL: https://github.com/apache/spark/pull/30407#discussion_r525861673 ## File path: sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveUtils.scala ## @@ -96,17 +96,18 @@ private[spark] object HiveUtils extends Loggi

[GitHub] [spark] SparkQA commented on pull request #30164: [SPARK-32919][SHUFFLE][test-maven][test-hadoop2.7] Driver side changes for coordinating push based shuffle by selecting external shuffle serv

SparkQA commented on pull request #30164: URL: https://github.com/apache/spark/pull/30164#issuecomment-729489017 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/35858/ -

[GitHub] [spark] LuciferYang edited a comment on pull request #30234: [SPARK-33285][CORE][SQL] Fix deprecated compilation warnings of "Auto-application to () is deprecated" in Scala 2.13

LuciferYang edited a comment on pull request #30234: URL: https://github.com/apache/spark/pull/30234#issuecomment-729487796 seems all case has been fixed except for `RDD.isEmpty` relevant case This is an automated message fro

[GitHub] [spark] LuciferYang commented on pull request #30234: [SPARK-33285][CORE][SQL] Fix deprecated compilation warnings of "Auto-application to () is deprecated" in Scala 2.13

LuciferYang commented on pull request #30234: URL: https://github.com/apache/spark/pull/30234#issuecomment-729487796 seems all case fixed except for `RDD.isEmpty` relevant case This is an automated message from the Apache Git

[GitHub] [spark] gengliangwang commented on a change in pull request #30407: [SPARK-32852][SQL][Doc] Revise the documentation of spark.sql.hive.metastore.jars

gengliangwang commented on a change in pull request #30407: URL: https://github.com/apache/spark/pull/30407#discussion_r525860237 ## File path: sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveUtils.scala ## @@ -95,18 +95,19 @@ private[spark] object HiveUtils extends Loggi

[GitHub] [spark] dongjoon-hyun commented on pull request #30234: [SPARK-33285][CORE][SQL] Fix deprecated compilation warnings of "Auto-application to () is deprecated" in Scala 2.13

dongjoon-hyun commented on pull request #30234: URL: https://github.com/apache/spark/pull/30234#issuecomment-729487327 Sorry, but what do you mean? > It seems that except for RDD.isEmpty has been fixed. This is an automate

[GitHub] [spark] LuciferYang commented on pull request #30234: [SPARK-33285][CORE][SQL] Fix deprecated compilation warnings of "Auto-application to () is deprecated" in Scala 2.13

LuciferYang commented on pull request #30234: URL: https://github.com/apache/spark/pull/30234#issuecomment-729486540 @dongjoon-hyun It seems that except for `RDD.isEmpty` has been fixed. This is an automated message f

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #30407: [SPARK-32852][SQL][Doc] Revise the documentation of spark.sql.hive.metastore.jars

dongjoon-hyun commented on a change in pull request #30407: URL: https://github.com/apache/spark/pull/30407#discussion_r525859433 ## File path: sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveUtils.scala ## @@ -95,18 +95,19 @@ private[spark] object HiveUtils extends Loggi

[GitHub] [spark] dongjoon-hyun commented on pull request #30391: [SPARK-33464][INFRA] Add/remove (un)necessary cache and restructure GitHub Actions yaml

dongjoon-hyun commented on pull request #30391: URL: https://github.com/apache/spark/pull/30391#issuecomment-729485211 The position is up to the agreement between @HyukjinKwon and @mridulm . This is an automated message from

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30164: [SPARK-32919][SHUFFLE][test-maven][test-hadoop2.7] Driver side changes for coordinating push based shuffle by selecting externa

AmplabJenkins removed a comment on pull request #30164: URL: https://github.com/apache/spark/pull/30164#issuecomment-729483958 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #30164: [SPARK-32919][SHUFFLE][test-maven][test-hadoop2.7] Driver side changes for coordinating push based shuffle by selecting external shuffl

AmplabJenkins commented on pull request #30164: URL: https://github.com/apache/spark/pull/30164#issuecomment-729483958 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] SparkQA commented on pull request #30164: [SPARK-32919][SHUFFLE][test-maven][test-hadoop2.7] Driver side changes for coordinating push based shuffle by selecting external shuffle serv

SparkQA commented on pull request #30164: URL: https://github.com/apache/spark/pull/30164#issuecomment-729483939 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/35856/ ---

[GitHub] [spark] HeartSaVioR commented on pull request #28841: [SPARK-31962][SQL] Provide modifiedAfter and modifiedBefore options when filtering from a batch-based file data source

HeartSaVioR commented on pull request #28841: URL: https://github.com/apache/spark/pull/28841#issuecomment-729483537 Thanks @cchighman for great efforts during so far, and sorry to make you struggle with the review process. I'll take this over based on the current state of the PR and addre

[GitHub] [spark] gengliangwang commented on pull request #30407: [SPARK-32852][Doc] Revise the documentation of spark.sql.hive.metastore.jars

gengliangwang commented on pull request #30407: URL: https://github.com/apache/spark/pull/30407#issuecomment-729483340 cc @AngersZh @cloud-fan This is an automated message from the Apache Git Service. To respond to the m

[GitHub] [spark] gengliangwang opened a new pull request #30407: [SPARK-32852][Doc] Revise the documentation of spark.sql.hive.metastore.jars

gengliangwang opened a new pull request #30407: URL: https://github.com/apache/spark/pull/30407 ### What changes were proposed in this pull request? This is a follow-up for https://github.com/apache/spark/pull/29881. It revises the documentation of the configuration `spark

[GitHub] [spark] SparkQA commented on pull request #30164: [SPARK-32919][SHUFFLE][test-maven][test-hadoop2.7] Driver side changes for coordinating push based shuffle by selecting external shuffle serv

SparkQA commented on pull request #30164: URL: https://github.com/apache/spark/pull/30164#issuecomment-729480460 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/35857/ -

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30355: [SPARK-32907][ML][PYTHON] Adaptively blockify instances - AFT,LiR,LoR

AmplabJenkins removed a comment on pull request #30355: URL: https://github.com/apache/spark/pull/30355#issuecomment-729475604 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30403: [SPARK-33448][SQL] Migrate CACHE/UNCACHE TABLE command to use UnresolvedTableOrView to resolve the identifier

AmplabJenkins removed a comment on pull request #30403: URL: https://github.com/apache/spark/pull/30403#issuecomment-729471388 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/131

[GitHub] [spark] SparkQA removed a comment on pull request #30355: [SPARK-32907][ML][PYTHON] Adaptively blockify instances - AFT,LiR,LoR

SparkQA removed a comment on pull request #30355: URL: https://github.com/apache/spark/pull/30355#issuecomment-729434913 **[Test build #131252 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/131252/testReport)** for PR 30355 at commit [`48b2814`](https://gi

[GitHub] [spark] SparkQA removed a comment on pull request #30234: [SPARK-33285][CORE][SQL] Fix deprecated compilation warnings of "Auto-application to () is deprecated" in Scala 2.13

SparkQA removed a comment on pull request #30234: URL: https://github.com/apache/spark/pull/30234#issuecomment-729391035 **[Test build #131244 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/131244/testReport)** for PR 30234 at commit [`2082c3b`](https://gi

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30234: [SPARK-33285][CORE][SQL] Fix deprecated compilation warnings of "Auto-application to () is deprecated" in Scala 2.13

AmplabJenkins removed a comment on pull request #30234: URL: https://github.com/apache/spark/pull/30234#issuecomment-729477538 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] SparkQA removed a comment on pull request #30403: [SPARK-33448][SQL] Migrate CACHE/UNCACHE TABLE command to use UnresolvedTableOrView to resolve the identifier

SparkQA removed a comment on pull request #30403: URL: https://github.com/apache/spark/pull/30403#issuecomment-72944 **[Test build #131246 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/131246/testReport)** for PR 30403 at commit [`a0687b3`](https://gi

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30351: [SPARK-33441][BUILD] Add unused-imports compilation check and remove all unused-imports

AmplabJenkins removed a comment on pull request #30351: URL: https://github.com/apache/spark/pull/30351#issuecomment-729469598 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30260: [SPARK-33354][SQL] New explicit cast syntax rules in ANSI mode

AmplabJenkins removed a comment on pull request #30260: URL: https://github.com/apache/spark/pull/30260#issuecomment-729466252 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30403: [SPARK-33448][SQL] Migrate CACHE/UNCACHE TABLE command to use UnresolvedTableOrView to resolve the identifier

AmplabJenkins removed a comment on pull request #30403: URL: https://github.com/apache/spark/pull/30403#issuecomment-729471375 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To r

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30355: [SPARK-32907][ML][PYTHON] Adaptively blockify instances - AFT,LiR,LoR

AmplabJenkins removed a comment on pull request #30355: URL: https://github.com/apache/spark/pull/30355#issuecomment-729473406 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30405: [SPARK-33476][CORE] Generalize ExecutorSource to expose user-given file system schemes

AmplabJenkins removed a comment on pull request #30405: URL: https://github.com/apache/spark/pull/30405#issuecomment-729460845 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] SparkQA commented on pull request #30393: [SPARK-24554][PYTHON][SQL] Add MapType support for PySpark with Arrow

SparkQA commented on pull request #30393: URL: https://github.com/apache/spark/pull/30393#issuecomment-729477702 **[Test build #131260 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/131260/testReport)** for PR 30393 at commit [`b257470`](https://github.com

[GitHub] [spark] AmplabJenkins commented on pull request #30234: [SPARK-33285][CORE][SQL] Fix deprecated compilation warnings of "Auto-application to () is deprecated" in Scala 2.13

AmplabJenkins commented on pull request #30234: URL: https://github.com/apache/spark/pull/30234#issuecomment-729477538 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] otterc commented on a change in pull request #30312: [WIP][SPARK-32917][SHUFFLE][CORE][test-maven][test-hadoop2.7] Adds support for executors to push shuffle blocks after successful m

otterc commented on a change in pull request #30312: URL: https://github.com/apache/spark/pull/30312#discussion_r525853164 ## File path: core/src/main/scala/org/apache/spark/shuffle/PushShuffleSupport.scala ## @@ -0,0 +1,466 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] [spark] SparkQA commented on pull request #30234: [SPARK-33285][CORE][SQL] Fix deprecated compilation warnings of "Auto-application to () is deprecated" in Scala 2.13

SparkQA commented on pull request #30234: URL: https://github.com/apache/spark/pull/30234#issuecomment-729476281 **[Test build #131244 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/131244/testReport)** for PR 30234 at commit [`2082c3b`](https://github.co

[GitHub] [spark] AmplabJenkins commented on pull request #30355: [SPARK-32907][ML][PYTHON] Adaptively blockify instances - AFT,LiR,LoR

AmplabJenkins commented on pull request #30355: URL: https://github.com/apache/spark/pull/30355#issuecomment-729475594 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] SparkQA commented on pull request #30355: [SPARK-32907][ML][PYTHON] Adaptively blockify instances - AFT,LiR,LoR

SparkQA commented on pull request #30355: URL: https://github.com/apache/spark/pull/30355#issuecomment-729475571 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/35855/ ---

[GitHub] [spark] SparkQA commented on pull request #30405: [SPARK-33476][CORE] Generalize ExecutorSource to expose user-given file system schemes

SparkQA commented on pull request #30405: URL: https://github.com/apache/spark/pull/30405#issuecomment-729473677 **[Test build #131259 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/131259/testReport)** for PR 30405 at commit [`8ca8624`](https://github.com

[GitHub] [spark] AmplabJenkins commented on pull request #30355: [SPARK-32907][ML][PYTHON] Adaptively blockify instances - AFT,LiR,LoR

AmplabJenkins commented on pull request #30355: URL: https://github.com/apache/spark/pull/30355#issuecomment-729473406 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] SparkQA commented on pull request #30355: [SPARK-32907][ML][PYTHON] Adaptively blockify instances - AFT,LiR,LoR

SparkQA commented on pull request #30355: URL: https://github.com/apache/spark/pull/30355#issuecomment-729472700 **[Test build #131252 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/131252/testReport)** for PR 30355 at commit [`48b2814`](https://github.co

[GitHub] [spark] imback82 commented on a change in pull request #30403: [SPARK-33448][SQL] Migrate CACHE/UNCACHE TABLE command to use UnresolvedTableOrView to resolve the identifier

imback82 commented on a change in pull request #30403: URL: https://github.com/apache/spark/pull/30403#discussion_r525849567 ## File path: sql/hive/src/test/scala/org/apache/spark/sql/hive/CachedTableSuite.scala ## @@ -113,7 +113,7 @@ class CachedTableSuite extends QueryTest w

[GitHub] [spark] BryanCutler commented on a change in pull request #30393: [SPARK-24554][PYTHON][SQL] Add MapType support for PySpark with Arrow

BryanCutler commented on a change in pull request #30393: URL: https://github.com/apache/spark/pull/30393#discussion_r525849381 ## File path: python/docs/source/user_guide/arrow_pandas.rst ## @@ -341,7 +341,7 @@ Supported SQL Types .. currentmodule:: pyspark.sql.types -Cur

[GitHub] [spark] SparkQA commented on pull request #30406: [SPARK-33473][SQL] Extend interpreted subexpression elimination to other interpreted projections

SparkQA commented on pull request #30406: URL: https://github.com/apache/spark/pull/30406#issuecomment-729471571 **[Test build #131258 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/131258/testReport)** for PR 30406 at commit [`ee4b070`](https://github.com

[GitHub] [spark] AmplabJenkins commented on pull request #30403: [SPARK-33448][SQL] Migrate CACHE/UNCACHE TABLE command to use UnresolvedTableOrView to resolve the identifier

AmplabJenkins commented on pull request #30403: URL: https://github.com/apache/spark/pull/30403#issuecomment-729471375 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] SparkQA commented on pull request #30403: [SPARK-33448][SQL] Migrate CACHE/UNCACHE TABLE command to use UnresolvedTableOrView to resolve the identifier

SparkQA commented on pull request #30403: URL: https://github.com/apache/spark/pull/30403#issuecomment-729470662 **[Test build #131246 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/131246/testReport)** for PR 30403 at commit [`a0687b3`](https://github.co

[GitHub] [spark] AmplabJenkins commented on pull request #30351: [SPARK-33441][BUILD] Add unused-imports compilation check and remove all unused-imports

AmplabJenkins commented on pull request #30351: URL: https://github.com/apache/spark/pull/30351#issuecomment-729469598 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] SparkQA commented on pull request #30351: [SPARK-33441][BUILD] Add unused-imports compilation check and remove all unused-imports

SparkQA commented on pull request #30351: URL: https://github.com/apache/spark/pull/30351#issuecomment-729469567 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/35854/ ---

[GitHub] [spark] SparkQA commented on pull request #30164: [SPARK-32919][SHUFFLE][test-maven][test-hadoop2.7] Driver side changes for coordinating push based shuffle by selecting external shuffle serv

SparkQA commented on pull request #30164: URL: https://github.com/apache/spark/pull/30164#issuecomment-729469271 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/35856/ -

[GitHub] [spark] BryanCutler commented on a change in pull request #30393: [SPARK-24554][PYTHON][SQL] Add MapType support for PySpark with Arrow

BryanCutler commented on a change in pull request #30393: URL: https://github.com/apache/spark/pull/30393#discussion_r525847003 ## File path: python/docs/source/user_guide/arrow_pandas.rst ## @@ -341,7 +341,7 @@ Supported SQL Types .. currentmodule:: pyspark.sql.types -Cur

[GitHub] [spark] BryanCutler commented on a change in pull request #30393: [SPARK-24554][PYTHON][SQL] Add MapType support for PySpark with Arrow

BryanCutler commented on a change in pull request #30393: URL: https://github.com/apache/spark/pull/30393#discussion_r525846522 ## File path: python/pyspark/sql/pandas/types.py ## @@ -306,3 +322,23 @@ def _check_series_convert_timestamps_tz_local(s, timezone): `pandas.

[GitHub] [spark] AmplabJenkins commented on pull request #30260: [SPARK-33354][SQL] New explicit cast syntax rules in ANSI mode

AmplabJenkins commented on pull request #30260: URL: https://github.com/apache/spark/pull/30260#issuecomment-729466252 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] SparkQA commented on pull request #30260: [SPARK-33354][SQL] New explicit cast syntax rules in ANSI mode

SparkQA commented on pull request #30260: URL: https://github.com/apache/spark/pull/30260#issuecomment-729466242 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/35853/ ---

[GitHub] [spark] maropu commented on pull request #30406: [SPARK-33473][SQL] Extend interpreted subexpression elimination to other interpreted projections

maropu commented on pull request #30406: URL: https://github.com/apache/spark/pull/30406#issuecomment-729464155 Looks fine if the tests pass. This is an automated message from the Apache Git Service. To respond to the message

[GitHub] [spark] SparkQA commented on pull request #30355: [SPARK-32907][ML][PYTHON] Adaptively blockify instances - AFT,LiR,LoR

SparkQA commented on pull request #30355: URL: https://github.com/apache/spark/pull/30355#issuecomment-729462035 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/35855/ -

[GitHub] [spark] maropu commented on a change in pull request #29725: [SPARK-32096][SQL] Improve sorting performance of Spark SQL window function by removing window partition key from sort order

maropu commented on a change in pull request #29725: URL: https://github.com/apache/spark/pull/29725#discussion_r525841241 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/SortExec.scala ## @@ -42,6 +43,139 @@ case class SortExec( global: Boolean,

[GitHub] [spark] otterc commented on a change in pull request #30312: [WIP][SPARK-32917][SHUFFLE][CORE][test-maven][test-hadoop2.7] Adds support for executors to push shuffle blocks after successful m

otterc commented on a change in pull request #30312: URL: https://github.com/apache/spark/pull/30312#discussion_r525838001 ## File path: core/src/main/scala/org/apache/spark/shuffle/PushShuffleSupport.scala ## @@ -0,0 +1,466 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] [spark] dongjoon-hyun commented on pull request #30405: [SPARK-33476][CORE] Generalize ExecutorSource to expose user-given file system schemes

dongjoon-hyun commented on pull request #30405: URL: https://github.com/apache/spark/pull/30405#issuecomment-729461077 All comments are addressed. (@HyukjinKwon , @viirya , @mridulm ) This is an automated message from the Apa

[GitHub] [spark] AmplabJenkins commented on pull request #30405: [SPARK-33476][CORE] Generalize ExecutorSource to expose user-given file system schemes

AmplabJenkins commented on pull request #30405: URL: https://github.com/apache/spark/pull/30405#issuecomment-729460845 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] SparkQA commented on pull request #30405: [SPARK-33476][CORE] Generalize ExecutorSource to expose user-given file system schemes

SparkQA commented on pull request #30405: URL: https://github.com/apache/spark/pull/30405#issuecomment-729460822 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/35852/ ---

[GitHub] [spark] maropu commented on a change in pull request #29725: [SPARK-32096][SQL] Improve sorting performance of Spark SQL window function by removing window partition key from sort order

maropu commented on a change in pull request #29725: URL: https://github.com/apache/spark/pull/29725#discussion_r525840579 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/exchange/EnsureRequirements.scala ## @@ -124,7 +125,18 @@ case class EnsureRequiremen

[GitHub] [spark] dongjoon-hyun edited a comment on pull request #30164: [SPARK-32919][SHUFFLE][test-maven][test-hadoop2.7] Driver side changes for coordinating push based shuffle by selecting external

dongjoon-hyun edited a comment on pull request #30164: URL: https://github.com/apache/spark/pull/30164#issuecomment-729460106 +1, LGTM. (only editorial comments.) Thank you all! This is an automated message from the Apache

[GitHub] [spark] dongjoon-hyun commented on pull request #30164: [SPARK-32919][SHUFFLE][test-maven][test-hadoop2.7] Driver side changes for coordinating push based shuffle by selecting external shuffl

dongjoon-hyun commented on pull request #30164: URL: https://github.com/apache/spark/pull/30164#issuecomment-729460106 +1, LGTM. (only editorial comments.) This is an automated message from the Apache Git Service. To respond

[GitHub] [spark] maropu commented on a change in pull request #29725: [SPARK-32096][SQL] Improve sorting performance of Spark SQL window function by removing window partition key from sort order

maropu commented on a change in pull request #29725: URL: https://github.com/apache/spark/pull/29725#discussion_r525834434 ## File path: sql/core/src/main/java/org/apache/spark/sql/execution/UnsafeExternalRowWindowSorter.java ## @@ -0,0 +1,453 @@ +/* + * Licensed to the Apache

[GitHub] [spark] SparkQA commented on pull request #30351: [SPARK-33441][BUILD] Add unused-imports compilation check and remove all unused-imports

SparkQA commented on pull request #30351: URL: https://github.com/apache/spark/pull/30351#issuecomment-729458991 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/35854/ -

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30396: [MINOR][SQL][DOCS] Update schema_of_csv and schema_of_json doc

AmplabJenkins removed a comment on pull request #30396: URL: https://github.com/apache/spark/pull/30396#issuecomment-729457164 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] SparkQA commented on pull request #30406: [SPARK-33473][SQL] Extend interpreted subexpression elimination to other interpreted projections

SparkQA commented on pull request #30406: URL: https://github.com/apache/spark/pull/30406#issuecomment-729457313 **[Test build #131257 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/131257/testReport)** for PR 30406 at commit [`37e3dca`](https://github.com

[GitHub] [spark] AmplabJenkins commented on pull request #30396: [MINOR][SQL][DOCS] Update schema_of_csv and schema_of_json doc

AmplabJenkins commented on pull request #30396: URL: https://github.com/apache/spark/pull/30396#issuecomment-729457164 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] otterc commented on a change in pull request #30312: [WIP][SPARK-32917][SHUFFLE][CORE][test-maven][test-hadoop2.7] Adds support for executors to push shuffle blocks after successful m

otterc commented on a change in pull request #30312: URL: https://github.com/apache/spark/pull/30312#discussion_r525838001 ## File path: core/src/main/scala/org/apache/spark/shuffle/PushShuffleSupport.scala ## @@ -0,0 +1,466 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] [spark] SparkQA removed a comment on pull request #30396: [MINOR][SQL][DOCS] Update schema_of_csv and schema_of_json doc

SparkQA removed a comment on pull request #30396: URL: https://github.com/apache/spark/pull/30396#issuecomment-729321498 **[Test build #131239 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/131239/testReport)** for PR 30396 at commit [`1224ba9`](https://gi

[GitHub] [spark] SparkQA commented on pull request #30396: [MINOR][SQL][DOCS] Update schema_of_csv and schema_of_json doc

SparkQA commented on pull request #30396: URL: https://github.com/apache/spark/pull/30396#issuecomment-729456105 **[Test build #131239 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/131239/testReport)** for PR 30396 at commit [`1224ba9`](https://github.co

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #30405: [SPARK-33476][CORE] Generalize ExecutorSource to expose user-given file system schemes

dongjoon-hyun commented on a change in pull request #30405: URL: https://github.com/apache/spark/pull/30405#discussion_r525837460 ## File path: core/src/main/scala/org/apache/spark/executor/Executor.scala ## @@ -110,7 +110,9 @@ private[spark] class Executor( .build()

[GitHub] [spark] mridulm commented on a change in pull request #30405: [SPARK-33476][CORE] Generalize ExecutorSource to expose user-given file system schemes

mridulm commented on a change in pull request #30405: URL: https://github.com/apache/spark/pull/30405#discussion_r525836701 ## File path: core/src/main/scala/org/apache/spark/executor/Executor.scala ## @@ -110,7 +110,9 @@ private[spark] class Executor( .build() Exe

[GitHub] [spark] SparkQA commented on pull request #30260: [SPARK-33354][SQL] New explicit cast syntax rules in ANSI mode

SparkQA commented on pull request #30260: URL: https://github.com/apache/spark/pull/30260#issuecomment-729454825 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/35853/ -

[GitHub] [spark] c21 commented on a change in pull request #29000: [SPARK-27194][SPARK-29302][SQL] Fix commit collision in dynamic partition overwrite mode

c21 commented on a change in pull request #29000: URL: https://github.com/apache/spark/pull/29000#discussion_r525836484 ## File path: core/src/main/scala/org/apache/spark/internal/io/FileCommitProtocol.scala ## @@ -169,4 +169,8 @@ object FileCommitProtocol extends Logging {

[GitHub] [spark] Victsm commented on a change in pull request #30164: [SPARK-32919][SHUFFLE][test-maven][test-hadoop2.7] Driver side changes for coordinating push based shuffle by selecting external s

Victsm commented on a change in pull request #30164: URL: https://github.com/apache/spark/pull/30164#discussion_r525835969 ## File path: core/src/main/scala/org/apache/spark/scheduler/SchedulerBackend.scala ## @@ -92,4 +93,16 @@ private[spark] trait SchedulerBackend { */

[GitHub] [spark] dongjoon-hyun commented on pull request #30405: [SPARK-33476][CORE] Generalize ExecutorSource to expose user-given file system schemes

dongjoon-hyun commented on pull request #30405: URL: https://github.com/apache/spark/pull/30405#issuecomment-729453805 Thank you, @HyukjinKwon and @viirya . I'll mention this new conf in the `monitoring.md` according to your advice.

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #30164: [SPARK-32919][SHUFFLE][test-maven][test-hadoop2.7] Driver side changes for coordinating push based shuffle by selecting ext

dongjoon-hyun commented on a change in pull request #30164: URL: https://github.com/apache/spark/pull/30164#discussion_r525835108 ## File path: resource-managers/yarn/src/main/scala/org/apache/spark/scheduler/cluster/YarnSchedulerBackend.scala ## @@ -161,6 +173,35 @@ private[s

[GitHub] [spark] BryanCutler commented on pull request #30393: [SPARK-24554][PYTHON][SQL] Add MapType support for PySpark with Arrow

BryanCutler commented on pull request #30393: URL: https://github.com/apache/spark/pull/30393#issuecomment-729452568 > BTW I believe we should also update the docs :-) Thanks for reminding me! I'll do that now.. This i

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #30164: [SPARK-32919][SHUFFLE][test-maven][test-hadoop2.7] Driver side changes for coordinating push based shuffle by selecting ext

dongjoon-hyun commented on a change in pull request #30164: URL: https://github.com/apache/spark/pull/30164#discussion_r525834706 ## File path: resource-managers/yarn/src/main/scala/org/apache/spark/scheduler/cluster/YarnSchedulerBackend.scala ## @@ -161,6 +173,35 @@ private[s

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #30164: [SPARK-32919][SHUFFLE][test-maven][test-hadoop2.7] Driver side changes for coordinating push based shuffle by selecting ext

dongjoon-hyun commented on a change in pull request #30164: URL: https://github.com/apache/spark/pull/30164#discussion_r525834706 ## File path: resource-managers/yarn/src/main/scala/org/apache/spark/scheduler/cluster/YarnSchedulerBackend.scala ## @@ -161,6 +173,35 @@ private[s

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30404: [SPARK-33475][BUILD] Bump ANTLR runtime version to 4.8-1

AmplabJenkins removed a comment on pull request #30404: URL: https://github.com/apache/spark/pull/30404#issuecomment-729450454 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30400: [SPARK-33469][SQL] Add current_timezone function

AmplabJenkins removed a comment on pull request #30400: URL: https://github.com/apache/spark/pull/30400#issuecomment-729441891 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8

  1   2   3   4   5   6   7   8   >