[GitHub] [spark] AmplabJenkins commented on pull request #31348: [SPARK-34245][CORE] Ensure Master removes executors that failed to send finished state

AmplabJenkins commented on pull request #31348: URL: https://github.com/apache/spark/pull/31348#issuecomment-773964544 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins commented on pull request #29087: [SPARK-28227][SQL] Support projection, aggregate/window functions, and lateral view in the TRANSFORM clause

AmplabJenkins commented on pull request #29087: URL: https://github.com/apache/spark/pull/29087#issuecomment-773964538 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39511/

[GitHub] [spark] AmplabJenkins commented on pull request #30650: [SPARK-24818][CORE] Support delay scheduling for barrier execution

AmplabJenkins commented on pull request #30650: URL: https://github.com/apache/spark/pull/30650#issuecomment-773964550 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134923/

[GitHub] [spark] SparkQA removed a comment on pull request #30650: [SPARK-24818][CORE] Support delay scheduling for barrier execution

SparkQA removed a comment on pull request #30650: URL: https://github.com/apache/spark/pull/30650#issuecomment-773876021 **[Test build #134923 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134923/testReport)** for PR 30650 at commit

[GitHub] [spark] SparkQA removed a comment on pull request #31348: [SPARK-34245][CORE] Ensure Master removes executors that failed to send finished state

SparkQA removed a comment on pull request #31348: URL: https://github.com/apache/spark/pull/31348#issuecomment-773899913 **[Test build #134925 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134925/testReport)** for PR 31348 at commit

[GitHub] [spark] SparkQA commented on pull request #30650: [SPARK-24818][CORE] Support delay scheduling for barrier execution

SparkQA commented on pull request #30650: URL: https://github.com/apache/spark/pull/30650#issuecomment-773961581 **[Test build #134923 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134923/testReport)** for PR 30650 at commit

[GitHub] [spark] SparkQA commented on pull request #31348: [SPARK-34245][CORE] Ensure Master removes executors that failed to send finished state

SparkQA commented on pull request #31348: URL: https://github.com/apache/spark/pull/31348#issuecomment-773961452 **[Test build #134925 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134925/testReport)** for PR 31348 at commit

[GitHub] [spark] SparkQA commented on pull request #31482: [SPARK-34346][CORE][SQL][3.1] io.file.buffer.size set by spark.buffer.size will override by loading hive-site.xml accidentally may cause perf

SparkQA commented on pull request #31482: URL: https://github.com/apache/spark/pull/31482#issuecomment-773961391 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39515/

[GitHub] [spark] SparkQA commented on pull request #31133: [SPARK-26836][SQL] Supporting Avro schema evolution for partitioned Hive tables with "avro.schema.literal"

SparkQA commented on pull request #31133: URL: https://github.com/apache/spark/pull/31133#issuecomment-773871059 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39502/

[GitHub] [spark] SparkQA commented on pull request #31471: [SPARK-34355][SQL] Add log and time cost for commit job

SparkQA commented on pull request #31471: URL: https://github.com/apache/spark/pull/31471#issuecomment-773870939 **[Test build #134906 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134906/testReport)** for PR 31471 at commit

[GitHub] [spark] SparkQA commented on pull request #31486: [SPARK-34359][SQL][3.1] Add a legacy config to restore the output schema of SHOW DATABASES

SparkQA commented on pull request #31486: URL: https://github.com/apache/spark/pull/31486#issuecomment-773957672 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39507/

[GitHub] [spark] SparkQA removed a comment on pull request #30957: [SPARK-31937][SQL] Support processing ArrayType/MapType/StructType data using no-serde mode script transform

SparkQA removed a comment on pull request #30957: URL: https://github.com/apache/spark/pull/30957#issuecomment-773823173 **[Test build #134915 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134915/testReport)** for PR 30957 at commit

[GitHub] [spark] jaceklaskowski commented on a change in pull request #31488: [SPARK-34376][SQL] Support regexp as a SQL function

jaceklaskowski commented on a change in pull request #31488: URL: https://github.com/apache/spark/pull/31488#discussion_r570878907 ## File path: sql/core/src/test/resources/sql-tests/inputs/regexp-functions.sql ## @@ -46,4 +46,9 @@ SELECT regexp_replace('healthy, wealthy, and

[GitHub] [spark] SparkQA commented on pull request #31348: [SPARK-34245][CORE] Ensure Master removes executors that failed to send finished state

SparkQA commented on pull request #31348: URL: https://github.com/apache/spark/pull/31348#issuecomment-773952868 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39508/

[GitHub] [spark] SparkQA commented on pull request #29087: [SPARK-28227][SQL] Support projection, aggregate/window functions, and lateral view in the TRANSFORM clause

SparkQA commented on pull request #29087: URL: https://github.com/apache/spark/pull/29087#issuecomment-773955989 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39511/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31487: [SPARK-34375][CORE][K8S][TEST] Replaces 'Mockito.initMocks' with 'Mockito.openMocks'

AmplabJenkins removed a comment on pull request #31487: URL: https://github.com/apache/spark/pull/31487#issuecomment-773932866 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39512/

[GitHub] [spark] SparkQA commented on pull request #30957: [SPARK-31937][SQL] Support processing ArrayType/MapType/StructType data using no-serde mode script transform

SparkQA commented on pull request #30957: URL: https://github.com/apache/spark/pull/30957#issuecomment-773955494 **[Test build #134915 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134915/testReport)** for PR 30957 at commit

[GitHub] [spark] SparkQA commented on pull request #31258: [SPARK-34168] [SQL] Support DPP in AQE when the join is Broadcast hash join at the beginning

SparkQA commented on pull request #31258: URL: https://github.com/apache/spark/pull/31258#issuecomment-773946008 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39509/

[GitHub] [spark] SparkQA commented on pull request #31284: [SPARK-34167][SQL]Reading parquet with IntDecimal written as a LongDecimal blows up

SparkQA commented on pull request #31284: URL: https://github.com/apache/spark/pull/31284#issuecomment-773581777 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

[GitHub] [spark] SparkQA commented on pull request #31482: [SPARK-34346][CORE][SQL][3.1] io.file.buffer.size set by spark.buffer.size will override by loading hive-site.xml accidentally may cause perf

SparkQA commented on pull request #31482: URL: https://github.com/apache/spark/pull/31482#issuecomment-773943412 **[Test build #134932 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134932/testReport)** for PR 31482 at commit

[GitHub] [spark] SparkQA commented on pull request #30869: [SPARK-33865][SQL] When HiveDDL, we need check avro schema too

SparkQA commented on pull request #30869: URL: https://github.com/apache/spark/pull/30869#issuecomment-773946758 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39510/

[GitHub] [spark] AmplabJenkins commented on pull request #30650: [SPARK-24818][CORE] Support delay scheduling for barrier execution

AmplabJenkins commented on pull request #30650: URL: https://github.com/apache/spark/pull/30650#issuecomment-773931656 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39505/

[GitHub] [spark] SparkQA commented on pull request #31483: [SPARK-33434][PYTHON][DOCS] Added RuntimeConfig to PySpark docs

SparkQA commented on pull request #31483: URL: https://github.com/apache/spark/pull/31483#issuecomment-773851149 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

[GitHub] [spark] dongjoon-hyun commented on pull request #31467: [SPARK-33212][FOLLOW-UP][BUILD] Uses provided properties for Hadoop client dependencies in root pom

dongjoon-hyun commented on pull request #31467: URL: https://github.com/apache/spark/pull/31467#issuecomment-773562256 Thank you, @HyukjinKwon and @sunchao ! This is an automated message from the Apache Git Service. To

[GitHub] [spark] github-actions[bot] commented on pull request #28938: [WIP][SPARK-32118][SQL] Use fine-grained read write lock for each database in HiveExternalCatalog

github-actions[bot] commented on pull request #28938: URL: https://github.com/apache/spark/pull/28938#issuecomment-773702470 We're closing this PR because it hasn't been updated in a while. This isn't a judgement on the merit of the PR in any way. It's just a way of keeping the PR queue

[GitHub] [spark] AmplabJenkins commented on pull request #31485: [SPARK-SQL][34137] Update suquery's stats when build LogicalPlan's stats

AmplabJenkins commented on pull request #31485: URL: https://github.com/apache/spark/pull/31485#issuecomment-773931655 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39506/

[GitHub] [spark] AmplabJenkins commented on pull request #31245: [SPARK-34157][SQL] Unify output of SHOW TABLES and pass output attributes properly

AmplabJenkins commented on pull request #31245: URL: https://github.com/apache/spark/pull/31245#issuecomment-773931658 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39504/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31470: [SPARK-34354][SQL] Fix failure when apply CostBasedJoinReorder on self-join

AmplabJenkins removed a comment on pull request #31470: URL: https://github.com/apache/spark/pull/31470#issuecomment-773222379 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] yaooqinn commented on a change in pull request #31471: [SPARK-34355][SQL] Add log and time cost for commit job

yaooqinn commented on a change in pull request #31471: URL: https://github.com/apache/spark/pull/31471#discussion_r570211414 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/FileFormatWriter.scala ## @@ -217,8 +217,12 @@ object FileFormatWriter

[GitHub] [spark] SparkQA removed a comment on pull request #31455: [SPARK-34342][SQL] Format DateLiteral and TimestampLiteral toString

SparkQA removed a comment on pull request #31455: URL: https://github.com/apache/spark/pull/31455#issuecomment-773307449 **[Test build #134875 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134875/testReport)** for PR 31455 at commit

[GitHub] [spark] AmplabJenkins commented on pull request #31468: [SPARK-34353][SQL] CollectLimitExec avoid shuffle if input rdd has single partition

AmplabJenkins commented on pull request #31468: URL: https://github.com/apache/spark/pull/31468#issuecomment-773222393 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] wzhfy edited a comment on pull request #30965: [SPARK-33935][SQL] Fix CBO cost function

wzhfy edited a comment on pull request #30965: URL: https://github.com/apache/spark/pull/30965#issuecomment-773864841 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #31348: [SPARK-34245][CORE] Ensure Master removes executors that failed to send finished state

SparkQA commented on pull request #31348: URL: https://github.com/apache/spark/pull/31348#issuecomment-773934403 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39508/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31245: [SPARK-34157][SQL] Unify output of SHOW TABLES and pass output attributes properly

AmplabJenkins removed a comment on pull request #31245: URL: https://github.com/apache/spark/pull/31245#issuecomment-773931658 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39504/

[GitHub] [spark] AmplabJenkins commented on pull request #31472: [SPARK-34356][ML] OVR transform fix potential column conflict

AmplabJenkins commented on pull request #31472: URL: https://github.com/apache/spark/pull/31472#issuecomment-773287510 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] github-actions[bot] closed pull request #30019: [SPARK-33135][CORE] Use listLocatedStatus from FileSystem implementations

github-actions[bot] closed pull request #30019: URL: https://github.com/apache/spark/pull/30019 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

[GitHub] [spark] attilapiros commented on a change in pull request #31133: [SPARK-26836][SQL] Supporting Avro schema evolution for partitioned Hive tables

attilapiros commented on a change in pull request #31133: URL: https://github.com/apache/spark/pull/31133#discussion_r570304183 ## File path: sql/hive/src/main/scala/org/apache/spark/sql/hive/TableReader.scala ## @@ -388,6 +394,9 @@ private[hive] object HiveTableUtil {

[GitHub] [spark] SparkQA commented on pull request #31245: [SPARK-34157][SQL] Unify output of SHOW TABLES and pass output attributes properly

SparkQA commented on pull request #31245: URL: https://github.com/apache/spark/pull/31245#issuecomment-773952816 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39513/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31471: [SPARK-34355][SQL] Add log and time cost for commit job

AmplabJenkins removed a comment on pull request #31471: URL: https://github.com/apache/spark/pull/31471#issuecomment-773255967 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] SparkQA commented on pull request #31488: [SPARK-34376][SQL] Support regexp as a SQL function

SparkQA commented on pull request #31488: URL: https://github.com/apache/spark/pull/31488#issuecomment-773937463 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39514/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31284: [SPARK-34167][SQL]Reading parquet with IntDecimal written as a LongDecimal blows up

AmplabJenkins removed a comment on pull request #31284: URL: https://github.com/apache/spark/pull/31284#issuecomment-773632692 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30650: [SPARK-24818][CORE] Support delay scheduling for barrier execution

AmplabJenkins removed a comment on pull request #30650: URL: https://github.com/apache/spark/pull/30650#issuecomment-773931656 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39505/

[GitHub] [spark] MaxGekk commented on pull request #31475: [SPARK-34360][SQL] Support table truncation by v2 Table Catalogs

MaxGekk commented on pull request #31475: URL: https://github.com/apache/spark/pull/31475#issuecomment-773557994 @cloud-fan @HyukjinKwon Could you review this, please. This is an automated message from the Apache Git

[GitHub] [spark] SparkQA commented on pull request #31488: [SPARK-34376][SQL] Support regexp as a SQL function

SparkQA commented on pull request #31488: URL: https://github.com/apache/spark/pull/31488#issuecomment-773952200 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39514/

[GitHub] [spark] AngersZhuuuu commented on a change in pull request #31204: [SPARK-26399][WEBUI][CORE] Add new stage-level REST APIs and parameters

AngersZh commented on a change in pull request #31204: URL: https://github.com/apache/spark/pull/31204#discussion_r570724448 ## File path: core/src/main/scala/org/apache/spark/status/AppStatusStore.scala ## @@ -138,14 +211,63 @@ private[spark] class AppStatusStore( }

[GitHub] [spark] AmplabJenkins commented on pull request #31483: [SPARK-33434][PYTHON][DOCS] Added RuntimeConfig to PySpark docs

AmplabJenkins commented on pull request #31483: URL: https://github.com/apache/spark/pull/31483#issuecomment-773839682 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins commented on pull request #31413: [SPARK-32985][SQL] Decouple bucket scan and bucket filter pruning for data source v1

AmplabJenkins commented on pull request #31413: URL: https://github.com/apache/spark/pull/31413#issuecomment-773111768 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] saikocat edited a comment on pull request #31473: [SPARK-34357][SQL] Map JDBC SQL TIME type to TimestampType with time portion fixed regardless of timezone

saikocat edited a comment on pull request #31473: URL: https://github.com/apache/spark/pull/31473#issuecomment-773280147 > @saikocat Have you confirmed all the integration tests like `PostgresIntegrationSuite` pass? > Jenkins and GA don't run them so we need to confirm by ourselves.

[GitHub] [spark] SparkQA commented on pull request #31445: [SPARK-34334][K8S] Correctly identify timed out pending pod requests as excess request

SparkQA commented on pull request #31445: URL: https://github.com/apache/spark/pull/31445#issuecomment-773404117 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

[GitHub] [spark] HyukjinKwon commented on pull request #31460: [SPARK-34346][CORE][SQL] io.file.buffer.size set by spark.buffer.size will override by loading hive-site.xml accidentally may cause perf

HyukjinKwon commented on pull request #31460: URL: https://github.com/apache/spark/pull/31460#issuecomment-773711647 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

[GitHub] [spark] SparkQA removed a comment on pull request #31448: [SPARK-28137][SQL] Data Type Formatting Functions: `to_number`.

SparkQA removed a comment on pull request #31448: URL: https://github.com/apache/spark/pull/31448#issuecomment-773036893 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #31471: [SPARK-34355][SQL] Add log and time cost for commit job

dongjoon-hyun commented on a change in pull request #31471: URL: https://github.com/apache/spark/pull/31471#discussion_r570646739 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/FileFormatWriter.scala ## @@ -217,8 +217,12 @@ object

[GitHub] [spark] SparkQA commented on pull request #31413: [SPARK-32985][SQL] Decouple bucket scan and bucket filter pruning for data source v1

SparkQA commented on pull request #31413: URL: https://github.com/apache/spark/pull/31413#issuecomment-773083192 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

[GitHub] [spark] SparkQA commented on pull request #31478: [SPARK-34371][SQL][TESTS] Run the datetime rebasing tests for Parquet datasource v1 and v2

SparkQA commented on pull request #31478: URL: https://github.com/apache/spark/pull/31478#issuecomment-773661391 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

[GitHub] [spark] HeartSaVioR edited a comment on pull request #31464: [SPARK-34339][CORE][SQL] Expose the number of total paths in Utils.buildLocationMetadata()

HeartSaVioR edited a comment on pull request #31464: URL: https://github.com/apache/spark/pull/31464#issuecomment-773072398 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] c21 commented on pull request #29625: [SPARK-24528][SQL] Add support to read multiple sorted bucket files for data source v1

c21 commented on pull request #29625: URL: https://github.com/apache/spark/pull/29625#issuecomment-773571433 @rahij - yes I am. Will raise a PR soon, thanks. This is an automated message from the Apache Git Service. To

[GitHub] [spark] SparkQA removed a comment on pull request #31394: [SPARK-34291][ML] LSH hashDistance optimization

SparkQA removed a comment on pull request #31394: URL: https://github.com/apache/spark/pull/31394#issuecomment-773118982 **[Test build #134862 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134862/testReport)** for PR 31394 at commit

[GitHub] [spark] AmplabJenkins commented on pull request #31456: [SPARK-34343][SQL][TESTS] Add missing test for some non-array types in PostgreSQL

AmplabJenkins commented on pull request #31456: URL: https://github.com/apache/spark/pull/31456#issuecomment-773603415 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] maropu commented on pull request #31463: [PYTHON][MINOR] Fix docstring of join

maropu commented on pull request #31463: URL: https://github.com/apache/spark/pull/31463#issuecomment-773700214 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

[GitHub] [spark] cloud-fan edited a comment on pull request #31474: [SPARK-34359][SQL] add a legacy config to restore the output schema of SHOW DATABASES

cloud-fan edited a comment on pull request #31474: URL: https://github.com/apache/spark/pull/31474#issuecomment-773350647 cc @imback82 @beliefer @AngersZh @HyukjinKwon This is an automated message from the Apache Git

[GitHub] [spark] c21 commented on a change in pull request #31413: [SPARK-32985][SQL] Decouple bucket scan and bucket filter pruning for data source v1

c21 commented on a change in pull request #31413: URL: https://github.com/apache/spark/pull/31413#discussion_r570716893 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/DataSourceScanExec.scala ## @@ -591,20 +590,41 @@ case class FileSourceScanExec(

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31485: [SPARK-SQL][34137] Update suquery's stats when build LogicalPlan's stats

AmplabJenkins removed a comment on pull request #31485: URL: https://github.com/apache/spark/pull/31485#issuecomment-773931655 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39506/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31481: [SQL][MINOR][TEST][3.1] Re-enable some DS v2 char/varchar test

AmplabJenkins removed a comment on pull request #31481: URL: https://github.com/apache/spark/pull/31481#issuecomment-773931660 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134909/

[GitHub] [spark] AmplabJenkins commented on pull request #31487: [SPARK-34375][CORE][K8S][TEST] Replaces 'Mockito.initMocks' with 'Mockito.openMocks'

AmplabJenkins commented on pull request #31487: URL: https://github.com/apache/spark/pull/31487#issuecomment-773932866 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39512/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30957: [SPARK-31937][SQL] Support processing ArrayType/MapType/StructType data using no-serde mode script transform

AmplabJenkins removed a comment on pull request #30957: URL: https://github.com/apache/spark/pull/30957#issuecomment-773931654 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134912/

[GitHub] [spark] SparkQA commented on pull request #29087: [SPARK-28227][SQL] Support projection, aggregate/window functions, and lateral view in the TRANSFORM clause

SparkQA commented on pull request #29087: URL: https://github.com/apache/spark/pull/29087#issuecomment-773934846 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39511/

[GitHub] [spark] AmplabJenkins commented on pull request #31482: [SPARK-34346][CORE][SQL][3.1] io.file.buffer.size set by spark.buffer.size will override by loading hive-site.xml accidentally may caus

AmplabJenkins commented on pull request #31482: URL: https://github.com/apache/spark/pull/31482#issuecomment-773839238 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] attilapiros commented on pull request #31445: [SPARK-34334][K8S] Correctly identify timed out pending pod requests as excess request

attilapiros commented on pull request #31445: URL: https://github.com/apache/spark/pull/31445#issuecomment-773596593 jenkins retest this please This is an automated message from the Apache Git Service. To respond to the

[GitHub] [spark] SparkQA removed a comment on pull request #31477: [SPARK-34369][SQL][WEBUI] Track number of pairs processed out of Join.

SparkQA removed a comment on pull request #31477: URL: https://github.com/apache/spark/pull/31477#issuecomment-773692087 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA removed a comment on pull request #31482: [SPARK-34346][CORE][SQL] io.file.buffer.size set by spark.buffer.size will override by loading hive-site.xml accidentally may cause p

SparkQA removed a comment on pull request #31482: URL: https://github.com/apache/spark/pull/31482#issuecomment-773819012 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #31487: [SPARK-34375][CORE][K8S][TEST] Replaces 'Mockito.initMocks' with 'Mockito.openMocks'

SparkQA commented on pull request #31487: URL: https://github.com/apache/spark/pull/31487#issuecomment-773902011 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

[GitHub] [spark] AmplabJenkins commented on pull request #31467: [SPARK-33212][FOLLOW-UP][BUILD] Uses provided properties for Hadoop client dependencies in root pom

AmplabJenkins commented on pull request #31467: URL: https://github.com/apache/spark/pull/31467#issuecomment-773111772 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins commented on pull request #29185: [SPARK-32384][CORE] repartitionAndSortWithinPartitions avoid shuffle with same partitioner

AmplabJenkins commented on pull request #29185: URL: https://github.com/apache/spark/pull/29185#issuecomment-773222385 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31413: [SPARK-32985][SQL] Decouple bucket scan and bucket filter pruning for data source v1

AmplabJenkins removed a comment on pull request #31413: URL: https://github.com/apache/spark/pull/31413#issuecomment-773111768 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] HyukjinKwon closed pull request #31467: [SPARK-33212][FOLLOW-UP][BUILD] Uses provided properties for Hadoop client dependencies in root pom

HyukjinKwon closed pull request #31467: URL: https://github.com/apache/spark/pull/31467 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [spark] SparkQA removed a comment on pull request #31384: [SPARK-31816][SQL][DOCS] Added high level description about JDBC connection providers for users/developers

SparkQA removed a comment on pull request #31384: URL: https://github.com/apache/spark/pull/31384#issuecomment-773389086 **[Test build #134880 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134880/testReport)** for PR 31384 at commit

[GitHub] [spark] SparkQA removed a comment on pull request #31284: [SPARK-34167][SQL]Reading parquet with IntDecimal written as a LongDecimal blows up

SparkQA removed a comment on pull request #31284: URL: https://github.com/apache/spark/pull/31284#issuecomment-773581777 **[Test build #134888 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134888/testReport)** for PR 31284 at commit

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #31133: [SPARK-26836][SQL] Supporting Avro schema evolution for partitioned Hive tables with "avro.schema.literal"

dongjoon-hyun commented on a change in pull request #31133: URL: https://github.com/apache/spark/pull/31133#discussion_r570554343 ## File path: sql/hive/src/main/scala/org/apache/spark/sql/hive/TableReader.scala ## @@ -388,6 +393,7 @@ private[hive] object HiveTableUtil {

[GitHub] [spark] maropu closed pull request #31456: [SPARK-34343][SQL][TESTS] Add missing test for some non-array types in PostgreSQL

maropu closed pull request #31456: URL: https://github.com/apache/spark/pull/31456 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [spark] wzhfy commented on pull request #30965: [SPARK-33935][SQL] Fix CBO cost function

wzhfy commented on pull request #30965: URL: https://github.com/apache/spark/pull/30965#issuecomment-773864841 @tanelk Hi, sorry to see this so late. The reason to use a relative value for rowCount and size, is to normalize them in a similar scale while comparing cost. Otherwise,

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31466: [SPARK-34352][SQL] Improve SQLQueryTestSuite so as could run on windows system

AmplabJenkins removed a comment on pull request #31466: URL: https://github.com/apache/spark/pull/31466#issuecomment-773146059 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] yaooqinn commented on pull request #31488: [SPARK-34376][SQL] Support regexp as a SQL function

yaooqinn commented on pull request #31488: URL: https://github.com/apache/spark/pull/31488#issuecomment-773910330 cc @cloud-fan @maropu thanks for checking This is an automated message from the Apache Git Service. To respond

[GitHub] [spark] AmplabJenkins commented on pull request #31469: [MINOR][ML] Param Validation should throw IllegalArgumentException

AmplabJenkins commented on pull request #31469: URL: https://github.com/apache/spark/pull/31469#issuecomment-773189276 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] cloud-fan commented on a change in pull request #31440: [SPARK-34331][SQL] Speed up DS v2 metadata col resolution

cloud-fan commented on a change in pull request #31440: URL: https://github.com/apache/spark/pull/31440#discussion_r570797933 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala ## @@ -964,11 +964,37 @@ class Analyzer(override val

[GitHub] [spark] yaooqinn commented on pull request #31482: [SPARK-34346][CORE][SQL] io.file.buffer.size set by spark.buffer.size will override by loading hive-site.xml accidentally may cause perf reg

yaooqinn commented on pull request #31482: URL: https://github.com/apache/spark/pull/31482#issuecomment-773812166 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

[GitHub] [spark] rdblue commented on a change in pull request #31451: [WIP][SPARK-34338][SQL] Report metrics from Datasource v2 scan

rdblue commented on a change in pull request #31451: URL: https://github.com/apache/spark/pull/31451#discussion_r570437357 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/DataSourceV2ScanExecBase.scala ## @@ -32,8 +32,13 @@ import

[GitHub] [spark] cloud-fan commented on a change in pull request #31473: [SPARK-34357][SQL] Map JDBC SQL TIME type to TimestampType with time portion fixed regardless of timezone

cloud-fan commented on a change in pull request #31473: URL: https://github.com/apache/spark/pull/31473#discussion_r570204612 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/jdbc/JdbcUtils.scala ## @@ -470,6 +455,27 @@ object JdbcUtils extends

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31480: [SPARK-32384][CORE] repartitionAndSortWithinPartitions avoid shuffle with same partitioner

AmplabJenkins removed a comment on pull request #31480: URL: https://github.com/apache/spark/pull/31480#issuecomment-773813064 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] maropu commented on a change in pull request #30957: [SPARK-31937][SQL] Support processing ArrayType/MapType/StructType data using no-serde mode script transform

maropu commented on a change in pull request #30957: URL: https://github.com/apache/spark/pull/30957#discussion_r570718578 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/CatalystTypeConverters.scala ## @@ -174,6 +174,7 @@ object CatalystTypeConverters

[GitHub] [spark] srowen commented on a change in pull request #31472: [SPARK-34356][ML] OVR transform fix potential column conflict

srowen commented on a change in pull request #31472: URL: https://github.com/apache/spark/pull/31472#discussion_r570224680 ## File path: mllib/src/main/scala/org/apache/spark/ml/classification/OneVsRest.scala ## @@ -185,71 +185,56 @@ final class OneVsRestModel private[ml] (

[GitHub] [spark] ron8hu commented on a change in pull request #31204: [SPARK-26399][WEBUI][CORE] Add new stage-level REST APIs and parameters

ron8hu commented on a change in pull request #31204: URL: https://github.com/apache/spark/pull/31204#discussion_r570721165 ## File path: core/src/main/scala/org/apache/spark/status/AppStatusStore.scala ## @@ -138,14 +211,63 @@ private[spark] class AppStatusStore( } }

[GitHub] [spark] AmplabJenkins commented on pull request #31464: [SPARK-34339][CORE][SQL] Expose the number of total paths in Utils.buildLocationMetadata()

AmplabJenkins commented on pull request #31464: URL: https://github.com/apache/spark/pull/31464#issuecomment-773081377 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AngersZhuuuu commented on pull request #31179: [SPARK-34113][SQL] Use metric data update metadata statistic's size and rowCount

AngersZh commented on pull request #31179: URL: https://github.com/apache/spark/pull/31179#issuecomment-773113604 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31469: [MINOR][ML] Param Validation should throw IllegalArgumentException

AmplabJenkins removed a comment on pull request #31469: URL: https://github.com/apache/spark/pull/31469#issuecomment-773189276 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] SparkQA commented on pull request #31413: [SPARK-32985][SQL] Decouple bucket scan and bucket filter pruning for data source v1

SparkQA commented on pull request #31413: URL: https://github.com/apache/spark/pull/31413#issuecomment-773915494 **[Test build #134910 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134910/testReport)** for PR 31413 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28885: [SPARK-29375][SPARK-28940][SPARK-32041][SQL] Whole plan exchange and subquery reuse

AmplabJenkins removed a comment on pull request #28885: URL: https://github.com/apache/spark/pull/28885#issuecomment-773499553 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] cloud-fan commented on a change in pull request #31474: [SPARK-34359][SQL] add a legacy config to restore the output schema of SHOW DATABASES

cloud-fan commented on a change in pull request #31474: URL: https://github.com/apache/spark/pull/31474#discussion_r570267617 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/v2Commands.scala ## @@ -325,11 +325,13 @@ case class

[GitHub] [spark] mridulm commented on pull request #30650: [SPARK-24818][CORE] Support delay scheduling for barrier execution

mridulm commented on pull request #30650: URL: https://github.com/apache/spark/pull/30650#issuecomment-773512448 > Before this PR, we always recommend users disable delay scheduling by setting delay to 0 as a workaround in the error message. That will set it for all stages ... I was

[GitHub] [spark] SparkQA commented on pull request #31394: [SPARK-34291][ML] LSH hashDistance optimization

SparkQA commented on pull request #31394: URL: https://github.com/apache/spark/pull/31394#issuecomment-773118982 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

[GitHub] [spark] SparkQA removed a comment on pull request #31480: [SPARK-32384][CORE] repartitionAndSortWithinPartitions avoid shuffle with same partitioner

SparkQA removed a comment on pull request #31480: URL: https://github.com/apache/spark/pull/31480#issuecomment-773777161 **[Test build #134903 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134903/testReport)** for PR 31480 at commit

<    1   2   3   4   5   6   7   8   >