[GitHub] [spark] AmplabJenkins commented on pull request #29064: [SPARK-32272][SQL] Add SQL standard command SET TIME ZONE

AmplabJenkins commented on pull request #29064: URL: https://github.com/apache/spark/pull/29064#issuecomment-657584893 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29086: [SPARK-32292][SPARK-32252][INFRA] Run the relevant tests only in GitHub Actions

AmplabJenkins removed a comment on pull request #29086: URL: https://github.com/apache/spark/pull/29086#issuecomment-657584643 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29064: [SPARK-32272][SQL] Add SQL standard command SET TIME ZONE

AmplabJenkins removed a comment on pull request #29064: URL: https://github.com/apache/spark/pull/29064#issuecomment-657584893 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To

[GitHub] [spark] AmplabJenkins commented on pull request #29086: [SPARK-32292][SPARK-32252][INFRA] Run the relevant tests only in GitHub Actions

AmplabJenkins commented on pull request #29086: URL: https://github.com/apache/spark/pull/29086#issuecomment-657584643 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA removed a comment on pull request #29064: [SPARK-32272][SQL] Add SQL standard command SET TIME ZONE

SparkQA removed a comment on pull request #29064: URL: https://github.com/apache/spark/pull/29064#issuecomment-657423952 **[Test build #125759 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/125759/testReport)** for PR 29064 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29045: [SPARK-32234][SQL] Spark sql commands are failing on selecting the orc tables

AmplabJenkins removed a comment on pull request #29045: URL: https://github.com/apache/spark/pull/29045#issuecomment-657583612 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] SparkQA removed a comment on pull request #29086: [SPARK-32292][SPARK-32252][INFRA] Run the relevant tests only in GitHub Actions

SparkQA removed a comment on pull request #29086: URL: https://github.com/apache/spark/pull/29086#issuecomment-657493419 **[Test build #125769 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/125769/testReport)** for PR 29086 at commit

[GitHub] [spark] SparkQA commented on pull request #29064: [SPARK-32272][SQL] Add SQL standard command SET TIME ZONE

SparkQA commented on pull request #29064: URL: https://github.com/apache/spark/pull/29064#issuecomment-657584303 **[Test build #125759 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/125759/testReport)** for PR 29064 at commit

[GitHub] [spark] SparkQA commented on pull request #29086: [SPARK-32292][SPARK-32252][INFRA] Run the relevant tests only in GitHub Actions

SparkQA commented on pull request #29086: URL: https://github.com/apache/spark/pull/29086#issuecomment-657583690 **[Test build #125769 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/125769/testReport)** for PR 29086 at commit

[GitHub] [spark] SparkQA removed a comment on pull request #29045: [SPARK-32234][SQL] Spark sql commands are failing on selecting the orc tables

SparkQA removed a comment on pull request #29045: URL: https://github.com/apache/spark/pull/29045#issuecomment-657423953 **[Test build #125760 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/125760/testReport)** for PR 29045 at commit

[GitHub] [spark] AmplabJenkins commented on pull request #29045: [SPARK-32234][SQL] Spark sql commands are failing on selecting the orc tables

AmplabJenkins commented on pull request #29045: URL: https://github.com/apache/spark/pull/29045#issuecomment-657583612 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #29045: [SPARK-32234][SQL] Spark sql commands are failing on selecting the orc tables

SparkQA commented on pull request #29045: URL: https://github.com/apache/spark/pull/29045#issuecomment-657582876 **[Test build #125760 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/125760/testReport)** for PR 29045 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29084: [SPARK-32220][SQL][FOLLOW-UP]SHUFFLE_REPLICATE_NL Hint should not change Non-Cartesian Product join result

AmplabJenkins removed a comment on pull request #29084: URL: https://github.com/apache/spark/pull/29084#issuecomment-657582167 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #29084: [SPARK-32220][SQL][FOLLOW-UP]SHUFFLE_REPLICATE_NL Hint should not change Non-Cartesian Product join result

AmplabJenkins commented on pull request #29084: URL: https://github.com/apache/spark/pull/29084#issuecomment-657582167 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA removed a comment on pull request #29084: [SPARK-32220][SQL][FOLLOW-UP]SHUFFLE_REPLICATE_NL Hint should not change Non-Cartesian Product join result

SparkQA removed a comment on pull request #29084: URL: https://github.com/apache/spark/pull/29084#issuecomment-657423819 **[Test build #125757 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/125757/testReport)** for PR 29084 at commit

[GitHub] [spark] gatorsmile commented on a change in pull request #29070: [SPARK-32220][SQL][3.0]SHUFFLE_REPLICATE_NL Hint should not change Non-Cartesian Product join result

gatorsmile commented on a change in pull request #29070: URL: https://github.com/apache/spark/pull/29070#discussion_r453675359 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/SparkStrategies.scala ## @@ -286,7 +286,7 @@ abstract class SparkStrategies

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29075: [SPARK-32284][SQL] Avoid expanding too many CNF predicates in partition pruning

AmplabJenkins removed a comment on pull request #29075: URL: https://github.com/apache/spark/pull/29075#issuecomment-657581254 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] SparkQA commented on pull request #29084: [SPARK-32220][SQL][FOLLOW-UP]SHUFFLE_REPLICATE_NL Hint should not change Non-Cartesian Product join result

SparkQA commented on pull request #29084: URL: https://github.com/apache/spark/pull/29084#issuecomment-657581455 **[Test build #125757 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/125757/testReport)** for PR 29084 at commit

[GitHub] [spark] AmplabJenkins commented on pull request #29075: [SPARK-32284][SQL] Avoid expanding too many CNF predicates in partition pruning

AmplabJenkins commented on pull request #29075: URL: https://github.com/apache/spark/pull/29075#issuecomment-657581254 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA removed a comment on pull request #29075: [SPARK-32284][SQL] Avoid expanding too many CNF predicates in partition pruning

SparkQA removed a comment on pull request #29075: URL: https://github.com/apache/spark/pull/29075#issuecomment-657423860 **[Test build #125758 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/125758/testReport)** for PR 29075 at commit

[GitHub] [spark] SparkQA commented on pull request #29075: [SPARK-32284][SQL] Avoid expanding too many CNF predicates in partition pruning

SparkQA commented on pull request #29075: URL: https://github.com/apache/spark/pull/29075#issuecomment-657580499 **[Test build #125758 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/125758/testReport)** for PR 29075 at commit

[GitHub] [spark] tgravescs commented on pull request #28874: [SPARK-32036] Replace references to blacklist/whitelist language with more appropriate terminology, excluding the blacklisting feature.

tgravescs commented on pull request #28874: URL: https://github.com/apache/spark/pull/28874#issuecomment-657580234 @xkrogen can you up merge? This is an automated message from the Apache Git Service. To respond to the

[GitHub] [spark] Ngone51 commented on a change in pull request #29014: [SPARK-32199][SPARK-32198] Reduce job failures during decommissioning

Ngone51 commented on a change in pull request #29014: URL: https://github.com/apache/spark/pull/29014#discussion_r453650410 ## File path: core/src/main/scala/org/apache/spark/scheduler/DAGScheduler.scala ## @@ -1767,8 +1767,17 @@ private[spark] class DAGScheduler(

[GitHub] [spark] yaooqinn commented on a change in pull request #29064: [SPARK-32272][SQL] Add SQL standard command SET TIME ZONE

yaooqinn commented on a change in pull request #29064: URL: https://github.com/apache/spark/pull/29064#discussion_r453659953 ## File path: sql/core/src/test/resources/sql-tests/results/timezone.sql.out ## @@ -0,0 +1,121 @@ +-- Automatically generated by SQLQueryTestSuite +--

[GitHub] [spark] AngersZhuuuu commented on a change in pull request #29085: [SPARK-32106][SQL]Implement SparkScriptTransformationExec in sql/core

AngersZh commented on a change in pull request #29085: URL: https://github.com/apache/spark/pull/29085#discussion_r453659431 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/SparkScriptTransformationExec.scala ## @@ -0,0 +1,187 @@ +/* + * Licensed to

[GitHub] [spark] cloud-fan commented on a change in pull request #29064: [SPARK-32272][SQL] Add SQL standard command SET TIME ZONE

cloud-fan commented on a change in pull request #29064: URL: https://github.com/apache/spark/pull/29064#discussion_r453654237 ## File path: sql/core/src/test/resources/sql-tests/results/timezone.sql.out ## @@ -0,0 +1,121 @@ +-- Automatically generated by SQLQueryTestSuite +--

[GitHub] [spark] cloud-fan commented on a change in pull request #29064: [SPARK-32272][SQL] Add SQL standard command SET TIME ZONE

cloud-fan commented on a change in pull request #29064: URL: https://github.com/apache/spark/pull/29064#discussion_r453653986 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/SparkSqlParser.scala ## @@ -90,6 +93,51 @@ class SparkSqlAstBuilder(conf:

[GitHub] [spark] cloud-fan commented on pull request #29085: [SPARK-32106][SQL]Implement SparkScriptTransformationExec in sql/core

cloud-fan commented on pull request #29085: URL: https://github.com/apache/spark/pull/29085#issuecomment-657564284 At least we should define how to convert catalyst values to strings, right? `UnsafeArray.toString` just gives you meaningless binary string.

[GitHub] [spark] cloud-fan commented on a change in pull request #29085: [SPARK-32106][SQL]Implement SparkScriptTransformationExec in sql/core

cloud-fan commented on a change in pull request #29085: URL: https://github.com/apache/spark/pull/29085#discussion_r453650629 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/SparkScriptTransformationExec.scala ## @@ -0,0 +1,187 @@ +/* + * Licensed to the

[GitHub] [spark] SparkQA commented on pull request #28363: [SPARK-27188][SS] FileStreamSink: provide a new option to have retention on output files

SparkQA commented on pull request #28363: URL: https://github.com/apache/spark/pull/28363#issuecomment-657561120 **[Test build #125772 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/125772/testReport)** for PR 28363 at commit

[GitHub] [spark] cloud-fan commented on a change in pull request #28840: [SPARK-31999][SQL] Add REFRESH FUNCTION command

cloud-fan commented on a change in pull request #28840: URL: https://github.com/apache/spark/pull/28840#discussion_r453643288 ## File path: docs/sql-ref-syntax-aux-cache-refresh-function.md ## @@ -0,0 +1,60 @@ +--- +layout: global +title: REFRESH FUNCTION +displayTitle:

[GitHub] [spark] southernriver edited a comment on pull request #7324: [SPARK-8955][SQL] Replace a duplicated initialize() in HiveGenericUDTF with new one

southernriver edited a comment on pull request #7324: URL: https://github.com/apache/spark/pull/7324#issuecomment-657560083 ``` HiveGenericUDTF deprecated initialize(ObjectInspector[] argOIs) HiveGenericUDTF latest initialize(StructObjectInspector argOIs) ``` I face the same

[GitHub] [spark] southernriver commented on pull request #7324: [SPARK-8955][SQL] Replace a duplicated initialize() in HiveGenericUDTF with new one

southernriver commented on pull request #7324: URL: https://github.com/apache/spark/pull/7324#issuecomment-657560083 ``` HiveGenericUDTF deprecated initialize(ObjectInspector[] argOIs) HiveGenericUDTF latest initialize(StructObjectInspector argOIs) ``` I face the same

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29084: [SPARK-32220][SQL][FOLLOW-UP]SHUFFLE_REPLICATE_NL Hint should not change Non-Cartesian Product join result

AmplabJenkins removed a comment on pull request #29084: URL: https://github.com/apache/spark/pull/29084#issuecomment-657557837 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28363: [SPARK-27188][SS] FileStreamSink: provide a new option to have retention on output files

AmplabJenkins removed a comment on pull request #28363: URL: https://github.com/apache/spark/pull/28363#issuecomment-657557970 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29087: [SPARK-28227][SQL] Support TRANSFORM with aggregation

AmplabJenkins removed a comment on pull request #29087: URL: https://github.com/apache/spark/pull/29087#issuecomment-657557829 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #29087: [SPARK-28227][SQL] Support TRANSFORM with aggregation

AmplabJenkins commented on pull request #29087: URL: https://github.com/apache/spark/pull/29087#issuecomment-657557829 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins commented on pull request #28363: [SPARK-27188][SS] FileStreamSink: provide a new option to have retention on output files

AmplabJenkins commented on pull request #28363: URL: https://github.com/apache/spark/pull/28363#issuecomment-657557970 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins commented on pull request #29084: [SPARK-32220][SQL][FOLLOW-UP]SHUFFLE_REPLICATE_NL Hint should not change Non-Cartesian Product join result

AmplabJenkins commented on pull request #29084: URL: https://github.com/apache/spark/pull/29084#issuecomment-657557837 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] GuoPhilipse commented on a change in pull request #29056: [SPARK-31753][SQL][DOCS] Add missing keywords in the SQL docs

GuoPhilipse commented on a change in pull request #29056: URL: https://github.com/apache/spark/pull/29056#discussion_r453643327 ## File path: docs/sql-ref-syntax-qry-select.md ## @@ -159,3 +159,6 @@ SELECT [ hints , ... ] [ ALL | DISTINCT ] { named_expression [ , ... ] } *

[GitHub] [spark] HeartSaVioR commented on pull request #28363: [SPARK-27188][SS] FileStreamSink: provide a new option to have retention on output files

HeartSaVioR commented on pull request #28363: URL: https://github.com/apache/spark/pull/28363#issuecomment-657557338 retest this, please This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [spark] SparkQA commented on pull request #29087: [SPARK-28227][SQL] Support TRANSFORM with aggregation

SparkQA commented on pull request #29087: URL: https://github.com/apache/spark/pull/29087#issuecomment-657557064 **[Test build #125770 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/125770/testReport)** for PR 29087 at commit

[GitHub] [spark] SparkQA commented on pull request #29084: [SPARK-32220][SQL][FOLLOW-UP]SHUFFLE_REPLICATE_NL Hint should not change Non-Cartesian Product join result

SparkQA commented on pull request #29084: URL: https://github.com/apache/spark/pull/29084#issuecomment-657557130 **[Test build #125771 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/125771/testReport)** for PR 29084 at commit

[GitHub] [spark] AngersZhuuuu commented on pull request #29087: [SPARK-28227][SQL] Support TRANSFORM with aggregation

AngersZh commented on pull request #29087: URL: https://github.com/apache/spark/pull/29087#issuecomment-657556438 cc @alfozan @HyukjinKwon @wangyum @cloud-fan This is an automated message from the Apache Git Service.

[GitHub] [spark] AngersZhuuuu opened a new pull request #29087: [SPARK-28227][SQL] Support TRANSFORM with aggregation

AngersZh opened a new pull request #29087: URL: https://github.com/apache/spark/pull/29087 ### What changes were proposed in this pull request? For Spark SQL, it can't support sql like : ``` SELECT TRANSFORM ( d2, max(d1) as maxd1, cast(sum(d3) as string)) USING 'cat' AS

[GitHub] [spark] maropu commented on pull request #29084: [SPARK-32220][SQL][FOLLOW-UP]SHUFFLE_REPLICATE_NL Hint should not change Non-Cartesian Product join result

maropu commented on pull request #29084: URL: https://github.com/apache/spark/pull/29084#issuecomment-657555210 retest this please This is an automated message from the Apache Git Service. To respond to the message, please

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29084: [SPARK-32220][SQL][FOLLOW-UP]SHUFFLE_REPLICATE_NL Hint should not change Non-Cartesian Product join result

AmplabJenkins removed a comment on pull request #29084: URL: https://github.com/apache/spark/pull/29084#issuecomment-657554467 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29084: [SPARK-32220][SQL][FOLLOW-UP]SHUFFLE_REPLICATE_NL Hint should not change Non-Cartesian Product join result

AmplabJenkins removed a comment on pull request #29084: URL: https://github.com/apache/spark/pull/29084#issuecomment-657554456 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To

[GitHub] [spark] AmplabJenkins commented on pull request #29084: [SPARK-32220][SQL][FOLLOW-UP]SHUFFLE_REPLICATE_NL Hint should not change Non-Cartesian Product join result

AmplabJenkins commented on pull request #29084: URL: https://github.com/apache/spark/pull/29084#issuecomment-657554456 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA removed a comment on pull request #29084: [SPARK-32220][SQL][FOLLOW-UP]SHUFFLE_REPLICATE_NL Hint should not change Non-Cartesian Product join result

SparkQA removed a comment on pull request #29084: URL: https://github.com/apache/spark/pull/29084#issuecomment-657400399 **[Test build #125754 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/125754/testReport)** for PR 29084 at commit

[GitHub] [spark] AngersZhuuuu commented on pull request #29085: [SPARK-32106][SQL]Implement SparkScriptTransformationExec in sql/core

AngersZh commented on pull request #29085: URL: https://github.com/apache/spark/pull/29085#issuecomment-657553668 @alfozan Hi, alfozan, I know that in facebook using script transform a lot, in your case, do you will use script transform with serde?

[GitHub] [spark] SparkQA commented on pull request #29084: [SPARK-32220][SQL][FOLLOW-UP]SHUFFLE_REPLICATE_NL Hint should not change Non-Cartesian Product join result

SparkQA commented on pull request #29084: URL: https://github.com/apache/spark/pull/29084#issuecomment-657553770 **[Test build #125754 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/125754/testReport)** for PR 29084 at commit

[GitHub] [spark] yaooqinn commented on a change in pull request #29064: [SPARK-32272][SQL] Add and extend SQL standard command SET TIME ZONE

yaooqinn commented on a change in pull request #29064: URL: https://github.com/apache/spark/pull/29064#discussion_r453639230 ## File path: sql/core/src/test/resources/sql-tests/results/timezone.sql.out ## @@ -0,0 +1,121 @@ +-- Automatically generated by SQLQueryTestSuite +--

[GitHub] [spark] AngersZhuuuu commented on pull request #29085: [SPARK-32106][SQL]Implement SparkScriptTransformationExec in sql/core

AngersZh commented on pull request #29085: URL: https://github.com/apache/spark/pull/29085#issuecomment-657550973 > can you explain the serde part? How can we do script transformation in sql/core without the hive serde lib? In most case, we won't use script with serde, so we

[GitHub] [spark] Ngone51 commented on a change in pull request #29014: [SPARK-32199][SPARK-32198] Reduce job failures during decommissioning

Ngone51 commented on a change in pull request #29014: URL: https://github.com/apache/spark/pull/29014#discussion_r453635661 ## File path: core/src/main/scala/org/apache/spark/scheduler/ExecutorLossReason.scala ## @@ -54,10 +54,13 @@ private [spark] object LossReasonPending

[GitHub] [spark] Ngone51 commented on a change in pull request #29032: [SPARK-32217] Plumb whether a worker would also be decommissioned along with executor

Ngone51 commented on a change in pull request #29032: URL: https://github.com/apache/spark/pull/29032#discussion_r453633702 ## File path: core/src/main/scala/org/apache/spark/scheduler/TaskSchedulerImpl.scala ## @@ -136,6 +136,8 @@ private[spark] class TaskSchedulerImpl(

[GitHub] [spark] cloud-fan commented on pull request #29085: [SPARK-32106][SQL]Implement SparkScriptTransformationExec in sql/core

cloud-fan commented on pull request #29085: URL: https://github.com/apache/spark/pull/29085#issuecomment-657547338 can you explain the serde part? How can we do script transformation in sql/core without the hive serde lib?

[GitHub] [spark] cloud-fan commented on a change in pull request #29085: [SPARK-32106][SQL]Implement SparkScriptTransformationExec in sql/core

cloud-fan commented on a change in pull request #29085: URL: https://github.com/apache/spark/pull/29085#discussion_r453631470 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/SparkScriptTransformationExec.scala ## @@ -0,0 +1,187 @@ +/* + * Licensed to the

[GitHub] [spark] cloud-fan commented on a change in pull request #29085: [SPARK-32106][SQL]Implement SparkScriptTransformationExec in sql/core

cloud-fan commented on a change in pull request #29085: URL: https://github.com/apache/spark/pull/29085#discussion_r453631470 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/SparkScriptTransformationExec.scala ## @@ -0,0 +1,187 @@ +/* + * Licensed to the

[GitHub] [spark] revans2 commented on pull request #29067: [SPARK-32274][SQL] Make SQL cache serialization pluggable

revans2 commented on pull request #29067: URL: https://github.com/apache/spark/pull/29067#issuecomment-657544856 @maropu > You cannot use the data source V2 interface for your purpose? We want to produce a transparent replacement for `.cache`, `.persist` and the SQL `CACHE`

[GitHub] [spark] Ngone51 commented on a change in pull request #29032: [SPARK-32217] Plumb whether a worker would also be decommissioned along with executor

Ngone51 commented on a change in pull request #29032: URL: https://github.com/apache/spark/pull/29032#discussion_r453623144 ## File path: core/src/main/scala/org/apache/spark/scheduler/DecommissionInfo.scala ## @@ -0,0 +1,27 @@ +/* + * Licensed to the Apache Software

[GitHub] [spark] WinkerDu commented on pull request #29000: [SPARK-27194][SPARK-29302][SQL] Fix commit collision in dynamic partition overwrite mode

WinkerDu commented on pull request #29000: URL: https://github.com/apache/spark/pull/29000#issuecomment-657538307 retest this please @cloud-fan @dongjoon-hyun This is an automated message from the Apache Git Service. To

[GitHub] [spark] Ngone51 commented on a change in pull request #29032: [SPARK-32217] Plumb whether a worker would also be decommissioned along with executor

Ngone51 commented on a change in pull request #29032: URL: https://github.com/apache/spark/pull/29032#discussion_r453614859 ## File path: core/src/main/scala/org/apache/spark/scheduler/DecommissionInfo.scala ## @@ -0,0 +1,27 @@ +/* + * Licensed to the Apache Software

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28957: [SPARK-32138] Drop Python 2.7, 3.4 and 3.5

AmplabJenkins removed a comment on pull request #28957: URL: https://github.com/apache/spark/pull/28957#issuecomment-657530078 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28957: [SPARK-32138] Drop Python 2.7, 3.4 and 3.5

AmplabJenkins removed a comment on pull request #28957: URL: https://github.com/apache/spark/pull/28957#issuecomment-657530072 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To

[GitHub] [spark] AmplabJenkins commented on pull request #28957: [SPARK-32138] Drop Python 2.7, 3.4 and 3.5

AmplabJenkins commented on pull request #28957: URL: https://github.com/apache/spark/pull/28957#issuecomment-657530072 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA removed a comment on pull request #28957: [SPARK-32138] Drop Python 2.7, 3.4 and 3.5

SparkQA removed a comment on pull request #28957: URL: https://github.com/apache/spark/pull/28957#issuecomment-657459581 **[Test build #125765 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/125765/testReport)** for PR 28957 at commit

[GitHub] [spark] SparkQA commented on pull request #28957: [SPARK-32138] Drop Python 2.7, 3.4 and 3.5

SparkQA commented on pull request #28957: URL: https://github.com/apache/spark/pull/28957#issuecomment-657529596 **[Test build #125765 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/125765/testReport)** for PR 28957 at commit

[GitHub] [spark] Ngone51 commented on a change in pull request #29032: [SPARK-32217] Plumb whether a worker would also be decommissioned along with executor

Ngone51 commented on a change in pull request #29032: URL: https://github.com/apache/spark/pull/29032#discussion_r453605397 ## File path: core/src/main/scala/org/apache/spark/scheduler/cluster/CoarseGrainedSchedulerBackend.scala ## @@ -418,7 +418,8 @@ class

[GitHub] [spark] revans2 commented on a change in pull request #29067: [SPARK-32274][SQL] Make SQL cache serialization pluggable

revans2 commented on a change in pull request #29067: URL: https://github.com/apache/spark/pull/29067#discussion_r453602842 ## File path: sql/core/src/test/scala/org/apache/spark/sql/execution/columnar/InMemoryColumnarQuerySuite.scala ## @@ -474,12 +484,8 @@ class

[GitHub] [spark] zhengruifeng commented on pull request #29018: [SPARK-32202][ML][WIP] tree models auto infer compact integer type

zhengruifeng commented on pull request #29018: URL: https://github.com/apache/spark/pull/29018#issuecomment-657505193 I remove the specialization in methods and kryo registration, since they had ignorable impact on the performance. Then I retest on more config, and it seems that

[GitHub] [spark] HyukjinKwon commented on a change in pull request #29086: [SPARK-32292][SPARK-32252][INFRA] Run the relevant tests only in GitHub Actions

HyukjinKwon commented on a change in pull request #29086: URL: https://github.com/apache/spark/pull/29086#discussion_r453573370 ## File path: dev/run-tests.py ## @@ -589,43 +627,74 @@ def main(): # /home/jenkins/anaconda2/envs/py36/bin os.environ["PATH"] =

[GitHub] [spark] HyukjinKwon commented on a change in pull request #29086: [SPARK-32292][SPARK-32252][INFRA] Run the relevant tests only in GitHub Actions

HyukjinKwon commented on a change in pull request #29086: URL: https://github.com/apache/spark/pull/29086#discussion_r453572801 ## File path: dev/run-tests.py ## @@ -589,43 +627,74 @@ def main(): # /home/jenkins/anaconda2/envs/py36/bin os.environ["PATH"] =

[GitHub] [spark] AmplabJenkins commented on pull request #29086: [SPARK-32292][SPARK-32252][INFRA] Run the relevant tests only in GitHub Actions

AmplabJenkins commented on pull request #29086: URL: https://github.com/apache/spark/pull/29086#issuecomment-657493848 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29086: [SPARK-32292][SPARK-32252][INFRA] Run the relevant tests only in GitHub Actions

AmplabJenkins removed a comment on pull request #29086: URL: https://github.com/apache/spark/pull/29086#issuecomment-657493848 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] SparkQA commented on pull request #29086: [SPARK-32292][SPARK-32252][INFRA] Run the relevant tests only in GitHub Actions

SparkQA commented on pull request #29086: URL: https://github.com/apache/spark/pull/29086#issuecomment-657493419 **[Test build #125769 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/125769/testReport)** for PR 29086 at commit

[GitHub] [spark] HyukjinKwon opened a new pull request #29086: [SPARK-32292][SPARK-32252][INFRA] Run the relevant tests only in GitHub Actions

HyukjinKwon opened a new pull request #29086: URL: https://github.com/apache/spark/pull/29086 ### What changes were proposed in this pull request? This PR mainly proposes to run only relevant tests just like Jenkins PR builder does. Currently, GitHub Actions always run full tests

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28957: [SPARK-32138] Drop Python 2.7, 3.4 and 3.5

AmplabJenkins removed a comment on pull request #28957: URL: https://github.com/apache/spark/pull/28957#issuecomment-657486973 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28957: [SPARK-32138] Drop Python 2.7, 3.4 and 3.5

AmplabJenkins removed a comment on pull request #28957: URL: https://github.com/apache/spark/pull/28957#issuecomment-657486961 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To

[GitHub] [spark] AmplabJenkins commented on pull request #28957: [SPARK-32138] Drop Python 2.7, 3.4 and 3.5

AmplabJenkins commented on pull request #28957: URL: https://github.com/apache/spark/pull/28957#issuecomment-657486961 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AngersZhuuuu commented on pull request #29085: [SPARK-32106][SQL]Implement SparkScriptTransformationExec in sql/core

AngersZh commented on pull request #29085: URL: https://github.com/apache/spark/pull/29085#issuecomment-657487168 cc @cloud-fan @maropu @HyukjinKwon This is an automated message from the Apache Git Service. To respond

[GitHub] [spark] SparkQA removed a comment on pull request #28957: [SPARK-32138] Drop Python 2.7, 3.4 and 3.5

SparkQA removed a comment on pull request #28957: URL: https://github.com/apache/spark/pull/28957#issuecomment-657403743 **[Test build #125755 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/125755/testReport)** for PR 28957 at commit

[GitHub] [spark] peter-toth commented on a change in pull request #29053: [SPARK-32241][SQL] Remove empty children of union

peter-toth commented on a change in pull request #29053: URL: https://github.com/apache/spark/pull/29053#discussion_r453548369 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/PropagateEmptyRelation.scala ## @@ -50,8 +50,24 @@ object

[GitHub] [spark] SparkQA commented on pull request #28957: [SPARK-32138] Drop Python 2.7, 3.4 and 3.5

SparkQA commented on pull request #28957: URL: https://github.com/apache/spark/pull/28957#issuecomment-657486175 **[Test build #125755 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/125755/testReport)** for PR 28957 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29085: [SPARK-32106][SQL]Implement SparkScriptTransformationExec in sql/core

AmplabJenkins removed a comment on pull request #29085: URL: https://github.com/apache/spark/pull/29085#issuecomment-657480868 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #29085: [SPARK-32106][SQL]Implement SparkScriptTransformationExec in sql/core

AmplabJenkins commented on pull request #29085: URL: https://github.com/apache/spark/pull/29085#issuecomment-657480868 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins removed a comment on pull request #27735: [SPARK-30985][k8s] Support propagating SPARK_CONF_DIR files to driver and executor pods.

AmplabJenkins removed a comment on pull request #27735: URL: https://github.com/apache/spark/pull/27735#issuecomment-657471298 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] SparkQA commented on pull request #29085: [SPARK-32106][SQL]Implement SparkScriptTransformationExec in sql/core

SparkQA commented on pull request #29085: URL: https://github.com/apache/spark/pull/29085#issuecomment-657480292 **[Test build #125768 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/125768/testReport)** for PR 29085 at commit

[GitHub] [spark] AmplabJenkins commented on pull request #27735: [SPARK-30985][k8s] Support propagating SPARK_CONF_DIR files to driver and executor pods.

AmplabJenkins commented on pull request #27735: URL: https://github.com/apache/spark/pull/27735#issuecomment-657480218 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AngersZhuuuu opened a new pull request #29085: [SPARK-32106][SQL]Implement SparkScriptTransformationExec in sql/core

AngersZh opened a new pull request #29085: URL: https://github.com/apache/spark/pull/29085 ### What changes were proposed in this pull request? Implement script transformation in sql/core module * Renamed file `hive/execution/ScriptTransformationExec` to

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29053: [SPARK-32241][SQL] Remove empty children of union

AmplabJenkins removed a comment on pull request #29053: URL: https://github.com/apache/spark/pull/29053#issuecomment-657473159 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #29053: [SPARK-32241][SQL] Remove empty children of union

AmplabJenkins commented on pull request #29053: URL: https://github.com/apache/spark/pull/29053#issuecomment-657473159 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #29053: [SPARK-32241][SQL] Remove empty children of union

SparkQA commented on pull request #29053: URL: https://github.com/apache/spark/pull/29053#issuecomment-657472527 **[Test build #125767 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/125767/testReport)** for PR 29053 at commit

[GitHub] [spark] peter-toth commented on a change in pull request #29053: [SPARK-32241][SQL] Remove empty children of union

peter-toth commented on a change in pull request #29053: URL: https://github.com/apache/spark/pull/29053#discussion_r453548369 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/PropagateEmptyRelation.scala ## @@ -50,8 +50,24 @@ object

[GitHub] [spark] AmplabJenkins removed a comment on pull request #27735: [SPARK-30985][k8s] Support propagating SPARK_CONF_DIR files to driver and executor pods.

AmplabJenkins removed a comment on pull request #27735: URL: https://github.com/apache/spark/pull/27735#issuecomment-657471290 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To

[GitHub] [spark] AmplabJenkins commented on pull request #27735: [SPARK-30985][k8s] Support propagating SPARK_CONF_DIR files to driver and executor pods.

AmplabJenkins commented on pull request #27735: URL: https://github.com/apache/spark/pull/27735#issuecomment-657471290 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] cloud-fan commented on a change in pull request #29064: [SPARK-32272][SQL] Add and extend SQL standard command SET TIME ZONE

cloud-fan commented on a change in pull request #29064: URL: https://github.com/apache/spark/pull/29064#discussion_r453546490 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/SparkSqlParser.scala ## @@ -90,6 +93,51 @@ class SparkSqlAstBuilder(conf:

[GitHub] [spark] beliefer commented on a change in pull request #27428: [SPARK-30276][SQL] Support Filter expression allows simultaneous use of DISTINCT

beliefer commented on a change in pull request #27428: URL: https://github.com/apache/spark/pull/27428#discussion_r453544249 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/RewriteDistinctAggregates.scala ## @@ -102,23 +102,126 @@ import

[GitHub] [spark] SparkQA removed a comment on pull request #27735: [SPARK-30985][k8s] Support propagating SPARK_CONF_DIR files to driver and executor pods.

SparkQA removed a comment on pull request #27735: URL: https://github.com/apache/spark/pull/27735#issuecomment-657459685 **[Test build #125766 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/125766/testReport)** for PR 27735 at commit

[GitHub] [spark] AmplabJenkins commented on pull request #27735: [SPARK-30985][k8s] Support propagating SPARK_CONF_DIR files to driver and executor pods.

AmplabJenkins commented on pull request #27735: URL: https://github.com/apache/spark/pull/27735#issuecomment-657468027 This is an automated message from the Apache Git Service. To respond to the message, please log on to

<    1   2   3   4   5   6   7   >