[GitHub] [spark] cloud-fan commented on a change in pull request #29107: [SPARK-32308][SQL] Move by-name resolution logic of unionByName from API code to analysis phase

cloud-fan commented on a change in pull request #29107: URL: https://github.com/apache/spark/pull/29107#discussion_r459212340 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala ## @@ -1387,10 +1388,11 @@ class Analyzer( i.c

[GitHub] [spark] cloud-fan commented on a change in pull request #29167: [SPARK-32374][SQL] Disallow setting properties when creating temporary views

cloud-fan commented on a change in pull request #29167: URL: https://github.com/apache/spark/pull/29167#discussion_r459210115 ## File path: sql/core/src/test/scala/org/apache/spark/sql/execution/SQLViewSuite.scala ## @@ -266,6 +266,16 @@ abstract class SQLViewSuite extends Que

[GitHub] [spark] cloud-fan commented on a change in pull request #29167: [SPARK-32374][SQL] Disallow setting properties when creating temporary views

cloud-fan commented on a change in pull request #29167: URL: https://github.com/apache/spark/pull/29167#discussion_r459209763 ## File path: sql/core/src/test/scala/org/apache/spark/sql/execution/SQLViewSuite.scala ## @@ -266,6 +266,16 @@ abstract class SQLViewSuite extends Que

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #29196: [SPARK-32398][TESTS][CORE][STREAMING][SQL][ML] Update to scalatest 3.2.0 for Scala 2.13.3+

dongjoon-hyun commented on a change in pull request #29196: URL: https://github.com/apache/spark/pull/29196#discussion_r459209060 ## File path: pom.xml ## @@ -892,7 +907,25 @@ org.scalatest scalatest_${scala.binary.version} -3.0.8 +3.2.

[GitHub] [spark] cloud-fan commented on a change in pull request #29104: [SPARK-32290][SQL] SingleColumn Null Aware Anti Join Optimize

cloud-fan commented on a change in pull request #29104: URL: https://github.com/apache/spark/pull/29104#discussion_r459208970 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/joins/BroadcastHashJoinExec.scala ## @@ -454,6 +478,28 @@ case class BroadcastHash

[GitHub] [spark] asfgit closed pull request #29032: [SPARK-32217] Plumb whether a worker would also be decommissioned along with executor

asfgit closed pull request #29032: URL: https://github.com/apache/spark/pull/29032 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to t

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #29191: [SPARK-32364][SQL][FOLLOWUP] Add toMap to return originalMap

dongjoon-hyun commented on a change in pull request #29191: URL: https://github.com/apache/spark/pull/29191#discussion_r459207445 ## File path: sql/catalyst/src/main/scala-2.12/org/apache/spark/sql/catalyst/util/CaseInsensitiveMap.scala ## @@ -52,6 +52,8 @@ class CaseInsensiti

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #29191: [SPARK-32364][SQL][FOLLOWUP] Add toMap to return originalMap

dongjoon-hyun commented on a change in pull request #29191: URL: https://github.com/apache/spark/pull/29191#discussion_r459207445 ## File path: sql/catalyst/src/main/scala-2.12/org/apache/spark/sql/catalyst/util/CaseInsensitiveMap.scala ## @@ -52,6 +52,8 @@ class CaseInsensiti

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #29191: [SPARK-32364][SQL][FOLLOWUP] Add toMap to return originalMap

dongjoon-hyun commented on a change in pull request #29191: URL: https://github.com/apache/spark/pull/29191#discussion_r459206761 ## File path: sql/core/src/main/scala/org/apache/spark/sql/DataFrameWriter.scala ## @@ -288,7 +288,7 @@ final class DataFrameWriter[T] private[sql](

[GitHub] [spark] SparkQA commented on pull request #29198: [SPARK-32401][SQL] Migrate function related commands to new resolution framework

SparkQA commented on pull request #29198: URL: https://github.com/apache/spark/pull/29198#issuecomment-662810236 **[Test build #126377 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/126377/testReport)** for PR 29198 at commit [`38756fc`](https://github.com

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29198: [SPARK-32401][SQL] Migrate function related commands to new resolution framework

AmplabJenkins removed a comment on pull request #29198: URL: https://github.com/apache/spark/pull/29198#issuecomment-662810535 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] LantaoJin commented on pull request #29062: [SPARK-32237][SQL] Resolve hint in CTE

LantaoJin commented on pull request #29062: URL: https://github.com/apache/spark/pull/29062#issuecomment-662810277 Thanks. I will file a backport PR. This is an automated message from the Apache Git Service. To respond to the

[GitHub] [spark] imback82 commented on a change in pull request #28840: [SPARK-31999][SQL] Add REFRESH FUNCTION command

imback82 commented on a change in pull request #28840: URL: https://github.com/apache/spark/pull/28840#discussion_r459205275 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/connector/catalog/LookupCatalog.scala ## @@ -155,4 +155,31 @@ private[sql] trait LookupCa

[GitHub] [spark] AmplabJenkins commented on pull request #29198: [SPARK-32401][SQL] Migrate function related commands to new resolution framework

AmplabJenkins commented on pull request #29198: URL: https://github.com/apache/spark/pull/29198#issuecomment-662810535 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] imback82 commented on a change in pull request #29198: [SPARK-32401][SQL] Migrate function related commands to new resolution framework

imback82 commented on a change in pull request #29198: URL: https://github.com/apache/spark/pull/29198#discussion_r459204665 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/parser/AstBuilder.scala ## @@ -3650,12 +3652,24 @@ class AstBuilder(conf: SQLCon

[GitHub] [spark] dongjoon-hyun commented on pull request #29191: [SPARK-32364][SQL][FOLLOWUP] Add toMap to return originalMap

dongjoon-hyun commented on pull request #29191: URL: https://github.com/apache/spark/pull/29191#issuecomment-662809395 Thank you, @cloud-fan and @maropu . This is an automated message from the Apache Git Service. To respond

[GitHub] [spark] holdenk commented on pull request #29032: [SPARK-32217] Plumb whether a worker would also be decommissioned along with executor

holdenk commented on pull request #29032: URL: https://github.com/apache/spark/pull/29032#issuecomment-662808939 Merged, thanks everyone. I don't know if we have a good principle around naming host v. worker with the deprecation.

[GitHub] [spark] imback82 opened a new pull request #29198: [SPARK-32401][SQL] Migrate function related commands to new resolution framework

imback82 opened a new pull request #29198: URL: https://github.com/apache/spark/pull/29198 ### What changes were proposed in this pull request? This PR proposes to migrate the following function related commands to the new resolution framework: - CREATE FUNCTION - DROP

[GitHub] [spark] cloud-fan commented on pull request #29032: [SPARK-32217] Plumb whether a worker would also be decommissioned along with executor

cloud-fan commented on pull request #29032: URL: https://github.com/apache/spark/pull/29032#issuecomment-662805220 I have a general question (not related to this PR): We have deprecated the usage of "multi-workers on one host", so host and worker should be the same thing. However, in the c

[GitHub] [spark] cloud-fan commented on a change in pull request #29166: [SPARK-32280][SPARK-32372][SQL] ResolveReferences.dedupRight should only rewrite attributes for ancestor nodes of the conflict

cloud-fan commented on a change in pull request #29166: URL: https://github.com/apache/spark/pull/29166#discussion_r459199272 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala ## @@ -1237,20 +1250,48 @@ class Analyzer( if (c

[GitHub] [spark] cloud-fan commented on a change in pull request #29166: [SPARK-32280][SPARK-32372][SQL] ResolveReferences.dedupRight should only rewrite attributes for ancestor nodes of the conflict

cloud-fan commented on a change in pull request #29166: URL: https://github.com/apache/spark/pull/29166#discussion_r459198562 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala ## @@ -1237,20 +1250,48 @@ class Analyzer( if (c

[GitHub] [spark] cloud-fan commented on a change in pull request #29166: [SPARK-32280][SPARK-32372][SQL] ResolveReferences.dedupRight should only rewrite attributes for ancestor nodes of the conflict

cloud-fan commented on a change in pull request #29166: URL: https://github.com/apache/spark/pull/29166#discussion_r459198329 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala ## @@ -1237,20 +1250,48 @@ class Analyzer( if (c

[GitHub] [spark] kevinjmh commented on pull request #28901: [SPARK-32064][SQL] Supporting create temporary table

kevinjmh commented on pull request #28901: URL: https://github.com/apache/spark/pull/28901#issuecomment-662802864 how about to use `CACHE TABLE` command to do that? This is an automated message from the Apache Git Service. To

[GitHub] [spark] cloud-fan commented on a change in pull request #29191: [SPARK-32364][SQL][FOLLOWUP] Add toMap to return originalMap

cloud-fan commented on a change in pull request #29191: URL: https://github.com/apache/spark/pull/29191#discussion_r459197779 ## File path: sql/core/src/main/scala/org/apache/spark/sql/DataFrameWriter.scala ## @@ -288,7 +288,7 @@ final class DataFrameWriter[T] private[sql](ds:

[GitHub] [spark] cloud-fan commented on a change in pull request #29191: [SPARK-32364][SQL][FOLLOWUP] Add toMap to return originalMap

cloud-fan commented on a change in pull request #29191: URL: https://github.com/apache/spark/pull/29191#discussion_r459197666 ## File path: sql/catalyst/src/main/scala-2.12/org/apache/spark/sql/catalyst/util/CaseInsensitiveMap.scala ## @@ -52,6 +52,8 @@ class CaseInsensitiveMa

[GitHub] [spark] cloud-fan commented on pull request #29191: [SPARK-32364][SQL][FOLLOWUP] Add toMap to return originalMap

cloud-fan commented on pull request #29191: URL: https://github.com/apache/spark/pull/29191#issuecomment-662802631 Let's define the behavior more clear: 1. DFReader/Writer options are case insensitive, so later option should override prior ones if the key equals case-insensitively. This

[GitHub] [spark] AngersZhuuuu commented on pull request #29085: [SPARK-32106][SQL]Implement SparkScriptTransformationExec in sql/core

AngersZh commented on pull request #29085: URL: https://github.com/apache/spark/pull/29085#issuecomment-662801978 > Looks almost okay now, so could you split this PR into pieces? I think its somewhat big fro reviews. For example; > > 1. More refactoring PR for `HiveScriptTransfo

[GitHub] [spark] AngersZhuuuu commented on a change in pull request #29085: [SPARK-32106][SQL]Implement SparkScriptTransformationExec in sql/core

AngersZh commented on a change in pull request #29085: URL: https://github.com/apache/spark/pull/29085#discussion_r459194715 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/BaseScriptTransformationExec.scala ## @@ -87,17 +181,72 @@ trait BaseScriptTran

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29085: [SPARK-32106][SQL]Implement SparkScriptTransformationExec in sql/core

AmplabJenkins removed a comment on pull request #29085: URL: https://github.com/apache/spark/pull/29085#issuecomment-662799097 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #29085: [SPARK-32106][SQL]Implement SparkScriptTransformationExec in sql/core

AmplabJenkins commented on pull request #29085: URL: https://github.com/apache/spark/pull/29085#issuecomment-662799097 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29197: [SPARK-32395][SQL] Commit output files after task attempt succeed in dynamic partition datawriter

AmplabJenkins removed a comment on pull request #29197: URL: https://github.com/apache/spark/pull/29197#issuecomment-662798617 Can one of the admins verify this patch? This is an automated message from the Apache Git Service.

[GitHub] [spark] AmplabJenkins commented on pull request #29197: [SPARK-32395][SQL] Commit output files after task attempt succeed in dynamic partition datawriter

AmplabJenkins commented on pull request #29197: URL: https://github.com/apache/spark/pull/29197#issuecomment-662798935 Can one of the admins verify this patch? This is an automated message from the Apache Git Service. To resp

[GitHub] [spark] SparkQA commented on pull request #29085: [SPARK-32106][SQL]Implement SparkScriptTransformationExec in sql/core

SparkQA commented on pull request #29085: URL: https://github.com/apache/spark/pull/29085#issuecomment-662798740 **[Test build #126376 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/126376/testReport)** for PR 29085 at commit [`03d3409`](https://github.com

[GitHub] [spark] AmplabJenkins commented on pull request #29197: [SPARK-32395][SQL] Commit output files after task attempt succeed in dynamic partition datawriter

AmplabJenkins commented on pull request #29197: URL: https://github.com/apache/spark/pull/29197#issuecomment-662798617 Can one of the admins verify this patch? This is an automated message from the Apache Git Service. To resp

[GitHub] [spark] cloud-fan commented on pull request #29062: [SPARK-32237][SQL] Resolve hint in CTE

cloud-fan commented on pull request #29062: URL: https://github.com/apache/spark/pull/29062#issuecomment-662798239 merged to master, thanks! @LantaoJin can you send a backport to 3.0? This is an automated message from

[GitHub] [spark] wankunde opened a new pull request #29197: [SPARK-32395][SQL] Commit output files after task attempt succeed in dynamic partition datawriter

wankunde opened a new pull request #29197: URL: https://github.com/apache/spark/pull/29197 # What changes were proposed in this pull request? Generally, distributed jobs have two stages of committing files: committing task's output files and committing job's output files. If one attempt

[GitHub] [spark] cloud-fan closed pull request #29062: [SPARK-32237][SQL] Resolve hint in CTE

cloud-fan closed pull request #29062: URL: https://github.com/apache/spark/pull/29062 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go t

[GitHub] [spark] cloud-fan commented on pull request #29194: [SPARK-30616][SQL][FOLLOW-UP] Use only config key name in the config doc.

cloud-fan commented on pull request #29194: URL: https://github.com/apache/spark/pull/29194#issuecomment-662797390 thanks, merging to master! This is an automated message from the Apache Git Service. To respond to the message

[GitHub] [spark] cloud-fan closed pull request #29194: [SPARK-30616][SQL][FOLLOW-UP] Use only config key name in the config doc.

cloud-fan closed pull request #29194: URL: https://github.com/apache/spark/pull/29194 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go t

[GitHub] [spark] AngersZhuuuu commented on a change in pull request #29085: [SPARK-32106][SQL]Implement SparkScriptTransformationExec in sql/core

AngersZh commented on a change in pull request #29085: URL: https://github.com/apache/spark/pull/29085#discussion_r459189448 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/BaseScriptTransformationExec.scala ## @@ -87,17 +181,72 @@ trait BaseScriptTran

[GitHub] [spark] AngersZhuuuu commented on a change in pull request #29085: [SPARK-32106][SQL]Implement SparkScriptTransformationExec in sql/core

AngersZh commented on a change in pull request #29085: URL: https://github.com/apache/spark/pull/29085#discussion_r459189448 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/BaseScriptTransformationExec.scala ## @@ -87,17 +181,72 @@ trait BaseScriptTran

[GitHub] [spark] AngersZhuuuu commented on a change in pull request #29085: [SPARK-32106][SQL]Implement SparkScriptTransformationExec in sql/core

AngersZh commented on a change in pull request #29085: URL: https://github.com/apache/spark/pull/29085#discussion_r459189586 ## File path: sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveInspectors.scala ## @@ -1063,6 +1063,9 @@ private[hive] trait HiveInspectors {

[GitHub] [spark] maropu commented on pull request #29085: [SPARK-32106][SQL]Implement SparkScriptTransformationExec in sql/core

maropu commented on pull request #29085: URL: https://github.com/apache/spark/pull/29085#issuecomment-662794540 Looks almost okay now, so could you split this PR into pieces? I think its somewhat big fro reviews. For example; 1. More refactoring PR for `HiveScriptTransformationExec` a

[GitHub] [spark] maropu commented on a change in pull request #29085: [SPARK-32106][SQL]Implement SparkScriptTransformationExec in sql/core

maropu commented on a change in pull request #29085: URL: https://github.com/apache/spark/pull/29085#discussion_r459188140 ## File path: sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveInspectors.scala ## @@ -1063,6 +1063,9 @@ private[hive] trait HiveInspectors {

[GitHub] [spark] maropu commented on a change in pull request #29085: [SPARK-32106][SQL]Implement SparkScriptTransformationExec in sql/core

maropu commented on a change in pull request #29085: URL: https://github.com/apache/spark/pull/29085#discussion_r459188225 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/BaseScriptTransformationExec.scala ## @@ -87,17 +181,72 @@ trait BaseScriptTransforma

[GitHub] [spark] Ngone51 commented on a change in pull request #29166: [SPARK-32280][SPARK-32372][SQL] ResolveReferences.dedupRight should only rewrite attributes for ancestor nodes of the conflict p

Ngone51 commented on a change in pull request #29166: URL: https://github.com/apache/spark/pull/29166#discussion_r459187090 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala ## @@ -1237,20 +1250,79 @@ class Analyzer( if (con

[GitHub] [spark] maropu commented on a change in pull request #29085: [SPARK-32106][SQL]Implement SparkScriptTransformationExec in sql/core

maropu commented on a change in pull request #29085: URL: https://github.com/apache/spark/pull/29085#discussion_r459186587 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/BaseScriptTransformationExec.scala ## @@ -87,17 +181,72 @@ trait BaseScriptTransforma

[GitHub] [spark] viirya commented on a change in pull request #29166: [SPARK-32280][SPARK-32372][SQL] ResolveReferences.dedupRight should only rewrite attributes for ancestor nodes of the conflict pl

viirya commented on a change in pull request #29166: URL: https://github.com/apache/spark/pull/29166#discussion_r459186345 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala ## @@ -1237,20 +1250,79 @@ class Analyzer( if (conf

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29166: [SPARK-32280][SPARK-32372][SQL] ResolveReferences.dedupRight should only rewrite attributes for ancestor nodes of the conflict

AmplabJenkins removed a comment on pull request #29166: URL: https://github.com/apache/spark/pull/29166#issuecomment-662791663 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #29166: [SPARK-32280][SPARK-32372][SQL] ResolveReferences.dedupRight should only rewrite attributes for ancestor nodes of the conflict plan

AmplabJenkins commented on pull request #29166: URL: https://github.com/apache/spark/pull/29166#issuecomment-662791663 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29192: [SPARK-32393][SQL] Fix postgres bpchar array support

AmplabJenkins removed a comment on pull request #29192: URL: https://github.com/apache/spark/pull/29192#issuecomment-662791661 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #29192: [SPARK-32393][SQL] Fix postgres bpchar array support

AmplabJenkins commented on pull request #29192: URL: https://github.com/apache/spark/pull/29192#issuecomment-662791661 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] SparkQA commented on pull request #29166: [SPARK-32280][SPARK-32372][SQL] ResolveReferences.dedupRight should only rewrite attributes for ancestor nodes of the conflict plan

SparkQA commented on pull request #29166: URL: https://github.com/apache/spark/pull/29166#issuecomment-662791379 **[Test build #126375 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/126375/testReport)** for PR 29166 at commit [`f73f186`](https://github.com

[GitHub] [spark] SparkQA commented on pull request #29192: [SPARK-32393][SQL] Fix postgres bpchar array support

SparkQA commented on pull request #29192: URL: https://github.com/apache/spark/pull/29192#issuecomment-662791349 **[Test build #126374 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/126374/testReport)** for PR 29192 at commit [`1000377`](https://github.com

[GitHub] [spark] Ngone51 commented on a change in pull request #29166: [SPARK-32280][SPARK-32372][SQL] ResolveReferences.dedupRight should only rewrite attributes for ancestor nodes of the conflict p

Ngone51 commented on a change in pull request #29166: URL: https://github.com/apache/spark/pull/29166#discussion_r459185548 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala ## @@ -1237,20 +1250,79 @@ class Analyzer( if (con

[GitHub] [spark] Ngone51 commented on a change in pull request #29166: [SPARK-32280][SPARK-32372][SQL] ResolveReferences.dedupRight should only rewrite attributes for ancestor nodes of the conflict p

Ngone51 commented on a change in pull request #29166: URL: https://github.com/apache/spark/pull/29166#discussion_r459185098 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala ## @@ -1237,20 +1250,79 @@ class Analyzer( if (con

[GitHub] [spark] HyukjinKwon commented on a change in pull request #29188: [SPARK-32179][SPARK-32188][PYTHON][DOCS] Replace and redesign the documentation base

HyukjinKwon commented on a change in pull request #29188: URL: https://github.com/apache/spark/pull/29188#discussion_r459184480 ## File path: dev/requirements.txt ## @@ -3,3 +3,4 @@ jira==1.0.3 PyGithub==1.26.0 Unidecode==0.04.19 sphinx +pydata_sphinx_theme Review comment:

[GitHub] [spark] maropu edited a comment on pull request #29192: [SPARK-32393][SQL] Fix postgres bpchar array support

maropu edited a comment on pull request #29192: URL: https://github.com/apache/spark/pull/29192#issuecomment-662779093 `bpchar` is internally used in postgresql [1][2], but users can use it in DDL; - [1] https://www.postgresql.org/docs/current/datatype-character.html - [2] https://w

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29192: [SPARK-32393][SQL] Fix postgres bpchar array support

AmplabJenkins removed a comment on pull request #29192: URL: https://github.com/apache/spark/pull/29192#issuecomment-662557759 Can one of the admins verify this patch? This is an automated message from the Apache Git Service.

[GitHub] [spark] holdenk commented on pull request #29032: [SPARK-32217] Plumb whether a worker would also be decommissioned along with executor

holdenk commented on pull request #29032: URL: https://github.com/apache/spark/pull/29032#issuecomment-662790280 Cool I’ll try merge this after dinner, thanks everyone for working on and reviewing this 👍 This is an automated

[GitHub] [spark] maropu commented on pull request #29192: [SPARK-32393][SQL] Fix postgres bpchar array support

maropu commented on pull request #29192: URL: https://github.com/apache/spark/pull/29192#issuecomment-662790228 ok to test This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29085: [SPARK-32106][SQL]Implement SparkScriptTransformationExec in sql/core

AmplabJenkins removed a comment on pull request #29085: URL: https://github.com/apache/spark/pull/29085#issuecomment-662790096 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #29085: [SPARK-32106][SQL]Implement SparkScriptTransformationExec in sql/core

AmplabJenkins commented on pull request #29085: URL: https://github.com/apache/spark/pull/29085#issuecomment-662790096 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] HyukjinKwon commented on a change in pull request #29188: [SPARK-32179][SPARK-32188][PYTHON][DOCS] Replace and redesign the documentation base

HyukjinKwon commented on a change in pull request #29188: URL: https://github.com/apache/spark/pull/29188#discussion_r459184480 ## File path: dev/requirements.txt ## @@ -3,3 +3,4 @@ jira==1.0.3 PyGithub==1.26.0 Unidecode==0.04.19 sphinx +pydata_sphinx_theme Review comment:

[GitHub] [spark] SparkQA commented on pull request #29085: [SPARK-32106][SQL]Implement SparkScriptTransformationExec in sql/core

SparkQA commented on pull request #29085: URL: https://github.com/apache/spark/pull/29085#issuecomment-662789794 **[Test build #126373 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/126373/testReport)** for PR 29085 at commit [`7f3cff8`](https://github.com

[GitHub] [spark] agrawaldevesh commented on pull request #29032: [SPARK-32217] Plumb whether a worker would also be decommissioned along with executor

agrawaldevesh commented on pull request #29032: URL: https://github.com/apache/spark/pull/29032#issuecomment-662789786 @holdenk... this is ready to go in. Thanks ! This is an automated message from the Apache Git Service. To

[GitHub] [spark] HyukjinKwon commented on pull request #29188: [SPARK-32179][SPARK-32188][PYTHON][DOCS] Replace and redesign the documentation base

HyukjinKwon commented on pull request #29188: URL: https://github.com/apache/spark/pull/29188#issuecomment-662789431 > I wonder if there is some non-hacky way to organize functions into logical groups, similarly to what ScalaDoc does. I tried hard but looked difficult to do. I will t

[GitHub] [spark] AmplabJenkins commented on pull request #29085: [SPARK-32106][SQL]Implement SparkScriptTransformationExec in sql/core

AmplabJenkins commented on pull request #29085: URL: https://github.com/apache/spark/pull/29085#issuecomment-662788634 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29085: [SPARK-32106][SQL]Implement SparkScriptTransformationExec in sql/core

AmplabJenkins removed a comment on pull request #29085: URL: https://github.com/apache/spark/pull/29085#issuecomment-662788634 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AngersZhuuuu commented on a change in pull request #29085: [SPARK-32106][SQL]Implement SparkScriptTransformationExec in sql/core

AngersZh commented on a change in pull request #29085: URL: https://github.com/apache/spark/pull/29085#discussion_r459182797 ## File path: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/parser/PlanParserSuite.scala ## @@ -1031,4 +1031,96 @@ class PlanParserSuite

[GitHub] [spark] SparkQA commented on pull request #29085: [SPARK-32106][SQL]Implement SparkScriptTransformationExec in sql/core

SparkQA commented on pull request #29085: URL: https://github.com/apache/spark/pull/29085#issuecomment-662788394 **[Test build #126372 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/126372/testReport)** for PR 29085 at commit [`be80c27`](https://github.com

[GitHub] [spark] AngersZhuuuu commented on pull request #29085: [SPARK-32106][SQL]Implement SparkScriptTransformationExec in sql/core

AngersZh commented on pull request #29085: URL: https://github.com/apache/spark/pull/29085#issuecomment-662787840 All done This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AngersZhuuuu commented on a change in pull request #29085: [SPARK-32106][SQL]Implement SparkScriptTransformationExec in sql/core

AngersZh commented on a change in pull request #29085: URL: https://github.com/apache/spark/pull/29085#discussion_r459182038 ## File path: sql/hive/src/main/scala/org/apache/spark/sql/hive/execution/HiveScriptTransformationExec.scala ## @@ -54,87 +52,110 @@ case class Hive

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29194: [SPARK-30616][SQL][FOLLOW-UP] Use only config key name in the config doc.

AmplabJenkins removed a comment on pull request #29194: URL: https://github.com/apache/spark/pull/29194#issuecomment-662787285 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #29194: [SPARK-30616][SQL][FOLLOW-UP] Use only config key name in the config doc.

AmplabJenkins commented on pull request #29194: URL: https://github.com/apache/spark/pull/29194#issuecomment-662787285 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] SparkQA removed a comment on pull request #29194: [SPARK-30616][SQL][FOLLOW-UP] Use only config key name in the config doc.

SparkQA removed a comment on pull request #29194: URL: https://github.com/apache/spark/pull/29194#issuecomment-662702681 **[Test build #126361 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/126361/testReport)** for PR 29194 at commit [`f9b8691`](https://gi

[GitHub] [spark] SparkQA commented on pull request #29194: [SPARK-30616][SQL][FOLLOW-UP] Use only config key name in the config doc.

SparkQA commented on pull request #29194: URL: https://github.com/apache/spark/pull/29194#issuecomment-662786999 **[Test build #126361 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/126361/testReport)** for PR 29194 at commit [`f9b8691`](https://github.co

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29196: [SPARK-32398][TESTS][CORE][STREAMING][SQL][ML] Update to scalatest 3.2.0 for Scala 2.13.3+

AmplabJenkins removed a comment on pull request #29196: URL: https://github.com/apache/spark/pull/29196#issuecomment-662785780 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #29196: [SPARK-32398][TESTS][CORE][STREAMING][SQL][ML] Update to scalatest 3.2.0 for Scala 2.13.3+

AmplabJenkins commented on pull request #29196: URL: https://github.com/apache/spark/pull/29196#issuecomment-662785780 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] Ngone51 commented on a change in pull request #29166: [SPARK-32280][SPARK-32372][SQL] ResolveReferences.dedupRight should only rewrite attributes for ancestor nodes of the conflict p

Ngone51 commented on a change in pull request #29166: URL: https://github.com/apache/spark/pull/29166#discussion_r459179732 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala ## @@ -1237,20 +1250,79 @@ class Analyzer( if (con

[GitHub] [spark] SparkQA commented on pull request #29196: [SPARK-32398][TESTS][CORE][STREAMING][SQL][ML] Update to scalatest 3.2.0 for Scala 2.13.3+

SparkQA commented on pull request #29196: URL: https://github.com/apache/spark/pull/29196#issuecomment-662785553 **[Test build #126371 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/126371/testReport)** for PR 29196 at commit [`9c12088`](https://github.com

[GitHub] [spark] AngersZhuuuu commented on a change in pull request #29085: [SPARK-32106][SQL]Implement SparkScriptTransformationExec in sql/core

AngersZh commented on a change in pull request #29085: URL: https://github.com/apache/spark/pull/29085#discussion_r459178460 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/SparkStrategies.scala ## @@ -532,6 +532,21 @@ abstract class SparkStrategies ex

[GitHub] [spark] maropu edited a comment on pull request #29192: [SPARK-32393][SQL] Fix postgres bpchar array support

maropu edited a comment on pull request #29192: URL: https://github.com/apache/spark/pull/29192#issuecomment-662784282 btw, thanks for your first contribution, @kujon ! This is an automated message from the Apache Git Service

[GitHub] [spark] maropu commented on pull request #29192: [SPARK-32393][SQL] Fix postgres bpchar array support

maropu commented on pull request #29192: URL: https://github.com/apache/spark/pull/29192#issuecomment-662784282 btw, thanks for your contribution, @kujon ! This is an automated message from the Apache Git Service. To respond

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29159: [SPARK-32310][ML][PySpark][3.0] ML params default value parity

AmplabJenkins removed a comment on pull request #29159: URL: https://github.com/apache/spark/pull/29159#issuecomment-662782378 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #29159: [SPARK-32310][ML][PySpark][3.0] ML params default value parity

AmplabJenkins commented on pull request #29159: URL: https://github.com/apache/spark/pull/29159#issuecomment-662782378 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] SparkQA removed a comment on pull request #29159: [SPARK-32310][ML][PySpark][3.0] ML params default value parity

SparkQA removed a comment on pull request #29159: URL: https://github.com/apache/spark/pull/29159#issuecomment-662767092 **[Test build #126367 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/126367/testReport)** for PR 29159 at commit [`52dd9b4`](https://gi

[GitHub] [spark] SparkQA commented on pull request #29159: [SPARK-32310][ML][PySpark][3.0] ML params default value parity

SparkQA commented on pull request #29159: URL: https://github.com/apache/spark/pull/29159#issuecomment-662782139 **[Test build #126367 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/126367/testReport)** for PR 29159 at commit [`52dd9b4`](https://github.co

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29193: [SPARK-32003][CORE][3.0] When external shuffle service is used, unregister outputs for executor on fetch failure after executor

AmplabJenkins removed a comment on pull request #29193: URL: https://github.com/apache/spark/pull/29193#issuecomment-662781417 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/126

[GitHub] [spark] AngersZhuuuu commented on a change in pull request #29085: [SPARK-32106][SQL]Implement SparkScriptTransformationExec in sql/core

AngersZh commented on a change in pull request #29085: URL: https://github.com/apache/spark/pull/29085#discussion_r459175410 ## File path: sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveInspectors.scala ## @@ -1063,6 +1063,9 @@ private[hive] trait HiveInspectors {

[GitHub] [spark] AmplabJenkins commented on pull request #29193: [SPARK-32003][CORE][3.0] When external shuffle service is used, unregister outputs for executor on fetch failure after executor is lost

AmplabJenkins commented on pull request #29193: URL: https://github.com/apache/spark/pull/29193#issuecomment-662781414 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29196: [SPARK-32398][TESTS][CORE][STREAMING][SQL][ML] Update to scalatest 3.2.0 for Scala 2.13.3+

AmplabJenkins removed a comment on pull request #29196: URL: https://github.com/apache/spark/pull/29196#issuecomment-662781211 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/126

[GitHub] [spark] SparkQA removed a comment on pull request #29193: [SPARK-32003][CORE][3.0] When external shuffle service is used, unregister outputs for executor on fetch failure after executor is lo

SparkQA removed a comment on pull request #29193: URL: https://github.com/apache/spark/pull/29193#issuecomment-662780801 **[Test build #126370 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/126370/testReport)** for PR 29193 at commit [`e54f221`](https://gi

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29193: [SPARK-32003][CORE][3.0] When external shuffle service is used, unregister outputs for executor on fetch failure after executor

AmplabJenkins removed a comment on pull request #29193: URL: https://github.com/apache/spark/pull/29193#issuecomment-662781414 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To r

[GitHub] [spark] SparkQA commented on pull request #29193: [SPARK-32003][CORE][3.0] When external shuffle service is used, unregister outputs for executor on fetch failure after executor is lost

SparkQA commented on pull request #29193: URL: https://github.com/apache/spark/pull/29193#issuecomment-662781403 **[Test build #126370 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/126370/testReport)** for PR 29193 at commit [`e54f221`](https://github.co

[GitHub] [spark] SparkQA removed a comment on pull request #29196: [SPARK-32398][TESTS][CORE][STREAMING][SQL][ML] Update to scalatest 3.2.0 for Scala 2.13.3+

SparkQA removed a comment on pull request #29196: URL: https://github.com/apache/spark/pull/29196#issuecomment-662780783 **[Test build #126369 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/126369/testReport)** for PR 29196 at commit [`5b20b65`](https://gi

[GitHub] [spark] AngersZhuuuu commented on a change in pull request #29085: [SPARK-32106][SQL]Implement SparkScriptTransformationExec in sql/core

AngersZh commented on a change in pull request #29085: URL: https://github.com/apache/spark/pull/29085#discussion_r459175002 ## File path: sql/core/src/test/scala/org/apache/spark/sql/execution/SparkScriptTransformationSuite.scala ## @@ -0,0 +1,103 @@ +/* + * Licensed to t

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29196: [SPARK-32398][TESTS][CORE][STREAMING][SQL][ML] Update to scalatest 3.2.0 for Scala 2.13.3+

AmplabJenkins removed a comment on pull request #29196: URL: https://github.com/apache/spark/pull/29196#issuecomment-662781049 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29193: [SPARK-32003][CORE][3.0] When external shuffle service is used, unregister outputs for executor on fetch failure after executor

AmplabJenkins removed a comment on pull request #29193: URL: https://github.com/apache/spark/pull/29193#issuecomment-662781036 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #29196: [SPARK-32398][TESTS][CORE][STREAMING][SQL][ML] Update to scalatest 3.2.0 for Scala 2.13.3+

AmplabJenkins commented on pull request #29196: URL: https://github.com/apache/spark/pull/29196#issuecomment-662781205 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

<    1   2   3   4   5   6   7   8   9   10   >