[GitHub] [spark] AmplabJenkins removed a comment on pull request #29107: [SPARK-32308][SQL] Move by-name resolution logic of unionByName from API code to analysis phase

AmplabJenkins removed a comment on pull request #29107: URL: https://github.com/apache/spark/pull/29107#issuecomment-662831333 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29190: [do not review][testing master PR build][discard] comment change

AmplabJenkins removed a comment on pull request #29190: URL: https://github.com/apache/spark/pull/29190#issuecomment-662831354 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #29190: [do not review][testing master PR build][discard] comment change

AmplabJenkins commented on pull request #29190: URL: https://github.com/apache/spark/pull/29190#issuecomment-662831354 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins commented on pull request #29107: [SPARK-32308][SQL] Move by-name resolution logic of unionByName from API code to analysis phase

AmplabJenkins commented on pull request #29107: URL: https://github.com/apache/spark/pull/29107#issuecomment-662831333 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #29190: [do not review][testing master PR build][discard] comment change

SparkQA commented on pull request #29190: URL: https://github.com/apache/spark/pull/29190#issuecomment-662831076 **[Test build #126387 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/126387/testReport)** for PR 29190 at commit

[GitHub] [spark] cloud-fan commented on pull request #29032: [SPARK-32217] Plumb whether a worker would also be decommissioned along with executor

cloud-fan commented on pull request #29032: URL: https://github.com/apache/spark/pull/29032#issuecomment-662829893 I agree `host` is better as it's general to any deployment model. I think we should only use `worker` when it refers to the standalone worker. Maybe we can start following

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #29191: [SPARK-32364][SQL][FOLLOWUP] Add toMap to return originalMap

dongjoon-hyun commented on a change in pull request #29191: URL: https://github.com/apache/spark/pull/29191#discussion_r459225681 ## File path: sql/core/src/main/scala/org/apache/spark/sql/DataFrameReader.scala ## @@ -361,7 +361,7 @@ class DataFrameReader

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29191: [SPARK-32364][SQL][FOLLOWUP] Add toMap to return originalMap

AmplabJenkins removed a comment on pull request #29191: URL: https://github.com/apache/spark/pull/29191#issuecomment-662829543 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #29191: [SPARK-32364][SQL][FOLLOWUP] Add toMap to return originalMap

AmplabJenkins commented on pull request #29191: URL: https://github.com/apache/spark/pull/29191#issuecomment-662829543 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #29191: [SPARK-32364][SQL][FOLLOWUP] Add toMap to return originalMap

dongjoon-hyun commented on a change in pull request #29191: URL: https://github.com/apache/spark/pull/29191#discussion_r459225681 ## File path: sql/core/src/main/scala/org/apache/spark/sql/DataFrameReader.scala ## @@ -361,7 +361,7 @@ class DataFrameReader

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #29191: [SPARK-32364][SQL][FOLLOWUP] Add toMap to return originalMap

dongjoon-hyun commented on a change in pull request #29191: URL: https://github.com/apache/spark/pull/29191#discussion_r459225681 ## File path: sql/core/src/main/scala/org/apache/spark/sql/DataFrameReader.scala ## @@ -361,7 +361,7 @@ class DataFrameReader

[GitHub] [spark] SparkQA commented on pull request #29191: [SPARK-32364][SQL][FOLLOWUP] Add toMap to return originalMap

SparkQA commented on pull request #29191: URL: https://github.com/apache/spark/pull/29191#issuecomment-662829231 **[Test build #126386 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/126386/testReport)** for PR 29191 at commit

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #29191: [SPARK-32364][SQL][FOLLOWUP] Add toMap to return originalMap

dongjoon-hyun commented on a change in pull request #29191: URL: https://github.com/apache/spark/pull/29191#discussion_r459225681 ## File path: sql/core/src/main/scala/org/apache/spark/sql/DataFrameReader.scala ## @@ -361,7 +361,7 @@ class DataFrameReader

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #29191: [SPARK-32364][SQL][FOLLOWUP] Add toMap to return originalMap

dongjoon-hyun commented on a change in pull request #29191: URL: https://github.com/apache/spark/pull/29191#discussion_r459225681 ## File path: sql/core/src/main/scala/org/apache/spark/sql/DataFrameReader.scala ## @@ -361,7 +361,7 @@ class DataFrameReader

[GitHub] [spark] agrawaldevesh commented on pull request #29032: [SPARK-32217] Plumb whether a worker would also be decommissioned along with executor

agrawaldevesh commented on pull request #29032: URL: https://github.com/apache/spark/pull/29032#issuecomment-662828785 > Merged, thanks everyone. > > I don't know if we have a good principle around naming host v. worker with the deprecation. Thanks @holdenk for championing

[GitHub] [spark] cloud-fan commented on a change in pull request #29191: [SPARK-32364][SQL][FOLLOWUP] Add toMap to return originalMap

cloud-fan commented on a change in pull request #29191: URL: https://github.com/apache/spark/pull/29191#discussion_r459223512 ## File path: sql/core/src/main/scala/org/apache/spark/sql/DataFrameReader.scala ## @@ -361,7 +361,7 @@ class DataFrameReader

[GitHub] [spark] agrawaldevesh commented on a change in pull request #29104: [SPARK-32290][SQL] SingleColumn Null Aware Anti Join Optimize

agrawaldevesh commented on a change in pull request #29104: URL: https://github.com/apache/spark/pull/29104#discussion_r459223391 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/joins/BroadcastHashJoinExec.scala ## @@ -454,6 +478,28 @@ case class

[GitHub] [spark] cloud-fan commented on a change in pull request #29191: [SPARK-32364][SQL][FOLLOWUP] Add toMap to return originalMap

cloud-fan commented on a change in pull request #29191: URL: https://github.com/apache/spark/pull/29191#discussion_r459223066 ## File path: sql/core/src/main/scala/org/apache/spark/sql/DataFrameReader.scala ## @@ -361,7 +361,7 @@ class DataFrameReader

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29191: [SPARK-32364][SQL][FOLLOWUP] Add toMap to return originalMap

AmplabJenkins removed a comment on pull request #29191: URL: https://github.com/apache/spark/pull/29191#issuecomment-662825995 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29014: [SPARK-32199][SPARK-32198] Reduce job failures during decommissioning

AmplabJenkins removed a comment on pull request #29014: URL: https://github.com/apache/spark/pull/29014#issuecomment-662825993 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #29191: [SPARK-32364][SQL][FOLLOWUP] Add toMap to return originalMap

AmplabJenkins commented on pull request #29191: URL: https://github.com/apache/spark/pull/29191#issuecomment-662825995 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins commented on pull request #29014: [SPARK-32199][SPARK-32198] Reduce job failures during decommissioning

AmplabJenkins commented on pull request #29014: URL: https://github.com/apache/spark/pull/29014#issuecomment-662825993 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #29191: [SPARK-32364][SQL][FOLLOWUP] Add toMap to return originalMap

SparkQA commented on pull request #29191: URL: https://github.com/apache/spark/pull/29191#issuecomment-662825656 **[Test build #126384 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/126384/testReport)** for PR 29191 at commit

[GitHub] [spark] SparkQA commented on pull request #29014: [SPARK-32199][SPARK-32198] Reduce job failures during decommissioning

SparkQA commented on pull request #29014: URL: https://github.com/apache/spark/pull/29014#issuecomment-662825676 **[Test build #126385 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/126385/testReport)** for PR 29014 at commit

[GitHub] [spark] HyukjinKwon commented on pull request #29191: [SPARK-32364][SQL][FOLLOWUP] Add toMap to return originalMap

HyukjinKwon commented on pull request #29191: URL: https://github.com/apache/spark/pull/29191#issuecomment-662824703 Looks good This is an automated message from the Apache Git Service. To respond to the message, please log

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29191: [SPARK-32364][SQL][FOLLOWUP] Add toMap to return originalMap

AmplabJenkins removed a comment on pull request #29191: URL: https://github.com/apache/spark/pull/29191#issuecomment-662824244 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #29191: [SPARK-32364][SQL][FOLLOWUP] Add toMap to return originalMap

AmplabJenkins commented on pull request #29191: URL: https://github.com/apache/spark/pull/29191#issuecomment-662824244 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #29191: [SPARK-32364][SQL][FOLLOWUP] Add toMap to return originalMap

SparkQA commented on pull request #29191: URL: https://github.com/apache/spark/pull/29191#issuecomment-662824010 **[Test build #126383 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/126383/testReport)** for PR 29191 at commit

[GitHub] [spark] HyukjinKwon commented on pull request #29196: [SPARK-32398][TESTS][CORE][STREAMING][SQL][ML] Update to scalatest 3.2.0 for Scala 2.13.3+

HyukjinKwon commented on pull request #29196: URL: https://github.com/apache/spark/pull/29196#issuecomment-662824076 Yeah, we still should address https://github.com/apache/spark/pull/29196#discussion_r459209060 but the changes here look good.

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #29191: [SPARK-32364][SQL][FOLLOWUP] Add toMap to return originalMap

dongjoon-hyun commented on a change in pull request #29191: URL: https://github.com/apache/spark/pull/29191#discussion_r459219673 ## File path: sql/core/src/main/scala/org/apache/spark/sql/DataFrameWriter.scala ## @@ -288,7 +288,7 @@ final class DataFrameWriter[T]

[GitHub] [spark] SparkQA commented on pull request #29199: [SPARK-32105][SQL][FOLLOWUP]Refactor current ScriptTransformationExec…

SparkQA commented on pull request #29199: URL: https://github.com/apache/spark/pull/29199#issuecomment-662822321 **[Test build #126382 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/126382/testReport)** for PR 29199 at commit

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #29191: [SPARK-32364][SQL][FOLLOWUP] Add toMap to return originalMap

dongjoon-hyun commented on a change in pull request #29191: URL: https://github.com/apache/spark/pull/29191#discussion_r459218262 ## File path: sql/core/src/main/scala/org/apache/spark/sql/DataFrameWriter.scala ## @@ -288,7 +288,7 @@ final class DataFrameWriter[T]

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29199: [SPARK-32105][SQL][FOLLOWUP]Refactor current ScriptTransformationExec…

AmplabJenkins removed a comment on pull request #29199: URL: https://github.com/apache/spark/pull/29199#issuecomment-662820990 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #29199: [SPARK-32105][SQL][FOLLOWUP]Refactor current ScriptTransformationExec…

AmplabJenkins commented on pull request #29199: URL: https://github.com/apache/spark/pull/29199#issuecomment-662820990 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #29199: [SPARK-32105][SQL][FOLLOWUP]Refactor current ScriptTransformationExec…

SparkQA commented on pull request #29199: URL: https://github.com/apache/spark/pull/29199#issuecomment-662820692 **[Test build #126381 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/126381/testReport)** for PR 29199 at commit

[GitHub] [spark] kujon commented on pull request #29192: [SPARK-32393][SQL] Fix postgres bpchar array support

kujon commented on pull request #29192: URL: https://github.com/apache/spark/pull/29192#issuecomment-662819772 > Could you add tests in PostgresIntegrationSuite then check if the test can pass on your local env? (Note: our testing framework, Jenkins, does not run it). will do!

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29196: [SPARK-32398][TESTS][CORE][STREAMING][SQL][ML] Update to scalatest 3.2.0 for Scala 2.13.3+

AmplabJenkins removed a comment on pull request #29196: URL: https://github.com/apache/spark/pull/29196#issuecomment-662819514 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29199: [SPARK-32105][SQL][FOLLOWUP]Refactor current ScriptTransformationExec…

AmplabJenkins removed a comment on pull request #29199: URL: https://github.com/apache/spark/pull/29199#issuecomment-662819301 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #29196: [SPARK-32398][TESTS][CORE][STREAMING][SQL][ML] Update to scalatest 3.2.0 for Scala 2.13.3+

AmplabJenkins commented on pull request #29196: URL: https://github.com/apache/spark/pull/29196#issuecomment-662819514 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29186: [SPARK-32386][SS][TESTS] Fix temp view leaking in Structured Streaming tests

AmplabJenkins removed a comment on pull request #29186: URL: https://github.com/apache/spark/pull/29186#issuecomment-662819334 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #29199: [SPARK-32105][SQL][FOLLOWUP]Refactor current ScriptTransformationExec…

AmplabJenkins commented on pull request #29199: URL: https://github.com/apache/spark/pull/29199#issuecomment-662819301 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29167: [SPARK-32374][SQL] Disallow setting properties when creating temporary views

AmplabJenkins removed a comment on pull request #29167: URL: https://github.com/apache/spark/pull/29167#issuecomment-662819369 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AngersZhuuuu commented on pull request #29199: [SPARK-32105][SQL][FOLLOWUP]Refactor current ScriptTransformationExec…

AngersZh commented on pull request #29199: URL: https://github.com/apache/spark/pull/29199#issuecomment-662819271 FYI @maropu This is an automated message from the Apache Git Service. To respond to the message, please

[GitHub] [spark] AmplabJenkins commented on pull request #29186: [SPARK-32386][SS][TESTS] Fix temp view leaking in Structured Streaming tests

AmplabJenkins commented on pull request #29186: URL: https://github.com/apache/spark/pull/29186#issuecomment-662819334 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA removed a comment on pull request #29196: [SPARK-32398][TESTS][CORE][STREAMING][SQL][ML] Update to scalatest 3.2.0 for Scala 2.13.3+

SparkQA removed a comment on pull request #29196: URL: https://github.com/apache/spark/pull/29196#issuecomment-662785553 **[Test build #126371 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/126371/testReport)** for PR 29196 at commit

[GitHub] [spark] AmplabJenkins commented on pull request #29167: [SPARK-32374][SQL] Disallow setting properties when creating temporary views

AmplabJenkins commented on pull request #29167: URL: https://github.com/apache/spark/pull/29167#issuecomment-662819369 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #29196: [SPARK-32398][TESTS][CORE][STREAMING][SQL][ML] Update to scalatest 3.2.0 for Scala 2.13.3+

SparkQA commented on pull request #29196: URL: https://github.com/apache/spark/pull/29196#issuecomment-662819159 **[Test build #126371 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/126371/testReport)** for PR 29196 at commit

[GitHub] [spark] SparkQA commented on pull request #29186: [SPARK-32386][SS][TESTS] Fix temp view leaking in Structured Streaming tests

SparkQA commented on pull request #29186: URL: https://github.com/apache/spark/pull/29186#issuecomment-662819020 **[Test build #126379 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/126379/testReport)** for PR 29186 at commit

[GitHub] [spark] SparkQA commented on pull request #29167: [SPARK-32374][SQL] Disallow setting properties when creating temporary views

SparkQA commented on pull request #29167: URL: https://github.com/apache/spark/pull/29167#issuecomment-662819073 **[Test build #126380 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/126380/testReport)** for PR 29167 at commit

[GitHub] [spark] viirya commented on a change in pull request #29166: [SPARK-32280][SPARK-32372][SQL] ResolveReferences.dedupRight should only rewrite attributes for ancestor nodes of the conflict pl

viirya commented on a change in pull request #29166: URL: https://github.com/apache/spark/pull/29166#discussion_r459214912 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala ## @@ -1237,20 +1250,79 @@ class Analyzer( if

[GitHub] [spark] SparkQA commented on pull request #29199: [SPARK-32105][SQL][FOLLOWUP]Refactor current ScriptTransformationExec…

SparkQA commented on pull request #29199: URL: https://github.com/apache/spark/pull/29199#issuecomment-662818996 **[Test build #126378 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/126378/testReport)** for PR 29199 at commit

[GitHub] [spark] AngersZhuuuu opened a new pull request #29199: [SPARK-32105][SQL][FOLLOWUP]Refactor current ScriptTransformationExec…

AngersZh opened a new pull request #29199: URL: https://github.com/apache/spark/pull/29199 # What changes were proposed in this pull request? - Extract common Script IOSchema `ScriptTransformationIOSchema` - avoid repeated judgement extract process output row method

[GitHub] [spark] xuanyuanking commented on pull request #29186: [SPARK-32386][SS][TESTS] Fix temp view leaking in Structured Streaming tests

xuanyuanking commented on pull request #29186: URL: https://github.com/apache/spark/pull/29186#issuecomment-662818538 retest this please This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [spark] HyukjinKwon closed pull request #29195: [SPARK-32338][SQL][PYSPARK][FOLLOW-UP] Update slice to accept Column for start and length.

HyukjinKwon closed pull request #29195: URL: https://github.com/apache/spark/pull/29195 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [spark] cloud-fan commented on a change in pull request #29191: [SPARK-32364][SQL][FOLLOWUP] Add toMap to return originalMap

cloud-fan commented on a change in pull request #29191: URL: https://github.com/apache/spark/pull/29191#discussion_r459213960 ## File path: sql/core/src/main/scala/org/apache/spark/sql/DataFrameWriter.scala ## @@ -288,7 +288,7 @@ final class DataFrameWriter[T] private[sql](ds:

[GitHub] [spark] cloud-fan commented on a change in pull request #29191: [SPARK-32364][SQL][FOLLOWUP] Add toMap to return originalMap

cloud-fan commented on a change in pull request #29191: URL: https://github.com/apache/spark/pull/29191#discussion_r459213858 ## File path: sql/catalyst/src/main/scala-2.12/org/apache/spark/sql/catalyst/util/CaseInsensitiveMap.scala ## @@ -52,6 +52,8 @@ class

[GitHub] [spark] HyukjinKwon commented on pull request #29195: [SPARK-32338][SQL][PYSPARK][FOLLOW-UP] Update slice to accept Column for start and length.

HyukjinKwon commented on pull request #29195: URL: https://github.com/apache/spark/pull/29195#issuecomment-662817669 Merged to master. This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [spark] imback82 commented on a change in pull request #29167: [SPARK-32374][SQL] Disallow setting properties when creating temporary views

imback82 commented on a change in pull request #29167: URL: https://github.com/apache/spark/pull/29167#discussion_r459212899 ## File path: sql/core/src/test/scala/org/apache/spark/sql/execution/SQLViewSuite.scala ## @@ -266,6 +266,16 @@ abstract class SQLViewSuite extends

[GitHub] [spark] cloud-fan commented on a change in pull request #29107: [SPARK-32308][SQL] Move by-name resolution logic of unionByName from API code to analysis phase

cloud-fan commented on a change in pull request #29107: URL: https://github.com/apache/spark/pull/29107#discussion_r459212692 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/basicLogicalOperators.scala ## @@ -271,7 +281,7 @@ case class

[GitHub] [spark] cloud-fan commented on a change in pull request #29107: [SPARK-32308][SQL] Move by-name resolution logic of unionByName from API code to analysis phase

cloud-fan commented on a change in pull request #29107: URL: https://github.com/apache/spark/pull/29107#discussion_r459212340 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala ## @@ -1387,10 +1388,11 @@ class Analyzer(

[GitHub] [spark] cloud-fan commented on a change in pull request #29167: [SPARK-32374][SQL] Disallow setting properties when creating temporary views

cloud-fan commented on a change in pull request #29167: URL: https://github.com/apache/spark/pull/29167#discussion_r459210115 ## File path: sql/core/src/test/scala/org/apache/spark/sql/execution/SQLViewSuite.scala ## @@ -266,6 +266,16 @@ abstract class SQLViewSuite extends

[GitHub] [spark] cloud-fan commented on a change in pull request #29167: [SPARK-32374][SQL] Disallow setting properties when creating temporary views

cloud-fan commented on a change in pull request #29167: URL: https://github.com/apache/spark/pull/29167#discussion_r459209763 ## File path: sql/core/src/test/scala/org/apache/spark/sql/execution/SQLViewSuite.scala ## @@ -266,6 +266,16 @@ abstract class SQLViewSuite extends

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #29196: [SPARK-32398][TESTS][CORE][STREAMING][SQL][ML] Update to scalatest 3.2.0 for Scala 2.13.3+

dongjoon-hyun commented on a change in pull request #29196: URL: https://github.com/apache/spark/pull/29196#discussion_r459209060 ## File path: pom.xml ## @@ -892,7 +907,25 @@ org.scalatest scalatest_${scala.binary.version} -3.0.8 +

[GitHub] [spark] cloud-fan commented on a change in pull request #29104: [SPARK-32290][SQL] SingleColumn Null Aware Anti Join Optimize

cloud-fan commented on a change in pull request #29104: URL: https://github.com/apache/spark/pull/29104#discussion_r459208970 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/joins/BroadcastHashJoinExec.scala ## @@ -454,6 +478,28 @@ case class

[GitHub] [spark] asfgit closed pull request #29032: [SPARK-32217] Plumb whether a worker would also be decommissioned along with executor

asfgit closed pull request #29032: URL: https://github.com/apache/spark/pull/29032 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #29191: [SPARK-32364][SQL][FOLLOWUP] Add toMap to return originalMap

dongjoon-hyun commented on a change in pull request #29191: URL: https://github.com/apache/spark/pull/29191#discussion_r459207445 ## File path: sql/catalyst/src/main/scala-2.12/org/apache/spark/sql/catalyst/util/CaseInsensitiveMap.scala ## @@ -52,6 +52,8 @@ class

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #29191: [SPARK-32364][SQL][FOLLOWUP] Add toMap to return originalMap

dongjoon-hyun commented on a change in pull request #29191: URL: https://github.com/apache/spark/pull/29191#discussion_r459207445 ## File path: sql/catalyst/src/main/scala-2.12/org/apache/spark/sql/catalyst/util/CaseInsensitiveMap.scala ## @@ -52,6 +52,8 @@ class

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #29191: [SPARK-32364][SQL][FOLLOWUP] Add toMap to return originalMap

dongjoon-hyun commented on a change in pull request #29191: URL: https://github.com/apache/spark/pull/29191#discussion_r459206761 ## File path: sql/core/src/main/scala/org/apache/spark/sql/DataFrameWriter.scala ## @@ -288,7 +288,7 @@ final class DataFrameWriter[T]

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29198: [SPARK-32401][SQL] Migrate function related commands to new resolution framework

AmplabJenkins removed a comment on pull request #29198: URL: https://github.com/apache/spark/pull/29198#issuecomment-662810535 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] SparkQA commented on pull request #29198: [SPARK-32401][SQL] Migrate function related commands to new resolution framework

SparkQA commented on pull request #29198: URL: https://github.com/apache/spark/pull/29198#issuecomment-662810236 **[Test build #126377 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/126377/testReport)** for PR 29198 at commit

[GitHub] [spark] LantaoJin commented on pull request #29062: [SPARK-32237][SQL] Resolve hint in CTE

LantaoJin commented on pull request #29062: URL: https://github.com/apache/spark/pull/29062#issuecomment-662810277 Thanks. I will file a backport PR. This is an automated message from the Apache Git Service. To respond to

[GitHub] [spark] imback82 commented on a change in pull request #28840: [SPARK-31999][SQL] Add REFRESH FUNCTION command

imback82 commented on a change in pull request #28840: URL: https://github.com/apache/spark/pull/28840#discussion_r459205275 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/connector/catalog/LookupCatalog.scala ## @@ -155,4 +155,31 @@ private[sql] trait

[GitHub] [spark] AmplabJenkins commented on pull request #29198: [SPARK-32401][SQL] Migrate function related commands to new resolution framework

AmplabJenkins commented on pull request #29198: URL: https://github.com/apache/spark/pull/29198#issuecomment-662810535 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] imback82 commented on a change in pull request #29198: [SPARK-32401][SQL] Migrate function related commands to new resolution framework

imback82 commented on a change in pull request #29198: URL: https://github.com/apache/spark/pull/29198#discussion_r459204665 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/parser/AstBuilder.scala ## @@ -3650,12 +3652,24 @@ class AstBuilder(conf:

[GitHub] [spark] dongjoon-hyun commented on pull request #29191: [SPARK-32364][SQL][FOLLOWUP] Add toMap to return originalMap

dongjoon-hyun commented on pull request #29191: URL: https://github.com/apache/spark/pull/29191#issuecomment-662809395 Thank you, @cloud-fan and @maropu . This is an automated message from the Apache Git Service. To respond

[GitHub] [spark] holdenk commented on pull request #29032: [SPARK-32217] Plumb whether a worker would also be decommissioned along with executor

holdenk commented on pull request #29032: URL: https://github.com/apache/spark/pull/29032#issuecomment-662808939 Merged, thanks everyone. I don't know if we have a good principle around naming host v. worker with the deprecation.

[GitHub] [spark] imback82 opened a new pull request #29198: [SPARK-32401][SQL] Migrate function related commands to new resolution framework

imback82 opened a new pull request #29198: URL: https://github.com/apache/spark/pull/29198 ### What changes were proposed in this pull request? This PR proposes to migrate the following function related commands to the new resolution framework: - CREATE FUNCTION - DROP

[GitHub] [spark] cloud-fan commented on pull request #29032: [SPARK-32217] Plumb whether a worker would also be decommissioned along with executor

cloud-fan commented on pull request #29032: URL: https://github.com/apache/spark/pull/29032#issuecomment-662805220 I have a general question (not related to this PR): We have deprecated the usage of "multi-workers on one host", so host and worker should be the same thing. However, in the

[GitHub] [spark] cloud-fan commented on a change in pull request #29166: [SPARK-32280][SPARK-32372][SQL] ResolveReferences.dedupRight should only rewrite attributes for ancestor nodes of the conflict

cloud-fan commented on a change in pull request #29166: URL: https://github.com/apache/spark/pull/29166#discussion_r459199272 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala ## @@ -1237,20 +1250,48 @@ class Analyzer( if

[GitHub] [spark] cloud-fan commented on a change in pull request #29166: [SPARK-32280][SPARK-32372][SQL] ResolveReferences.dedupRight should only rewrite attributes for ancestor nodes of the conflict

cloud-fan commented on a change in pull request #29166: URL: https://github.com/apache/spark/pull/29166#discussion_r459198562 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala ## @@ -1237,20 +1250,48 @@ class Analyzer( if

[GitHub] [spark] cloud-fan commented on a change in pull request #29166: [SPARK-32280][SPARK-32372][SQL] ResolveReferences.dedupRight should only rewrite attributes for ancestor nodes of the conflict

cloud-fan commented on a change in pull request #29166: URL: https://github.com/apache/spark/pull/29166#discussion_r459198329 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala ## @@ -1237,20 +1250,48 @@ class Analyzer( if

[GitHub] [spark] kevinjmh commented on pull request #28901: [SPARK-32064][SQL] Supporting create temporary table

kevinjmh commented on pull request #28901: URL: https://github.com/apache/spark/pull/28901#issuecomment-662802864 how about to use `CACHE TABLE` command to do that? This is an automated message from the Apache Git Service.

[GitHub] [spark] cloud-fan commented on a change in pull request #29191: [SPARK-32364][SQL][FOLLOWUP] Add toMap to return originalMap

cloud-fan commented on a change in pull request #29191: URL: https://github.com/apache/spark/pull/29191#discussion_r459197779 ## File path: sql/core/src/main/scala/org/apache/spark/sql/DataFrameWriter.scala ## @@ -288,7 +288,7 @@ final class DataFrameWriter[T] private[sql](ds:

[GitHub] [spark] cloud-fan commented on a change in pull request #29191: [SPARK-32364][SQL][FOLLOWUP] Add toMap to return originalMap

cloud-fan commented on a change in pull request #29191: URL: https://github.com/apache/spark/pull/29191#discussion_r459197666 ## File path: sql/catalyst/src/main/scala-2.12/org/apache/spark/sql/catalyst/util/CaseInsensitiveMap.scala ## @@ -52,6 +52,8 @@ class

[GitHub] [spark] cloud-fan commented on pull request #29191: [SPARK-32364][SQL][FOLLOWUP] Add toMap to return originalMap

cloud-fan commented on pull request #29191: URL: https://github.com/apache/spark/pull/29191#issuecomment-662802631 Let's define the behavior more clear: 1. DFReader/Writer options are case insensitive, so later option should override prior ones if the key equals case-insensitively. This

[GitHub] [spark] AngersZhuuuu commented on pull request #29085: [SPARK-32106][SQL]Implement SparkScriptTransformationExec in sql/core

AngersZh commented on pull request #29085: URL: https://github.com/apache/spark/pull/29085#issuecomment-662801978 > Looks almost okay now, so could you split this PR into pieces? I think its somewhat big fro reviews. For example; > > 1. More refactoring PR for

[GitHub] [spark] AngersZhuuuu commented on a change in pull request #29085: [SPARK-32106][SQL]Implement SparkScriptTransformationExec in sql/core

AngersZh commented on a change in pull request #29085: URL: https://github.com/apache/spark/pull/29085#discussion_r459194715 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/BaseScriptTransformationExec.scala ## @@ -87,17 +181,72 @@ trait

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29085: [SPARK-32106][SQL]Implement SparkScriptTransformationExec in sql/core

AmplabJenkins removed a comment on pull request #29085: URL: https://github.com/apache/spark/pull/29085#issuecomment-662799097 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #29085: [SPARK-32106][SQL]Implement SparkScriptTransformationExec in sql/core

AmplabJenkins commented on pull request #29085: URL: https://github.com/apache/spark/pull/29085#issuecomment-662799097 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29197: [SPARK-32395][SQL] Commit output files after task attempt succeed in dynamic partition datawriter

AmplabJenkins removed a comment on pull request #29197: URL: https://github.com/apache/spark/pull/29197#issuecomment-662798617 Can one of the admins verify this patch? This is an automated message from the Apache Git

[GitHub] [spark] AmplabJenkins commented on pull request #29197: [SPARK-32395][SQL] Commit output files after task attempt succeed in dynamic partition datawriter

AmplabJenkins commented on pull request #29197: URL: https://github.com/apache/spark/pull/29197#issuecomment-662798935 Can one of the admins verify this patch? This is an automated message from the Apache Git Service. To

[GitHub] [spark] SparkQA commented on pull request #29085: [SPARK-32106][SQL]Implement SparkScriptTransformationExec in sql/core

SparkQA commented on pull request #29085: URL: https://github.com/apache/spark/pull/29085#issuecomment-662798740 **[Test build #126376 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/126376/testReport)** for PR 29085 at commit

[GitHub] [spark] AmplabJenkins commented on pull request #29197: [SPARK-32395][SQL] Commit output files after task attempt succeed in dynamic partition datawriter

AmplabJenkins commented on pull request #29197: URL: https://github.com/apache/spark/pull/29197#issuecomment-662798617 Can one of the admins verify this patch? This is an automated message from the Apache Git Service. To

[GitHub] [spark] cloud-fan commented on pull request #29062: [SPARK-32237][SQL] Resolve hint in CTE

cloud-fan commented on pull request #29062: URL: https://github.com/apache/spark/pull/29062#issuecomment-662798239 merged to master, thanks! @LantaoJin can you send a backport to 3.0? This is an automated message from

[GitHub] [spark] wankunde opened a new pull request #29197: [SPARK-32395][SQL] Commit output files after task attempt succeed in dynamic partition datawriter

wankunde opened a new pull request #29197: URL: https://github.com/apache/spark/pull/29197 # What changes were proposed in this pull request? Generally, distributed jobs have two stages of committing files: committing task's output files and committing job's output files. If one attempt

[GitHub] [spark] cloud-fan closed pull request #29062: [SPARK-32237][SQL] Resolve hint in CTE

cloud-fan closed pull request #29062: URL: https://github.com/apache/spark/pull/29062 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [spark] cloud-fan commented on pull request #29194: [SPARK-30616][SQL][FOLLOW-UP] Use only config key name in the config doc.

cloud-fan commented on pull request #29194: URL: https://github.com/apache/spark/pull/29194#issuecomment-662797390 thanks, merging to master! This is an automated message from the Apache Git Service. To respond to the

[GitHub] [spark] cloud-fan closed pull request #29194: [SPARK-30616][SQL][FOLLOW-UP] Use only config key name in the config doc.

cloud-fan closed pull request #29194: URL: https://github.com/apache/spark/pull/29194 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [spark] AngersZhuuuu commented on a change in pull request #29085: [SPARK-32106][SQL]Implement SparkScriptTransformationExec in sql/core

AngersZh commented on a change in pull request #29085: URL: https://github.com/apache/spark/pull/29085#discussion_r459189448 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/BaseScriptTransformationExec.scala ## @@ -87,17 +181,72 @@ trait

[GitHub] [spark] AngersZhuuuu commented on a change in pull request #29085: [SPARK-32106][SQL]Implement SparkScriptTransformationExec in sql/core

AngersZh commented on a change in pull request #29085: URL: https://github.com/apache/spark/pull/29085#discussion_r459189448 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/BaseScriptTransformationExec.scala ## @@ -87,17 +181,72 @@ trait

  1   2   3   4   5   6   7   8   9   10   >