[GitHub] [spark] HyukjinKwon commented on pull request #29685: [SPARK-32824][Core] Improve the error message when the user forgets the .amount in a resource config

HyukjinKwon commented on pull request #29685: URL: https://github.com/apache/spark/pull/29685#issuecomment-690911326 Yeah, there was no conflict and tests passed (https://github.com/apache/spark/commit/e86d90b21d4b3d6658b3cb6dd30daafb32b0c1bd).

[GitHub] [spark] HyukjinKwon commented on pull request #29286: [WIP][SPARK-21708][BUILD] Migrate build to sbt 1.x

HyukjinKwon commented on pull request #29286: URL: https://github.com/apache/spark/pull/29286#issuecomment-690910244 @gemelen can you set it here https://github.com/apache/spark/blob/master/project/SparkBuild.scala#L1010? Th

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29721: [SPARK-32851][SQL][TEST] Tests should fail if errors happen when generating expr code

AmplabJenkins removed a comment on pull request #29721: URL: https://github.com/apache/spark/pull/29721#issuecomment-690902072 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #29721: [SPARK-32851][SQL][TEST] Tests should fail if errors happen when generating expr code

AmplabJenkins commented on pull request #29721: URL: https://github.com/apache/spark/pull/29721#issuecomment-690902072 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] SparkQA commented on pull request #29721: [SPARK-32851][SQL][TEST] Tests should fail if errors happen when generating expr code

SparkQA commented on pull request #29721: URL: https://github.com/apache/spark/pull/29721#issuecomment-690901671 **[Test build #128555 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/128555/testReport)** for PR 29721 at commit [`0484de4`](https://github.com

[GitHub] [spark] karolchmist edited a comment on pull request #28545: [SPARK-30090][SHELL] Adapt Spark REPL to Scala 2.13

karolchmist edited a comment on pull request #28545: URL: https://github.com/apache/spark/pull/28545#issuecomment-690077928 Hello @srowen , for me the PR is ready to be reviewed. I extracted a Scala 2.12/2.13 specific code to separate directories, as the interface of `ILoop` changed.

[GitHub] [spark] maropu commented on a change in pull request #29721: [SPARK-32851][SQL][TEST] Tests should fail if errors happen when generating expr code

maropu commented on a change in pull request #29721: URL: https://github.com/apache/spark/pull/29721#discussion_r486797845 ## File path: sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/HiveQuerySuite.scala ## @@ -104,6 +104,16 @@ class HiveQuerySuite extends HiveCo

[GitHub] [spark] HyukjinKwon commented on a change in pull request #29721: [SPARK-32851][SQL][TEST] Tests should fail if errors happen when generating expr code

HyukjinKwon commented on a change in pull request #29721: URL: https://github.com/apache/spark/pull/29721#discussion_r486797352 ## File path: sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/HiveQuerySuite.scala ## @@ -104,6 +104,16 @@ class HiveQuerySuite extends H

[GitHub] [spark] HyukjinKwon commented on a change in pull request #29720: [SPARK-32849][PYSPARK] Add default values for non-required keys when creating StructType

HyukjinKwon commented on a change in pull request #29720: URL: https://github.com/apache/spark/pull/29720#discussion_r486794746 ## File path: python/pyspark/sql/types.py ## @@ -305,7 +305,7 @@ def jsonValue(self): @classmethod def fromJson(cls, json): Review comment:

[GitHub] [spark] HyukjinKwon commented on a change in pull request #29720: [SPARK-32849][PYSPARK] Add default values for non-required keys when creating StructType

HyukjinKwon commented on a change in pull request #29720: URL: https://github.com/apache/spark/pull/29720#discussion_r486794529 ## File path: python/pyspark/sql/types.py ## @@ -305,7 +305,7 @@ def jsonValue(self): @classmethod def fromJson(cls, json): return

[GitHub] [spark] liaoaoyuan97 commented on a change in pull request #21164: [SPARK-24098][SQL] ScriptTransformationExec should wait process exiting before output iterator finish

liaoaoyuan97 commented on a change in pull request #21164: URL: https://github.com/apache/spark/pull/21164#discussion_r486793639 ## File path: sql/hive/src/main/scala/org/apache/spark/sql/hive/execution/ScriptTransformationExec.scala ## @@ -137,13 +137,12 @@ case class ScriptT

[GitHub] [spark] HyukjinKwon edited a comment on pull request #29706: [SPARK-32839][WINDOWS] Make Spark scripts working with the spaces in paths on Windows

HyukjinKwon edited a comment on pull request #29706: URL: https://github.com/apache/spark/pull/29706#issuecomment-690896035 I'll merge this in few days. I am not sure who to ask review. Maybe @srowen. .. This is an automated

[GitHub] [spark] HyukjinKwon commented on pull request #29706: [SPARK-32839][WINDOWS] Make Spark scripts working with the spaces in paths on Windows

HyukjinKwon commented on pull request #29706: URL: https://github.com/apache/spark/pull/29706#issuecomment-690896035 I'll merge this in few days. I am not sure who to ask review. This is an automated message from the Apache G

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29722: [SPARK-32850][CORE] Simply the RPC message flow of decommission

AmplabJenkins removed a comment on pull request #29722: URL: https://github.com/apache/spark/pull/29722#issuecomment-690894738 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/128

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29673: [SPARK-32816][SQL] Fix analyzer bug when aggregating multiple distinct DECIMAL columns

AmplabJenkins removed a comment on pull request #29673: URL: https://github.com/apache/spark/pull/29673#issuecomment-690894654 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29585: [SPARK-32741][SQL] Check if the same ExprId refers to the unique attribute in logical plans

AmplabJenkins removed a comment on pull request #29585: URL: https://github.com/apache/spark/pull/29585#issuecomment-690894643 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29722: [SPARK-32850][CORE] Simply the RPC message flow of decommission

AmplabJenkins removed a comment on pull request #29722: URL: https://github.com/apache/spark/pull/29722#issuecomment-690894733 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To r

[GitHub] [spark] AmplabJenkins commented on pull request #29722: [SPARK-32850][CORE] Simply the RPC message flow of decommission

AmplabJenkins commented on pull request #29722: URL: https://github.com/apache/spark/pull/29722#issuecomment-690894733 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] AmplabJenkins commented on pull request #29673: [SPARK-32816][SQL] Fix analyzer bug when aggregating multiple distinct DECIMAL columns

AmplabJenkins commented on pull request #29673: URL: https://github.com/apache/spark/pull/29673#issuecomment-690894654 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] AmplabJenkins commented on pull request #29585: [SPARK-32741][SQL] Check if the same ExprId refers to the unique attribute in logical plans

AmplabJenkins commented on pull request #29585: URL: https://github.com/apache/spark/pull/29585#issuecomment-690894643 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] SparkQA removed a comment on pull request #29722: [SPARK-32850][CORE] Simply the RPC message flow of decommission

SparkQA removed a comment on pull request #29722: URL: https://github.com/apache/spark/pull/29722#issuecomment-690851353 **[Test build #128548 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/128548/testReport)** for PR 29722 at commit [`3bca7c0`](https://gi

[GitHub] [spark] SparkQA commented on pull request #29585: [SPARK-32741][SQL] Check if the same ExprId refers to the unique attribute in logical plans

SparkQA commented on pull request #29585: URL: https://github.com/apache/spark/pull/29585#issuecomment-690894145 **[Test build #128554 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/128554/testReport)** for PR 29585 at commit [`e014a13`](https://github.com

[GitHub] [spark] SparkQA commented on pull request #29722: [SPARK-32850][CORE] Simply the RPC message flow of decommission

SparkQA commented on pull request #29722: URL: https://github.com/apache/spark/pull/29722#issuecomment-690894221 **[Test build #128548 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/128548/testReport)** for PR 29722 at commit [`3bca7c0`](https://github.co

[GitHub] [spark] SparkQA removed a comment on pull request #29673: [SPARK-32816][SQL] Fix analyzer bug when aggregating multiple distinct DECIMAL columns

SparkQA removed a comment on pull request #29673: URL: https://github.com/apache/spark/pull/29673#issuecomment-690804618 **[Test build #128540 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/128540/testReport)** for PR 29673 at commit [`8510ff9`](https://gi

[GitHub] [spark] SparkQA commented on pull request #29673: [SPARK-32816][SQL] Fix analyzer bug when aggregating multiple distinct DECIMAL columns

SparkQA commented on pull request #29673: URL: https://github.com/apache/spark/pull/29673#issuecomment-690893672 **[Test build #128540 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/128540/testReport)** for PR 29673 at commit [`8510ff9`](https://github.co

[GitHub] [spark] AmplabJenkins commented on pull request #29565: [SPARK-24994][SQL] Add UnwrapCastInBinaryComparison optimizer to simplify literal types

AmplabJenkins commented on pull request #29565: URL: https://github.com/apache/spark/pull/29565#issuecomment-690893067 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29565: [SPARK-24994][SQL] Add UnwrapCastInBinaryComparison optimizer to simplify literal types

AmplabJenkins removed a comment on pull request #29565: URL: https://github.com/apache/spark/pull/29565#issuecomment-690893067 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] SparkQA removed a comment on pull request #29565: [SPARK-24994][SQL] Add UnwrapCastInBinaryComparison optimizer to simplify literal types

SparkQA removed a comment on pull request #29565: URL: https://github.com/apache/spark/pull/29565#issuecomment-690802886 **[Test build #128539 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/128539/testReport)** for PR 29565 at commit [`ec88961`](https://gi

[GitHub] [spark] SparkQA commented on pull request #29565: [SPARK-24994][SQL] Add UnwrapCastInBinaryComparison optimizer to simplify literal types

SparkQA commented on pull request #29565: URL: https://github.com/apache/spark/pull/29565#issuecomment-690892299 **[Test build #128539 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/128539/testReport)** for PR 29565 at commit [`ec88961`](https://github.co

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29723: [SPARK-32853][SQL] consecutive save/load calls in DataFrame/StreamReader/Writer should not fail

AmplabJenkins removed a comment on pull request #29723: URL: https://github.com/apache/spark/pull/29723#issuecomment-690889630 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #29723: [SPARK-32853][SQL] consecutive save/load calls in DataFrame/StreamReader/Writer should not fail

AmplabJenkins commented on pull request #29723: URL: https://github.com/apache/spark/pull/29723#issuecomment-690889630 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29724: [SPARK-32854][SS] Minor code and doc improvement for stream-stream join

AmplabJenkins removed a comment on pull request #29724: URL: https://github.com/apache/spark/pull/29724#issuecomment-690889562 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #29724: [SPARK-32854][SS] Minor code and doc improvement for stream-stream join

AmplabJenkins commented on pull request #29724: URL: https://github.com/apache/spark/pull/29724#issuecomment-690889562 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] SparkQA commented on pull request #29724: [SPARK-32854][SS] Minor code and doc improvement for stream-stream join

SparkQA commented on pull request #29724: URL: https://github.com/apache/spark/pull/29724#issuecomment-690889136 **[Test build #128552 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/128552/testReport)** for PR 29724 at commit [`069ad73`](https://github.com

[GitHub] [spark] SparkQA commented on pull request #29723: [SPARK-32853][SQL] consecutive save/load calls in DataFrame/StreamReader/Writer should not fail

SparkQA commented on pull request #29723: URL: https://github.com/apache/spark/pull/29723#issuecomment-690889186 **[Test build #128553 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/128553/testReport)** for PR 29723 at commit [`b8b38ec`](https://github.com

[GitHub] [spark] c21 opened a new pull request #29724: [SPARK-32854][SS] Minor code and doc improvement for stream-stream join

c21 opened a new pull request #29724: URL: https://github.com/apache/spark/pull/29724 ### What changes were proposed in this pull request? Several minor code and documentation improvement for stream-stream join. Specifically: * Remove extending from `SparkPlan`, as ext

[GitHub] [spark] c21 commented on pull request #29724: [SPARK-32854][SS] Minor code and doc improvement for stream-stream join

c21 commented on pull request #29724: URL: https://github.com/apache/spark/pull/29724#issuecomment-690887740 cc @cloud-fan and @sameeragarwal if you have time to take a look, thanks. This is an automated message from the Apac

[GitHub] [spark] cloud-fan opened a new pull request #29723: [SPARK-32853][SQL] consecutive save/load calls in DataFrame/StreamReader/Writer should not fail

cloud-fan opened a new pull request #29723: URL: https://github.com/apache/spark/pull/29723 ### What changes were proposed in this pull request? This is a followup of https://github.com/apache/spark/pull/29328 In https://github.com/apache/spark/pull/29328 , we forbid th

[GitHub] [spark] cloud-fan commented on pull request #29723: [SPARK-32853][SQL] consecutive save/load calls in DataFrame/StreamReader/Writer should not fail

cloud-fan commented on pull request #29723: URL: https://github.com/apache/spark/pull/29723#issuecomment-690887291 @imback82 @dongjoon-hyun This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [spark] dongjoon-hyun closed pull request #29707: [SPARK-32832][SS][2.4] Use CaseInsensitiveMap for DataStreamReader/Writer options

dongjoon-hyun closed pull request #29707: URL: https://github.com/apache/spark/pull/29707 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [spark] LantaoJin commented on pull request #25840: [SPARK-29166][SQL] Add parameters to limit the number of dynamic partitions for data source table

LantaoJin commented on pull request #25840: URL: https://github.com/apache/spark/pull/25840#issuecomment-690879012 ping @dongjoon-hyun @viirya . This is an automated message from the Apache Git Service. To respond to the mes

[GitHub] [spark] AmplabJenkins removed a comment on pull request #25840: [SPARK-29166][SQL] Add parameters to limit the number of dynamic partitions for data source table

AmplabJenkins removed a comment on pull request #25840: URL: https://github.com/apache/spark/pull/25840#issuecomment-690877941 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #25840: [SPARK-29166][SQL] Add parameters to limit the number of dynamic partitions for data source table

AmplabJenkins commented on pull request #25840: URL: https://github.com/apache/spark/pull/25840#issuecomment-690877941 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] SparkQA commented on pull request #25840: [SPARK-29166][SQL] Add parameters to limit the number of dynamic partitions for data source table

SparkQA commented on pull request #25840: URL: https://github.com/apache/spark/pull/25840#issuecomment-690877598 **[Test build #128551 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/128551/testReport)** for PR 25840 at commit [`f78ff9c`](https://github.com

[GitHub] [spark] linar-jether commented on pull request #29719: [SPARK-32846][SQL][PYTHON] Support createDataFrame from an RDD of pd.DataFrames

linar-jether commented on pull request #29719: URL: https://github.com/apache/spark/pull/29719#issuecomment-690876526 Thank you @HyukjinKwon, issue is that this only applies to dataframes, this means that only spark supported types can be input to `.mapInPandas`. so this does not cover

[GitHub] [spark] LantaoJin commented on pull request #25840: [SPARK-29166][SQL] Add parameters to limit the number of dynamic partitions for data source table

LantaoJin commented on pull request #25840: URL: https://github.com/apache/spark/pull/25840#issuecomment-690876701 retest this please This is an automated message from the Apache Git Service. To respond to the message, please

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29564: [WIP][SPARK-32708] Query optimization fails to reuse exchange with DataSourceV2

AmplabJenkins removed a comment on pull request #29564: URL: https://github.com/apache/spark/pull/29564#issuecomment-690874399 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #29564: [WIP][SPARK-32708] Query optimization fails to reuse exchange with DataSourceV2

AmplabJenkins commented on pull request #29564: URL: https://github.com/apache/spark/pull/29564#issuecomment-690874399 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] SparkQA removed a comment on pull request #29564: [WIP][SPARK-32708] Query optimization fails to reuse exchange with DataSourceV2

SparkQA removed a comment on pull request #29564: URL: https://github.com/apache/spark/pull/29564#issuecomment-690818602 **[Test build #128543 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/128543/testReport)** for PR 29564 at commit [`8b864e7`](https://gi

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29677: [SPARK-32820][SQL] Remove redundant shuffle exchanges inserted by EnsureRequirements

AmplabJenkins removed a comment on pull request #29677: URL: https://github.com/apache/spark/pull/29677#issuecomment-690873703 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] SparkQA commented on pull request #29564: [WIP][SPARK-32708] Query optimization fails to reuse exchange with DataSourceV2

SparkQA commented on pull request #29564: URL: https://github.com/apache/spark/pull/29564#issuecomment-690873976 **[Test build #128543 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/128543/testReport)** for PR 29564 at commit [`8b864e7`](https://github.co

[GitHub] [spark] AmplabJenkins commented on pull request #29677: [SPARK-32820][SQL] Remove redundant shuffle exchanges inserted by EnsureRequirements

AmplabJenkins commented on pull request #29677: URL: https://github.com/apache/spark/pull/29677#issuecomment-690873703 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] SparkQA commented on pull request #29677: [SPARK-32820][SQL] Remove redundant shuffle exchanges inserted by EnsureRequirements

SparkQA commented on pull request #29677: URL: https://github.com/apache/spark/pull/29677#issuecomment-690873403 **[Test build #128550 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/128550/testReport)** for PR 29677 at commit [`5680d48`](https://github.com

[GitHub] [spark] AngersZhuuuu commented on pull request #29692: [SPARK-32830][SQL] Optimize Skewed BroadcastNestedLoopJoin with AQE

AngersZh commented on pull request #29692: URL: https://github.com/apache/spark/pull/29692#issuecomment-690873256 > > after stream side executed, we will get the raw count of each partition and judge if it's skewed seriously, if skewed seriously and volume is large, repartition stream

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29706: [SPARK-32839][WINDOWS] Make Spark scripts working with the spaces in paths on Windows

AmplabJenkins removed a comment on pull request #29706: URL: https://github.com/apache/spark/pull/29706#issuecomment-690872268 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #29706: [SPARK-32839][WINDOWS] Make Spark scripts working with the spaces in paths on Windows

AmplabJenkins commented on pull request #29706: URL: https://github.com/apache/spark/pull/29706#issuecomment-690872268 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] sarutak commented on a change in pull request #29677: [SPARK-32820][SQL] Remove redundant shuffle exchanges inserted by EnsureRequirements

sarutak commented on a change in pull request #29677: URL: https://github.com/apache/spark/pull/29677#discussion_r486772296 ## File path: sql/core/src/test/scala/org/apache/spark/sql/execution/PlannerSuite.scala ## @@ -994,6 +994,38 @@ class PlannerSuite extends SharedSparkSes

[GitHub] [spark] SparkQA removed a comment on pull request #29706: [SPARK-32839][WINDOWS] Make Spark scripts working with the spaces in paths on Windows

SparkQA removed a comment on pull request #29706: URL: https://github.com/apache/spark/pull/29706#issuecomment-690824615 **[Test build #128544 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/128544/testReport)** for PR 29706 at commit [`4fda201`](https://gi

[GitHub] [spark] SparkQA commented on pull request #29706: [SPARK-32839][WINDOWS] Make Spark scripts working with the spaces in paths on Windows

SparkQA commented on pull request #29706: URL: https://github.com/apache/spark/pull/29706#issuecomment-690871694 **[Test build #128544 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/128544/testReport)** for PR 29706 at commit [`4fda201`](https://github.co

[GitHub] [spark] cloud-fan commented on pull request #29692: [SPARK-32830][SQL] Optimize Skewed BroadcastNestedLoopJoin with AQE

cloud-fan commented on pull request #29692: URL: https://github.com/apache/spark/pull/29692#issuecomment-690871544 > after stream side executed, we will get the raw count of each partition and judge if it's skewed seriously, if skewed seriously and volume is large, repartition stream side

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29564: [WIP][SPARK-32708] Query optimization fails to reuse exchange with DataSourceV2

AmplabJenkins removed a comment on pull request #29564: URL: https://github.com/apache/spark/pull/29564#issuecomment-690867723 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #29564: [WIP][SPARK-32708] Query optimization fails to reuse exchange with DataSourceV2

AmplabJenkins commented on pull request #29564: URL: https://github.com/apache/spark/pull/29564#issuecomment-690867723 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] SparkQA removed a comment on pull request #29564: [WIP][SPARK-32708] Query optimization fails to reuse exchange with DataSourceV2

SparkQA removed a comment on pull request #29564: URL: https://github.com/apache/spark/pull/29564#issuecomment-690809986 **[Test build #128541 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/128541/testReport)** for PR 29564 at commit [`98483c8`](https://gi

[GitHub] [spark] SparkQA commented on pull request #29564: [WIP][SPARK-32708] Query optimization fails to reuse exchange with DataSourceV2

SparkQA commented on pull request #29564: URL: https://github.com/apache/spark/pull/29564#issuecomment-690867354 **[Test build #128541 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/128541/testReport)** for PR 29564 at commit [`98483c8`](https://github.co

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29585: [SPARK-32741][SQL] Check if the same ExprId refers to the unique attribute in logical plans

AmplabJenkins removed a comment on pull request #29585: URL: https://github.com/apache/spark/pull/29585#issuecomment-690863833 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/128

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29316: [SPARK-32508][SQL] Disallow empty part col values in partition spec before static partition writing

AmplabJenkins removed a comment on pull request #29316: URL: https://github.com/apache/spark/pull/29316#issuecomment-690863644 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] SparkQA removed a comment on pull request #29585: [SPARK-32741][SQL] Check if the same ExprId refers to the unique attribute in logical plans

SparkQA removed a comment on pull request #29585: URL: https://github.com/apache/spark/pull/29585#issuecomment-690814370 **[Test build #128542 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/128542/testReport)** for PR 29585 at commit [`097bbbe`](https://gi

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29585: [SPARK-32741][SQL] Check if the same ExprId refers to the unique attribute in logical plans

AmplabJenkins removed a comment on pull request #29585: URL: https://github.com/apache/spark/pull/29585#issuecomment-690863825 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To r

[GitHub] [spark] AmplabJenkins commented on pull request #29585: [SPARK-32741][SQL] Check if the same ExprId refers to the unique attribute in logical plans

AmplabJenkins commented on pull request #29585: URL: https://github.com/apache/spark/pull/29585#issuecomment-690863825 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] SparkQA commented on pull request #29585: [SPARK-32741][SQL] Check if the same ExprId refers to the unique attribute in logical plans

SparkQA commented on pull request #29585: URL: https://github.com/apache/spark/pull/29585#issuecomment-690863709 **[Test build #128542 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/128542/testReport)** for PR 29585 at commit [`097bbbe`](https://github.co

[GitHub] [spark] AmplabJenkins commented on pull request #29316: [SPARK-32508][SQL] Disallow empty part col values in partition spec before static partition writing

AmplabJenkins commented on pull request #29316: URL: https://github.com/apache/spark/pull/29316#issuecomment-690863644 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] SparkQA commented on pull request #29316: [SPARK-32508][SQL] Disallow empty part col values in partition spec before static partition writing

SparkQA commented on pull request #29316: URL: https://github.com/apache/spark/pull/29316#issuecomment-690863425 **[Test build #128549 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/128549/testReport)** for PR 29316 at commit [`65f781a`](https://github.com

[GitHub] [spark] viirya commented on a change in pull request #29587: [SPARK-32376][SQL] Make unionByName null-filling behavior work with struct columns

viirya commented on a change in pull request #29587: URL: https://github.com/apache/spark/pull/29587#discussion_r486762671 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/ResolveUnion.scala ## @@ -17,29 +17,101 @@ package org.apache.spark.sq

[GitHub] [spark] viirya commented on a change in pull request #29587: [SPARK-32376][SQL] Make unionByName null-filling behavior work with struct columns

viirya commented on a change in pull request #29587: URL: https://github.com/apache/spark/pull/29587#discussion_r486761719 ## File path: sql/core/src/test/scala/org/apache/spark/sql/DataFrameSetOperationsSuite.scala ## @@ -536,4 +536,71 @@ class DataFrameSetOperationsSuite ext

[GitHub] [spark] cloud-fan commented on a change in pull request #29572: [SPARK-32730][SQL] Improve LeftSemi and Existence SortMergeJoin right side buffering

cloud-fan commented on a change in pull request #29572: URL: https://github.com/apache/spark/pull/29572#discussion_r486756148 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/joins/SortMergeJoinExec.scala ## @@ -251,7 +251,8 @@ case class SortMergeJoinExec(

[GitHub] [spark] AmplabJenkins commented on pull request #29722: [SPARK-32850][CORE] Simply the RPC message flow of decommission

AmplabJenkins commented on pull request #29722: URL: https://github.com/apache/spark/pull/29722#issuecomment-690851701 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29722: [SPARK-32850][CORE] Simply the RPC message flow of decommission

AmplabJenkins removed a comment on pull request #29722: URL: https://github.com/apache/spark/pull/29722#issuecomment-690851701 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] Ngone51 commented on pull request #29722: [SPARK-32850][CORE] Simply the RPC message flow of decommission

Ngone51 commented on pull request #29722: URL: https://github.com/apache/spark/pull/29722#issuecomment-690851376 @cloud-fan @agrawaldevesh @holdenk Please take a look, thanks! This is an automated message from the Apache

[GitHub] [spark] SparkQA commented on pull request #29722: [SPARK-32850][CORE] Simply the RPC message flow of decommission

SparkQA commented on pull request #29722: URL: https://github.com/apache/spark/pull/29722#issuecomment-690851353 **[Test build #128548 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/128548/testReport)** for PR 29722 at commit [`3bca7c0`](https://github.com

[GitHub] [spark] Ngone51 opened a new pull request #29722: [SPARK-32850][CORE] Simply the RPC message flow of decommission

Ngone51 opened a new pull request #29722: URL: https://github.com/apache/spark/pull/29722 ### What changes were proposed in this pull request? This PR cleans up the RPC message flow among the multiple decommission use cases, it includes changes: * Keep `Worker`

[GitHub] [spark] dongjoon-hyun commented on pull request #29707: [SPARK-32832][SS][2.4] Use CaseInsensitiveMap for DataStreamReader/Writer options

dongjoon-hyun commented on pull request #29707: URL: https://github.com/apache/spark/pull/29707#issuecomment-690850790 Thank you all. Merged to branch-2.4. This is an automated message from the Apache Git Service. To respond

[GitHub] [spark] HeartSaVioR commented on pull request #29066: [WIP][SPARK-23889] DataSourceV2: required sorting and clustering for writes

HeartSaVioR commented on pull request #29066: URL: https://github.com/apache/spark/pull/29066#issuecomment-690848690 This is definitely a major missing piece on DSv2 compared to DSv1, as DSv1 writer can deal with Dataframe directly, hence able to do arbitrary changes (including repartition

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29721: [SPARK-32851][SQL][TEST] Tests should fail if errors happen when generating expr code

AmplabJenkins removed a comment on pull request #29721: URL: https://github.com/apache/spark/pull/29721#issuecomment-690846042 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #29721: [SPARK-32851][SQL][TEST] Tests should fail if errors happen when generating expr code

AmplabJenkins commented on pull request #29721: URL: https://github.com/apache/spark/pull/29721#issuecomment-690846042 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] SparkQA commented on pull request #29721: [SPARK-32851][SQL][TEST] Tests should fail if errors happen when generating expr code

SparkQA commented on pull request #29721: URL: https://github.com/apache/spark/pull/29721#issuecomment-690845696 **[Test build #128547 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/128547/testReport)** for PR 29721 at commit [`8b9b5e4`](https://github.com

[GitHub] [spark] maropu opened a new pull request #29721: [SPARK-32851][SQL][TEST] Tests should fail if errors happen when generating expr code

maropu opened a new pull request #29721: URL: https://github.com/apache/spark/pull/29721 ### What changes were proposed in this pull request? This PR intends to set `CODEGEN_ONLY` at `CODEGEN_FACTORY_MODE` in test spark context so that tests can fail if errors happen when gen

[GitHub] [spark] maropu closed pull request #29713: [SPARK-32677][SQL][DOCS][MINOR] Improve code comment in CreateFunctionCommand

maropu closed pull request #29713: URL: https://github.com/apache/spark/pull/29713 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to t

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29695: [SPARK-32833][SQL] [WIP]JDBC V2 Datasource aggregate push down

AmplabJenkins removed a comment on pull request #29695: URL: https://github.com/apache/spark/pull/29695#issuecomment-690832730 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #29695: [SPARK-32833][SQL] [WIP]JDBC V2 Datasource aggregate push down

AmplabJenkins commented on pull request #29695: URL: https://github.com/apache/spark/pull/29695#issuecomment-690832730 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] SparkQA commented on pull request #29695: [SPARK-32833][SQL] [WIP]JDBC V2 Datasource aggregate push down

SparkQA commented on pull request #29695: URL: https://github.com/apache/spark/pull/29695#issuecomment-690832325 **[Test build #128546 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/128546/testReport)** for PR 29695 at commit [`b11b24b`](https://github.com

[GitHub] [spark] maropu closed pull request #29716: [SPARK-32840][SQL][3.0] Invalid interval value can happen to be just adhesive with the unit

maropu closed pull request #29716: URL: https://github.com/apache/spark/pull/29716 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to t

[GitHub] [spark] maropu closed pull request #29681: [SPARK-32822][SQL] Change the number of partitions to zero when a range is empty with WholeStageCodegen disabled or falled back

maropu closed pull request #29681: URL: https://github.com/apache/spark/pull/29681 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to t

[GitHub] [spark] beliefer commented on pull request #29626: [SPARK-32777][SQL] Aggregation support aggregate function with multiple foldable expressions.

beliefer commented on pull request #29626: URL: https://github.com/apache/spark/pull/29626#issuecomment-690826597 @cloud-fan Thanks This is an automated message from the Apache Git Service. To respond to the message, please l

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29695: [SPARK-32833][SQL] [WIP]JDBC V2 Datasource aggregate push down

AmplabJenkins removed a comment on pull request #29695: URL: https://github.com/apache/spark/pull/29695#issuecomment-690825087 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #29695: [SPARK-32833][SQL] [WIP]JDBC V2 Datasource aggregate push down

AmplabJenkins commented on pull request #29695: URL: https://github.com/apache/spark/pull/29695#issuecomment-690825087 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] SparkQA commented on pull request #29695: [SPARK-32833][SQL] [WIP]JDBC V2 Datasource aggregate push down

SparkQA commented on pull request #29695: URL: https://github.com/apache/spark/pull/29695#issuecomment-690824646 **[Test build #128545 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/128545/testReport)** for PR 29695 at commit [`a30fd6d`](https://github.com

[GitHub] [spark] SparkQA commented on pull request #29706: [SPARK-32839][WINDOWS] Make Spark scripts working with the spaces in paths on Windows

SparkQA commented on pull request #29706: URL: https://github.com/apache/spark/pull/29706#issuecomment-690824615 **[Test build #128544 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/128544/testReport)** for PR 29706 at commit [`4fda201`](https://github.com

[GitHub] [spark] maropu closed pull request #29714: [SPARK-32828][SQL][3.0] Cast from a derived user-defined type to a base type

maropu closed pull request #29714: URL: https://github.com/apache/spark/pull/29714 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to t

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29706: [SPARK-32839][WINDOWS] Make Spark scripts working with the spaces in paths on Windows

AmplabJenkins removed a comment on pull request #29706: URL: https://github.com/apache/spark/pull/29706#issuecomment-690822986 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #29706: [SPARK-32839][WINDOWS] Make Spark scripts working with the spaces in paths on Windows

AmplabJenkins commented on pull request #29706: URL: https://github.com/apache/spark/pull/29706#issuecomment-690822986 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

  1   2   3   4   5   6   7   8   >