[GitHub] [spark] dbtsai commented on pull request #33695: [SPARK-34863][SQL] Support complex types for Parquet vectorized reader

2021-08-11 Thread GitBox
dbtsai commented on pull request #33695: URL: https://github.com/apache/spark/pull/33695#issuecomment-897394930 Thank you, @sunchao for great work. Do we have benchmark result? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

[GitHub] [spark] yaooqinn commented on a change in pull request #33706: [SPARK-36477][SQL] Inferring schema from JSON file shall respect ignoreCorruptFiles and handle CharConversionException

2021-08-11 Thread GitBox
yaooqinn commented on a change in pull request #33706: URL: https://github.com/apache/spark/pull/33706#discussion_r687435652 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/json/JsonInferSchema.scala ## @@ -68,21 +73,27 @@ private[sql] class JsonInferSc

[GitHub] [spark] yaooqinn commented on a change in pull request #33706: [SPARK-36477][SQL] Inferring schema from JSON file shall respect ignoreCorruptFiles and handle CharConversionException

2021-08-11 Thread GitBox
yaooqinn commented on a change in pull request #33706: URL: https://github.com/apache/spark/pull/33706#discussion_r687435652 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/json/JsonInferSchema.scala ## @@ -68,21 +73,27 @@ private[sql] class JsonInferSc

[GitHub] [spark] zhuqi-lucas commented on a change in pull request #33617: [SPARK-35548][CORE][SHUFFLE] Handling new attempt has started error message in BlockPushErrorHandler in client

2021-08-11 Thread GitBox
zhuqi-lucas commented on a change in pull request #33617: URL: https://github.com/apache/spark/pull/33617#discussion_r687427336 ## File path: common/network-shuffle/src/main/java/org/apache/spark/network/shuffle/RemoteBlockPushResolver.java ## @@ -513,12 +511,17 @@ public Merg

[GitHub] [spark] zhuqi-lucas commented on a change in pull request #33617: [SPARK-35548][CORE][SHUFFLE] Handling new attempt has started error message in BlockPushErrorHandler in client

2021-08-11 Thread GitBox
zhuqi-lucas commented on a change in pull request #33617: URL: https://github.com/apache/spark/pull/33617#discussion_r687425590 ## File path: common/network-common/src/main/java/org/apache/spark/network/server/BlockPushNonFatalFailure.java ## @@ -39,6 +39,14 @@ public stati

[GitHub] [spark] zhuqi-lucas commented on a change in pull request #33617: [SPARK-35548][CORE][SHUFFLE] Handling new attempt has started error message in BlockPushErrorHandler in client

2021-08-11 Thread GitBox
zhuqi-lucas commented on a change in pull request #33617: URL: https://github.com/apache/spark/pull/33617#discussion_r687424570 ## File path: common/network-common/src/main/java/org/apache/spark/network/server/BlockPushNonFatalFailure.java ## @@ -146,12 +159,24 @@ public stati

[GitHub] [spark] cloud-fan commented on a change in pull request #33665: [SPARK-36428][SQL] the seconds parameter of make_timestamp should accept integer type

2021-08-11 Thread GitBox
cloud-fan commented on a change in pull request #33665: URL: https://github.com/apache/spark/pull/33665#discussion_r687422415 ## File path: sql/core/src/test/resources/sql-tests/results/ansi/timestamp.sql.out ## @@ -107,6 +107,40 @@ java.time.DateTimeException The fraction of

[GitHub] [spark] cloud-fan commented on a change in pull request #33665: [SPARK-36428][SQL] the seconds parameter of make_timestamp should accept integer type

2021-08-11 Thread GitBox
cloud-fan commented on a change in pull request #33665: URL: https://github.com/apache/spark/pull/33665#discussion_r687422062 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/datetimeExpressions.scala ## @@ -2617,30 +2628,41 @@ case class Mak

[GitHub] [spark] AmplabJenkins commented on pull request #33713: [SPARK-36483][CORE][TESTS] Fix intermittent test failures at Netty 4.1.52+

2021-08-11 Thread GitBox
AmplabJenkins commented on pull request #33713: URL: https://github.com/apache/spark/pull/33713#issuecomment-897381233 Can one of the admins verify this patch? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL a

[GitHub] [spark] SparkQA commented on pull request #33665: [SPARK-36428][SQL] the seconds parameter of make_timestamp should accept integer type

2021-08-11 Thread GitBox
SparkQA commented on pull request #33665: URL: https://github.com/apache/spark/pull/33665#issuecomment-897381089 **[Test build #142366 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/142366/testReport)** for PR 33665 at commit [`deaeb76`](https://github.com

[GitHub] [spark] cloud-fan commented on a change in pull request #33665: [SPARK-36428][SQL] the seconds parameter of make_timestamp should accept integer type

2021-08-11 Thread GitBox
cloud-fan commented on a change in pull request #33665: URL: https://github.com/apache/spark/pull/33665#discussion_r687421613 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/datetimeExpressions.scala ## @@ -2495,7 +2495,9 @@ case class MakeT

[GitHub] [spark] SparkQA commented on pull request #33695: [SPARK-34863][SQL] Support complex types for Parquet vectorized reader

2021-08-11 Thread GitBox
SparkQA commented on pull request #33695: URL: https://github.com/apache/spark/pull/33695#issuecomment-897380587 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/46873/ -- This is an automated message from the Apache

[GitHub] [spark] beliefer commented on pull request #33665: [SPARK-36428][SQL] the seconds parameter of make_timestamp should accept integer type

2021-08-11 Thread GitBox
beliefer commented on pull request #33665: URL: https://github.com/apache/spark/pull/33665#issuecomment-897379780 > @beliefer can you fix the code conflicts? thanks! OK -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub a

[GitHub] [spark] AmplabJenkins commented on pull request #33704: [SPARK-36353][SQL][3.1] RemoveNoopOperators should keep output schema

2021-08-11 Thread GitBox
AmplabJenkins commented on pull request #33704: URL: https://github.com/apache/spark/pull/33704#issuecomment-897379628 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/142359/ -- This

[GitHub] [spark] SparkQA commented on pull request #33704: [SPARK-36353][SQL][3.1] RemoveNoopOperators should keep output schema

2021-08-11 Thread GitBox
SparkQA commented on pull request #33704: URL: https://github.com/apache/spark/pull/33704#issuecomment-897378635 **[Test build #142359 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/142359/testReport)** for PR 33704 at commit [`5b1ec2c`](https://github.co

[GitHub] [spark] AmplabJenkins commented on pull request #33705: [SPARK-36353][SQL][3.0] RemoveNoopOperators should keep output schema

2021-08-11 Thread GitBox
AmplabJenkins commented on pull request #33705: URL: https://github.com/apache/spark/pull/33705#issuecomment-897378289 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/142358/ -- This

[GitHub] [spark] SparkQA commented on pull request #33705: [SPARK-36353][SQL][3.0] RemoveNoopOperators should keep output schema

2021-08-11 Thread GitBox
SparkQA commented on pull request #33705: URL: https://github.com/apache/spark/pull/33705#issuecomment-897376750 **[Test build #142358 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/142358/testReport)** for PR 33705 at commit [`225b612`](https://github.co

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33706: [SPARK-36477][SQL] Inferring schema from JSON file shall respect ignoreCorruptFiles and handle CharConversionException

2021-08-11 Thread GitBox
AmplabJenkins removed a comment on pull request #33706: URL: https://github.com/apache/spark/pull/33706#issuecomment-897170120 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33717: [SPARK-36486][SQL] Support cast type constructed same string to day time interval

2021-08-11 Thread GitBox
AmplabJenkins removed a comment on pull request #33717: URL: https://github.com/apache/spark/pull/33717#issuecomment-897375698 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/46871/

[GitHub] [spark] AmplabJenkins commented on pull request #33706: [SPARK-36477][SQL] Inferring schema from JSON file shall respect ignoreCorruptFiles and handle CharConversionException

2021-08-11 Thread GitBox
AmplabJenkins commented on pull request #33706: URL: https://github.com/apache/spark/pull/33706#issuecomment-897375696 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/46872/ -- T

[GitHub] [spark] AmplabJenkins commented on pull request #33717: [SPARK-36486][SQL] Support cast type constructed same string to day time interval

2021-08-11 Thread GitBox
AmplabJenkins commented on pull request #33717: URL: https://github.com/apache/spark/pull/33717#issuecomment-897375698 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/46871/ -- T

[GitHub] [spark] cloud-fan closed pull request #33711: Fix bug in disable unnecessary bucketed scan

2021-08-11 Thread GitBox
cloud-fan closed pull request #33711: URL: https://github.com/apache/spark/pull/33711 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsu

[GitHub] [spark] cloud-fan commented on pull request #33711: Fix bug in disable unnecessary bucketed scan

2021-08-11 Thread GitBox
cloud-fan commented on pull request #33711: URL: https://github.com/apache/spark/pull/33711#issuecomment-897374324 The fix LGTM. If a table has only one bucket, it can satisfy `AllTuples` and no shuffle is needed for aggregate without group keys. However, if we disable bucket scan, then `A

[GitHub] [spark] zhuqi-lucas commented on a change in pull request #33617: [SPARK-35548][CORE][SHUFFLE] Handling new attempt has started error message in BlockPushErrorHandler in client

2021-08-11 Thread GitBox
zhuqi-lucas commented on a change in pull request #33617: URL: https://github.com/apache/spark/pull/33617#discussion_r687412274 ## File path: common/network-shuffle/src/main/java/org/apache/spark/network/shuffle/RemoteBlockPushResolver.java ## @@ -514,12 +512,12 @@ public Merg

[GitHub] [spark] SparkQA commented on pull request #33706: [SPARK-36477][SQL] Inferring schema from JSON file shall respect ignoreCorruptFiles and handle CharConversionException

2021-08-11 Thread GitBox
SparkQA commented on pull request #33706: URL: https://github.com/apache/spark/pull/33706#issuecomment-897371317 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/46872/ -- This is an automated message from the A

[GitHub] [spark] zwangsheng commented on pull request #33718: [CORE] modify exit executor log logic

2021-08-11 Thread GitBox
zwangsheng commented on pull request #33718: URL: https://github.com/apache/spark/pull/33718#issuecomment-897368922 > can you file a JIRA please? Also it would be great to show the console output you got from the manual test. OK, i will do it later. -- This is an automated message

[GitHub] [spark] HyukjinKwon commented on pull request #33718: [CORE] modify exit executor log logic

2021-08-11 Thread GitBox
HyukjinKwon commented on pull request #33718: URL: https://github.com/apache/spark/pull/33718#issuecomment-897368348 can you file a JIRA please? Also it would be great to show the console output you got from the manual test. -- This is an automated message from the Apache Git Service. To

[GitHub] [spark] cloud-fan commented on a change in pull request #33711: Fix bug in disable unnecessary bucketed scan

2021-08-11 Thread GitBox
cloud-fan commented on a change in pull request #33711: URL: https://github.com/apache/spark/pull/33711#discussion_r687407918 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/bucketing/DisableUnnecessaryBucketedScan.scala ## @@ -120,7 +120,7 @@ object Disab

[GitHub] [spark] HyukjinKwon commented on pull request #33711: Fix bug in disable unnecessary bucketed scan

2021-08-11 Thread GitBox
HyukjinKwon commented on pull request #33711: URL: https://github.com/apache/spark/pull/33711#issuecomment-897368109 Can you file a JIRA please -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to th

[GitHub] [spark] SparkQA commented on pull request #33717: [SPARK-36486][SQL] Support cast type constructed same string to day time interval

2021-08-11 Thread GitBox
SparkQA commented on pull request #33717: URL: https://github.com/apache/spark/pull/33717#issuecomment-897366695 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/46871/ -- This is an automated message from the A

[GitHub] [spark] SparkQA commented on pull request #33695: [SPARK-34863][SQL] Support complex types for Parquet vectorized reader

2021-08-11 Thread GitBox
SparkQA commented on pull request #33695: URL: https://github.com/apache/spark/pull/33695#issuecomment-897362383 **[Test build #142365 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/142365/testReport)** for PR 33695 at commit [`9aec74a`](https://github.com

[GitHub] [spark] SparkQA commented on pull request #33706: [SPARK-36477][SQL] Inferring schema from JSON file shall respect ignoreCorruptFiles and handle CharConversionException

2021-08-11 Thread GitBox
SparkQA commented on pull request #33706: URL: https://github.com/apache/spark/pull/33706#issuecomment-897357620 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/46872/ -- This is an automated message from the Apache

[GitHub] [spark] cloud-fan commented on pull request #33665: [SPARK-36428][SQL] the seconds parameter of make_timestamp should accept integer type

2021-08-11 Thread GitBox
cloud-fan commented on pull request #33665: URL: https://github.com/apache/spark/pull/33665#issuecomment-897356316 @beliefer can you fix the code conflicts? thanks! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33716: [SPARK-36485][SQL] Support cast type constructed string as year month interval

2021-08-11 Thread GitBox
AmplabJenkins removed a comment on pull request #33716: URL: https://github.com/apache/spark/pull/33716#issuecomment-897353585 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/46869/

[GitHub] [spark] AmplabJenkins commented on pull request #33716: [SPARK-36485][SQL] Support cast type constructed string as year month interval

2021-08-11 Thread GitBox
AmplabJenkins commented on pull request #33716: URL: https://github.com/apache/spark/pull/33716#issuecomment-897353585 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/46869/ -- T

[GitHub] [spark] c21 commented on pull request #33698: [SPARK-32986][SQL] Add bucketed scan info in query plan of data source v1

2021-08-11 Thread GitBox
c21 commented on pull request #33698: URL: https://github.com/apache/spark/pull/33698#issuecomment-897352972 Thank you @dongjoon-hyun and @cloud-fan for review! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

[GitHub] [spark] HyukjinKwon commented on pull request #33706: [SPARK-36477][SQL] Inferring schema from JSON file shall respect ignoreCorruptFiles and handle CharConversionException

2021-08-11 Thread GitBox
HyukjinKwon commented on pull request #33706: URL: https://github.com/apache/spark/pull/33706#issuecomment-897348923 cc @MaxGekk would you mind taking a look please when you find some time? -- This is an automated message from the Apache Git Service. To respond to the message, please log

[GitHub] [spark] HyukjinKwon commented on a change in pull request #33706: [SPARK-36477][SQL] Inferring schema from JSON file shall respect ignoreCorruptFiles and handle CharConversionException

2021-08-11 Thread GitBox
HyukjinKwon commented on a change in pull request #33706: URL: https://github.com/apache/spark/pull/33706#discussion_r687388095 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/json/JsonInferSchema.scala ## @@ -68,21 +73,27 @@ private[sql] class JsonInfe

[GitHub] [spark] SparkQA commented on pull request #33717: [SPARK-36486][SQL] Support cast type constructed same string to day time interval

2021-08-11 Thread GitBox
SparkQA commented on pull request #33717: URL: https://github.com/apache/spark/pull/33717#issuecomment-897348019 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/46871/ -- This is an automated message from the Apache

[GitHub] [spark] cloud-fan closed pull request #33707: [SPARK-36479][SQL][TEST] Improve datetime test coverage in SQL files

2021-08-11 Thread GitBox
cloud-fan closed pull request #33707: URL: https://github.com/apache/spark/pull/33707 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsu

[GitHub] [spark] cloud-fan commented on pull request #33707: [SPARK-36479][SQL][TEST] Improve datetime test coverage in SQL files

2021-08-11 Thread GitBox
cloud-fan commented on pull request #33707: URL: https://github.com/apache/spark/pull/33707#issuecomment-897346637 The GA failure is unrelated, I'm merging this to master/3.2, thanks for the review! @gengliangwang let's fix the discovered bugs. -- This is an automated message from

[GitHub] [spark] cloud-fan commented on a change in pull request #30706: [SPARK-33779][SQL] DataSource V2: API to request distribution and ordering on write

2021-08-11 Thread GitBox
cloud-fan commented on a change in pull request #30706: URL: https://github.com/apache/spark/pull/30706#discussion_r687384463 ## File path: sql/catalyst/src/main/java/org/apache/spark/sql/connector/distributions/ClusteredDistribution.java ## @@ -0,0 +1,35 @@ +/* + * Licensed t

[GitHub] [spark] SparkQA commented on pull request #33716: [SPARK-36485][SQL] Support cast type constructed string as year month interval

2021-08-11 Thread GitBox
SparkQA commented on pull request #33716: URL: https://github.com/apache/spark/pull/33716#issuecomment-897343430 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/46869/ -- This is an automated message from the A

[GitHub] [spark] SparkQA commented on pull request #33639: [SPARK-34952][SQL] Aggregate (Min/Max/Count) push down for Parquet

2021-08-11 Thread GitBox
SparkQA commented on pull request #33639: URL: https://github.com/apache/spark/pull/33639#issuecomment-897342579 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/46870/ -- This is an automated message from the A

[GitHub] [spark] AmplabJenkins commented on pull request #33639: [SPARK-34952][SQL] Aggregate (Min/Max/Count) push down for Parquet

2021-08-11 Thread GitBox
AmplabJenkins commented on pull request #33639: URL: https://github.com/apache/spark/pull/33639#issuecomment-897342602 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/46870/ -- T

[GitHub] [spark] SparkQA commented on pull request #33706: [SPARK-36477][SQL] Inferring schema from JSON file shall respect ignoreCorruptFiles and handle CharConversionException

2021-08-11 Thread GitBox
SparkQA commented on pull request #33706: URL: https://github.com/apache/spark/pull/33706#issuecomment-897341563 **[Test build #142364 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/142364/testReport)** for PR 33706 at commit [`7a65467`](https://github.com

[GitHub] [spark] yaooqinn commented on a change in pull request #33706: [SPARK-36477][SQL] Inferring schema from JSON file shall respect ignoreCorruptFiles and handle CharConversionException

2021-08-11 Thread GitBox
yaooqinn commented on a change in pull request #33706: URL: https://github.com/apache/spark/pull/33706#discussion_r687379665 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/json/JsonInferSchema.scala ## @@ -68,21 +73,25 @@ private[sql] class JsonInferSc

[GitHub] [spark] HyukjinKwon commented on a change in pull request #33706: [SPARK-36477][SQL] Inferring schema from JSON file shall respect ignoreCorruptFiles and handle IOE

2021-08-11 Thread GitBox
HyukjinKwon commented on a change in pull request #33706: URL: https://github.com/apache/spark/pull/33706#discussion_r687376730 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/json/JsonInferSchema.scala ## @@ -68,21 +73,25 @@ private[sql] class JsonInfe

[GitHub] [spark] SparkQA commented on pull request #33717: [SPARK-36486][SQL] Support cast type constructed same string to day time interval

2021-08-11 Thread GitBox
SparkQA commented on pull request #33717: URL: https://github.com/apache/spark/pull/33717#issuecomment-897335662 **[Test build #142363 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/142363/testReport)** for PR 33717 at commit [`91cdc0a`](https://github.com

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33713: [SPARK-36483][CORE][TESTS] Fix intermittent test failures at Netty 4.1.52+

2021-08-11 Thread GitBox
AmplabJenkins removed a comment on pull request #33713: URL: https://github.com/apache/spark/pull/33713#issuecomment-897334074 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/142357/ -

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33665: [SPARK-36428][SQL] the seconds parameter of make_timestamp should accept integer type

2021-08-11 Thread GitBox
AmplabJenkins removed a comment on pull request #33665: URL: https://github.com/apache/spark/pull/33665#issuecomment-897334077 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/46868/

[GitHub] [spark] AmplabJenkins commented on pull request #33718: [CORE] modify exit executor log logic

2021-08-11 Thread GitBox
AmplabJenkins commented on pull request #33718: URL: https://github.com/apache/spark/pull/33718#issuecomment-897334405 Can one of the admins verify this patch? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL a

[GitHub] [spark] AmplabJenkins commented on pull request #33713: [SPARK-36483][CORE][TESTS] Fix intermittent test failures at Netty 4.1.52+

2021-08-11 Thread GitBox
AmplabJenkins commented on pull request #33713: URL: https://github.com/apache/spark/pull/33713#issuecomment-897334074 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/142357/ -- This

[GitHub] [spark] AmplabJenkins commented on pull request #33665: [SPARK-36428][SQL] the seconds parameter of make_timestamp should accept integer type

2021-08-11 Thread GitBox
AmplabJenkins commented on pull request #33665: URL: https://github.com/apache/spark/pull/33665#issuecomment-897334077 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/46868/ -- T

[GitHub] [spark] Ngone51 commented on a change in pull request #33617: [SPARK-35548][CORE][SHUFFLE] Handling new attempt has started error message in BlockPushErrorHandler in client

2021-08-11 Thread GitBox
Ngone51 commented on a change in pull request #33617: URL: https://github.com/apache/spark/pull/33617#discussion_r687372545 ## File path: common/network-shuffle/src/main/java/org/apache/spark/network/shuffle/RemoteBlockPushResolver.java ## @@ -514,12 +512,12 @@ public MergeSta

[GitHub] [spark] Ngone51 commented on a change in pull request #33617: [SPARK-35548][CORE][SHUFFLE] Handling new attempt has started error message in BlockPushErrorHandler in client

2021-08-11 Thread GitBox
Ngone51 commented on a change in pull request #33617: URL: https://github.com/apache/spark/pull/33617#discussion_r687372088 ## File path: common/network-shuffle/src/main/java/org/apache/spark/network/shuffle/RemoteBlockPushResolver.java ## @@ -514,12 +512,12 @@ public MergeSta

[GitHub] [spark] zwangsheng opened a new pull request #33718: [CORE] modify exit executor log logic

2021-08-11 Thread GitBox
zwangsheng opened a new pull request #33718: URL: https://github.com/apache/spark/pull/33718 ### What changes were proposed in this pull request? Adjust the log logic of CoarseGrainedExecutorBackend ### Why are the changes needed? When exit executor with system code 0, coars

[GitHub] [spark] SparkQA commented on pull request #33716: [SPARK-36485][SQL] Support cast type constructed string as year month interval

2021-08-11 Thread GitBox
SparkQA commented on pull request #33716: URL: https://github.com/apache/spark/pull/33716#issuecomment-897330711 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/46869/ -- This is an automated message from the Apache

[GitHub] [spark] Ngone51 commented on a change in pull request #33615: [SPARK-36374][SHUFFLE][DOC] Push-based shuffle high level user documentation

2021-08-11 Thread GitBox
Ngone51 commented on a change in pull request #33615: URL: https://github.com/apache/spark/pull/33615#discussion_r687326402 ## File path: docs/configuration.md ## @@ -3152,3 +3152,109 @@ The stage level scheduling feature allows users to specify task and executor res This is

[GitHub] [spark] SparkQA removed a comment on pull request #33713: [SPARK-36483][CORE][TESTS] Fix intermittent test failures at Netty 4.1.52+

2021-08-11 Thread GitBox
SparkQA removed a comment on pull request #33713: URL: https://github.com/apache/spark/pull/33713#issuecomment-897269970 **[Test build #142357 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/142357/testReport)** for PR 33713 at commit [`3fc5953`](https://gi

[GitHub] [spark] SparkQA commented on pull request #33639: [SPARK-34952][SQL] Aggregate (Min/Max/Count) push down for Parquet

2021-08-11 Thread GitBox
SparkQA commented on pull request #33639: URL: https://github.com/apache/spark/pull/33639#issuecomment-897329719 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/46870/ -- This is an automated message from the Apache

[GitHub] [spark] AngersZhuuuu commented on pull request #33717: [SPARK-36486][SQL] Support cast type constructed same string to day time interval

2021-08-11 Thread GitBox
AngersZh commented on pull request #33717: URL: https://github.com/apache/spark/pull/33717#issuecomment-897328583 FYi @MaxGekk @cloud-fan -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [spark] AngersZhuuuu opened a new pull request #33717: [SPARK-36486][SQL] Support cast type constructed same string to day time interval

2021-08-11 Thread GitBox
AngersZh opened a new pull request #33717: URL: https://github.com/apache/spark/pull/33717 ### What changes were proposed in this pull request? Spark support type constructed string as day time interval such as ``` interval '1 day 2 hour' ``` And PGSQL support ```

[GitHub] [spark] SparkQA commented on pull request #33665: [SPARK-36428][SQL] the seconds parameter of make_timestamp should accept integer type

2021-08-11 Thread GitBox
SparkQA commented on pull request #33665: URL: https://github.com/apache/spark/pull/33665#issuecomment-897327996 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/46868/ -- This is an automated message from the A

[GitHub] [spark] SparkQA commented on pull request #33713: [SPARK-36483][CORE][TESTS] Fix intermittent test failures at Netty 4.1.52+

2021-08-11 Thread GitBox
SparkQA commented on pull request #33713: URL: https://github.com/apache/spark/pull/33713#issuecomment-897325003 **[Test build #142357 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/142357/testReport)** for PR 33713 at commit [`3fc5953`](https://github.co

[GitHub] [spark] SparkQA commented on pull request #33704: [SPARK-36353][SQL][3.1] RemoveNoopOperators should keep output schema

2021-08-11 Thread GitBox
SparkQA commented on pull request #33704: URL: https://github.com/apache/spark/pull/33704#issuecomment-897323305 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/46867/ -- This is an automated message from the A

[GitHub] [spark] AmplabJenkins commented on pull request #33704: [SPARK-36353][SQL][3.1] RemoveNoopOperators should keep output schema

2021-08-11 Thread GitBox
AmplabJenkins commented on pull request #33704: URL: https://github.com/apache/spark/pull/33704#issuecomment-897323317 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/46867/ -- T

[GitHub] [spark] AmplabJenkins commented on pull request #33705: [SPARK-36353][SQL][3.0] RemoveNoopOperators should keep output schema

2021-08-11 Thread GitBox
AmplabJenkins commented on pull request #33705: URL: https://github.com/apache/spark/pull/33705#issuecomment-897317621 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/46866/ -- T

[GitHub] [spark] SparkQA commented on pull request #33705: [SPARK-36353][SQL][3.0] RemoveNoopOperators should keep output schema

2021-08-11 Thread GitBox
SparkQA commented on pull request #33705: URL: https://github.com/apache/spark/pull/33705#issuecomment-897317598 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/46866/ -- This is an automated message from the A

[GitHub] [spark] SparkQA commented on pull request #33716: [SPARK-36485][SQL] Support cast type constructed string as year month interval

2021-08-11 Thread GitBox
SparkQA commented on pull request #33716: URL: https://github.com/apache/spark/pull/33716#issuecomment-897316420 **[Test build #142362 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/142362/testReport)** for PR 33716 at commit [`2f7b98b`](https://github.com

[GitHub] [spark] mridulm commented on a change in pull request #33617: [SPARK-35548][CORE][SHUFFLE] Handling new attempt has started error message in BlockPushErrorHandler in client

2021-08-11 Thread GitBox
mridulm commented on a change in pull request #33617: URL: https://github.com/apache/spark/pull/33617#discussion_r687339894 ## File path: common/network-common/src/main/java/org/apache/spark/network/server/TransportRequestHandler.java ## @@ -249,8 +249,17 @@ public String getI

[GitHub] [spark] HeartSaVioR commented on a change in pull request #33708: [SPARK-36480][SS] SessionWindowStateStoreSaveExec should not filter input rows against watermark

2021-08-11 Thread GitBox
HeartSaVioR commented on a change in pull request #33708: URL: https://github.com/apache/spark/pull/33708#discussion_r687337199 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/statefulOperators.scala ## @@ -642,9 +647,7 @@ case class SessionWindo

[GitHub] [spark] HeartSaVioR commented on pull request #33708: [SPARK-36480][SS] SessionWindowStateStoreSaveExec should not filter input rows against watermark

2021-08-11 Thread GitBox
HeartSaVioR commented on pull request #33708: URL: https://github.com/apache/spark/pull/33708#issuecomment-897314969 Thanks @viirya for the quick reviewing and merging! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

[GitHub] [spark] HeartSaVioR commented on a change in pull request #33708: [SPARK-36480][SS] SessionWindowStateStoreSaveExec should not filter input rows against watermark

2021-08-11 Thread GitBox
HeartSaVioR commented on a change in pull request #33708: URL: https://github.com/apache/spark/pull/33708#discussion_r687337199 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/statefulOperators.scala ## @@ -642,9 +647,7 @@ case class SessionWindo

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33713: [SPARK-36483][CORE][TESTS] Fix intermittent test failures at Netty 4.1.52+

2021-08-11 Thread GitBox
AmplabJenkins removed a comment on pull request #33713: URL: https://github.com/apache/spark/pull/33713#issuecomment-897313601 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/46865/

[GitHub] [spark] SparkQA commented on pull request #33639: [SPARK-34952][SQL] Aggregate (Min/Max/Count) push down for Parquet

2021-08-11 Thread GitBox
SparkQA commented on pull request #33639: URL: https://github.com/apache/spark/pull/33639#issuecomment-897314324 **[Test build #142361 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/142361/testReport)** for PR 33639 at commit [`fc70672`](https://github.com

[GitHub] [spark] AngersZhuuuu commented on pull request #33716: [SPARK-36485][SQL] Support cast type constructed string as year month interval

2021-08-11 Thread GitBox
AngersZh commented on pull request #33716: URL: https://github.com/apache/spark/pull/33716#issuecomment-897313987 ping @cloud-fan @MaxGekk -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to th

[GitHub] [spark] AngersZhuuuu opened a new pull request #33716: [SPARK-36485][SQL] Support cast type constructed string as year month interval

2021-08-11 Thread GitBox
AngersZh opened a new pull request #33716: URL: https://github.com/apache/spark/pull/33716 ### What changes were proposed in this pull request? Spark support type constructed string as year month interval such as ``` interval '1 year 2 month' interval '3 year' ``` And

[GitHub] [spark] AmplabJenkins commented on pull request #33713: [SPARK-36483][CORE][TESTS] Fix intermittent test failures at Netty 4.1.52+

2021-08-11 Thread GitBox
AmplabJenkins commented on pull request #33713: URL: https://github.com/apache/spark/pull/33713#issuecomment-897313601 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/46865/ -- T

[GitHub] [spark] viirya closed pull request #33708: [SPARK-36480][SS] SessionWindowStateStoreSaveExec should not filter input rows against watermark

2021-08-11 Thread GitBox
viirya closed pull request #33708: URL: https://github.com/apache/spark/pull/33708 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubsc

[GitHub] [spark] viirya commented on pull request #33708: [SPARK-36480][SS] SessionWindowStateStoreSaveExec should not filter input rows against watermark

2021-08-11 Thread GitBox
viirya commented on pull request #33708: URL: https://github.com/apache/spark/pull/33708#issuecomment-897313406 Thanks for fixing this. Merging to master/3.2. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL ab

[GitHub] [spark] zhuqi-lucas commented on a change in pull request #33617: [SPARK-35548][CORE][SHUFFLE] Handling new attempt has started error message in BlockPushErrorHandler in client

2021-08-11 Thread GitBox
zhuqi-lucas commented on a change in pull request #33617: URL: https://github.com/apache/spark/pull/33617#discussion_r687331572 ## File path: common/network-shuffle/src/main/java/org/apache/spark/network/shuffle/RemoteBlockPushResolver.java ## @@ -395,19 +395,17 @@ void delete

[GitHub] [spark] SparkQA commented on pull request #33704: [SPARK-36353][SQL][3.1] RemoveNoopOperators should keep output schema

2021-08-11 Thread GitBox
SparkQA commented on pull request #33704: URL: https://github.com/apache/spark/pull/33704#issuecomment-897309980 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/46867/ -- This is an automated message from the Apache

[GitHub] [spark] SparkQA removed a comment on pull request #33709: [WIP][SPARK-36418][SQL] Use CAST in parsing of dates/timestamps with default pattern

2021-08-11 Thread GitBox
SparkQA removed a comment on pull request #33709: URL: https://github.com/apache/spark/pull/33709#issuecomment-897196922 **[Test build #142353 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/142353/testReport)** for PR 33709 at commit [`8da6961`](https://gi

[GitHub] [spark] SparkQA commented on pull request #33665: [SPARK-36428][SQL] the seconds parameter of make_timestamp should accept integer type

2021-08-11 Thread GitBox
SparkQA commented on pull request #33665: URL: https://github.com/apache/spark/pull/33665#issuecomment-897309534 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/46868/ -- This is an automated message from the Apache

[GitHub] [spark] SparkQA commented on pull request #33705: [SPARK-36353][SQL][3.0] RemoveNoopOperators should keep output schema

2021-08-11 Thread GitBox
SparkQA commented on pull request #33705: URL: https://github.com/apache/spark/pull/33705#issuecomment-897308319 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/46866/ -- This is an automated message from the Apache

[GitHub] [spark] viirya commented on a change in pull request #33708: [SPARK-36480][SS] SessionWindowStateStoreSaveExec should not filter input rows against watermark

2021-08-11 Thread GitBox
viirya commented on a change in pull request #33708: URL: https://github.com/apache/spark/pull/33708#discussion_r687328633 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/statefulOperators.scala ## @@ -642,9 +647,7 @@ case class SessionWindowStat

[GitHub] [spark] SparkQA commented on pull request #33713: [SPARK-36483][CORE][TESTS] Fix intermittent test failures at Netty 4.1.52+

2021-08-11 Thread GitBox
SparkQA commented on pull request #33713: URL: https://github.com/apache/spark/pull/33713#issuecomment-897304375 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/46865/ -- This is an automated message from the A

[GitHub] [spark] huaxingao commented on a change in pull request #33639: [SPARK-34952][SQL] Aggregate (Min/Max/Count) push down for Parquet

2021-08-11 Thread GitBox
huaxingao commented on a change in pull request #33639: URL: https://github.com/apache/spark/pull/33639#discussion_r687325426 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetUtils.scala ## @@ -127,4 +145,260 @@ object ParquetUtil

[GitHub] [spark] dongjoon-hyun commented on pull request #33698: [SPARK-32986][SQL] Add bucketed scan info in query plan of data source v1

2021-08-11 Thread GitBox
dongjoon-hyun commented on pull request #33698: URL: https://github.com/apache/spark/pull/33698#issuecomment-897303742 Thank you, @c21 and @cloud-fan . -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [spark] huaxingao commented on a change in pull request #33639: [SPARK-34952][SQL] Aggregate (Min/Max/Count) push down for Parquet

2021-08-11 Thread GitBox
huaxingao commented on a change in pull request #33639: URL: https://github.com/apache/spark/pull/33639#discussion_r687325363 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetUtils.scala ## @@ -127,4 +145,260 @@ object ParquetUtil

[GitHub] [spark] dongjoon-hyun commented on pull request #33698: [SPARK-32986][SQL] Add bucketed scan info in query plan of data source v1

2021-08-11 Thread GitBox
dongjoon-hyun commented on pull request #33698: URL: https://github.com/apache/spark/pull/33698#issuecomment-897303657 Merged to master for Apache Spark 3.3.0 according to the issue type, `Improvement`. -- This is an automated message from the Apache Git Service. To respond to the messag

[GitHub] [spark] huaxingao commented on a change in pull request #33639: [SPARK-34952][SQL] Aggregate (Min/Max/Count) push down for Parquet

2021-08-11 Thread GitBox
huaxingao commented on a change in pull request #33639: URL: https://github.com/apache/spark/pull/33639#discussion_r687325296 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetUtils.scala ## @@ -127,4 +145,260 @@ object ParquetUtil

[GitHub] [spark] huaxingao commented on a change in pull request #33639: [SPARK-34952][SQL] Aggregate (Min/Max/Count) push down for Parquet

2021-08-11 Thread GitBox
huaxingao commented on a change in pull request #33639: URL: https://github.com/apache/spark/pull/33639#discussion_r687325170 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/parquet/ParquetPartitionReaderFactory.scala ## @@ -80,43 +84,90 @@

[GitHub] [spark] huaxingao commented on a change in pull request #33639: [SPARK-34952][SQL] Aggregate (Min/Max/Count) push down for Parquet

2021-08-11 Thread GitBox
huaxingao commented on a change in pull request #33639: URL: https://github.com/apache/spark/pull/33639#discussion_r687325073 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetUtils.scala ## @@ -127,4 +145,260 @@ object ParquetUtil

[GitHub] [spark] dongjoon-hyun closed pull request #33698: [SPARK-32986][SQL] Add bucketed scan info in query plan of data source v1

2021-08-11 Thread GitBox
dongjoon-hyun closed pull request #33698: URL: https://github.com/apache/spark/pull/33698 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-

[GitHub] [spark] huaxingao commented on a change in pull request #33639: [SPARK-34952][SQL] Aggregate (Min/Max/Count) push down for Parquet

2021-08-11 Thread GitBox
huaxingao commented on a change in pull request #33639: URL: https://github.com/apache/spark/pull/33639#discussion_r687324970 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetUtils.scala ## @@ -127,4 +145,260 @@ object ParquetUtil

[GitHub] [spark] huaxingao commented on a change in pull request #33639: [SPARK-34952][SQL] Aggregate (Min/Max/Count) push down for Parquet

2021-08-11 Thread GitBox
huaxingao commented on a change in pull request #33639: URL: https://github.com/apache/spark/pull/33639#discussion_r687324911 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala ## @@ -870,6 +870,13 @@ object SQLConf { .checkValue(thres

[GitHub] [spark] AmplabJenkins commented on pull request #33709: [WIP][SPARK-36418][SQL] Use CAST in parsing of dates/timestamps with default pattern

2021-08-11 Thread GitBox
AmplabJenkins commented on pull request #33709: URL: https://github.com/apache/spark/pull/33709#issuecomment-897301618 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/142353/ -- This

[GitHub] [spark] SparkQA commented on pull request #33709: [WIP][SPARK-36418][SQL] Use CAST in parsing of dates/timestamps with default pattern

2021-08-11 Thread GitBox
SparkQA commented on pull request #33709: URL: https://github.com/apache/spark/pull/33709#issuecomment-897301098 **[Test build #142353 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/142353/testReport)** for PR 33709 at commit [`8da6961`](https://github.co

  1   2   3   4   5   6   7   >