[GitHub] [spark] SparkQA removed a comment on pull request #33176: [SPARK-35975][SQL] New configuration `spark.sql.timestampType` for the default timestamp type

2021-07-01 Thread GitBox
SparkQA removed a comment on pull request #33176: URL: https://github.com/apache/spark/pull/33176#issuecomment-872396248 **[Test build #140527 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140527/testReport)** for PR 33176 at commit

[GitHub] [spark] AmplabJenkins commented on pull request #33176: [SPARK-35975][SQL] New configuration `spark.sql.timestampType` for the default timestamp type

2021-07-01 Thread GitBox
AmplabJenkins commented on pull request #33176: URL: https://github.com/apache/spark/pull/33176#issuecomment-872545780 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/140527/ -- This

[GitHub] [spark] dongjoon-hyun commented on pull request #33157: [WIP][SPARK-35954] Upgrade Apache Curator Dependency to 4.2.0

2021-07-01 Thread GitBox
dongjoon-hyun commented on pull request #33157: URL: https://github.com/apache/spark/pull/33157#issuecomment-872545708 Thank you for working on this, @nirousseau . > as expected, it breaks tests, I will try to setup a local dev env and fix library changes asap. -- This is

[GitHub] [spark] dongjoon-hyun commented on pull request #33176: [SPARK-35975][SQL] New configuration `spark.sql.timestampType` for the default timestamp type

2021-07-01 Thread GitBox
dongjoon-hyun commented on pull request #33176: URL: https://github.com/apache/spark/pull/33176#issuecomment-872545331 Thank you, @gengliangwang and @MaxGekk ! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

[GitHub] [spark] dongjoon-hyun edited a comment on pull request #33160: [SPARK-35959][BUILD] Add a new Maven profile "no-shaded-hadoop-client" for Hadoop versions older than 3.2.2/3.3.1

2021-07-01 Thread GitBox
dongjoon-hyun edited a comment on pull request #33160: URL: https://github.com/apache/spark/pull/33160#issuecomment-872544781 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [spark] dongjoon-hyun commented on pull request #33160: [SPARK-35959][BUILD] Add a new Maven profile "no-shaded-hadoop-client" for Hadoop versions older than 3.2.2/3.3.1

2021-07-01 Thread GitBox
dongjoon-hyun commented on pull request #33160: URL: https://github.com/apache/spark/pull/33160#issuecomment-872544781 For Hadoop 2 build, I noticed that GitHub Action job used `sbt` directly. So, I verified that combination compilation manually. ``` $ ./build/sbt -Pyarn -Pmesos

[GitHub] [spark] SparkQA commented on pull request #33176: [SPARK-35975][SQL] New configuration `spark.sql.timestampType` for the default timestamp type

2021-07-01 Thread GitBox
SparkQA commented on pull request #33176: URL: https://github.com/apache/spark/pull/33176#issuecomment-872544771 **[Test build #140527 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140527/testReport)** for PR 33176 at commit

[GitHub] [spark] AmplabJenkins commented on pull request #32921: [SPARK-35779][SQL] Dynamic filtering for Data Source V2

2021-07-01 Thread GitBox
AmplabJenkins commented on pull request #32921: URL: https://github.com/apache/spark/pull/32921#issuecomment-872544597 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/45048/ --

[GitHub] [spark] SparkQA commented on pull request #32921: [SPARK-35779][SQL] Dynamic filtering for Data Source V2

2021-07-01 Thread GitBox
SparkQA commented on pull request #32921: URL: https://github.com/apache/spark/pull/32921#issuecomment-872544567 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/45048/ -- This is an automated message from the

[GitHub] [spark] SparkQA commented on pull request #33177: [SPARK-35955][SQL] Check for overflow in Average in ANSI mode

2021-07-01 Thread GitBox
SparkQA commented on pull request #33177: URL: https://github.com/apache/spark/pull/33177#issuecomment-872538721 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/45050/ -- This is an automated message from the Apache

[GitHub] [spark] rahulsmahadev commented on a change in pull request #33093: [SPARK-35897][SS] Support user defined initial state with flatMapGroupsWithState in Structured Streaming

2021-07-01 Thread GitBox
rahulsmahadev commented on a change in pull request #33093: URL: https://github.com/apache/spark/pull/33093#discussion_r662580556 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/UnsupportedOperationChecker.scala ## @@ -232,6 +238,10 @@ object

[GitHub] [spark] SparkQA commented on pull request #33093: [SPARK-35897][SS] Support user defined initial state with flatMapGroupsWithState in Structured Streaming

2021-07-01 Thread GitBox
SparkQA commented on pull request #33093: URL: https://github.com/apache/spark/pull/33093#issuecomment-872536986 **[Test build #140538 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140538/testReport)** for PR 33093 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33176: [SPARK-35975][SQL] New configuration `spark.sql.timestampType` for the default timestamp type

2021-07-01 Thread GitBox
AmplabJenkins removed a comment on pull request #33176: URL: https://github.com/apache/spark/pull/33176#issuecomment-872536048 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/45045/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33164: [SPARK-35958][CORE] Refactor SparkError.scala to SparkThrowable.java

2021-07-01 Thread GitBox
AmplabJenkins removed a comment on pull request #33164: URL: https://github.com/apache/spark/pull/33164#issuecomment-872536047 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/140533/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33160: [SPARK-35959][BUILD] Add a new Maven profile "no-shaded-hadoop-client" for Hadoop versions older than 3.2.2/3.3.1

2021-07-01 Thread GitBox
AmplabJenkins removed a comment on pull request #33160: URL: https://github.com/apache/spark/pull/33160#issuecomment-872536277 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/45049/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33070: [SPARK-35551][SQL] Handle the COUNT bug for lateral subqueries

2021-07-01 Thread GitBox
AmplabJenkins removed a comment on pull request #33070: URL: https://github.com/apache/spark/pull/33070#issuecomment-872536050 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/45047/

[GitHub] [spark] SparkQA commented on pull request #33160: [SPARK-35959][BUILD] Add a new Maven profile "no-shaded-hadoop-client" for Hadoop versions older than 3.2.2/3.3.1

2021-07-01 Thread GitBox
SparkQA commented on pull request #33160: URL: https://github.com/apache/spark/pull/33160#issuecomment-872536248 Kubernetes integration test unable to build dist. exiting with code: 1 URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/45049/ --

[GitHub] [spark] AmplabJenkins commented on pull request #33160: [SPARK-35959][BUILD] Add a new Maven profile "no-shaded-hadoop-client" for Hadoop versions older than 3.2.2/3.3.1

2021-07-01 Thread GitBox
AmplabJenkins commented on pull request #33160: URL: https://github.com/apache/spark/pull/33160#issuecomment-872536277 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/45049/ --

[GitHub] [spark] AmplabJenkins commented on pull request #33070: [SPARK-35551][SQL] Handle the COUNT bug for lateral subqueries

2021-07-01 Thread GitBox
AmplabJenkins commented on pull request #33070: URL: https://github.com/apache/spark/pull/33070#issuecomment-872536050 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/45047/ --

[GitHub] [spark] AmplabJenkins commented on pull request #33176: [SPARK-35975][SQL] New configuration `spark.sql.timestampType` for the default timestamp type

2021-07-01 Thread GitBox
AmplabJenkins commented on pull request #33176: URL: https://github.com/apache/spark/pull/33176#issuecomment-872536048 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/45045/ --

[GitHub] [spark] AmplabJenkins commented on pull request #33164: [SPARK-35958][CORE] Refactor SparkError.scala to SparkThrowable.java

2021-07-01 Thread GitBox
AmplabJenkins commented on pull request #33164: URL: https://github.com/apache/spark/pull/33164#issuecomment-872536047 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/140533/ -- This

[GitHub] [spark] SparkQA removed a comment on pull request #33164: [SPARK-35958][CORE] Refactor SparkError.scala to SparkThrowable.java

2021-07-01 Thread GitBox
SparkQA removed a comment on pull request #33164: URL: https://github.com/apache/spark/pull/33164#issuecomment-872468576 **[Test build #140533 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140533/testReport)** for PR 33164 at commit

[GitHub] [spark] SparkQA commented on pull request #33164: [SPARK-35958][CORE] Refactor SparkError.scala to SparkThrowable.java

2021-07-01 Thread GitBox
SparkQA commented on pull request #33164: URL: https://github.com/apache/spark/pull/33164#issuecomment-872534204 **[Test build #140533 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140533/testReport)** for PR 33164 at commit

[GitHub] [spark] karenfeng commented on pull request #33177: [SPARK-35955][SQL] Check for overflow in Average in ANSI mode

2021-07-01 Thread GitBox
karenfeng commented on pull request #33177: URL: https://github.com/apache/spark/pull/33177#issuecomment-872530300 > I don't think this is related to [SPARK-35855](https://issues.apache.org/jira/browse/SPARK-35855). Good catch @peter-toth - sorry, meant SPARK-35955 臘‍♀️ -- This

[GitHub] [spark] peter-toth commented on pull request #33177: [SPARK-35855][SQL] Check for overflow in Average in ANSI mode

2021-07-01 Thread GitBox
peter-toth commented on pull request #33177: URL: https://github.com/apache/spark/pull/33177#issuecomment-872529206 I don't think this is related to SPARK-35855. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

[GitHub] [spark] MaxGekk closed pull request #33176: [SPARK-35975][SQL] New configuration `spark.sql.timestampType` for the default timestamp type

2021-07-01 Thread GitBox
MaxGekk closed pull request #33176: URL: https://github.com/apache/spark/pull/33176 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [spark] MaxGekk commented on pull request #33176: [SPARK-35975][SQL] New configuration `spark.sql.timestampType` for the default timestamp type

2021-07-01 Thread GitBox
MaxGekk commented on pull request #33176: URL: https://github.com/apache/spark/pull/33176#issuecomment-872527626 +1, LGTM. Merging to master. Thank you, @gengliangwang and @cloud-fan @dongjoon-hyun for reviews. -- This is an automated message from the Apache Git Service. To respond to

[GitHub] [spark] SparkQA commented on pull request #32921: [SPARK-35779][SQL] Dynamic filtering for Data Source V2

2021-07-01 Thread GitBox
SparkQA commented on pull request #32921: URL: https://github.com/apache/spark/pull/32921#issuecomment-872527313 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/45048/ -- This is an automated message from the Apache

[GitHub] [spark] viirya commented on a change in pull request #32933: [SPARK-35785][SS] Cleanup support for RocksDB instance

2021-07-01 Thread GitBox
viirya commented on a change in pull request #32933: URL: https://github.com/apache/spark/pull/32933#discussion_r662567672 ## File path: sql/core/src/test/scala/org/apache/spark/sql/execution/streaming/state/RocksDBSuite.scala ## @@ -207,6 +273,133 @@ class RocksDBSuite

[GitHub] [spark] viirya commented on a change in pull request #32933: [SPARK-35785][SS] Cleanup support for RocksDB instance

2021-07-01 Thread GitBox
viirya commented on a change in pull request #32933: URL: https://github.com/apache/spark/pull/32933#discussion_r662563375 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/state/RocksDB.scala ## @@ -253,6 +253,13 @@ class RocksDB(

[GitHub] [spark] viirya commented on a change in pull request #32933: [SPARK-35785][SS] Cleanup support for RocksDB instance

2021-07-01 Thread GitBox
viirya commented on a change in pull request #32933: URL: https://github.com/apache/spark/pull/32933#discussion_r662564040 ## File path: sql/core/src/test/scala/org/apache/spark/sql/execution/streaming/state/RocksDBSuite.scala ## @@ -207,6 +273,133 @@ class RocksDBSuite

[GitHub] [spark] SparkQA commented on pull request #33070: [SPARK-35551][SQL] Handle the COUNT bug for lateral subqueries

2021-07-01 Thread GitBox
SparkQA commented on pull request #33070: URL: https://github.com/apache/spark/pull/33070#issuecomment-872519917 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/45047/ -- This is an automated message from the

[GitHub] [spark] kotlovs commented on pull request #33154: [SPARK-35949][CORE]Fixes bug for sparkContext stopped on client mode

2021-07-01 Thread GitBox
kotlovs commented on pull request #33154: URL: https://github.com/apache/spark/pull/33154#issuecomment-872517103 Thanks for detailed explanation. Now I understand your problem. You create a server inside the app and await user's requests. The similar thing occurs when ThriftServer is

[GitHub] [spark] SparkQA commented on pull request #33176: [SPARK-35975][SQL] New configuration `spark.sql.timestampType` for the default timestamp type

2021-07-01 Thread GitBox
SparkQA commented on pull request #33176: URL: https://github.com/apache/spark/pull/33176#issuecomment-872514118 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/45045/ -- This is an automated message from the

[GitHub] [spark] SparkQA commented on pull request #33160: [SPARK-35959][BUILD] Add a new Maven profile "no-shaded-hadoop-client" for Hadoop versions older than 3.2.2/3.3.1

2021-07-01 Thread GitBox
SparkQA commented on pull request #33160: URL: https://github.com/apache/spark/pull/33160#issuecomment-872509503 **[Test build #140537 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140537/testReport)** for PR 33160 at commit

[GitHub] [spark] SparkQA commented on pull request #33177: [SPARK-35855][SQL] Use CheckOverflowInSum for Average

2021-07-01 Thread GitBox
SparkQA commented on pull request #33177: URL: https://github.com/apache/spark/pull/33177#issuecomment-872509496 **[Test build #140536 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140536/testReport)** for PR 33177 at commit

[GitHub] [spark] karenfeng opened a new pull request #33177: [SPARK-35855][SQL] Use CheckOverflowInSum for Average

2021-07-01 Thread GitBox
karenfeng opened a new pull request #33177: URL: https://github.com/apache/spark/pull/33177 ### What changes were proposed in this pull request? Fixes decimal overflow issues for decimal average in ANSI mode, so that overflows throw an exception rather than returning null.

[GitHub] [spark] SparkQA commented on pull request #32921: [SPARK-35779][SQL] Dynamic filtering for Data Source V2

2021-07-01 Thread GitBox
SparkQA commented on pull request #32921: URL: https://github.com/apache/spark/pull/32921#issuecomment-872503244 **[Test build #140535 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140535/testReport)** for PR 32921 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33160: [SPARK-35959][BUILD] Add a new Maven profile "no-shaded-hadoop-client" for Hadoop versions older than 3.2.2/3.3.1

2021-07-01 Thread GitBox
AmplabJenkins removed a comment on pull request #33160: URL: https://github.com/apache/spark/pull/33160#issuecomment-872501806 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/45044/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33176: [SPARK-35975][SQL] New configuration `spark.sql.timestampType` for the default timestamp type

2021-07-01 Thread GitBox
AmplabJenkins removed a comment on pull request #33176: URL: https://github.com/apache/spark/pull/33176#issuecomment-872501807 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/45043/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33164: [SPARK-35958][CORE] Refactor SparkError.scala to SparkThrowable.java

2021-07-01 Thread GitBox
AmplabJenkins removed a comment on pull request #33164: URL: https://github.com/apache/spark/pull/33164#issuecomment-872501808 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/45046/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33172: [SPARK-35968][SQL] Make sure partitions are not too small in AQE partition coalescing

2021-07-01 Thread GitBox
AmplabJenkins removed a comment on pull request #33172: URL: https://github.com/apache/spark/pull/33172#issuecomment-872501805 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/140525/

[GitHub] [spark] AmplabJenkins commented on pull request #33176: [SPARK-35975][SQL] New configuration `spark.sql.timestampType` for the default timestamp type

2021-07-01 Thread GitBox
AmplabJenkins commented on pull request #33176: URL: https://github.com/apache/spark/pull/33176#issuecomment-872501807 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/45043/ --

[GitHub] [spark] AmplabJenkins commented on pull request #33164: [SPARK-35958][CORE] Refactor SparkError.scala to SparkThrowable.java

2021-07-01 Thread GitBox
AmplabJenkins commented on pull request #33164: URL: https://github.com/apache/spark/pull/33164#issuecomment-872501808 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/45046/ --

[GitHub] [spark] AmplabJenkins commented on pull request #33160: [SPARK-35959][BUILD] Add a new Maven profile "no-shaded-hadoop-client" for Hadoop versions older than 3.2.2/3.3.1

2021-07-01 Thread GitBox
AmplabJenkins commented on pull request #33160: URL: https://github.com/apache/spark/pull/33160#issuecomment-872501806 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/45044/ --

[GitHub] [spark] AmplabJenkins commented on pull request #33172: [SPARK-35968][SQL] Make sure partitions are not too small in AQE partition coalescing

2021-07-01 Thread GitBox
AmplabJenkins commented on pull request #33172: URL: https://github.com/apache/spark/pull/33172#issuecomment-872501805 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/140525/ -- This

[GitHub] [spark] SparkQA commented on pull request #33070: [SPARK-35551][SQL] Handle the COUNT bug for lateral subqueries

2021-07-01 Thread GitBox
SparkQA commented on pull request #33070: URL: https://github.com/apache/spark/pull/33070#issuecomment-872500134 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/45047/ -- This is an automated message from the Apache

[GitHub] [spark] SparkQA removed a comment on pull request #33172: [SPARK-35968][SQL] Make sure partitions are not too small in AQE partition coalescing

2021-07-01 Thread GitBox
SparkQA removed a comment on pull request #33172: URL: https://github.com/apache/spark/pull/33172#issuecomment-872349676 **[Test build #140525 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140525/testReport)** for PR 33172 at commit

[GitHub] [spark] SparkQA commented on pull request #33172: [SPARK-35968][SQL] Make sure partitions are not too small in AQE partition coalescing

2021-07-01 Thread GitBox
SparkQA commented on pull request #33172: URL: https://github.com/apache/spark/pull/33172#issuecomment-872498952 **[Test build #140525 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140525/testReport)** for PR 33172 at commit

[GitHub] [spark] aokolnychyi commented on a change in pull request #32921: [SPARK-35779][SQL] Dynamic filtering for Data Source V2

2021-07-01 Thread GitBox
aokolnychyi commented on a change in pull request #32921: URL: https://github.com/apache/spark/pull/32921#discussion_r662543704 ## File path: sql/catalyst/src/test/scala/org/apache/spark/sql/connector/catalog/InMemoryTable.scala ## @@ -245,21 +246,76 @@ class InMemoryTable(

[GitHub] [spark] SparkQA commented on pull request #33176: [SPARK-35975][SQL] New configuration `spark.sql.timestampType` for the default timestamp type

2021-07-01 Thread GitBox
SparkQA commented on pull request #33176: URL: https://github.com/apache/spark/pull/33176#issuecomment-872495652 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/45045/ -- This is an automated message from the Apache

[GitHub] [spark] aokolnychyi commented on a change in pull request #32921: [SPARK-35779][SQL] Dynamic filtering for Data Source V2

2021-07-01 Thread GitBox
aokolnychyi commented on a change in pull request #32921: URL: https://github.com/apache/spark/pull/32921#discussion_r662540019 ## File path: sql/catalyst/src/test/scala/org/apache/spark/sql/connector/catalog/InMemoryTable.scala ## @@ -245,21 +246,76 @@ class InMemoryTable(

[GitHub] [spark] SparkQA commented on pull request #33164: [SPARK-35958][CORE] Refactor SparkError.scala to SparkThrowable.java

2021-07-01 Thread GitBox
SparkQA commented on pull request #33164: URL: https://github.com/apache/spark/pull/33164#issuecomment-872494529 Kubernetes integration test unable to build dist. exiting with code: 1 URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/45046/ --

[GitHub] [spark] aokolnychyi commented on a change in pull request #32921: [SPARK-35779][SQL] Dynamic filtering for Data Source V2

2021-07-01 Thread GitBox
aokolnychyi commented on a change in pull request #32921: URL: https://github.com/apache/spark/pull/32921#discussion_r662536240 ## File path: sql/catalyst/src/test/scala/org/apache/spark/sql/connector/catalog/InMemoryTable.scala ## @@ -245,21 +246,76 @@ class InMemoryTable(

[GitHub] [spark] aokolnychyi commented on a change in pull request #32921: [SPARK-35779][SQL] Dynamic filtering for Data Source V2

2021-07-01 Thread GitBox
aokolnychyi commented on a change in pull request #32921: URL: https://github.com/apache/spark/pull/32921#discussion_r662536165 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/DataSourceStrategy.scala ## @@ -631,6 +631,25 @@ object

[GitHub] [spark] q2w commented on pull request #32902: [SPARK-35754][CORE] Add config to put migrating blocks on disk only

2021-07-01 Thread GitBox
q2w commented on pull request #32902: URL: https://github.com/apache/spark/pull/32902#issuecomment-872490128 > This would be inconsistent with the expectation of the storage level set on the RDD/DF. > Instead, what we can do is prefer disk in case it is available as an option (if

[GitHub] [spark] SparkQA commented on pull request #33160: [SPARK-35959][BUILD] Add a new Maven profile "no-shaded-hadoop-client" for Hadoop versions older than 3.2.2/3.3.1

2021-07-01 Thread GitBox
SparkQA commented on pull request #33160: URL: https://github.com/apache/spark/pull/33160#issuecomment-872489465 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/45044/ -- This is an automated message from the

[GitHub] [spark] aokolnychyi commented on a change in pull request #32921: [SPARK-35779][SQL] Dynamic filtering for Data Source V2

2021-07-01 Thread GitBox
aokolnychyi commented on a change in pull request #32921: URL: https://github.com/apache/spark/pull/32921#discussion_r662534188 ## File path: sql/catalyst/src/test/scala/org/apache/spark/sql/connector/catalog/InMemoryTable.scala ## @@ -245,21 +246,76 @@ class InMemoryTable(

[GitHub] [spark] aokolnychyi commented on a change in pull request #32921: [SPARK-35779][SQL] Dynamic filtering for Data Source V2

2021-07-01 Thread GitBox
aokolnychyi commented on a change in pull request #32921: URL: https://github.com/apache/spark/pull/32921#discussion_r662532114 ## File path: sql/catalyst/src/test/scala/org/apache/spark/sql/connector/catalog/InMemoryTable.scala ## @@ -245,21 +246,76 @@ class InMemoryTable(

[GitHub] [spark] SparkQA commented on pull request #33176: [SPARK-35975][SQL] New configuration `spark.sql.timestampType` for the default timestamp type

2021-07-01 Thread GitBox
SparkQA commented on pull request #33176: URL: https://github.com/apache/spark/pull/33176#issuecomment-872483003 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/45043/ -- This is an automated message from the

[GitHub] [spark] SparkQA removed a comment on pull request #33140: [SPARK-35881][SQL] Add support for columnar execution of final query stage in AdaptiveSparkPlanExec

2021-07-01 Thread GitBox
SparkQA removed a comment on pull request #33140: URL: https://github.com/apache/spark/pull/33140#issuecomment-872304365 **[Test build #140524 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140524/testReport)** for PR 33140 at commit

[GitHub] [spark] aokolnychyi commented on a change in pull request #32921: [SPARK-35779][SQL] Dynamic filtering for Data Source V2

2021-07-01 Thread GitBox
aokolnychyi commented on a change in pull request #32921: URL: https://github.com/apache/spark/pull/32921#discussion_r662523099 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/DataSourceStrategy.scala ## @@ -631,6 +631,25 @@ object

[GitHub] [spark] aokolnychyi commented on a change in pull request #32921: [SPARK-35779][SQL] Dynamic filtering for Data Source V2

2021-07-01 Thread GitBox
aokolnychyi commented on a change in pull request #32921: URL: https://github.com/apache/spark/pull/32921#discussion_r662522905 ## File path: sql/catalyst/src/test/scala/org/apache/spark/sql/connector/catalog/InMemoryTable.scala ## @@ -245,21 +246,76 @@ class InMemoryTable(

[GitHub] [spark] AmplabJenkins commented on pull request #33140: [SPARK-35881][SQL] Add support for columnar execution of final query stage in AdaptiveSparkPlanExec

2021-07-01 Thread GitBox
AmplabJenkins commented on pull request #33140: URL: https://github.com/apache/spark/pull/33140#issuecomment-872474797 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/140524/ -- This

[GitHub] [spark] SparkQA commented on pull request #33140: [SPARK-35881][SQL] Add support for columnar execution of final query stage in AdaptiveSparkPlanExec

2021-07-01 Thread GitBox
SparkQA commented on pull request #33140: URL: https://github.com/apache/spark/pull/33140#issuecomment-872473469 **[Test build #140524 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140524/testReport)** for PR 33140 at commit

[GitHub] [spark] viirya commented on pull request #32921: [SPARK-35779][SQL] Dynamic filtering for Data Source V2

2021-07-01 Thread GitBox
viirya commented on pull request #32921: URL: https://github.com/apache/spark/pull/32921#issuecomment-872472840 Thank you, @cloud-fan! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [spark] zsxwing commented on a change in pull request #33093: [SPARK-35897][SS] Support user defined initial state with flatMapGroupsWithState in Structured Streaming

2021-07-01 Thread GitBox
zsxwing commented on a change in pull request #33093: URL: https://github.com/apache/spark/pull/33093#discussion_r662519707 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/SparkStrategies.scala ## @@ -561,11 +561,13 @@ abstract class SparkStrategies

[GitHub] [spark] cloud-fan commented on a change in pull request #33070: [SPARK-35551][SQL] Handle the COUNT bug for lateral subqueries

2021-07-01 Thread GitBox
cloud-fan commented on a change in pull request #33070: URL: https://github.com/apache/spark/pull/33070#discussion_r662519357 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/DecorrelateInnerQuery.scala ## @@ -212,14 +214,38 @@ object

[GitHub] [spark] SparkQA commented on pull request #33070: [SPARK-35551][SQL] Handle the COUNT bug for lateral subqueries

2021-07-01 Thread GitBox
SparkQA commented on pull request #33070: URL: https://github.com/apache/spark/pull/33070#issuecomment-872470969 **[Test build #140534 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140534/testReport)** for PR 33070 at commit

[GitHub] [spark] cloud-fan commented on a change in pull request #32944: [SPARK-35794][SQL] Allow custom plugin for AQE cost evaluator

2021-07-01 Thread GitBox
cloud-fan commented on a change in pull request #32944: URL: https://github.com/apache/spark/pull/32944#discussion_r662517806 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/adaptive/simpleCosting.scala ## @@ -38,7 +38,7 @@ case class SimpleCost(value:

[GitHub] [spark] cloud-fan commented on a change in pull request #32944: [SPARK-35794][SQL] Allow custom plugin for AQE cost evaluator

2021-07-01 Thread GitBox
cloud-fan commented on a change in pull request #32944: URL: https://github.com/apache/spark/pull/32944#discussion_r662517185 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/adaptive/costing.scala ## @@ -17,16 +17,32 @@ package

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33070: [SPARK-35551][SQL] Handle the COUNT bug for lateral subqueries

2021-07-01 Thread GitBox
AmplabJenkins removed a comment on pull request #33070: URL: https://github.com/apache/spark/pull/33070#issuecomment-872182395 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/140492/

[GitHub] [spark] cloud-fan commented on a change in pull request #32944: [SPARK-35794][SQL] Allow custom plugin for AQE cost evaluator

2021-07-01 Thread GitBox
cloud-fan commented on a change in pull request #32944: URL: https://github.com/apache/spark/pull/32944#discussion_r662517185 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/adaptive/costing.scala ## @@ -17,16 +17,32 @@ package

[GitHub] [spark] SparkQA commented on pull request #33164: [SPARK-35958][CORE] Refactor SparkError.scala to SparkThrowable.java

2021-07-01 Thread GitBox
SparkQA commented on pull request #33164: URL: https://github.com/apache/spark/pull/33164#issuecomment-872468576 **[Test build #140533 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140533/testReport)** for PR 33164 at commit

[GitHub] [spark] SparkQA commented on pull request #33176: [SPARK-35975][SQL] New configuration `spark.sql.timestampType` for the default timestamp type

2021-07-01 Thread GitBox
SparkQA commented on pull request #33176: URL: https://github.com/apache/spark/pull/33176#issuecomment-872468498 **[Test build #140532 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140532/testReport)** for PR 33176 at commit

[GitHub] [spark] SparkQA commented on pull request #33160: [SPARK-35959][BUILD] Add a new Maven profile "no-shaded-hadoop-client" for Hadoop versions older than 3.2.2/3.3.1

2021-07-01 Thread GitBox
SparkQA commented on pull request #33160: URL: https://github.com/apache/spark/pull/33160#issuecomment-872468183 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/45044/ -- This is an automated message from the Apache

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33172: [SPARK-35968][SQL] Make sure partitions are not too small in AQE partition coalescing

2021-07-01 Thread GitBox
AmplabJenkins removed a comment on pull request #33172: URL: https://github.com/apache/spark/pull/33172#issuecomment-872466517 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33160: [SPARK-35959][BUILD] Add a new Maven profile "no-shaded-hadoop-client" for Hadoop versions older than 3.2.2/3.3.1

2021-07-01 Thread GitBox
AmplabJenkins removed a comment on pull request #33160: URL: https://github.com/apache/spark/pull/33160#issuecomment-872100431 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33170: [SPARK-35967][SQL] Update nullability based on column statistics

2021-07-01 Thread GitBox
AmplabJenkins removed a comment on pull request #33170: URL: https://github.com/apache/spark/pull/33170#issuecomment-872315783 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32959: [SPARK-35780][SQL] Support DATE/TIMESTAMP literals across the full range

2021-07-01 Thread GitBox
AmplabJenkins removed a comment on pull request #32959: URL: https://github.com/apache/spark/pull/32959#issuecomment-872441521 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33095: [SPARK-35339][PYTHON] Improve unit tests for data-type-based basic operations

2021-07-01 Thread GitBox
AmplabJenkins removed a comment on pull request #33095: URL: https://github.com/apache/spark/pull/33095#issuecomment-872402017 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [spark] AmplabJenkins commented on pull request #32959: [SPARK-35780][SQL] Support DATE/TIMESTAMP literals across the full range

2021-07-01 Thread GitBox
AmplabJenkins commented on pull request #32959: URL: https://github.com/apache/spark/pull/32959#issuecomment-872466520 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/45040/ --

[GitHub] [spark] AmplabJenkins commented on pull request #33170: [SPARK-35967][SQL] Update nullability based on column statistics

2021-07-01 Thread GitBox
AmplabJenkins commented on pull request #33170: URL: https://github.com/apache/spark/pull/33170#issuecomment-872466521 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/140520/ -- This

[GitHub] [spark] AmplabJenkins commented on pull request #33172: [SPARK-35968][SQL] Make sure partitions are not too small in AQE partition coalescing

2021-07-01 Thread GitBox
AmplabJenkins commented on pull request #33172: URL: https://github.com/apache/spark/pull/33172#issuecomment-872466518 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

[GitHub] [spark] AmplabJenkins commented on pull request #33095: [SPARK-35339][PYTHON] Improve unit tests for data-type-based basic operations

2021-07-01 Thread GitBox
AmplabJenkins commented on pull request #33095: URL: https://github.com/apache/spark/pull/33095#issuecomment-872466515 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/45041/ --

[GitHub] [spark] AmplabJenkins commented on pull request #33160: [SPARK-35959][BUILD] Add a new Maven profile "no-shaded-hadoop-client" for Hadoop versions older than 3.2.2/3.3.1

2021-07-01 Thread GitBox
AmplabJenkins commented on pull request #33160: URL: https://github.com/apache/spark/pull/33160#issuecomment-872466516 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/140531/ -- This

[GitHub] [spark] cloud-fan commented on a change in pull request #32944: [SPARK-35794][SQL] Allow custom plugin for AQE cost evaluator

2021-07-01 Thread GitBox
cloud-fan commented on a change in pull request #32944: URL: https://github.com/apache/spark/pull/32944#discussion_r662513062 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala ## @@ -642,6 +642,13 @@ object SQLConf {

[GitHub] [spark] xkrogen commented on pull request #33153: [SPARK-35951][DOCS] Add since versions for Avro options in Documentation

2021-07-01 Thread GitBox
xkrogen commented on pull request #33153: URL: https://github.com/apache/spark/pull/33153#issuecomment-872464219 Great idea, thx @gengliangwang ! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [spark] cloud-fan commented on a change in pull request #32921: [SPARK-35779][SQL] Dynamic filtering for Data Source V2

2021-07-01 Thread GitBox
cloud-fan commented on a change in pull request #32921: URL: https://github.com/apache/spark/pull/32921#discussion_r662511594 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/DataSourceStrategy.scala ## @@ -631,6 +631,25 @@ object

[GitHub] [spark] cloud-fan commented on a change in pull request #32921: [SPARK-35779][SQL] Dynamic filtering for Data Source V2

2021-07-01 Thread GitBox
cloud-fan commented on a change in pull request #32921: URL: https://github.com/apache/spark/pull/32921#discussion_r662510629 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/DataSourceStrategy.scala ## @@ -631,6 +631,25 @@ object

[GitHub] [spark] SparkQA commented on pull request #33176: [SPARK-35975][SQL] New configuration `spark.sql.timestampType` for the default timestamp type

2021-07-01 Thread GitBox
SparkQA commented on pull request #33176: URL: https://github.com/apache/spark/pull/33176#issuecomment-872461862 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/45043/ -- This is an automated message from the Apache

[GitHub] [spark] cloud-fan commented on a change in pull request #32921: [SPARK-35779][SQL] Dynamic filtering for Data Source V2

2021-07-01 Thread GitBox
cloud-fan commented on a change in pull request #32921: URL: https://github.com/apache/spark/pull/32921#discussion_r662509830 ## File path: sql/catalyst/src/test/scala/org/apache/spark/sql/connector/catalog/InMemoryTable.scala ## @@ -245,21 +246,76 @@ class InMemoryTable(

[GitHub] [spark] SparkQA removed a comment on pull request #33172: [SPARK-35968][SQL] Make sure partitions are not too small in AQE partition coalescing

2021-07-01 Thread GitBox
SparkQA removed a comment on pull request #33172: URL: https://github.com/apache/spark/pull/33172#issuecomment-872304320 **[Test build #140523 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140523/testReport)** for PR 33172 at commit

[GitHub] [spark] SparkQA commented on pull request #33172: [SPARK-35968][SQL] Make sure partitions are not too small in AQE partition coalescing

2021-07-01 Thread GitBox
SparkQA commented on pull request #33172: URL: https://github.com/apache/spark/pull/33172#issuecomment-872457323 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

[GitHub] [spark] MaxGekk commented on a change in pull request #33176: [SPARK-35975][SQL] New configuration `spark.sql.timestampType` for the default timestamp type

2021-07-01 Thread GitBox
MaxGekk commented on a change in pull request #33176: URL: https://github.com/apache/spark/pull/33176#discussion_r662503528 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala ## @@ -2820,6 +2821,24 @@ object SQLConf { .booleanConf

[GitHub] [spark] cloud-fan commented on a change in pull request #32861: [SPARK-35710] [SQL] Support DPP + AQE when there is no reused broadcast exchange

2021-07-01 Thread GitBox
cloud-fan commented on a change in pull request #32861: URL: https://github.com/apache/spark/pull/32861#discussion_r662501923 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/adaptive/PlanAdaptiveDynamicPruningFilters.scala ## @@ -62,8 +63,19 @@ case class

[GitHub] [spark] cloud-fan commented on a change in pull request #32861: [SPARK-35710] [SQL] Support DPP + AQE when there is no reused broadcast exchange

2021-07-01 Thread GitBox
cloud-fan commented on a change in pull request #32861: URL: https://github.com/apache/spark/pull/32861#discussion_r662501158 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/adaptive/PlanAdaptiveDynamicPruningFilters.scala ## @@ -62,8 +63,19 @@ case class

[GitHub] [spark] c21 commented on pull request #33168: [SPARK-35965][DOCS] Add doc for ORC nested column vectorized reader

2021-07-01 Thread GitBox
c21 commented on pull request #33168: URL: https://github.com/apache/spark/pull/33168#issuecomment-872449629 Thank you @HyukjinKwon and @dongjoon-hyun for review! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

[GitHub] [spark] SparkQA commented on pull request #32959: [SPARK-35780][SQL] Support DATE/TIMESTAMP literals across the full range

2021-07-01 Thread GitBox
SparkQA commented on pull request #32959: URL: https://github.com/apache/spark/pull/32959#issuecomment-872446685 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/45040/ -- This is an automated message from the

[GitHub] [spark] SparkQA commented on pull request #33095: [SPARK-35339][PYTHON] Improve unit tests for data-type-based basic operations

2021-07-01 Thread GitBox
SparkQA commented on pull request #33095: URL: https://github.com/apache/spark/pull/33095#issuecomment-872445946 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/45041/ -- This is an automated message from the

<    1   2   3   4   5   6   7   8   9   10   >