[GitHub] [spark] cloud-fan commented on a change in pull request #34291: [SPARK-37020][SQL] DS V2 LIMIT push down

2021-10-26 Thread GitBox
cloud-fan commented on a change in pull request #34291: URL: https://github.com/apache/spark/pull/34291#discussion_r737161972 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/V2ScanRelationPushDown.scala ## @@ -36,7 +36,7 @@ object V2ScanRela

[GitHub] [spark] itholic commented on a change in pull request #34389: [SPARK-37036][PYTHON] Add util function to raise advice warning for pandas API on Spark.

2021-10-26 Thread GitBox
itholic commented on a change in pull request #34389: URL: https://github.com/apache/spark/pull/34389#discussion_r737158831 ## File path: python/pyspark/pandas/utils.py ## @@ -958,6 +958,17 @@ def compare_allow_null( return left.isNull() | right.isNull() | comp(left, right

[GitHub] [spark] SparkQA commented on pull request #34399: [SPARK-37031][SQL][TESTS][FOLLOWUP] Add a missing test to DescribeNamespaceSuite

2021-10-26 Thread GitBox
SparkQA commented on pull request #34399: URL: https://github.com/apache/spark/pull/34399#issuecomment-952595013 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49117/ -- This is an automated message from the Apache

[GitHub] [spark] itholic commented on a change in pull request #34389: [SPARK-37036][PYTHON] Add util function to raise advice warning for pandas API on Spark.

2021-10-26 Thread GitBox
itholic commented on a change in pull request #34389: URL: https://github.com/apache/spark/pull/34389#discussion_r737157078 ## File path: python/pyspark/pandas/frame.py ## @@ -4530,6 +4539,10 @@ def to_table( index_col: Optional[Union[str, List[str]]] = None,

[GitHub] [spark] SparkQA commented on pull request #34248: [SPARK-36647][SQL][TESTS] Push down Aggregate (Min/Max/Count) for Parquet if filter is on partition col

2021-10-26 Thread GitBox
SparkQA commented on pull request #34248: URL: https://github.com/apache/spark/pull/34248#issuecomment-952591848 **[Test build #144636 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/144636/testReport)** for PR 34248 at commit [`1293ae0`](https://github.co

[GitHub] [spark] LuciferYang commented on a change in pull request #34313: [SPARK-37013][SQL] Forbid `%0$` usage explicitly to ensure `format_string` has same behavior when using Java 8 and Java 17

2021-10-26 Thread GitBox
LuciferYang commented on a change in pull request #34313: URL: https://github.com/apache/spark/pull/34313#discussion_r737154105 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/stringExpressions.scala ## @@ -1617,6 +1617,8 @@ case class Parse

[GitHub] [spark] SparkQA commented on pull request #34389: [SPARK-37036][PYTHON] Add util function to raise advice warning for pandas API on Spark.

2021-10-26 Thread GitBox
SparkQA commented on pull request #34389: URL: https://github.com/apache/spark/pull/34389#issuecomment-952591214 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49114/ -- This is an automated message from the A

[GitHub] [spark] AngersZhuuuu commented on a change in pull request #34380: [SPARK-37082][SQL] Implements histogram_numeric aggregation function which supports partial aggregation.

2021-10-26 Thread GitBox
AngersZh commented on a change in pull request #34380: URL: https://github.com/apache/spark/pull/34380#discussion_r737152698 ## File path: sql/catalyst/src/main/java/org/apache/spark/sql/util/NumericHistogram.java ## @@ -0,0 +1,288 @@ +/** + * Licensed to the Apache Softwa

[GitHub] [spark] AngersZhuuuu commented on a change in pull request #34380: [SPARK-37082][SQL] Implements histogram_numeric aggregation function which supports partial aggregation.

2021-10-26 Thread GitBox
AngersZh commented on a change in pull request #34380: URL: https://github.com/apache/spark/pull/34380#discussion_r737152698 ## File path: sql/catalyst/src/main/java/org/apache/spark/sql/util/NumericHistogram.java ## @@ -0,0 +1,288 @@ +/** + * Licensed to the Apache Softwa

[GitHub] [spark] ueshin commented on a change in pull request #34389: [SPARK-37036][PYTHON] Add util function to raise advice warning for pandas API on Spark.

2021-10-26 Thread GitBox
ueshin commented on a change in pull request #34389: URL: https://github.com/apache/spark/pull/34389#discussion_r737151018 ## File path: python/pyspark/pandas/utils.py ## @@ -958,6 +958,17 @@ def compare_allow_null( return left.isNull() | right.isNull() | comp(left, right)

[GitHub] [spark] LuciferYang edited a comment on pull request #34376: [SPARK-37105][TEST] Pass all UTs in `sql/hive` with Java 17

2021-10-26 Thread GitBox
LuciferYang edited a comment on pull request #34376: URL: https://github.com/apache/spark/pull/34376#issuecomment-952587999 @srowen @dongjoon-hyun I tried 4 versions of JDK to test ``` mvn clean install -DskipTests -pl common/network-common -am mvn test -pl common/network-common

[GitHub] [spark] LuciferYang commented on pull request #34376: [SPARK-37105][TEST] Pass all UTs in `sql/hive` with Java 17

2021-10-26 Thread GitBox
LuciferYang commented on pull request #34376: URL: https://github.com/apache/spark/pull/34376#issuecomment-952587999 @srowen @dongjoon-hyun I tried 4 versions of JDK on my M1: - Oracle 17 x64 version: passed - Oracle 17 arm64 version: passed - zulu 17 x64 version: Can reproduce

[GitHub] [spark] SparkQA commented on pull request #34389: [SPARK-37036][PYTHON] Add util function to raise advice warning for pandas API on Spark.

2021-10-26 Thread GitBox
SparkQA commented on pull request #34389: URL: https://github.com/apache/spark/pull/34389#issuecomment-952585076 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49113/ -- This is an automated message from the A

[GitHub] [spark] cloud-fan commented on a change in pull request #34313: [SPARK-37013][SQL] Forbid `%0$` usage explicitly to ensure `format_string` has same behavior when using Java 8 and Java 17

2021-10-26 Thread GitBox
cloud-fan commented on a change in pull request #34313: URL: https://github.com/apache/spark/pull/34313#discussion_r737144189 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/stringExpressions.scala ## @@ -1617,6 +1617,8 @@ case class ParseUr

[GitHub] [spark] cloud-fan commented on a change in pull request #34380: [SPARK-37082][SQL] Implements histogram_numeric aggregation function which supports partial aggregation.

2021-10-26 Thread GitBox
cloud-fan commented on a change in pull request #34380: URL: https://github.com/apache/spark/pull/34380#discussion_r737142979 ## File path: sql/catalyst/src/main/java/org/apache/spark/sql/util/NumericHistogram.java ## @@ -0,0 +1,288 @@ +/** + * Licensed to the Apache Software

[GitHub] [spark] SparkQA commented on pull request #34380: [SPARK-37082][SQL] Implements histogram_numeric aggregation function which supports partial aggregation.

2021-10-26 Thread GitBox
SparkQA commented on pull request #34380: URL: https://github.com/apache/spark/pull/34380#issuecomment-952580017 **[Test build #144648 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/144648/testReport)** for PR 34380 at commit [`c7a8c4b`](https://github.com

[GitHub] [spark] AngersZhuuuu commented on a change in pull request #34380: [SPARK-37082][SQL] Implements histogram_numeric aggregation function which supports partial aggregation.

2021-10-26 Thread GitBox
AngersZh commented on a change in pull request #34380: URL: https://github.com/apache/spark/pull/34380#discussion_r737139587 ## File path: sql/catalyst/src/main/java/org/apache/spark/sql/util/NumericHistogram.java ## @@ -0,0 +1,288 @@ +/** + * Licensed to the Apache Softwa

[GitHub] [spark] ulysses-you commented on pull request #34398: [SPARK-37125][SQL] Support AnsiInterval radix sort

2021-10-26 Thread GitBox
ulysses-you commented on pull request #34398: URL: https://github.com/apache/spark/pull/34398#issuecomment-952578220 cc @MaxGekk @yaooqinn @cloud-fan @srowen if you have time to take a look -- This is an automated message from the Apache Git Service. To respond to the message, please

[GitHub] [spark] SparkQA commented on pull request #34380: [SPARK-37082][SQL] Implements histogram_numeric aggregation function which supports partial aggregation.

2021-10-26 Thread GitBox
SparkQA commented on pull request #34380: URL: https://github.com/apache/spark/pull/34380#issuecomment-952578256 **[Test build #144647 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/144647/testReport)** for PR 34380 at commit [`8d4504d`](https://github.com

[GitHub] [spark] AngersZhuuuu commented on a change in pull request #34380: [SPARK-37082][SQL] Implements histogram_numeric aggregation function which supports partial aggregation.

2021-10-26 Thread GitBox
AngersZh commented on a change in pull request #34380: URL: https://github.com/apache/spark/pull/34380#discussion_r737138980 ## File path: sql/catalyst/src/main/java/org/apache/spark/sql/util/NumericHistogram.java ## @@ -0,0 +1,288 @@ +/** + * Licensed to the Apache Softwa

[GitHub] [spark] SparkQA commented on pull request #34397: [SPARK-36348][PYTHON][FOLLOWUP] Complete test_astype for index

2021-10-26 Thread GitBox
SparkQA commented on pull request #34397: URL: https://github.com/apache/spark/pull/34397#issuecomment-952576721 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49115/ -- This is an automated message from the Apache

[GitHub] [spark] SparkQA commented on pull request #34291: [SPARK-37020][SQL] DS V2 LIMIT push down

2021-10-26 Thread GitBox
SparkQA commented on pull request #34291: URL: https://github.com/apache/spark/pull/34291#issuecomment-952576111 **[Test build #144646 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/144646/testReport)** for PR 34291 at commit [`86c020d`](https://github.co

[GitHub] [spark] AmplabJenkins commented on pull request #34291: [SPARK-37020][SQL] DS V2 LIMIT push down

2021-10-26 Thread GitBox
AmplabJenkins commented on pull request #34291: URL: https://github.com/apache/spark/pull/34291#issuecomment-952576144 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/144646/ -- This

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34291: [SPARK-37020][SQL] DS V2 LIMIT push down

2021-10-26 Thread GitBox
AmplabJenkins removed a comment on pull request #34291: URL: https://github.com/apache/spark/pull/34291#issuecomment-952576144 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/144646/ -

[GitHub] [spark] SparkQA removed a comment on pull request #34291: [SPARK-37020][SQL] DS V2 LIMIT push down

2021-10-26 Thread GitBox
SparkQA removed a comment on pull request #34291: URL: https://github.com/apache/spark/pull/34291#issuecomment-952569439 **[Test build #144646 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/144646/testReport)** for PR 34291 at commit [`86c020d`](https://gi

[GitHub] [spark] SparkQA commented on pull request #34291: [SPARK-37020][SQL] DS V2 LIMIT push down

2021-10-26 Thread GitBox
SparkQA commented on pull request #34291: URL: https://github.com/apache/spark/pull/34291#issuecomment-952569439 **[Test build #144646 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/144646/testReport)** for PR 34291 at commit [`86c020d`](https://github.com

[GitHub] [spark] SparkQA commented on pull request #34399: [SPARK-37031][SQL][TESTS][FOLLOWUP] Add a missing test to DescribeNamespaceSuite

2021-10-26 Thread GitBox
SparkQA commented on pull request #34399: URL: https://github.com/apache/spark/pull/34399#issuecomment-952569275 **[Test build #144645 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/144645/testReport)** for PR 34399 at commit [`4110c95`](https://github.com

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34291: [SPARK-37020][SQL] DS V2 LIMIT push down

2021-10-26 Thread GitBox
AmplabJenkins removed a comment on pull request #34291: URL: https://github.com/apache/spark/pull/34291#issuecomment-952568692 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/49116/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34398: [SPARK-37125][SQL] Support AnsiInterval radix sort

2021-10-26 Thread GitBox
AmplabJenkins removed a comment on pull request #34398: URL: https://github.com/apache/spark/pull/34398#issuecomment-952568689 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/49112/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34389: [SPARK-37036][PYTHON] Add util function to raise advice warning for pandas API on Spark.

2021-10-26 Thread GitBox
AmplabJenkins removed a comment on pull request #34389: URL: https://github.com/apache/spark/pull/34389#issuecomment-952568690 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34397: [SPARK-36348][PYTHON][FOLLOWUP] Complete test_astype for index

2021-10-26 Thread GitBox
AmplabJenkins removed a comment on pull request #34397: URL: https://github.com/apache/spark/pull/34397#issuecomment-952568688 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/144644/ -

[GitHub] [spark] AmplabJenkins commented on pull request #34397: [SPARK-36348][PYTHON][FOLLOWUP] Complete test_astype for index

2021-10-26 Thread GitBox
AmplabJenkins commented on pull request #34397: URL: https://github.com/apache/spark/pull/34397#issuecomment-952568688 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/144644/ -- This

[GitHub] [spark] AmplabJenkins commented on pull request #34291: [SPARK-37020][SQL] DS V2 LIMIT push down

2021-10-26 Thread GitBox
AmplabJenkins commented on pull request #34291: URL: https://github.com/apache/spark/pull/34291#issuecomment-952568692 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/49116/ -- T

[GitHub] [spark] AmplabJenkins commented on pull request #34398: [SPARK-37125][SQL] Support AnsiInterval radix sort

2021-10-26 Thread GitBox
AmplabJenkins commented on pull request #34398: URL: https://github.com/apache/spark/pull/34398#issuecomment-952568689 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/49112/ -- T

[GitHub] [spark] AmplabJenkins commented on pull request #34389: [SPARK-37036][PYTHON] Add util function to raise advice warning for pandas API on Spark.

2021-10-26 Thread GitBox
AmplabJenkins commented on pull request #34389: URL: https://github.com/apache/spark/pull/34389#issuecomment-952568690 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To un

[GitHub] [spark] SparkQA commented on pull request #34389: [SPARK-37036][PYTHON] Add util function to raise advice warning for pandas API on Spark.

2021-10-26 Thread GitBox
SparkQA commented on pull request #34389: URL: https://github.com/apache/spark/pull/34389#issuecomment-952565050 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49114/ -- This is an automated message from the Apache

[GitHub] [spark] SparkQA removed a comment on pull request #34397: [SPARK-36348][PYTHON][FOLLOWUP] Complete test_astype for index

2021-10-26 Thread GitBox
SparkQA removed a comment on pull request #34397: URL: https://github.com/apache/spark/pull/34397#issuecomment-952549477 **[Test build #144644 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/144644/testReport)** for PR 34397 at commit [`4ab3fee`](https://gi

[GitHub] [spark] SparkQA removed a comment on pull request #34389: [SPARK-37036][PYTHON] Add util function to raise advice warning for pandas API on Spark.

2021-10-26 Thread GitBox
SparkQA removed a comment on pull request #34389: URL: https://github.com/apache/spark/pull/34389#issuecomment-952543868 **[Test build #144643 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/144643/testReport)** for PR 34389 at commit [`fc950ce`](https://gi

[GitHub] [spark] LuciferYang commented on a change in pull request #34313: [SPARK-37013][SQL] Forbid `%0$` usage explicitly to ensure `format_string` has same behavior when using Java 8 and Java 17

2021-10-26 Thread GitBox
LuciferYang commented on a change in pull request #34313: URL: https://github.com/apache/spark/pull/34313#discussion_r737128931 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/stringExpressions.scala ## @@ -1617,6 +1617,8 @@ case class Parse

[GitHub] [spark] LuciferYang commented on a change in pull request #34313: [SPARK-37013][SQL] Forbid `%0$` usage explicitly to ensure `format_string` has same behavior when using Java 8 and Java 17

2021-10-26 Thread GitBox
LuciferYang commented on a change in pull request #34313: URL: https://github.com/apache/spark/pull/34313#discussion_r737128931 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/stringExpressions.scala ## @@ -1617,6 +1617,8 @@ case class Parse

[GitHub] [spark] HyukjinKwon closed pull request #34397: [SPARK-36348][PYTHON][FOLLOWUP] Complete test_astype for index

2021-10-26 Thread GitBox
HyukjinKwon closed pull request #34397: URL: https://github.com/apache/spark/pull/34397 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-un

[GitHub] [spark] HyukjinKwon commented on pull request #34397: [SPARK-36348][PYTHON][FOLLOWUP] Complete test_astype for index

2021-10-26 Thread GitBox
HyukjinKwon commented on pull request #34397: URL: https://github.com/apache/spark/pull/34397#issuecomment-952561659 Merged to master. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specifi

[GitHub] [spark] SparkQA commented on pull request #34397: [SPARK-36348][PYTHON][FOLLOWUP] Complete test_astype for index

2021-10-26 Thread GitBox
SparkQA commented on pull request #34397: URL: https://github.com/apache/spark/pull/34397#issuecomment-952561324 **[Test build #144644 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/144644/testReport)** for PR 34397 at commit [`4ab3fee`](https://github.co

[GitHub] [spark] SparkQA commented on pull request #34389: [SPARK-37036][PYTHON] Add util function to raise advice warning for pandas API on Spark.

2021-10-26 Thread GitBox
SparkQA commented on pull request #34389: URL: https://github.com/apache/spark/pull/34389#issuecomment-952560866 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49113/ -- This is an automated message from the Apache

[GitHub] [spark] LuciferYang commented on a change in pull request #34313: [SPARK-37013][SQL] Forbid `%0$` usage explicitly to ensure `format_string` has same behavior when using Java 8 and Java 17

2021-10-26 Thread GitBox
LuciferYang commented on a change in pull request #34313: URL: https://github.com/apache/spark/pull/34313#discussion_r737118939 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/stringExpressions.scala ## @@ -1617,6 +1617,8 @@ case class Parse

[GitHub] [spark] SparkQA commented on pull request #34389: [SPARK-37036][PYTHON] Add util function to raise advice warning for pandas API on Spark.

2021-10-26 Thread GitBox
SparkQA commented on pull request #34389: URL: https://github.com/apache/spark/pull/34389#issuecomment-952557157 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49111/ -- This is an automated message from the A

[GitHub] [spark] SparkQA commented on pull request #34389: [SPARK-37036][PYTHON] Add util function to raise advice warning for pandas API on Spark.

2021-10-26 Thread GitBox
SparkQA commented on pull request #34389: URL: https://github.com/apache/spark/pull/34389#issuecomment-952554723 **[Test build #144643 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/144643/testReport)** for PR 34389 at commit [`fc950ce`](https://github.co

[GitHub] [spark] SparkQA commented on pull request #34291: [SPARK-37020][SQL] DS V2 LIMIT push down

2021-10-26 Thread GitBox
SparkQA commented on pull request #34291: URL: https://github.com/apache/spark/pull/34291#issuecomment-952554611 Kubernetes integration test unable to build dist. exiting with code: 1 URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49116/ -- This

[GitHub] [spark] SparkQA commented on pull request #34398: [SPARK-37125][SQL] Support AnsiInterval radix sort

2021-10-26 Thread GitBox
SparkQA commented on pull request #34398: URL: https://github.com/apache/spark/pull/34398#issuecomment-952554028 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49112/ -- This is an automated message from the A

[GitHub] [spark] LuciferYang commented on a change in pull request #34313: [SPARK-37013][SQL] Forbid `%0$` usage explicitly to ensure `format_string` has same behavior when using Java 8 and Java 17

2021-10-26 Thread GitBox
LuciferYang commented on a change in pull request #34313: URL: https://github.com/apache/spark/pull/34313#discussion_r737118939 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/stringExpressions.scala ## @@ -1617,6 +1617,8 @@ case class Parse

[GitHub] [spark] LuciferYang commented on a change in pull request #34313: [SPARK-37013][SQL] Forbid `%0$` usage explicitly to ensure `format_string` has same behavior when using Java 8 and Java 17

2021-10-26 Thread GitBox
LuciferYang commented on a change in pull request #34313: URL: https://github.com/apache/spark/pull/34313#discussion_r737118939 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/stringExpressions.scala ## @@ -1617,6 +1617,8 @@ case class Parse

[GitHub] [spark] imback82 commented on a change in pull request #34399: [SPARK-37031][SQL][TESTS][FOLLOWUP] Add a missing test to DescribeNamespaceSuite

2021-10-26 Thread GitBox
imback82 commented on a change in pull request #34399: URL: https://github.com/apache/spark/pull/34399#discussion_r737116555 ## File path: sql/core/src/test/scala/org/apache/spark/sql/execution/command/v1/DescribeNamespaceSuite.scala ## @@ -50,6 +51,23 @@ trait DescribeNamespa

[GitHub] [spark] imback82 opened a new pull request #34399: [SPARK-37031][SQL][TESTS][FOLLOWUP] Add a missing test to DescribeNamespaceSuite

2021-10-26 Thread GitBox
imback82 opened a new pull request #34399: URL: https://github.com/apache/spark/pull/34399 ### What changes were proposed in this pull request? This PR proposes to add a missing test on "keeping the legacy output schema" to `DescribeNamespaceSuite`. (#31705 didn't seem to add

[GitHub] [spark] SparkQA commented on pull request #34397: [SPARK-36348][PYTHON][FOLLOWUP] Complete test_astype for index

2021-10-26 Thread GitBox
SparkQA commented on pull request #34397: URL: https://github.com/apache/spark/pull/34397#issuecomment-952549477 **[Test build #144644 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/144644/testReport)** for PR 34397 at commit [`4ab3fee`](https://github.com

[GitHub] [spark] viirya commented on a change in pull request #34298: [SPARK-34960][SQL] Aggregate push down for ORC

2021-10-26 Thread GitBox
viirya commented on a change in pull request #34298: URL: https://github.com/apache/spark/pull/34298#discussion_r737115431 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/AggregatePushDownUtils.scala ## @@ -0,0 +1,142 @@ +/* + * Licensed to the

[GitHub] [spark] HyukjinKwon commented on pull request #34397: [SPARK-36348][PYTHON][FOLLOWUP] Complete test_astype for index

2021-10-26 Thread GitBox
HyukjinKwon commented on pull request #34397: URL: https://github.com/apache/spark/pull/34397#issuecomment-952548229 retest this please -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specif

[GitHub] [spark] HyukjinKwon removed a comment on pull request #34397: [SPARK-36348][PYTHON][FOLLOWUP] Complete test_astype for index

2021-10-26 Thread GitBox
HyukjinKwon removed a comment on pull request #34397: URL: https://github.com/apache/spark/pull/34397#issuecomment-952511779 retest this pleasse -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to t

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34388: [SPARK-37115][SQL] HiveClientImpl should use shim to wrap all hive client calls

2021-10-26 Thread GitBox
AmplabJenkins removed a comment on pull request #34388: URL: https://github.com/apache/spark/pull/34388#issuecomment-952547318 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/144640/ -

[GitHub] [spark] AmplabJenkins commented on pull request #34388: [SPARK-37115][SQL] HiveClientImpl should use shim to wrap all hive client calls

2021-10-26 Thread GitBox
AmplabJenkins commented on pull request #34388: URL: https://github.com/apache/spark/pull/34388#issuecomment-952547318 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/144640/ -- This

[GitHub] [spark] SparkQA removed a comment on pull request #34388: [SPARK-37115][SQL] HiveClientImpl should use shim to wrap all hive client calls

2021-10-26 Thread GitBox
SparkQA removed a comment on pull request #34388: URL: https://github.com/apache/spark/pull/34388#issuecomment-952497190 **[Test build #144640 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/144640/testReport)** for PR 34388 at commit [`a888ae7`](https://gi

[GitHub] [spark] SparkQA commented on pull request #34388: [SPARK-37115][SQL] HiveClientImpl should use shim to wrap all hive client calls

2021-10-26 Thread GitBox
SparkQA commented on pull request #34388: URL: https://github.com/apache/spark/pull/34388#issuecomment-952546597 **[Test build #144640 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/144640/testReport)** for PR 34388 at commit [`a888ae7`](https://github.co

[GitHub] [spark] SparkQA commented on pull request #34389: [SPARK-37036][PYTHON] Add util function to raise advice warning for pandas API on Spark.

2021-10-26 Thread GitBox
SparkQA commented on pull request #34389: URL: https://github.com/apache/spark/pull/34389#issuecomment-952543868 **[Test build #144643 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/144643/testReport)** for PR 34389 at commit [`fc950ce`](https://github.com

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34388: [SPARK-37115][SQL] HiveClientImpl should use shim to wrap all hive client calls

2021-10-26 Thread GitBox
AmplabJenkins removed a comment on pull request #34388: URL: https://github.com/apache/spark/pull/34388#issuecomment-952542439 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/49110/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34298: [SPARK-34960][SQL] Aggregate push down for ORC

2021-10-26 Thread GitBox
AmplabJenkins removed a comment on pull request #34298: URL: https://github.com/apache/spark/pull/34298#issuecomment-952542508 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/144632/ -

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34397: [SPARK-36348][PYTHON][FOLLOWUP] Complete test_astype for index

2021-10-26 Thread GitBox
AmplabJenkins removed a comment on pull request #34397: URL: https://github.com/apache/spark/pull/34397#issuecomment-952542440 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/49109/

[GitHub] [spark] AmplabJenkins commented on pull request #34298: [SPARK-34960][SQL] Aggregate push down for ORC

2021-10-26 Thread GitBox
AmplabJenkins commented on pull request #34298: URL: https://github.com/apache/spark/pull/34298#issuecomment-952542508 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/144632/ -- This

[GitHub] [spark] AmplabJenkins commented on pull request #34388: [SPARK-37115][SQL] HiveClientImpl should use shim to wrap all hive client calls

2021-10-26 Thread GitBox
AmplabJenkins commented on pull request #34388: URL: https://github.com/apache/spark/pull/34388#issuecomment-952542439 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/49110/ -- T

[GitHub] [spark] AmplabJenkins commented on pull request #34397: [SPARK-36348][PYTHON][FOLLOWUP] Complete test_astype for index

2021-10-26 Thread GitBox
AmplabJenkins commented on pull request #34397: URL: https://github.com/apache/spark/pull/34397#issuecomment-952542440 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/49109/ -- T

[GitHub] [spark] SparkQA removed a comment on pull request #34298: [SPARK-34960][SQL] Aggregate push down for ORC

2021-10-26 Thread GitBox
SparkQA removed a comment on pull request #34298: URL: https://github.com/apache/spark/pull/34298#issuecomment-952424889 **[Test build #144632 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/144632/testReport)** for PR 34298 at commit [`9b8b9ef`](https://gi

[GitHub] [spark] SparkQA commented on pull request #34298: [SPARK-34960][SQL] Aggregate push down for ORC

2021-10-26 Thread GitBox
SparkQA commented on pull request #34298: URL: https://github.com/apache/spark/pull/34298#issuecomment-952541525 **[Test build #144632 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/144632/testReport)** for PR 34298 at commit [`9b8b9ef`](https://github.co

[GitHub] [spark] SparkQA commented on pull request #34389: [SPARK-37036][PYTHON] Add util function to raise advice warning for pandas API on Spark.

2021-10-26 Thread GitBox
SparkQA commented on pull request #34389: URL: https://github.com/apache/spark/pull/34389#issuecomment-952538900 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49111/ -- This is an automated message from the Apache

[GitHub] [spark] SparkQA commented on pull request #34398: [SPARK-37125][SQL] Support AnsiInterval radix sort

2021-10-26 Thread GitBox
SparkQA commented on pull request #34398: URL: https://github.com/apache/spark/pull/34398#issuecomment-952538865 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49112/ -- This is an automated message from the Apache

[GitHub] [spark] SparkQA commented on pull request #34388: [SPARK-37115][SQL] HiveClientImpl should use shim to wrap all hive client calls

2021-10-26 Thread GitBox
SparkQA commented on pull request #34388: URL: https://github.com/apache/spark/pull/34388#issuecomment-952537568 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49110/ -- This is an automated message from the A

[GitHub] [spark] SparkQA commented on pull request #34397: [SPARK-36348][PYTHON][FOLLOWUP] Complete test_astype for index

2021-10-26 Thread GitBox
SparkQA commented on pull request #34397: URL: https://github.com/apache/spark/pull/34397#issuecomment-952532797 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49109/ -- This is an automated message from the A

[GitHub] [spark] cloud-fan closed pull request #34305: [SPARK-37031][SQL][TESTS] Unify v1 and v2 DESCRIBE NAMESPACE tests

2021-10-26 Thread GitBox
cloud-fan closed pull request #34305: URL: https://github.com/apache/spark/pull/34305 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsu

[GitHub] [spark] cloud-fan commented on pull request #34305: [SPARK-37031][SQL][TESTS] Unify v1 and v2 DESCRIBE NAMESPACE tests

2021-10-26 Thread GitBox
cloud-fan commented on pull request #34305: URL: https://github.com/apache/spark/pull/34305#issuecomment-952523239 thanks, merging to master! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [spark] SparkQA commented on pull request #34398: [SPARK-37125][SQL] Support AnsiInterval radix sort

2021-10-26 Thread GitBox
SparkQA commented on pull request #34398: URL: https://github.com/apache/spark/pull/34398#issuecomment-952520197 **[Test build #144642 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/144642/testReport)** for PR 34398 at commit [`d472055`](https://github.com

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34375: [SPARK-37107][PYTHON] Inline type hints for python/pyspark/status.py

2021-10-26 Thread GitBox
AmplabJenkins removed a comment on pull request #34375: URL: https://github.com/apache/spark/pull/34375#issuecomment-952519237 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/49105/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34389: [SPARK-37036][PYTHON] Add util function to raise advice warning for pandas API on Spark.

2021-10-26 Thread GitBox
AmplabJenkins removed a comment on pull request #34389: URL: https://github.com/apache/spark/pull/34389#issuecomment-952505305 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34397: [SPARK-36348][PYTHON][FOLLOWUP] Complete test_astype for index

2021-10-26 Thread GitBox
AmplabJenkins removed a comment on pull request #34397: URL: https://github.com/apache/spark/pull/34397#issuecomment-952519239 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/144639/ -

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34248: [SPARK-36647][SQL][TESTS] Push down Aggregate (Min/Max/Count) for Parquet if filter is on partition col

2021-10-26 Thread GitBox
AmplabJenkins removed a comment on pull request #34248: URL: https://github.com/apache/spark/pull/34248#issuecomment-952519241 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/49106/

[GitHub] [spark] AmplabJenkins commented on pull request #34248: [SPARK-36647][SQL][TESTS] Push down Aggregate (Min/Max/Count) for Parquet if filter is on partition col

2021-10-26 Thread GitBox
AmplabJenkins commented on pull request #34248: URL: https://github.com/apache/spark/pull/34248#issuecomment-952519241 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/49106/ -- T

[GitHub] [spark] AmplabJenkins commented on pull request #34397: [SPARK-36348][PYTHON][FOLLOWUP] Complete test_astype for index

2021-10-26 Thread GitBox
AmplabJenkins commented on pull request #34397: URL: https://github.com/apache/spark/pull/34397#issuecomment-952519239 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/144639/ -- This

[GitHub] [spark] AmplabJenkins commented on pull request #34389: [SPARK-37036][PYTHON] Add util function to raise advice warning for pandas API on Spark.

2021-10-26 Thread GitBox
AmplabJenkins commented on pull request #34389: URL: https://github.com/apache/spark/pull/34389#issuecomment-952519246 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To un

[GitHub] [spark] AmplabJenkins commented on pull request #34375: [SPARK-37107][PYTHON] Inline type hints for python/pyspark/status.py

2021-10-26 Thread GitBox
AmplabJenkins commented on pull request #34375: URL: https://github.com/apache/spark/pull/34375#issuecomment-952519237 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/49105/ -- T

[GitHub] [spark] SparkQA commented on pull request #34389: [SPARK-37036][PYTHON] Add util function to raise advice warning for pandas API on Spark.

2021-10-26 Thread GitBox
SparkQA commented on pull request #34389: URL: https://github.com/apache/spark/pull/34389#issuecomment-952518768 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49108/ -- This is an automated message from the A

[GitHub] [spark] ulysses-you opened a new pull request #34398: [SPARK-37125][SQL] Support AnsiInterval radix sort

2021-10-26 Thread GitBox
ulysses-you opened a new pull request #34398: URL: https://github.com/apache/spark/pull/34398 ### What changes were proposed in this pull request? - Make `AnsiInterval` data type support radix sort in SQL. - Enhance the `SortSuite` by disable radix. ### Why are the c

[GitHub] [spark] SparkQA commented on pull request #34375: [SPARK-37107][PYTHON] Inline type hints for python/pyspark/status.py

2021-10-26 Thread GitBox
SparkQA commented on pull request #34375: URL: https://github.com/apache/spark/pull/34375#issuecomment-952516488 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49105/ -- This is an automated message from the A

[GitHub] [spark] SparkQA commented on pull request #34397: [SPARK-36348][PYTHON][FOLLOWUP] Complete test_astype for index

2021-10-26 Thread GitBox
SparkQA commented on pull request #34397: URL: https://github.com/apache/spark/pull/34397#issuecomment-952515095 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49109/ -- This is an automated message from the Apache

[GitHub] [spark] SparkQA commented on pull request #34248: [SPARK-36647][SQL][TESTS] Push down Aggregate (Min/Max/Count) for Parquet if filter is on partition col

2021-10-26 Thread GitBox
SparkQA commented on pull request #34248: URL: https://github.com/apache/spark/pull/34248#issuecomment-952514851 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49106/ -- This is an automated message from the A

[GitHub] [spark] SparkQA commented on pull request #34388: [SPARK-37115][SQL] HiveClientImpl should use shim to wrap all hive client calls

2021-10-26 Thread GitBox
SparkQA commented on pull request #34388: URL: https://github.com/apache/spark/pull/34388#issuecomment-952514767 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49110/ -- This is an automated message from the Apache

[GitHub] [spark] Yikun commented on a change in pull request #34213: [SPARK-36396][PYTHON] Implement DataFrame.cov

2021-10-26 Thread GitBox
Yikun commented on a change in pull request #34213: URL: https://github.com/apache/spark/pull/34213#discussion_r737085007 ## File path: python/pyspark/pandas/frame.py ## @@ -8201,6 +8202,185 @@ def update(self, other: "DataFrame", join: str = "left", overwrite: bool = True)

[GitHub] [spark] SparkQA commented on pull request #34389: [SPARK-37036][PYTHON] Add util function to raise advice warning for pandas API on Spark.

2021-10-26 Thread GitBox
SparkQA commented on pull request #34389: URL: https://github.com/apache/spark/pull/34389#issuecomment-952513304 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49104/ -- This is an automated message from the A

[GitHub] [spark] dchvn commented on pull request #34213: [SPARK-36396][PYTHON] Implement DataFrame.cov

2021-10-26 Thread GitBox
dchvn commented on pull request #34213: URL: https://github.com/apache/spark/pull/34213#issuecomment-952512142 Ping @HyukjinKwon , Many thanks! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to th

[GitHub] [spark] HyukjinKwon commented on pull request #34397: [SPARK-36348][PYTHON][FOLLOWUP] Complete test_astype for index

2021-10-26 Thread GitBox
HyukjinKwon commented on pull request #34397: URL: https://github.com/apache/spark/pull/34397#issuecomment-952511779 retest this pleasse -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the speci

[GitHub] [spark] LuciferYang edited a comment on pull request #34376: [SPARK-37105][TEST] Pass all UTs in `sql/hive` with Java 17

2021-10-26 Thread GitBox
LuciferYang edited a comment on pull request #34376: URL: https://github.com/apache/spark/pull/34376#issuecomment-952506879 @dongjoon-hyun and I test this module on a M1 ``` uname -a Darwin B00396921U 20.5.0 Darwin Kernel Version 20.5.0: Sat May 8 05:10:31 PDT 2021; root:

[GitHub] [spark] SparkQA commented on pull request #34389: [SPARK-37036][PYTHON] Add util function to raise advice warning for pandas API on Spark.

2021-10-26 Thread GitBox
SparkQA commented on pull request #34389: URL: https://github.com/apache/spark/pull/34389#issuecomment-952511336 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49107/ -- This is an automated message from the A

[GitHub] [spark] LuciferYang edited a comment on pull request #34376: [SPARK-37105][TEST] Pass all UTs in `sql/hive` with Java 17

2021-10-26 Thread GitBox
LuciferYang edited a comment on pull request #34376: URL: https://github.com/apache/spark/pull/34376#issuecomment-952510648 > This seems specific to M1? E.g. cl-plus-ssl/cl-plus-ssl#114 too. Seems like it's coming from the Azul jdk itself. Maybe something has to get updated on that end

[GitHub] [spark] LuciferYang commented on pull request #34376: [SPARK-37105][TEST] Pass all UTs in `sql/hive` with Java 17

2021-10-26 Thread GitBox
LuciferYang commented on pull request #34376: URL: https://github.com/apache/spark/pull/34376#issuecomment-952510648 > This seems specific to M1? E.g. cl-plus-ssl/cl-plus-ssl#114 too. Seems like it's coming from the Azul jdk itself. Maybe something has to get updated on that end It

[GitHub] [spark] srowen commented on pull request #34376: [SPARK-37105][TEST] Pass all UTs in `sql/hive` with Java 17

2021-10-26 Thread GitBox
srowen commented on pull request #34376: URL: https://github.com/apache/spark/pull/34376#issuecomment-952509642 This seems specific to M1? E.g. https://github.com/cl-plus-ssl/cl-plus-ssl/issues/114 too. Seems like it's coming from the Azul jdk itself. Maybe something has to get updated on

  1   2   3   4   5   >