[GitHub] [spark] AngersZhuuuu commented on issue #27861: [SPARK-30707][SQL]Window function set partitionSpec as order spec when orderSpec is empty
AngersZh commented on issue #27861: [SPARK-30707][SQL]Window function set partitionSpec as order spec when orderSpec is empty URL: https://github.com/apache/spark/pull/27861#issuecomment-597472224 @cloud-fan As I have tested in postgres SQL, when use window frame with `RANGE` won't add default order spec, when use window frame with `ROW`, it will add default order spec This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AngersZhuuuu edited a comment on issue #27861: [SPARK-30707][SQL]Window function set partitionSpec as order spec when orderSpec is empty
AngersZh edited a comment on issue #27861: [SPARK-30707][SQL]Window function set partitionSpec as order spec when orderSpec is empty URL: https://github.com/apache/spark/pull/27861#issuecomment-597472224 @cloud-fan As I have tested in postgres SQL, when use window frame with `RANGE` won't add default order spec, when use window frame with `ROW`, it will add default order spec We can add code to control this different behavior, but just a little strange This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AngersZhuuuu edited a comment on issue #27861: [SPARK-30707][SQL]Window function set partitionSpec as order spec when orderSpec is empty
AngersZh edited a comment on issue #27861: [SPARK-30707][SQL]Window function set partitionSpec as order spec when orderSpec is empty URL: https://github.com/apache/spark/pull/27861#issuecomment-597467691 ``` angerszhu=# select id, sum(num) over (partition by id range between 1 preceding and 1 following) angerszhu-# from s4 ; ERROR: RANGE with offset PRECEDING/FOLLOWING requires exactly one ORDER BY column LINE 1: select id, sum(num) over (partition by id ^ angerszhu=# angerszhu=# angerszhu=# select id, sum(num) over (partition by id rows between 1 preceding and 1 following) from s4 ; id | sum +- 1 | 3 2 | 3 3 | 6 4 | 7 5 | 9 6 | 9 (6 rows) angerszhu=# select id, sum(num) over (partition by id range between 1 preceding and 1 following) from s4 ; ERROR: RANGE with offset PRECEDING/FOLLOWING requires exactly one ORDER BY column LINE 1: select id, sum(num) over (partition by id ^ angerszhu=# select id, sum(num) over (partition by id range 1 preceding) from s4 ; ERROR: RANGE with offset PRECEDING/FOLLOWING requires exactly one ORDER BY column LINE 1: select id, sum(num) over (partition by id ^ angerszhu=# select id, sum(num) over (partition by id rows 1 preceding) from s4 ; id | sum +- 1 | 3 2 | 3 3 | 6 4 | 7 5 | 9 6 | 9 (6 rows) angerszhu=# ``` This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27617: [SPARK-30865][SQL] Refactor DateTimeUtils
AmplabJenkins removed a comment on issue #27617: [SPARK-30865][SQL] Refactor DateTimeUtils URL: https://github.com/apache/spark/pull/27617#issuecomment-597468829 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27617: [SPARK-30865][SQL] Refactor DateTimeUtils
AmplabJenkins removed a comment on issue #27617: [SPARK-30865][SQL] Refactor DateTimeUtils URL: https://github.com/apache/spark/pull/27617#issuecomment-597468834 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/24384/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27617: [SPARK-30865][SQL] Refactor DateTimeUtils
AmplabJenkins commented on issue #27617: [SPARK-30865][SQL] Refactor DateTimeUtils URL: https://github.com/apache/spark/pull/27617#issuecomment-597468829 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27617: [SPARK-30865][SQL] Refactor DateTimeUtils
AmplabJenkins commented on issue #27617: [SPARK-30865][SQL] Refactor DateTimeUtils URL: https://github.com/apache/spark/pull/27617#issuecomment-597468834 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/24384/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27759: [SPARK-31008][SQL]Support json_array_length function
AmplabJenkins removed a comment on issue #27759: [SPARK-31008][SQL]Support json_array_length function URL: https://github.com/apache/spark/pull/27759#issuecomment-597468283 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/119643/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27617: [SPARK-30865][SQL] Refactor DateTimeUtils
SparkQA commented on issue #27617: [SPARK-30865][SQL] Refactor DateTimeUtils URL: https://github.com/apache/spark/pull/27617#issuecomment-597468467 **[Test build #119653 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/119653/testReport)** for PR 27617 at commit [`1fd3aa2`](https://github.com/apache/spark/commit/1fd3aa22c81b7a55fc280c18b25b1904210238bc). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27759: [SPARK-31008][SQL]Support json_array_length function
AmplabJenkins commented on issue #27759: [SPARK-31008][SQL]Support json_array_length function URL: https://github.com/apache/spark/pull/27759#issuecomment-597468277 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27759: [SPARK-31008][SQL]Support json_array_length function
AmplabJenkins commented on issue #27759: [SPARK-31008][SQL]Support json_array_length function URL: https://github.com/apache/spark/pull/27759#issuecomment-597468283 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/119643/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27759: [SPARK-31008][SQL]Support json_array_length function
AmplabJenkins removed a comment on issue #27759: [SPARK-31008][SQL]Support json_array_length function URL: https://github.com/apache/spark/pull/27759#issuecomment-597468277 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27759: [SPARK-31008][SQL]Support json_array_length function
SparkQA commented on issue #27759: [SPARK-31008][SQL]Support json_array_length function URL: https://github.com/apache/spark/pull/27759#issuecomment-597467701 **[Test build #119643 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/119643/testReport)** for PR 27759 at commit [`5a860d7`](https://github.com/apache/spark/commit/5a860d7c9ef3704737fe4971c704eb5f3a15a295). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #27759: [SPARK-31008][SQL]Support json_array_length function
SparkQA removed a comment on issue #27759: [SPARK-31008][SQL]Support json_array_length function URL: https://github.com/apache/spark/pull/27759#issuecomment-597406322 **[Test build #119643 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/119643/testReport)** for PR 27759 at commit [`5a860d7`](https://github.com/apache/spark/commit/5a860d7c9ef3704737fe4971c704eb5f3a15a295). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AngersZhuuuu commented on issue #27861: [SPARK-30707][SQL]Window function set partitionSpec as order spec when orderSpec is empty
AngersZh commented on issue #27861: [SPARK-30707][SQL]Window function set partitionSpec as order spec when orderSpec is empty URL: https://github.com/apache/spark/pull/27861#issuecomment-597467691 ``` angerszhu=# select id, sum(num) over (partition by id angerszhu(# range between 1 preceding and 1 following) angerszhu-# from s4 ; ERROR: RANGE with offset PRECEDING/FOLLOWING requires exactly one ORDER BY column LINE 1: select id, sum(num) over (partition by id ^ angerszhu=# angerszhu=# angerszhu=# select id, sum(num) over (partition by id rows between 1 preceding and 1 following) from s4 ; id | sum +- 1 | 3 2 | 3 3 | 6 4 | 7 5 | 9 6 | 9 (6 rows) angerszhu=# select id, sum(num) over (partition by id range between 1 preceding and 1 following) from s4 ; ERROR: RANGE with offset PRECEDING/FOLLOWING requires exactly one ORDER BY column LINE 1: select id, sum(num) over (partition by id ^ angerszhu=# select id, sum(num) over (partition by id range 1 preceding) from s4 ; ERROR: RANGE with offset PRECEDING/FOLLOWING requires exactly one ORDER BY column LINE 1: select id, sum(num) over (partition by id ^ angerszhu=# select id, sum(num) over (partition by id rows 1 preceding) from s4 ; id | sum +- 1 | 3 2 | 3 3 | 6 4 | 7 5 | 9 6 | 9 (6 rows) angerszhu=# ``` This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27807: [SPARK-31076][SQL] Convert Catalyst's DATE/TIMESTAMP to Java Date/Timestamp via local date-time
AmplabJenkins removed a comment on issue #27807: [SPARK-31076][SQL] Convert Catalyst's DATE/TIMESTAMP to Java Date/Timestamp via local date-time URL: https://github.com/apache/spark/pull/27807#issuecomment-597466409 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/24383/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27807: [SPARK-31076][SQL] Convert Catalyst's DATE/TIMESTAMP to Java Date/Timestamp via local date-time
AmplabJenkins commented on issue #27807: [SPARK-31076][SQL] Convert Catalyst's DATE/TIMESTAMP to Java Date/Timestamp via local date-time URL: https://github.com/apache/spark/pull/27807#issuecomment-597466409 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/24383/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27807: [SPARK-31076][SQL] Convert Catalyst's DATE/TIMESTAMP to Java Date/Timestamp via local date-time
AmplabJenkins removed a comment on issue #27807: [SPARK-31076][SQL] Convert Catalyst's DATE/TIMESTAMP to Java Date/Timestamp via local date-time URL: https://github.com/apache/spark/pull/27807#issuecomment-597466405 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27807: [SPARK-31076][SQL] Convert Catalyst's DATE/TIMESTAMP to Java Date/Timestamp via local date-time
AmplabJenkins commented on issue #27807: [SPARK-31076][SQL] Convert Catalyst's DATE/TIMESTAMP to Java Date/Timestamp via local date-time URL: https://github.com/apache/spark/pull/27807#issuecomment-597466405 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27807: [SPARK-31076][SQL] Convert Catalyst's DATE/TIMESTAMP to Java Date/Timestamp via local date-time
SparkQA commented on issue #27807: [SPARK-31076][SQL] Convert Catalyst's DATE/TIMESTAMP to Java Date/Timestamp via local date-time URL: https://github.com/apache/spark/pull/27807#issuecomment-597466035 **[Test build #119652 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/119652/testReport)** for PR 27807 at commit [`407cc1f`](https://github.com/apache/spark/commit/407cc1fd8584f2a045b8db74cfe50023f13b7919). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27861: [SPARK-30707][SQL]Window function set partitionSpec as order spec when orderSpec is empty
AmplabJenkins commented on issue #27861: [SPARK-30707][SQL]Window function set partitionSpec as order spec when orderSpec is empty URL: https://github.com/apache/spark/pull/27861#issuecomment-597464125 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/24382/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27861: [SPARK-30707][SQL]Window function set partitionSpec as order spec when orderSpec is empty
AmplabJenkins removed a comment on issue #27861: [SPARK-30707][SQL]Window function set partitionSpec as order spec when orderSpec is empty URL: https://github.com/apache/spark/pull/27861#issuecomment-597464125 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/24382/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27861: [SPARK-30707][SQL]Window function set partitionSpec as order spec when orderSpec is empty
AmplabJenkins removed a comment on issue #27861: [SPARK-30707][SQL]Window function set partitionSpec as order spec when orderSpec is empty URL: https://github.com/apache/spark/pull/27861#issuecomment-597464120 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27861: [SPARK-30707][SQL]Window function set partitionSpec as order spec when orderSpec is empty
AmplabJenkins commented on issue #27861: [SPARK-30707][SQL]Window function set partitionSpec as order spec when orderSpec is empty URL: https://github.com/apache/spark/pull/27861#issuecomment-597464120 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AngersZhuuuu commented on a change in pull request #27861: [SPARK-30707][SQL]Window function set partitionSpec as order spec when orderSpec is empty
AngersZh commented on a change in pull request #27861: [SPARK-30707][SQL]Window function set partitionSpec as order spec when orderSpec is empty URL: https://github.com/apache/spark/pull/27861#discussion_r390763768 ## File path: sql/core/src/test/resources/sql-tests/results/postgreSQL/window_part3.sql.out ## @@ -243,9 +243,8 @@ from t1 where f1 = f2 -- !query schema struct<> -- !query output -org.apache.spark.sql.AnalysisException -cannot resolve '(PARTITION BY spark_catalog.default.t1.`f1` RANGE BETWEEN 1 PRECEDING AND 1 FOLLOWING)' due to data type mismatch: A range window frame cannot be used in an unordered window specification.; line 1 pos 24 Review comment: > is there any result changing after this PR? Yeah, will add order spec then won't meet this problem. Tried in postgres Sql, will throw exception, emmm. maybe these type sql we shouldn't add order spec by default? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27861: [SPARK-30707][SQL]Window function set partitionSpec as order spec when orderSpec is empty
SparkQA commented on issue #27861: [SPARK-30707][SQL]Window function set partitionSpec as order spec when orderSpec is empty URL: https://github.com/apache/spark/pull/27861#issuecomment-597463671 **[Test build #119651 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/119651/testReport)** for PR 27861 at commit [`62739e3`](https://github.com/apache/spark/commit/62739e3ca7502659b77580b7b7f7877127ab78c8). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AngersZhuuuu commented on a change in pull request #27861: [SPARK-30707][SQL]Window function set partitionSpec as order spec when orderSpec is empty
AngersZh commented on a change in pull request #27861: [SPARK-30707][SQL]Window function set partitionSpec as order spec when orderSpec is empty URL: https://github.com/apache/spark/pull/27861#discussion_r390763768 ## File path: sql/core/src/test/resources/sql-tests/results/postgreSQL/window_part3.sql.out ## @@ -243,9 +243,8 @@ from t1 where f1 = f2 -- !query schema struct<> -- !query output -org.apache.spark.sql.AnalysisException -cannot resolve '(PARTITION BY spark_catalog.default.t1.`f1` RANGE BETWEEN 1 PRECEDING AND 1 FOLLOWING)' due to data type mismatch: A range window frame cannot be used in an unordered window specification.; line 1 pos 24 Review comment: > is there any result changing after this PR? Yeah, will add order spec then won't meet this problem This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27867: [SPARK-31111][SQL][Tests] Fix interval output issue in ExtractBenchmark
AmplabJenkins removed a comment on issue #27867: [SPARK-3][SQL][Tests] Fix interval output issue in ExtractBenchmark URL: https://github.com/apache/spark/pull/27867#issuecomment-597461813 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/24381/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27867: [SPARK-31111][SQL][Tests] Fix interval output issue in ExtractBenchmark
AmplabJenkins removed a comment on issue #27867: [SPARK-3][SQL][Tests] Fix interval output issue in ExtractBenchmark URL: https://github.com/apache/spark/pull/27867#issuecomment-597461809 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27867: [SPARK-31111][SQL][Tests] Fix interval output issue in ExtractBenchmark
AmplabJenkins commented on issue #27867: [SPARK-3][SQL][Tests] Fix interval output issue in ExtractBenchmark URL: https://github.com/apache/spark/pull/27867#issuecomment-597461813 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/24381/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27867: [SPARK-31111][SQL][Tests] Fix interval output issue in ExtractBenchmark
AmplabJenkins commented on issue #27867: [SPARK-3][SQL][Tests] Fix interval output issue in ExtractBenchmark URL: https://github.com/apache/spark/pull/27867#issuecomment-597461809 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] cloud-fan commented on issue #27830: [SPARK-31030][SQL] Backward Compatibility for Parsing and formatting Datetime
cloud-fan commented on issue #27830: [SPARK-31030][SQL] Backward Compatibility for Parsing and formatting Datetime URL: https://github.com/apache/spark/pull/27830#issuecomment-597461388 thanks, merging to master/3.0! This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27867: [SPARK-31111][SQL][Tests] Fix interval output issue in ExtractBenchmark
SparkQA commented on issue #27867: [SPARK-3][SQL][Tests] Fix interval output issue in ExtractBenchmark URL: https://github.com/apache/spark/pull/27867#issuecomment-597461444 **[Test build #119650 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/119650/testReport)** for PR 27867 at commit [`14722a1`](https://github.com/apache/spark/commit/14722a1764ab729713f9cfdad18aa4363a78eec2). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] cloud-fan closed pull request #27830: [SPARK-31030][SQL] Backward Compatibility for Parsing and formatting Datetime
cloud-fan closed pull request #27830: [SPARK-31030][SQL] Backward Compatibility for Parsing and formatting Datetime URL: https://github.com/apache/spark/pull/27830 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] cloud-fan commented on a change in pull request #27861: [SPARK-30707][SQL]Window function set partitionSpec as order spec when orderSpec is empty
cloud-fan commented on a change in pull request #27861: [SPARK-30707][SQL]Window function set partitionSpec as order spec when orderSpec is empty URL: https://github.com/apache/spark/pull/27861#discussion_r390762519 ## File path: sql/core/src/test/resources/sql-tests/results/postgreSQL/window_part3.sql.out ## @@ -243,9 +243,8 @@ from t1 where f1 = f2 -- !query schema struct<> -- !query output -org.apache.spark.sql.AnalysisException -cannot resolve '(PARTITION BY spark_catalog.default.t1.`f1` RANGE BETWEEN 1 PRECEDING AND 1 FOLLOWING)' due to data type mismatch: A range window frame cannot be used in an unordered window specification.; line 1 pos 24 Review comment: is there any result changing after this PR? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] MaxGekk commented on a change in pull request #27807: [SPARK-31076][SQL] Convert Catalyst's DATE/TIMESTAMP to Java Date/Timestamp via local date-time
MaxGekk commented on a change in pull request #27807: [SPARK-31076][SQL]
Convert Catalyst's DATE/TIMESTAMP to Java Date/Timestamp via local date-time
URL: https://github.com/apache/spark/pull/27807#discussion_r390761064
##
File path:
sql/core/v1.2/src/main/java/org/apache/spark/sql/execution/datasources/orc/OrcColumnVector.java
##
@@ -136,7 +138,9 @@ public int getInt(int rowId) {
public long getLong(int rowId) {
int index = getRowIndex(rowId);
if (isTimestamp) {
- return timestampData.time[index] * 1000 + timestampData.nanos[index] /
1000 % 1000;
+ Timestamp ts = new Timestamp(timestampData.time[index]);
Review comment:
> Could you elaborate a little more what was the existing bug here?
ORC writer uses `DateTimeUtils.toJavaTimestamp`, see
https://github.com/apache/spark/blob/300ec1a74cb14867c22e616e657566d510426331/sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/orc/OrcSerializer.scala#L144-L148
but here we don't use opposite function `DateTimeUtils.fromJavaTimestamp`.
And the replaced hand-written code is not equal to `fromJavaTimestamp`
> Does this PR add a test coverage for this vectorized code path change?
I have to fix this place due test failures, see
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/119486/testReport/
The changes covered by the round trip test
https://github.com/apache/spark/blob/7782b61a31ba49cdeffc35a942bd365bb71b026d/sql/hive/src/test/scala/org/apache/spark/sql/sources/HadoopFsRelationTest.scala#L122
which runs by `OrcHadoopFsRelationSuite` for vectorized code path.
> can we add a round trip test in OrcQuerySuite to read/write date before
1582?
@cloud-fan The test I pointed out above generates random dates/timestamps
before 1582 with high probability.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] HyukjinKwon closed pull request #27663: [SPARK-30895]:The behavior of CsvToStructs should not depend on SQLConf.get
HyukjinKwon closed pull request #27663: [SPARK-30895]:The behavior of CsvToStructs should not depend on SQLConf.get URL: https://github.com/apache/spark/pull/27663 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on issue #27663: [SPARK-30895]:The behavior of CsvToStructs should not depend on SQLConf.get
HyukjinKwon commented on issue #27663: [SPARK-30895]:The behavior of CsvToStructs should not depend on SQLConf.get URL: https://github.com/apache/spark/pull/27663#issuecomment-597459066 Closing this per the discussion in the JIRA This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] HyukjinKwon closed pull request #27666: [SPARK-30896]:The behavior of JsonToStructs should not depend on SQLConf.get
HyukjinKwon closed pull request #27666: [SPARK-30896]:The behavior of JsonToStructs should not depend on SQLConf.get URL: https://github.com/apache/spark/pull/27666 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on issue #27666: [SPARK-30896]:The behavior of JsonToStructs should not depend on SQLConf.get
HyukjinKwon commented on issue #27666: [SPARK-30896]:The behavior of JsonToStructs should not depend on SQLConf.get URL: https://github.com/apache/spark/pull/27666#issuecomment-597459033 Closing this per the discussion in the JIRA This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] yaooqinn commented on a change in pull request #27867: [SPARK-31111][SQL][Tests] Fix interval output issue in ExtractBenchmark
yaooqinn commented on a change in pull request #27867:
[SPARK-3][SQL][Tests] Fix interval output issue in ExtractBenchmark
URL: https://github.com/apache/spark/pull/27867#discussion_r390757478
##
File path:
sql/core/src/test/scala/org/apache/spark/sql/execution/benchmark/ExtractBenchmark.scala
##
@@ -56,9 +56,11 @@ object ExtractBenchmark extends SqlBasedBenchmark {
}
}
- private def castExpr(from: String): String = from match {
+ private def castExpr(from: String, toStr: Boolean = false): String = from
match {
case "timestamp" => "cast(id as timestamp)"
case "date" => "cast(cast(id as timestamp) as date)"
+case "interval" if toStr => "cast((cast(cast(id as timestamp) as date) -
date'0001-01-01') + " +
Review comment:
good advice, thanks!
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] jiangxb1987 commented on a change in pull request #27768: [SPARK-31018][CORE] Deprecate support of multiple workers on the same host in Standalone
jiangxb1987 commented on a change in pull request #27768: [SPARK-31018][CORE] Deprecate support of multiple workers on the same host in Standalone URL: https://github.com/apache/spark/pull/27768#discussion_r390756919 ## File path: docs/hardware-provisioning.md ## @@ -63,10 +63,10 @@ Note that memory usage is greatly affected by storage level and serialization fo the [tuning guide](tuning.html) for tips on how to reduce it. Finally, note that the Java VM does not always behave well with more than 200 GiB of RAM. If you -purchase machines with more RAM than this, you can run _multiple worker JVMs per node_. In -Spark's [standalone mode](spark-standalone.html), you can set the number of workers per node -with the `SPARK_WORKER_INSTANCES` variable in `conf/spark-env.sh`, and the number of cores -per worker with `SPARK_WORKER_CORES`. +purchase machines with more RAM than this, you can launch multiple executors in a single node. In +Spark's [standalone mode](spark-standalone.html), a worker is responsible for launching multiple +executors according to its available memory and cores. And each executor will be launched in a Review comment: nit: `And` -> `and` This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] jiangxb1987 commented on a change in pull request #27768: [SPARK-31018][CORE] Deprecate support of multiple workers on the same host in Standalone
jiangxb1987 commented on a change in pull request #27768: [SPARK-31018][CORE] Deprecate support of multiple workers on the same host in Standalone URL: https://github.com/apache/spark/pull/27768#discussion_r390756668 ## File path: docs/core-migration-guide.md ## @@ -38,3 +38,5 @@ license: | - Event log file will be written as UTF-8 encoding, and Spark History Server will replay event log files as UTF-8 encoding. Previously Spark writes event log file as default charset of driver JVM process, so Spark History Server of Spark 2.x is needed to read the old event log files in case of incompatible encoding. - A new protocol for fetching shuffle blocks is used. It's recommended that external shuffle services be upgraded when running Spark 3.0 apps. Old external shuffle services can still be used by setting the configuration `spark.shuffle.useOldFetchProtocol` to `true`. Otherwise, Spark may run into errors with messages like `IllegalArgumentException: Unexpected message type: `. + +- `SPARK_WORKER_INSTANCES` is deprecated. It's recommended to launch multiple executors in one worker and launch one worker per node instead of launching multiple workers per node and launching one executor per worker. Review comment: You should mention this is for Standalone backend only. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27861: [SPARK-30707][SQL]Window function set partitionSpec as order spec when orderSpec is empty
AmplabJenkins removed a comment on issue #27861: [SPARK-30707][SQL]Window function set partitionSpec as order spec when orderSpec is empty URL: https://github.com/apache/spark/pull/27861#issuecomment-597453785 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/119649/ Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27861: [SPARK-30707][SQL]Window function set partitionSpec as order spec when orderSpec is empty
AmplabJenkins removed a comment on issue #27861: [SPARK-30707][SQL]Window function set partitionSpec as order spec when orderSpec is empty URL: https://github.com/apache/spark/pull/27861#issuecomment-597453780 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #27861: [SPARK-30707][SQL]Window function set partitionSpec as order spec when orderSpec is empty
SparkQA removed a comment on issue #27861: [SPARK-30707][SQL]Window function set partitionSpec as order spec when orderSpec is empty URL: https://github.com/apache/spark/pull/27861#issuecomment-597449397 **[Test build #119649 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/119649/testReport)** for PR 27861 at commit [`85e66d2`](https://github.com/apache/spark/commit/85e66d27b5bfb53c88e01bd487c636225ba75ed2). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27861: [SPARK-30707][SQL]Window function set partitionSpec as order spec when orderSpec is empty
SparkQA commented on issue #27861: [SPARK-30707][SQL]Window function set partitionSpec as order spec when orderSpec is empty URL: https://github.com/apache/spark/pull/27861#issuecomment-597453758 **[Test build #119649 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/119649/testReport)** for PR 27861 at commit [`85e66d2`](https://github.com/apache/spark/commit/85e66d27b5bfb53c88e01bd487c636225ba75ed2). * This patch **fails Spark unit tests**. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27861: [SPARK-30707][SQL]Window function set partitionSpec as order spec when orderSpec is empty
AmplabJenkins commented on issue #27861: [SPARK-30707][SQL]Window function set partitionSpec as order spec when orderSpec is empty URL: https://github.com/apache/spark/pull/27861#issuecomment-597453785 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/119649/ Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27861: [SPARK-30707][SQL]Window function set partitionSpec as order spec when orderSpec is empty
AmplabJenkins commented on issue #27861: [SPARK-30707][SQL]Window function set partitionSpec as order spec when orderSpec is empty URL: https://github.com/apache/spark/pull/27861#issuecomment-597453780 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] cloud-fan commented on a change in pull request #27867: [SPARK-31111][SQL][Tests] Fix interval output issue in ExtractBenchmark
cloud-fan commented on a change in pull request #27867:
[SPARK-3][SQL][Tests] Fix interval output issue in ExtractBenchmark
URL: https://github.com/apache/spark/pull/27867#discussion_r390754252
##
File path:
sql/core/src/test/scala/org/apache/spark/sql/execution/benchmark/ExtractBenchmark.scala
##
@@ -56,9 +56,11 @@ object ExtractBenchmark extends SqlBasedBenchmark {
}
}
- private def castExpr(from: String): String = from match {
+ private def castExpr(from: String, toStr: Boolean = false): String = from
match {
case "timestamp" => "cast(id as timestamp)"
case "date" => "cast(cast(id as timestamp) as date)"
+case "interval" if toStr => "cast((cast(cast(id as timestamp) as date) -
date'0001-01-01') + " +
Review comment:
Maybe we should not use the noop sink here, but
`df.queryExecution.toRDD.foreach(_ => ())`
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27861: [SPARK-30707][SQL]Window function set partitionSpec as order spec when orderSpec is empty
AmplabJenkins commented on issue #27861: [SPARK-30707][SQL]Window function set partitionSpec as order spec when orderSpec is empty URL: https://github.com/apache/spark/pull/27861#issuecomment-597449644 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/24380/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27861: [SPARK-30707][SQL]Window function set partitionSpec as order spec when orderSpec is empty
AmplabJenkins commented on issue #27861: [SPARK-30707][SQL]Window function set partitionSpec as order spec when orderSpec is empty URL: https://github.com/apache/spark/pull/27861#issuecomment-597449635 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27861: [SPARK-30707][SQL]Window function set partitionSpec as order spec when orderSpec is empty
AmplabJenkins removed a comment on issue #27861: [SPARK-30707][SQL]Window function set partitionSpec as order spec when orderSpec is empty URL: https://github.com/apache/spark/pull/27861#issuecomment-597449635 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27861: [SPARK-30707][SQL]Window function set partitionSpec as order spec when orderSpec is empty
AmplabJenkins removed a comment on issue #27861: [SPARK-30707][SQL]Window function set partitionSpec as order spec when orderSpec is empty URL: https://github.com/apache/spark/pull/27861#issuecomment-597449644 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/24380/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27861: [SPARK-30707][SQL]Window function set partitionSpec as order spec when orderSpec is empty
SparkQA commented on issue #27861: [SPARK-30707][SQL]Window function set partitionSpec as order spec when orderSpec is empty URL: https://github.com/apache/spark/pull/27861#issuecomment-597449397 **[Test build #119649 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/119649/testReport)** for PR 27861 at commit [`85e66d2`](https://github.com/apache/spark/commit/85e66d27b5bfb53c88e01bd487c636225ba75ed2). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27836: [SPARK-31009][SQL] Support json_object_keys function
AmplabJenkins removed a comment on issue #27836: [SPARK-31009][SQL] Support json_object_keys function URL: https://github.com/apache/spark/pull/27836#issuecomment-597448065 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/24379/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27861: [SPARK-30707][SQL]Window function set partitionSpec as order spec when orderSpec is empty
AmplabJenkins removed a comment on issue #27861: [SPARK-30707][SQL]Window function set partitionSpec as order spec when orderSpec is empty URL: https://github.com/apache/spark/pull/27861#issuecomment-597448066 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/24378/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27861: [SPARK-30707][SQL]Window function set partitionSpec as order spec when orderSpec is empty
AmplabJenkins removed a comment on issue #27861: [SPARK-30707][SQL]Window function set partitionSpec as order spec when orderSpec is empty URL: https://github.com/apache/spark/pull/27861#issuecomment-597448060 Build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27836: [SPARK-31009][SQL] Support json_object_keys function
AmplabJenkins removed a comment on issue #27836: [SPARK-31009][SQL] Support json_object_keys function URL: https://github.com/apache/spark/pull/27836#issuecomment-597448059 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27861: [SPARK-30707][SQL]Window function set partitionSpec as order spec when orderSpec is empty
AmplabJenkins commented on issue #27861: [SPARK-30707][SQL]Window function set partitionSpec as order spec when orderSpec is empty URL: https://github.com/apache/spark/pull/27861#issuecomment-597448060 Build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27861: [SPARK-30707][SQL]Window function set partitionSpec as order spec when orderSpec is empty
AmplabJenkins commented on issue #27861: [SPARK-30707][SQL]Window function set partitionSpec as order spec when orderSpec is empty URL: https://github.com/apache/spark/pull/27861#issuecomment-597448066 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/24378/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27836: [SPARK-31009][SQL] Support json_object_keys function
AmplabJenkins commented on issue #27836: [SPARK-31009][SQL] Support json_object_keys function URL: https://github.com/apache/spark/pull/27836#issuecomment-597448065 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/24379/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27836: [SPARK-31009][SQL] Support json_object_keys function
AmplabJenkins commented on issue #27836: [SPARK-31009][SQL] Support json_object_keys function URL: https://github.com/apache/spark/pull/27836#issuecomment-597448059 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27861: [SPARK-30707][SQL]Window function set partitionSpec as order spec when orderSpec is empty
SparkQA commented on issue #27861: [SPARK-30707][SQL]Window function set partitionSpec as order spec when orderSpec is empty URL: https://github.com/apache/spark/pull/27861#issuecomment-597447725 **[Test build #119647 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/119647/testReport)** for PR 27861 at commit [`487e533`](https://github.com/apache/spark/commit/487e5333405b0c8c6d8140cecda5a608e78cb745). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27836: [SPARK-31009][SQL] Support json_object_keys function
SparkQA commented on issue #27836: [SPARK-31009][SQL] Support json_object_keys function URL: https://github.com/apache/spark/pull/27836#issuecomment-597447744 **[Test build #119648 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/119648/testReport)** for PR 27836 at commit [`7a74b30`](https://github.com/apache/spark/commit/7a74b303d209b67adfcdb3b902daed5ff8c3793e). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27801: [SPARK-31047][SQL] Improve file listing for ViewFileSystem
AmplabJenkins removed a comment on issue #27801: [SPARK-31047][SQL] Improve file listing for ViewFileSystem URL: https://github.com/apache/spark/pull/27801#issuecomment-597443657 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/119639/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27801: [SPARK-31047][SQL] Improve file listing for ViewFileSystem
AmplabJenkins removed a comment on issue #27801: [SPARK-31047][SQL] Improve file listing for ViewFileSystem URL: https://github.com/apache/spark/pull/27801#issuecomment-597443653 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27801: [SPARK-31047][SQL] Improve file listing for ViewFileSystem
AmplabJenkins commented on issue #27801: [SPARK-31047][SQL] Improve file listing for ViewFileSystem URL: https://github.com/apache/spark/pull/27801#issuecomment-597443653 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27801: [SPARK-31047][SQL] Improve file listing for ViewFileSystem
AmplabJenkins commented on issue #27801: [SPARK-31047][SQL] Improve file listing for ViewFileSystem URL: https://github.com/apache/spark/pull/27801#issuecomment-597443657 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/119639/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #27801: [SPARK-31047][SQL] Improve file listing for ViewFileSystem
SparkQA removed a comment on issue #27801: [SPARK-31047][SQL] Improve file listing for ViewFileSystem URL: https://github.com/apache/spark/pull/27801#issuecomment-597379727 **[Test build #119639 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/119639/testReport)** for PR 27801 at commit [`c6b964e`](https://github.com/apache/spark/commit/c6b964e0be490401a4d72e36b124f9029fd7e396). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27801: [SPARK-31047][SQL] Improve file listing for ViewFileSystem
SparkQA commented on issue #27801: [SPARK-31047][SQL] Improve file listing for ViewFileSystem URL: https://github.com/apache/spark/pull/27801#issuecomment-597443157 **[Test build #119639 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/119639/testReport)** for PR 27801 at commit [`c6b964e`](https://github.com/apache/spark/commit/c6b964e0be490401a4d72e36b124f9029fd7e396). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] gatorsmile closed pull request #27833: [SPARK-31070][SQL] make skew join split skewed partitions more evenly
gatorsmile closed pull request #27833: [SPARK-31070][SQL] make skew join split skewed partitions more evenly URL: https://github.com/apache/spark/pull/27833 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] gatorsmile commented on issue #27833: [SPARK-31070][SQL] make skew join split skewed partitions more evenly
gatorsmile commented on issue #27833: [SPARK-31070][SQL] make skew join split skewed partitions more evenly URL: https://github.com/apache/spark/pull/27833#issuecomment-597443090 Thanks! Merged to master/3.0 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27803: [SPARK-31049][SQL] Support nested adjacent generators, e.g., explode(explode(v))
AmplabJenkins commented on issue #27803: [SPARK-31049][SQL] Support nested adjacent generators, e.g., explode(explode(v)) URL: https://github.com/apache/spark/pull/27803#issuecomment-597441984 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27803: [SPARK-31049][SQL] Support nested adjacent generators, e.g., explode(explode(v))
AmplabJenkins removed a comment on issue #27803: [SPARK-31049][SQL] Support nested adjacent generators, e.g., explode(explode(v)) URL: https://github.com/apache/spark/pull/27803#issuecomment-597441984 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27803: [SPARK-31049][SQL] Support nested adjacent generators, e.g., explode(explode(v))
AmplabJenkins commented on issue #27803: [SPARK-31049][SQL] Support nested adjacent generators, e.g., explode(explode(v)) URL: https://github.com/apache/spark/pull/27803#issuecomment-597441992 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/119638/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27803: [SPARK-31049][SQL] Support nested adjacent generators, e.g., explode(explode(v))
AmplabJenkins removed a comment on issue #27803: [SPARK-31049][SQL] Support nested adjacent generators, e.g., explode(explode(v)) URL: https://github.com/apache/spark/pull/27803#issuecomment-597441992 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/119638/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27803: [SPARK-31049][SQL] Support nested adjacent generators, e.g., explode(explode(v))
SparkQA commented on issue #27803: [SPARK-31049][SQL] Support nested adjacent generators, e.g., explode(explode(v)) URL: https://github.com/apache/spark/pull/27803#issuecomment-597441449 **[Test build #119638 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/119638/testReport)** for PR 27803 at commit [`0ed5509`](https://github.com/apache/spark/commit/0ed5509b286521d05234a672a8634c77f3150557). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27866: [SPARK-31110][DOCS][SQL] refine sql doc for SELECT
SparkQA commented on issue #27866: [SPARK-31110][DOCS][SQL] refine sql doc for SELECT URL: https://github.com/apache/spark/pull/27866#issuecomment-597441472 **[Test build #119646 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/119646/testReport)** for PR 27866 at commit [`d40afe2`](https://github.com/apache/spark/commit/d40afe2f10434144a993f4240e77c120cadd4ad2). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27866: [SPARK-31110][DOCS][SQL] refine sql doc for SELECT
AmplabJenkins commented on issue #27866: [SPARK-31110][DOCS][SQL] refine sql doc for SELECT URL: https://github.com/apache/spark/pull/27866#issuecomment-597441544 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27866: [SPARK-31110][DOCS][SQL] refine sql doc for SELECT
AmplabJenkins commented on issue #27866: [SPARK-31110][DOCS][SQL] refine sql doc for SELECT URL: https://github.com/apache/spark/pull/27866#issuecomment-597441551 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/119646/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #27803: [SPARK-31049][SQL] Support nested adjacent generators, e.g., explode(explode(v))
SparkQA removed a comment on issue #27803: [SPARK-31049][SQL] Support nested adjacent generators, e.g., explode(explode(v)) URL: https://github.com/apache/spark/pull/27803#issuecomment-597375193 **[Test build #119638 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/119638/testReport)** for PR 27803 at commit [`0ed5509`](https://github.com/apache/spark/commit/0ed5509b286521d05234a672a8634c77f3150557). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27866: [SPARK-31110][DOCS][SQL] refine sql doc for SELECT
AmplabJenkins removed a comment on issue #27866: [SPARK-31110][DOCS][SQL] refine sql doc for SELECT URL: https://github.com/apache/spark/pull/27866#issuecomment-597441551 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/119646/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27866: [SPARK-31110][DOCS][SQL] refine sql doc for SELECT
AmplabJenkins removed a comment on issue #27866: [SPARK-31110][DOCS][SQL] refine sql doc for SELECT URL: https://github.com/apache/spark/pull/27866#issuecomment-597441544 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #27866: [SPARK-31110][DOCS][SQL] refine sql doc for SELECT
SparkQA removed a comment on issue #27866: [SPARK-31110][DOCS][SQL] refine sql doc for SELECT URL: https://github.com/apache/spark/pull/27866#issuecomment-597439142 **[Test build #119646 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/119646/testReport)** for PR 27866 at commit [`d40afe2`](https://github.com/apache/spark/commit/d40afe2f10434144a993f4240e77c120cadd4ad2). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27832: [SPARK-31011][CORE] Log better message if SIGPWR is not supported while setting up decommission
AmplabJenkins removed a comment on issue #27832: [SPARK-31011][CORE] Log better message if SIGPWR is not supported while setting up decommission URL: https://github.com/apache/spark/pull/27832#issuecomment-597440951 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/119642/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27832: [SPARK-31011][CORE] Log better message if SIGPWR is not supported while setting up decommission
AmplabJenkins removed a comment on issue #27832: [SPARK-31011][CORE] Log better message if SIGPWR is not supported while setting up decommission URL: https://github.com/apache/spark/pull/27832#issuecomment-597440947 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] yaooqinn commented on a change in pull request #27867: [SPARK-31111][SQL][Tests] Fix interval output issue in ExtractBenchmark
yaooqinn commented on a change in pull request #27867:
[SPARK-3][SQL][Tests] Fix interval output issue in ExtractBenchmark
URL: https://github.com/apache/spark/pull/27867#discussion_r390743073
##
File path:
sql/core/src/test/scala/org/apache/spark/sql/execution/benchmark/ExtractBenchmark.scala
##
@@ -56,9 +56,11 @@ object ExtractBenchmark extends SqlBasedBenchmark {
}
}
- private def castExpr(from: String): String = from match {
+ private def castExpr(from: String, toStr: Boolean = false): String = from
match {
case "timestamp" => "cast(id as timestamp)"
case "date" => "cast(cast(id as timestamp) as date)"
+case "interval" if toStr => "cast((cast(cast(id as timestamp) as date) -
date'0001-01-01') + " +
Review comment:
We forbid interval as output schema only
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27832: [SPARK-31011][CORE] Log better message if SIGPWR is not supported while setting up decommission
AmplabJenkins commented on issue #27832: [SPARK-31011][CORE] Log better message if SIGPWR is not supported while setting up decommission URL: https://github.com/apache/spark/pull/27832#issuecomment-597440951 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/119642/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27832: [SPARK-31011][CORE] Log better message if SIGPWR is not supported while setting up decommission
AmplabJenkins commented on issue #27832: [SPARK-31011][CORE] Log better message if SIGPWR is not supported while setting up decommission URL: https://github.com/apache/spark/pull/27832#issuecomment-597440947 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] cloud-fan commented on issue #27627: [WIP][SPARK-28067][SQL] Fix incorrect results for decimal aggregate sum by returning null on decimal overflow
cloud-fan commented on issue #27627: [WIP][SPARK-28067][SQL] Fix incorrect results for decimal aggregate sum by returning null on decimal overflow URL: https://github.com/apache/spark/pull/27627#issuecomment-597440834 @skambha great analysis! I agree with you that we need another boolean flag in the sum aggregate buffer, but I'd like to make it simpler and only change it for decimals. How about we add a new expression `DecimalSum`? In which: 1. the buffer attributes are [sum, isEmpty] 2. initial value is [0, true] 3. the `updateExpression` should do: 3.1 update `isEmpty` to false 3.2 set `sum` to null if overflowed 3.3 do nothing if `sum` is already null. 4. the `mergeExpression` should do: 4.1 update `isEmpty` to false 4.2 if the input buffer's `isEmpty` is true, keep sum unchanged 4.3 if the input buffer's `isEmpty` is false, but `sum` is null, update its own `sum` to null 4.4 do nothing if `sum` is already null. 4.5 otherwise, add input buffer's `sum` 5. the `evaluateExpression` should do: 5.1 output null if `isEmpty` is true 5.2 fail if `sum` is null and ansi mode is on 5.3 otherwise, output the sum. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #27832: [SPARK-31011][CORE] Log better message if SIGPWR is not supported while setting up decommission
SparkQA removed a comment on issue #27832: [SPARK-31011][CORE] Log better message if SIGPWR is not supported while setting up decommission URL: https://github.com/apache/spark/pull/27832#issuecomment-597400479 **[Test build #119642 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/119642/testReport)** for PR 27832 at commit [`7ffc615`](https://github.com/apache/spark/commit/7ffc615e4ae44b88709c47beb02ef5c8310cd64f). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] viirya commented on a change in pull request #27866: [SPARK-31110][DOCS][SQL] refine sql doc for SELECT
viirya commented on a change in pull request #27866: [SPARK-31110][DOCS][SQL] refine sql doc for SELECT URL: https://github.com/apache/spark/pull/27866#discussion_r390669221 ## File path: docs/sql-ref-syntax-qry-select-groupby.md ## @@ -44,7 +44,7 @@ GROUP BY [ GROUPING SETS grouping_sets ] group_expression [ , group_expression [ set is specified by a list of comma-separated expressions in parentheses. Syntax: -( () | ( expression [ , ... ] ) ) +( '()' | ( expression [ , ... ] ) ) Review comment: The syntax here indicates one group set or more? In above the syntax is `GROUP BY [ GROUPING SETS grouping_sets ] group_expression [ , group_expression [ , ... ] ]`. For example, `group_expression` here is for just one group expression. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27832: [SPARK-31011][CORE] Log better message if SIGPWR is not supported while setting up decommission
SparkQA commented on issue #27832: [SPARK-31011][CORE] Log better message if SIGPWR is not supported while setting up decommission URL: https://github.com/apache/spark/pull/27832#issuecomment-597440486 **[Test build #119642 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/119642/testReport)** for PR 27832 at commit [`7ffc615`](https://github.com/apache/spark/commit/7ffc615e4ae44b88709c47beb02ef5c8310cd64f). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27866: [SPARK-31110][DOCS][SQL] refine sql doc for SELECT
AmplabJenkins removed a comment on issue #27866: [SPARK-31110][DOCS][SQL] refine sql doc for SELECT URL: https://github.com/apache/spark/pull/27866#issuecomment-597439379 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/24377/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27866: [SPARK-31110][DOCS][SQL] refine sql doc for SELECT
AmplabJenkins commented on issue #27866: [SPARK-31110][DOCS][SQL] refine sql doc for SELECT URL: https://github.com/apache/spark/pull/27866#issuecomment-597439379 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/24377/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27866: [SPARK-31110][DOCS][SQL] refine sql doc for SELECT
AmplabJenkins removed a comment on issue #27866: [SPARK-31110][DOCS][SQL] refine sql doc for SELECT URL: https://github.com/apache/spark/pull/27866#issuecomment-597439369 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27866: [SPARK-31110][DOCS][SQL] refine sql doc for SELECT
AmplabJenkins commented on issue #27866: [SPARK-31110][DOCS][SQL] refine sql doc for SELECT URL: https://github.com/apache/spark/pull/27866#issuecomment-597439369 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27866: [SPARK-31110][DOCS][SQL] refine sql doc for SELECT
SparkQA commented on issue #27866: [SPARK-31110][DOCS][SQL] refine sql doc for SELECT URL: https://github.com/apache/spark/pull/27866#issuecomment-597439142 **[Test build #119646 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/119646/testReport)** for PR 27866 at commit [`d40afe2`](https://github.com/apache/spark/commit/d40afe2f10434144a993f4240e77c120cadd4ad2). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] cloud-fan commented on a change in pull request #27866: [SPARK-31110][DOCS][SQL] refine sql doc for SELECT
cloud-fan commented on a change in pull request #27866:
[SPARK-31110][DOCS][SQL] refine sql doc for SELECT
URL: https://github.com/apache/spark/pull/27866#discussion_r390739939
##
File path: docs/sql-ref-syntax-qry-select.md
##
@@ -1,57 +1,61 @@
---
layout: global
title: SELECT
-displayTitle: SELECT
+displayTitle: SELECT
license: |
Licensed to the Apache Software Foundation (ASF) under one or more
contributor license agreements. See the NOTICE file distributed with
this work for additional information regarding copyright ownership.
The ASF licenses this file to You under the Apache License, Version 2.0
(the "License"); you may not use this file except in compliance with
the License. You may obtain a copy of the License at
-
+
http://www.apache.org/licenses/LICENSE-2.0
-
+
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
---
Spark supports a `SELECT` statement and conforms to the ANSI SQL standard.
Queries are
-used to retrieve result sets from one or more tables. The following section
+used to retrieve result sets from one or more tables. The following section
describes the overall query syntax and the sub-sections cover different
constructs
-of a query along with examples.
+of a query along with examples.
### Syntax
{% highlight sql %}
[ WITH with_query [ , ... ] ]
+select_statement [ { UNION | INTERSECT | EXCEPT } select_statement, ... ]
Review comment:
It's defined in `select_statement`:
https://github.com/apache/spark/pull/27866/files#diff-bc52347d9c1373190bce4389d3066ad5R40
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
