[GitHub] [spark] dongjoon-hyun commented on a change in pull request #25345: [SPARK-28609][DOC] Fix broken styles/links and make up-to-date
dongjoon-hyun commented on a change in pull request #25345: [SPARK-28609][DOC]
Fix broken styles/links and make up-to-date
URL: https://github.com/apache/spark/pull/25345#discussion_r310340352
##
File path: docs/structured-streaming-programming-guide.md
##
@@ -3047,12 +3047,6 @@ To run a supported query in continuous processing mode,
all you need to do is sp
{% highlight scala %}
import org.apache.spark.sql.streaming.Trigger
-spark
- .readStream
- .format("rate")
- .option("rowsPerSecond", "10")
- .option("")
-
Review comment:
This seems to be incomplete leftover. We had better remove this because the
other language tabs don't have this. After removing this, we will have `Kafka`
examples consistently across all languages.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on a change in pull request #25345: [SPARK-28609][DOC] Fix broken styles/links and make up-to-date
dongjoon-hyun commented on a change in pull request #25345: [SPARK-28609][DOC] Fix broken styles/links and make up-to-date URL: https://github.com/apache/spark/pull/25345#discussion_r310340367 ## File path: docs/submitting-applications.md ## @@ -182,7 +182,7 @@ The master URL passed to Spark can be in one of the following formats: k8s://HOST:PORT Connect to a Kubernetes cluster in cluster mode. Client mode is currently unsupported and will be supported in future releases. -The HOST and PORT refer to the [Kubernetes API Server](https://kubernetes.io/docs/reference/generated/kube-apiserver/). +The HOST and PORT refer to the https://kubernetes.io/docs/reference/generated/kube-apiserver/;>Kubernetes API Server. Review comment: Inside table, we need to use `a` tag like line 183. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on a change in pull request #25345: [SPARK-28609][DOC] Fix broken styles/links and make up-to-date
dongjoon-hyun commented on a change in pull request #25345: [SPARK-28609][DOC] Fix broken styles/links and make up-to-date URL: https://github.com/apache/spark/pull/25345#discussion_r310340310 ## File path: docs/structured-streaming-programming-guide.md ## @@ -2190,34 +2190,34 @@ The function offers a simple way to express your processing logic but does not a deduplicate generated data when failures cause reprocessing of some input data. For that situation you must specify the processing logic in an object. -1. The function takes a row as input. +- First, the function takes a row as input. Review comment: The following is the current screenshot of our 2.4.3 page. The enumeration doesn't work well with template. Also, the python example has redundant leading spaces inconsistently from the other Python examples.  This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on a change in pull request #25345: [SPARK-28609][DOC] Fix broken styles/links and make up-to-date
dongjoon-hyun commented on a change in pull request #25345: [SPARK-28609][DOC] Fix broken styles/links and make up-to-date URL: https://github.com/apache/spark/pull/25345#discussion_r310340261 ## File path: docs/structured-streaming-programming-guide.md ## @@ -1496,9 +1496,9 @@ Additional details on supported joins: - Joins can be cascaded, that is, you can do `df1.join(df2, ...).join(df3, ...).join(df4, )`. -- As of Spark 2.3, you can use joins only when the query is in Append output mode. Other output modes are not yet supported. +- As of Spark 2.4, you can use joins only when the query is in Append output mode. Other output modes are not yet supported. Review comment: For now, it's `2.4` because this patch will be backported to `branch-2.4`. We will update this later when we prepare 3.0.0. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on a change in pull request #25345: [SPARK-28609][DOC] Fix broken styles/links and make up-to-date
dongjoon-hyun commented on a change in pull request #25345: [SPARK-28609][DOC] Fix broken styles/links and make up-to-date URL: https://github.com/apache/spark/pull/25345#discussion_r310340174 ## File path: docs/structured-streaming-programming-guide.md ## @@ -27,7 +27,7 @@ Structured Streaming is a scalable and fault-tolerant stream processing engine b Internally, by default, Structured Streaming queries are processed using a *micro-batch processing* engine, which processes data streams as a series of small batch jobs thereby achieving end-to-end latencies as low as 100 milliseconds and exactly-once fault-tolerance guarantees. However, since Spark 2.3, we have introduced a new low-latency processing mode called **Continuous Processing**, which can achieve end-to-end latencies as low as 1 millisecond with at-least-once guarantees. Without changing the Dataset/DataFrame operations in your queries, you will be able to choose the mode based on your application requirements. -In this guide, we are going to walk you through the programming model and the APIs. We are going to explain the concepts mostly using the default micro-batch processing model, and then [later](#continuous-processing-experimental) discuss Continuous Processing model. First, let's start with a simple example of a Structured Streaming query - a streaming word count. +In this guide, we are going to walk you through the programming model and the APIs. We are going to explain the concepts mostly using the default micro-batch processing model, and then [later](#continuous-processing) discuss Continuous Processing model. First, let's start with a simple example of a Structured Streaming query - a streaming word count. Review comment: This fixes the broken link. Previously, it points like the following. - https://spark.apache.org/docs/latest/structured-streaming-programming-guide.html#continuous-processing-experimental This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on a change in pull request #25345: [SPARK-28609][DOC] Fix broken styles/links and make up-to-date
dongjoon-hyun commented on a change in pull request #25345: [SPARK-28609][DOC] Fix broken styles/links and make up-to-date URL: https://github.com/apache/spark/pull/25345#discussion_r310340174 ## File path: docs/structured-streaming-programming-guide.md ## @@ -27,7 +27,7 @@ Structured Streaming is a scalable and fault-tolerant stream processing engine b Internally, by default, Structured Streaming queries are processed using a *micro-batch processing* engine, which processes data streams as a series of small batch jobs thereby achieving end-to-end latencies as low as 100 milliseconds and exactly-once fault-tolerance guarantees. However, since Spark 2.3, we have introduced a new low-latency processing mode called **Continuous Processing**, which can achieve end-to-end latencies as low as 1 millisecond with at-least-once guarantees. Without changing the Dataset/DataFrame operations in your queries, you will be able to choose the mode based on your application requirements. -In this guide, we are going to walk you through the programming model and the APIs. We are going to explain the concepts mostly using the default micro-batch processing model, and then [later](#continuous-processing-experimental) discuss Continuous Processing model. First, let's start with a simple example of a Structured Streaming query - a streaming word count. +In this guide, we are going to walk you through the programming model and the APIs. We are going to explain the concepts mostly using the default micro-batch processing model, and then [later](#continuous-processing) discuss Continuous Processing model. First, let's start with a simple example of a Structured Streaming query - a streaming word count. Review comment: This fixes the broken link. Previously, it points like the following instead of that section. - https://spark.apache.org/docs/latest/structured-streaming-programming-guide.html#continuous-processing-experimental This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on a change in pull request #25345: [SPARK-28609][DOC] Fix broken styles/links and make up-to-date
dongjoon-hyun commented on a change in pull request #25345: [SPARK-28609][DOC] Fix broken styles/links and make up-to-date URL: https://github.com/apache/spark/pull/25345#discussion_r310340153 ## File path: docs/configuration.md ## @@ -696,7 +696,7 @@ Apart from these, the following properties are also available, and may be useful option --repositories or spark.jars.repositories will also be included. Useful for allowing Spark to resolve artifacts from behind a firewall e.g. via an in-house artifact server like Artifactory. Details on the settings file format can be -found at http://ant.apache.org/ivy/history/latest-milestone/settings.html +found at http://ant.apache.org/ivy/history/latest-milestone/settings.html;>Settings Files Review comment: Hyperlink is better than the text. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on a change in pull request #25345: [SPARK-28609][DOC] Fix broken styles/links and make up-to-date
dongjoon-hyun commented on a change in pull request #25345: [SPARK-28609][DOC] Fix broken styles/links and make up-to-date URL: https://github.com/apache/spark/pull/25345#discussion_r310340153 ## File path: docs/configuration.md ## @@ -696,7 +696,7 @@ Apart from these, the following properties are also available, and may be useful option --repositories or spark.jars.repositories will also be included. Useful for allowing Spark to resolve artifacts from behind a firewall e.g. via an in-house artifact server like Artifactory. Details on the settings file format can be -found at http://ant.apache.org/ivy/history/latest-milestone/settings.html +found at http://ant.apache.org/ivy/history/latest-milestone/settings.html;>Settings Files Review comment: Hyperlink is better than the text. Note that `Settings Files` is the title of that page. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on a change in pull request #25345: [SPARK-28609][DOC] Fix broken styles/links and make up-to-date
dongjoon-hyun commented on a change in pull request #25345: [SPARK-28609][DOC] Fix broken styles/links and make up-to-date URL: https://github.com/apache/spark/pull/25345#discussion_r310340174 ## File path: docs/structured-streaming-programming-guide.md ## @@ -27,7 +27,7 @@ Structured Streaming is a scalable and fault-tolerant stream processing engine b Internally, by default, Structured Streaming queries are processed using a *micro-batch processing* engine, which processes data streams as a series of small batch jobs thereby achieving end-to-end latencies as low as 100 milliseconds and exactly-once fault-tolerance guarantees. However, since Spark 2.3, we have introduced a new low-latency processing mode called **Continuous Processing**, which can achieve end-to-end latencies as low as 1 millisecond with at-least-once guarantees. Without changing the Dataset/DataFrame operations in your queries, you will be able to choose the mode based on your application requirements. -In this guide, we are going to walk you through the programming model and the APIs. We are going to explain the concepts mostly using the default micro-batch processing model, and then [later](#continuous-processing-experimental) discuss Continuous Processing model. First, let's start with a simple example of a Structured Streaming query - a streaming word count. +In this guide, we are going to walk you through the programming model and the APIs. We are going to explain the concepts mostly using the default micro-batch processing model, and then [later](#continuous-processing) discuss Continuous Processing model. First, let's start with a simple example of a Structured Streaming query - a streaming word count. Review comment: This fixes the broken link. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on a change in pull request #25345: [SPARK-28609][DOC] Fix broken styles/links and make up-to-date
dongjoon-hyun commented on a change in pull request #25345: [SPARK-28609][DOC]
Fix broken styles/links and make up-to-date
URL: https://github.com/apache/spark/pull/25345#discussion_r310340148
##
File path: docs/building-spark.md
##
@@ -241,8 +241,8 @@ The run-tests script also can be limited to a specific
Python version or a speci
To run the SparkR tests you will need to install the
[knitr](https://cran.r-project.org/package=knitr),
[rmarkdown](https://cran.r-project.org/package=rmarkdown),
[testthat](https://cran.r-project.org/package=testthat),
[e1071](https://cran.r-project.org/package=e1071) and
[survival](https://cran.r-project.org/package=survival) packages first:
-R -e "install.packages(c('knitr', 'rmarkdown', 'devtools', 'e1071',
'survival'), repos='https://cloud.r-project.org/')"
-R -e "devtools::install_version('testthat', version = '1.0.2',
repos='https://cloud.r-project.org/')"
+Rscript -e "install.packages(c('knitr', 'rmarkdown', 'devtools', 'e1071',
'survival'), repos='https://cloud.r-project.org/')"
+Rscript -e "devtools::install_version('testthat', version = '1.0.2',
repos='https://cloud.r-project.org/')"
Review comment:
This hides the verbose message and also is consistent with
`dev/create-release/spark-rm/Dockerfile`.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] albertusk95 closed pull request #25326: [SPARK-28590][PySpark][WIP] Add sort_stats Setter and Getter in Profiler
albertusk95 closed pull request #25326: [SPARK-28590][PySpark][WIP] Add sort_stats Setter and Getter in Profiler URL: https://github.com/apache/spark/pull/25326 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] albertusk95 commented on issue #25326: [SPARK-28590][PySpark][WIP] Add sort_stats Setter and Getter in Profiler
albertusk95 commented on issue #25326: [SPARK-28590][PySpark][WIP] Add sort_stats Setter and Getter in Profiler URL: https://github.com/apache/spark/pull/25326#issuecomment-517896324 well, I think I'll just implement it on the forked repo. Ok to close the PR. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] dongjoon-hyun opened a new pull request #25345: Init
dongjoon-hyun opened a new pull request #25345: Init URL: https://github.com/apache/spark/pull/25345 ## What changes were proposed in this pull request? (Please fill in changes proposed in this fix) ## How was this patch tested? (Please explain how this patch was tested. E.g. unit tests, integration tests, manual tests) (If this patch involves UI changes, please attach a screenshot; otherwise, remove this) Please review https://spark.apache.org/contributing.html before opening a pull request. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] WeichenXu123 commented on issue #25294: [WIP][SPARK-28476][SQL] Support ALTER DATABASE SET LOCATION
WeichenXu123 commented on issue #25294: [WIP][SPARK-28476][SQL] Support ALTER DATABASE SET LOCATION URL: https://github.com/apache/spark/pull/25294#issuecomment-517895635 Jenkins, retest this please. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] felixcheung commented on a change in pull request #25310: [SPARK-28578][INFRA] Improve Github pull request template
felixcheung commented on a change in pull request #25310: [SPARK-28578][INFRA] Improve Github pull request template URL: https://github.com/apache/spark/pull/25310#discussion_r310338919 ## File path: .github/PULL_REQUEST_TEMPLATE ## @@ -1,10 +1,32 @@ -## What changes were proposed in this pull request? + -(Please fill in changes proposed in this fix) +### What changes were proposed in this pull request? -## How was this patch tested? +### Why changes are needed? Review comment: `Why are the changes needed`? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] felixcheung commented on issue #25339: [SPARK-28606][INFRA] Update CRAN key to recover docker image generation
felixcheung commented on issue #25339: [SPARK-28606][INFRA] Update CRAN key to recover docker image generation URL: https://github.com/apache/spark/pull/25339#issuecomment-517893947 LGTM This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] maryannxue commented on issue #25308: [SPARK-28576][SQL] fix the dead lock issue when enable new adaptive execution
maryannxue commented on issue #25308: [SPARK-28576][SQL] fix the dead lock issue when enable new adaptive execution URL: https://github.com/apache/spark/pull/25308#issuecomment-517893672 I suggest we fix the deadlock issue by disabling the wrong UI update for subquery first. And we fix the UI issue right after. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] maryannxue commented on issue #25308: [SPARK-28576][SQL] fix the dead lock issue when enable new adaptive execution
maryannxue commented on issue #25308: [SPARK-28576][SQL] fix the dead lock issue when enable new adaptive execution URL: https://github.com/apache/spark/pull/25308#issuecomment-517893572 After an offline discussion with @cloud-fan , we found out that the reason why the stats did not get updated correctly to the subquery was that the changing plan of subquery could not be in sync with the metrics collection. It is like: 1. When the subquery is first submitted, the initial plan of the subquery is posted to UI. 2. As it is being executed, the exchange nodes within the subquery change too, because of the "copy" operation when creating a new stage. 3. The UI discards the metrics for the new Exchange node coz it doesn't "recognize it" yet but meanwhile keeps collecting metrics for the old Exchange node, which now does not exist in the new plan of the subquery. 4. The main query proceeds on to new stages or the final plan, updating the UI again, and now that subquery is updated with the new Exchange node, which makes the plan itself right, but not the metrics. To solve this, @cloud-fan proposed that we could add an interface in UI to update accumulator ID independently. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #25022: [SPARK-24695][SQL] Move `CalendarInterval` to org.apache.spark.sql.types package
AmplabJenkins removed a comment on issue #25022: [SPARK-24695][SQL] Move `CalendarInterval` to org.apache.spark.sql.types package URL: https://github.com/apache/spark/pull/25022#issuecomment-517892079 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/108594/ Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #25022: [SPARK-24695][SQL] Move `CalendarInterval` to org.apache.spark.sql.types package
AmplabJenkins removed a comment on issue #25022: [SPARK-24695][SQL] Move `CalendarInterval` to org.apache.spark.sql.types package URL: https://github.com/apache/spark/pull/25022#issuecomment-517892076 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #25022: [SPARK-24695][SQL] Move `CalendarInterval` to org.apache.spark.sql.types package
AmplabJenkins commented on issue #25022: [SPARK-24695][SQL] Move `CalendarInterval` to org.apache.spark.sql.types package URL: https://github.com/apache/spark/pull/25022#issuecomment-517892076 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #25022: [SPARK-24695][SQL] Move `CalendarInterval` to org.apache.spark.sql.types package
AmplabJenkins commented on issue #25022: [SPARK-24695][SQL] Move `CalendarInterval` to org.apache.spark.sql.types package URL: https://github.com/apache/spark/pull/25022#issuecomment-517892079 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/108594/ Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #25022: [SPARK-24695][SQL] Move `CalendarInterval` to org.apache.spark.sql.types package
SparkQA removed a comment on issue #25022: [SPARK-24695][SQL] Move `CalendarInterval` to org.apache.spark.sql.types package URL: https://github.com/apache/spark/pull/25022#issuecomment-517885375 **[Test build #108594 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/108594/testReport)** for PR 25022 at commit [`a07c79f`](https://github.com/apache/spark/commit/a07c79f430579295c17fc618d7722511652f3c4c). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #25022: [SPARK-24695][SQL] Move `CalendarInterval` to org.apache.spark.sql.types package
SparkQA commented on issue #25022: [SPARK-24695][SQL] Move `CalendarInterval` to org.apache.spark.sql.types package URL: https://github.com/apache/spark/pull/25022#issuecomment-517892006 **[Test build #108594 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/108594/testReport)** for PR 25022 at commit [`a07c79f`](https://github.com/apache/spark/commit/a07c79f430579295c17fc618d7722511652f3c4c). * This patch **fails Spark unit tests**. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] dongjinleekr commented on issue #22282: [SPARK-23539][SS] Add support for Kafka headers in Structured Streaming
dongjinleekr commented on issue #22282: [SPARK-23539][SS] Add support for Kafka headers in Structured Streaming URL: https://github.com/apache/spark/pull/22282#issuecomment-517891763 Ping @zsxwing @srowen This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #25335: [SPARK-28601][CORE][SQL] Use StandardCharsets.UTF_8 instead of "UTF-8" string representation, and get rid of UnsupportedEncodingException
AmplabJenkins commented on issue #25335: [SPARK-28601][CORE][SQL] Use StandardCharsets.UTF_8 instead of "UTF-8" string representation, and get rid of UnsupportedEncodingException URL: https://github.com/apache/spark/pull/25335#issuecomment-517891015 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #25335: [SPARK-28601][CORE][SQL] Use StandardCharsets.UTF_8 instead of "UTF-8" string representation, and get rid of UnsupportedEncodingExcept
AmplabJenkins removed a comment on issue #25335: [SPARK-28601][CORE][SQL] Use StandardCharsets.UTF_8 instead of "UTF-8" string representation, and get rid of UnsupportedEncodingException URL: https://github.com/apache/spark/pull/25335#issuecomment-517891015 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #25335: [SPARK-28601][CORE][SQL] Use StandardCharsets.UTF_8 instead of "UTF-8" string representation, and get rid of UnsupportedEncodingException
AmplabJenkins commented on issue #25335: [SPARK-28601][CORE][SQL] Use StandardCharsets.UTF_8 instead of "UTF-8" string representation, and get rid of UnsupportedEncodingException URL: https://github.com/apache/spark/pull/25335#issuecomment-517891018 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/108592/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #25335: [SPARK-28601][CORE][SQL] Use StandardCharsets.UTF_8 instead of "UTF-8" string representation, and get rid of UnsupportedEncodingExcept
AmplabJenkins removed a comment on issue #25335: [SPARK-28601][CORE][SQL] Use StandardCharsets.UTF_8 instead of "UTF-8" string representation, and get rid of UnsupportedEncodingException URL: https://github.com/apache/spark/pull/25335#issuecomment-517891018 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/108592/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #25335: [SPARK-28601][CORE][SQL] Use StandardCharsets.UTF_8 instead of "UTF-8" string representation, and get rid of UnsupportedEncodingException
SparkQA commented on issue #25335: [SPARK-28601][CORE][SQL] Use StandardCharsets.UTF_8 instead of "UTF-8" string representation, and get rid of UnsupportedEncodingException URL: https://github.com/apache/spark/pull/25335#issuecomment-517890919 **[Test build #108592 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/108592/testReport)** for PR 25335 at commit [`94bb1d6`](https://github.com/apache/spark/commit/94bb1d6b9caadb15b5e50c1710a488ca386597b7). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #25335: [SPARK-28601][CORE][SQL] Use StandardCharsets.UTF_8 instead of "UTF-8" string representation, and get rid of UnsupportedEncodingException
SparkQA removed a comment on issue #25335: [SPARK-28601][CORE][SQL] Use StandardCharsets.UTF_8 instead of "UTF-8" string representation, and get rid of UnsupportedEncodingException URL: https://github.com/apache/spark/pull/25335#issuecomment-517881537 **[Test build #108592 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/108592/testReport)** for PR 25335 at commit [`94bb1d6`](https://github.com/apache/spark/commit/94bb1d6b9caadb15b5e50c1710a488ca386597b7). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #25340: [SPARK-28393][SQL][PYTHON][TESTS] Convert and port 'pgSQL/join.sql' into UDF test base
AmplabJenkins removed a comment on issue #25340: [SPARK-28393][SQL][PYTHON][TESTS] Convert and port 'pgSQL/join.sql' into UDF test base URL: https://github.com/apache/spark/pull/25340#issuecomment-517890427 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/108589/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #25340: [SPARK-28393][SQL][PYTHON][TESTS] Convert and port 'pgSQL/join.sql' into UDF test base
AmplabJenkins removed a comment on issue #25340: [SPARK-28393][SQL][PYTHON][TESTS] Convert and port 'pgSQL/join.sql' into UDF test base URL: https://github.com/apache/spark/pull/25340#issuecomment-517890425 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #25340: [SPARK-28393][SQL][PYTHON][TESTS] Convert and port 'pgSQL/join.sql' into UDF test base
AmplabJenkins commented on issue #25340: [SPARK-28393][SQL][PYTHON][TESTS] Convert and port 'pgSQL/join.sql' into UDF test base URL: https://github.com/apache/spark/pull/25340#issuecomment-517890427 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/108589/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #25340: [SPARK-28393][SQL][PYTHON][TESTS] Convert and port 'pgSQL/join.sql' into UDF test base
AmplabJenkins commented on issue #25340: [SPARK-28393][SQL][PYTHON][TESTS] Convert and port 'pgSQL/join.sql' into UDF test base URL: https://github.com/apache/spark/pull/25340#issuecomment-517890425 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #25340: [SPARK-28393][SQL][PYTHON][TESTS] Convert and port 'pgSQL/join.sql' into UDF test base
SparkQA removed a comment on issue #25340: [SPARK-28393][SQL][PYTHON][TESTS] Convert and port 'pgSQL/join.sql' into UDF test base URL: https://github.com/apache/spark/pull/25340#issuecomment-517874211 **[Test build #108589 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/108589/testReport)** for PR 25340 at commit [`6f43011`](https://github.com/apache/spark/commit/6f430111391ce29b523a6a7d5023f325c06a64f4). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #25340: [SPARK-28393][SQL][PYTHON][TESTS] Convert and port 'pgSQL/join.sql' into UDF test base
SparkQA commented on issue #25340: [SPARK-28393][SQL][PYTHON][TESTS] Convert and port 'pgSQL/join.sql' into UDF test base URL: https://github.com/apache/spark/pull/25340#issuecomment-517890320 **[Test build #108589 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/108589/testReport)** for PR 25340 at commit [`6f43011`](https://github.com/apache/spark/commit/6f430111391ce29b523a6a7d5023f325c06a64f4). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #25342: [SPARK-28571][CORE][SHUFFLE] Use the shuffle writer plugin for the SortShuffleWriter
AmplabJenkins removed a comment on issue #25342: [SPARK-28571][CORE][SHUFFLE] Use the shuffle writer plugin for the SortShuffleWriter URL: https://github.com/apache/spark/pull/25342#issuecomment-517890185 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #25342: [SPARK-28571][CORE][SHUFFLE] Use the shuffle writer plugin for the SortShuffleWriter
AmplabJenkins removed a comment on issue #25342: [SPARK-28571][CORE][SHUFFLE] Use the shuffle writer plugin for the SortShuffleWriter URL: https://github.com/apache/spark/pull/25342#issuecomment-517890189 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/108591/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #25342: [SPARK-28571][CORE][SHUFFLE] Use the shuffle writer plugin for the SortShuffleWriter
AmplabJenkins commented on issue #25342: [SPARK-28571][CORE][SHUFFLE] Use the shuffle writer plugin for the SortShuffleWriter URL: https://github.com/apache/spark/pull/25342#issuecomment-517890189 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/108591/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #25342: [SPARK-28571][CORE][SHUFFLE] Use the shuffle writer plugin for the SortShuffleWriter
AmplabJenkins commented on issue #25342: [SPARK-28571][CORE][SHUFFLE] Use the shuffle writer plugin for the SortShuffleWriter URL: https://github.com/apache/spark/pull/25342#issuecomment-517890185 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #25342: [SPARK-28571][CORE][SHUFFLE] Use the shuffle writer plugin for the SortShuffleWriter
SparkQA removed a comment on issue #25342: [SPARK-28571][CORE][SHUFFLE] Use the shuffle writer plugin for the SortShuffleWriter URL: https://github.com/apache/spark/pull/25342#issuecomment-517880900 **[Test build #108591 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/108591/testReport)** for PR 25342 at commit [`2f9b4ca`](https://github.com/apache/spark/commit/2f9b4ca64f44bc15bd4527c661f42e96f64621d7). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #25342: [SPARK-28571][CORE][SHUFFLE] Use the shuffle writer plugin for the SortShuffleWriter
SparkQA commented on issue #25342: [SPARK-28571][CORE][SHUFFLE] Use the shuffle writer plugin for the SortShuffleWriter URL: https://github.com/apache/spark/pull/25342#issuecomment-517890102 **[Test build #108591 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/108591/testReport)** for PR 25342 at commit [`2f9b4ca`](https://github.com/apache/spark/commit/2f9b4ca64f44bc15bd4527c661f42e96f64621d7). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on issue #20677: [MINOR][DOC][SS] Event time can't be greater then processing time. 12:21, owl. Mistake…
dongjoon-hyun commented on issue #20677: [MINOR][DOC][SS] Event time can't be greater then processing time. 12:21, owl. Mistake… URL: https://github.com/apache/spark/pull/20677#issuecomment-517889387 As @srowen mentioned early [here](https://github.com/apache/spark/pull/20677#issuecomment-383589806), this illustrates the event time which is the value of column `timestamp`. So, the event time value is different from X-axis. I'll close this PR for now. Please reopen this if this is not the case. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] dongjoon-hyun closed pull request #20677: [MINOR][DOC][SS] Event time can't be greater then processing time. 12:21, owl. Mistake…
dongjoon-hyun closed pull request #20677: [MINOR][DOC][SS] Event time can't be greater then processing time. 12:21, owl. Mistake… URL: https://github.com/apache/spark/pull/20677 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] dongjoon-hyun edited a comment on issue #20677: [MINOR][DOC][SS] Event time can't be greater then processing time. 12:21, owl. Mistake…
dongjoon-hyun edited a comment on issue #20677: [MINOR][DOC][SS] Event time can't be greater then processing time. 12:21, owl. Mistake… URL: https://github.com/apache/spark/pull/20677#issuecomment-517888913 For me, it looks correct. Data `12:21, owl` arrives at 12:19 with the event time `12:21`, doesn't it? > Point 12:21 for owl can't have 12:19 processing time. Since Spark received the data, Spark processed it. The point will be assigned to the window `12:15 ~ 12:25` and `12:20 ~ 12:30` although those windows are hidden in the picture. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] dongjoon-hyun edited a comment on issue #20677: [MINOR][DOC][SS] Event time can't be greater then processing time. 12:21, owl. Mistake…
dongjoon-hyun edited a comment on issue #20677: [MINOR][DOC][SS] Event time can't be greater then processing time. 12:21, owl. Mistake… URL: https://github.com/apache/spark/pull/20677#issuecomment-517888913 For me, it looks correct. Data `12:21, owl` arrives at 12:19 with the event time `12:21`, doesn't it? > Point 12:21 for owl can't have 12:19 processing time. Since Spark received the data, Spark processed it. As a result, the point will be assigned to the window `12:15 ~ 12:25` and `12:20 ~ 12:30` although those windows are hidden in the picture. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] dongjoon-hyun edited a comment on issue #20677: [MINOR][DOC][SS] Event time can't be greater then processing time. 12:21, owl. Mistake…
dongjoon-hyun edited a comment on issue #20677: [MINOR][DOC][SS] Event time can't be greater then processing time. 12:21, owl. Mistake… URL: https://github.com/apache/spark/pull/20677#issuecomment-517888913 For me, it looks correct. Data `12:21, owl` arrives at 12:19 with the event time `12:21`, doesn't it? > Point 12:21 for owl can't have 12:19 processing time. The point will be assigned to the window `12:15 ~ 12:25` and `12:20 ~ 12:30` although those windows are hidden in the picture. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on issue #20677: [MINOR][DOC][SS] Event time can't be greater then processing time. 12:21, owl. Mistake…
dongjoon-hyun commented on issue #20677: [MINOR][DOC][SS] Event time can't be greater then processing time. 12:21, owl. Mistake… URL: https://github.com/apache/spark/pull/20677#issuecomment-517888913 For me, it looks correct. Data `12:21, owl` arrives at 12:19 with the event time `12:21`, doesn't it? > Point 12:21 for owl can't have 12:19 processing time. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #25341: [SPARK-28607][CORE][SHUFFLE] Don't store partition lengths twice.
AmplabJenkins removed a comment on issue #25341: [SPARK-28607][CORE][SHUFFLE] Don't store partition lengths twice. URL: https://github.com/apache/spark/pull/25341#issuecomment-517887056 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/108588/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #25341: [SPARK-28607][CORE][SHUFFLE] Don't store partition lengths twice.
AmplabJenkins removed a comment on issue #25341: [SPARK-28607][CORE][SHUFFLE] Don't store partition lengths twice. URL: https://github.com/apache/spark/pull/25341#issuecomment-517887055 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #25341: [SPARK-28607][CORE][SHUFFLE] Don't store partition lengths twice.
AmplabJenkins commented on issue #25341: [SPARK-28607][CORE][SHUFFLE] Don't store partition lengths twice. URL: https://github.com/apache/spark/pull/25341#issuecomment-517887056 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/108588/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #25341: [SPARK-28607][CORE][SHUFFLE] Don't store partition lengths twice.
AmplabJenkins commented on issue #25341: [SPARK-28607][CORE][SHUFFLE] Don't store partition lengths twice. URL: https://github.com/apache/spark/pull/25341#issuecomment-517887055 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #25341: [SPARK-28607][CORE][SHUFFLE] Don't store partition lengths twice.
SparkQA removed a comment on issue #25341: [SPARK-28607][CORE][SHUFFLE] Don't store partition lengths twice. URL: https://github.com/apache/spark/pull/25341#issuecomment-517874207 **[Test build #108588 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/108588/testReport)** for PR 25341 at commit [`3b26014`](https://github.com/apache/spark/commit/3b26014a2c6b0b8160e1c1fd925c2541c02a157e). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #25341: [SPARK-28607][CORE][SHUFFLE] Don't store partition lengths twice.
SparkQA commented on issue #25341: [SPARK-28607][CORE][SHUFFLE] Don't store partition lengths twice. URL: https://github.com/apache/spark/pull/25341#issuecomment-517886889 **[Test build #108588 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/108588/testReport)** for PR 25341 at commit [`3b26014`](https://github.com/apache/spark/commit/3b26014a2c6b0b8160e1c1fd925c2541c02a157e). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #25228: [SPARK-28472][SQL][TEST] Add test for thriftserver protocol versions
AmplabJenkins commented on issue #25228: [SPARK-28472][SQL][TEST] Add test for thriftserver protocol versions URL: https://github.com/apache/spark/pull/25228#issuecomment-517886762 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #25228: [SPARK-28472][SQL][TEST] Add test for thriftserver protocol versions
AmplabJenkins removed a comment on issue #25228: [SPARK-28472][SQL][TEST] Add test for thriftserver protocol versions URL: https://github.com/apache/spark/pull/25228#issuecomment-517886762 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #25228: [SPARK-28472][SQL][TEST] Add test for thriftserver protocol versions
SparkQA removed a comment on issue #25228: [SPARK-28472][SQL][TEST] Add test for thriftserver protocol versions URL: https://github.com/apache/spark/pull/25228#issuecomment-517884865 **[Test build #108593 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/108593/testReport)** for PR 25228 at commit [`1c9afae`](https://github.com/apache/spark/commit/1c9afae88de80a69c872488d520574f37ff95ec3). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #25228: [SPARK-28472][SQL][TEST] Add test for thriftserver protocol versions
AmplabJenkins commented on issue #25228: [SPARK-28472][SQL][TEST] Add test for thriftserver protocol versions URL: https://github.com/apache/spark/pull/25228#issuecomment-517886764 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/108593/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #25228: [SPARK-28472][SQL][TEST] Add test for thriftserver protocol versions
AmplabJenkins removed a comment on issue #25228: [SPARK-28472][SQL][TEST] Add test for thriftserver protocol versions URL: https://github.com/apache/spark/pull/25228#issuecomment-517886764 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/108593/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #25228: [SPARK-28472][SQL][TEST] Add test for thriftserver protocol versions
SparkQA commented on issue #25228: [SPARK-28472][SQL][TEST] Add test for thriftserver protocol versions URL: https://github.com/apache/spark/pull/25228#issuecomment-517886733 **[Test build #108593 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/108593/testReport)** for PR 25228 at commit [`1c9afae`](https://github.com/apache/spark/commit/1c9afae88de80a69c872488d520574f37ff95ec3). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #25022: [SPARK-24695][SQL] Move `CalendarInterval` to org.apache.spark.sql.types package
SparkQA commented on issue #25022: [SPARK-24695][SQL] Move `CalendarInterval` to org.apache.spark.sql.types package URL: https://github.com/apache/spark/pull/25022#issuecomment-517885375 **[Test build #108594 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/108594/testReport)** for PR 25022 at commit [`a07c79f`](https://github.com/apache/spark/commit/a07c79f430579295c17fc618d7722511652f3c4c). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #25228: [SPARK-28472][SQL][TEST] Add test for thriftserver protocol versions
AmplabJenkins removed a comment on issue #25228: [SPARK-28472][SQL][TEST] Add test for thriftserver protocol versions URL: https://github.com/apache/spark/pull/25228#issuecomment-517885236 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #25022: [SPARK-24695][SQL] Move `CalendarInterval` to org.apache.spark.sql.types package
AmplabJenkins removed a comment on issue #25022: [SPARK-24695][SQL] Move `CalendarInterval` to org.apache.spark.sql.types package URL: https://github.com/apache/spark/pull/25022#issuecomment-517885233 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/13682/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #25022: [SPARK-24695][SQL] Move `CalendarInterval` to org.apache.spark.sql.types package
AmplabJenkins removed a comment on issue #25022: [SPARK-24695][SQL] Move `CalendarInterval` to org.apache.spark.sql.types package URL: https://github.com/apache/spark/pull/25022#issuecomment-517885231 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #25228: [SPARK-28472][SQL][TEST] Add test for thriftserver protocol versions
AmplabJenkins removed a comment on issue #25228: [SPARK-28472][SQL][TEST] Add test for thriftserver protocol versions URL: https://github.com/apache/spark/pull/25228#issuecomment-517885238 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/13681/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #25228: [SPARK-28472][SQL][TEST] Add test for thriftserver protocol versions
AmplabJenkins commented on issue #25228: [SPARK-28472][SQL][TEST] Add test for thriftserver protocol versions URL: https://github.com/apache/spark/pull/25228#issuecomment-517885236 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #25022: [SPARK-24695][SQL] Move `CalendarInterval` to org.apache.spark.sql.types package
AmplabJenkins commented on issue #25022: [SPARK-24695][SQL] Move `CalendarInterval` to org.apache.spark.sql.types package URL: https://github.com/apache/spark/pull/25022#issuecomment-517885231 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #25228: [SPARK-28472][SQL][TEST] Add test for thriftserver protocol versions
AmplabJenkins commented on issue #25228: [SPARK-28472][SQL][TEST] Add test for thriftserver protocol versions URL: https://github.com/apache/spark/pull/25228#issuecomment-517885238 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/13681/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #25022: [SPARK-24695][SQL] Move `CalendarInterval` to org.apache.spark.sql.types package
AmplabJenkins commented on issue #25022: [SPARK-24695][SQL] Move `CalendarInterval` to org.apache.spark.sql.types package URL: https://github.com/apache/spark/pull/25022#issuecomment-517885233 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/13682/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #25022: [SPARK-24695][SQL] Move `CalendarInterval` to org.apache.spark.sql.types package
AmplabJenkins removed a comment on issue #25022: [SPARK-24695][SQL] Move `CalendarInterval` to org.apache.spark.sql.types package URL: https://github.com/apache/spark/pull/25022#issuecomment-517883541 Can one of the admins verify this patch? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on issue #25022: [SPARK-24695][SQL] Move `CalendarInterval` to org.apache.spark.sql.types package
HyukjinKwon commented on issue #25022: [SPARK-24695][SQL] Move `CalendarInterval` to org.apache.spark.sql.types package URL: https://github.com/apache/spark/pull/25022#issuecomment-517885081 ping @priyankagargnitk This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] shivsood commented on issue #25344: [WIP][SPARK-28151][SQL] Mapped ByteType to TinyINT for MsSQLServerDialect
shivsood commented on issue #25344: [WIP][SPARK-28151][SQL] Mapped ByteType to TinyINT for MsSQLServerDialect URL: https://github.com/apache/spark/pull/25344#issuecomment-517885054 @dongjoon-hyun @srowen This is PR for ByteType fix. I am seeing a catalyst exception post my fix. I will need your help/guidance on resolving this. I see that i fails in ValidateExternalType() with mismatched type , but i am not able to get to exact flow of events here. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on issue #25022: [SPARK-24695][SQL] Move `CalendarInterval` to org.apache.spark.sql.types package
HyukjinKwon commented on issue #25022: [SPARK-24695][SQL] Move `CalendarInterval` to org.apache.spark.sql.types package URL: https://github.com/apache/spark/pull/25022#issuecomment-517885062 ok to test This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on issue #25343: [SPARK-28582][PYTHON] Fix flaky test DaemonTests.do_termination_test which fail on Python 3.7
dongjoon-hyun commented on issue #25343: [SPARK-28582][PYTHON] Fix flaky test DaemonTests.do_termination_test which fail on Python 3.7 URL: https://github.com/apache/spark/pull/25343#issuecomment-517885061 Thank you so much for your help for release 2.4.x. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #25344: [WIP][SPARK-28151][SQL] Mapped ByteType to TinyINT for MsSQLServerDialect
AmplabJenkins commented on issue #25344: [WIP][SPARK-28151][SQL] Mapped ByteType to TinyINT for MsSQLServerDialect URL: https://github.com/apache/spark/pull/25344#issuecomment-517884834 Can one of the admins verify this patch? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #25344: [WIP][SPARK-28151][SQL] Mapped ByteType to TinyINT for MsSQLServerDialect
AmplabJenkins removed a comment on issue #25344: [WIP][SPARK-28151][SQL] Mapped ByteType to TinyINT for MsSQLServerDialect URL: https://github.com/apache/spark/pull/25344#issuecomment-517884622 Can one of the admins verify this patch? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #25228: [SPARK-28472][SQL][TEST] Add test for thriftserver protocol versions
SparkQA commented on issue #25228: [SPARK-28472][SQL][TEST] Add test for thriftserver protocol versions URL: https://github.com/apache/spark/pull/25228#issuecomment-517884865 **[Test build #108593 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/108593/testReport)** for PR 25228 at commit [`1c9afae`](https://github.com/apache/spark/commit/1c9afae88de80a69c872488d520574f37ff95ec3). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] gatorsmile commented on issue #25228: [SPARK-28472][SQL][TEST] Add test for thriftserver protocol versions
gatorsmile commented on issue #25228: [SPARK-28472][SQL][TEST] Add test for thriftserver protocol versions URL: https://github.com/apache/spark/pull/25228#issuecomment-517884768 retest this please This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #25344: [WIP][SPARK-28151][SQL] Mapped ByteType to TinyINT for MsSQLServerDialect
AmplabJenkins removed a comment on issue #25344: [WIP][SPARK-28151][SQL] Mapped ByteType to TinyINT for MsSQLServerDialect URL: https://github.com/apache/spark/pull/25344#issuecomment-517884589 Can one of the admins verify this patch? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #25344: [WIP][SPARK-28151][SQL] Mapped ByteType to TinyINT for MsSQLServerDialect
AmplabJenkins commented on issue #25344: [WIP][SPARK-28151][SQL] Mapped ByteType to TinyINT for MsSQLServerDialect URL: https://github.com/apache/spark/pull/25344#issuecomment-517884589 Can one of the admins verify this patch? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #25344: [WIP][SPARK-28151][SQL] Mapped ByteType to TinyINT for MsSQLServerDialect
AmplabJenkins commented on issue #25344: [WIP][SPARK-28151][SQL] Mapped ByteType to TinyINT for MsSQLServerDialect URL: https://github.com/apache/spark/pull/25344#issuecomment-517884622 Can one of the admins verify this patch? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] shivsood opened a new pull request #25344: [WIP][SPARK-2815][SQL] Mapped ByteType to TinyINT for MsSQLServerDialect
shivsood opened a new pull request #25344: [WIP][SPARK-2815][SQL] Mapped
ByteType to TinyINT for MsSQLServerDialect
URL: https://github.com/apache/spark/pull/25344
Writing dataframe with column type BYTETYPE fails when using JDBC connector
for SQL Server. The problem is due to
- (Write path) Incorrect mapping of BYTETYPE in getCommonJDBCType() in
jdbcutils.scala where BYTETYPE gets mapped to BYTE text. It should be mapped to
TINYINT
case ByteType => Option(JdbcType("BYTE", java.sql.Types.TINYINT))
In getCatalystType() ( JDBC to Catalyst type mapping) TINYINT is mapped to
INTEGER, while it should be mapped to BYTETYPE. Mapping to integer is ok from
the point of view of upcasting, but will lead to 4 byte allocation rather than
1 byte for BYTETYPE.
- (read path) Read path ends up calling makeGetter(dt: DataType, metadata:
Metadata). The function sets the value in RDD row. The value is set per the
data type. Here there is no mapping for BYTETYPE and thus results will result
in an error when getCatalystType() is fixed.
Note : These issues were found when reading/writing with SQLServer.
Error seen when writing table
```
(JDBC Write failed,com.microsoft.sqlserver.jdbc.SQLServerException: Column,
parameter, or variable #2: Cannot find data type BYTE.)
com.microsoft.sqlserver.jdbc.SQLServerException: Column, parameter, or
variable #2: Cannot find data type BYTE.
com.microsoft.sqlserver.jdbc.SQLServerException.makeFromDatabaseError(SQLServerException.java:254)
com.microsoft.sqlserver.jdbc.SQLServerStatement.getNextResult(SQLServerStatement.java:1608)
com.microsoft.sqlserver.jdbc.SQLServerStatement.doExecuteStatement(SQLServerStatement.java:859)
```
## What changes were proposed in this pull request?
Change mapping of BYTETYPE from BYTE to TinyINT.
(Please fill in changes proposed in this fix)
## How was this patch tested?
Path was tested with integration test by adding test case to
MsSqlServerIntegrationSuite.scala.
The fix fails with the following error when df.write is done with a
dataframe what contains a ByteType.
```
19/08/02 18:25:44 INFO Executor: Finished task 0.0 in stage 7.0 (TID 7).
1197 bytes result sent to driver
19/08/02 18:25:44 INFO TaskSetManager: Finished task 0.0 in stage 7.0 (TID
7) in 43 ms on localhost (executor driver) (1/2)
19/08/02 18:25:44 INFO CodeGenerator: Code generated in 14.586963 ms
19/08/02 18:25:44 ERROR Executor: Exception in task 1.0 in stage 7.0 (TID 8)
java.lang.RuntimeException: Error while encoding:
java.lang.RuntimeException: java.lang.Integer is not a valid external type for
schema of tinyint
if (assertnotnull(input[0, org.apache.spark.sql.Row, true]).isNullAt) null
else validateexternaltype(getexternalrowfield(assertnotnull(input[0,
org.apache.spark.sql.Row, true]), 0, serialNum), ByteType) AS serialNum#231
at
org.apache.spark.sql.catalyst.encoders.ExpressionEncoder.toRow(ExpressionEncoder.scala:344)
at
org.apache.spark.sql.SparkSession.$anonfun$createDataFrame$1(SparkSession.scala:367)
at scala.collection.Iterator$$anon$10.next(Iterator.scala:459)
at
org.apache.spark.sql.catalyst.expressions.GeneratedClass$GeneratedIteratorForCodegenStage1.processNext(Unknown
Source)
at
org.apache.spark.sql.execution.BufferedRowIterator.hasNext(BufferedRowIterator.java:43)
at
org.apache.spark.sql.execution.WholeStageCodegenExec$$anon$1.hasNext(WholeStageCodegenExec.scala:731)
at scala.collection.Iterator$$anon$10.hasNext(Iterator.scala:458)
at scala.collection.Iterator$$anon$10.hasNext(Iterator.scala:458)
at
org.apache.spark.sql.execution.datasources.jdbc.JdbcUtils$.savePartition(JdbcUtils.scala:662)
at
org.apache.spark.sql.execution.datasources.jdbc.JdbcUtils$.$anonfun$saveTable$1(JdbcUtils.scala:845)
```
Please review https://spark.apache.org/contributing.html before opening a
pull request.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on issue #25343: [SPARK-28582][PYTHON] Fix flaky test DaemonTests.do_termination_test which fail on Python 3.7
HyukjinKwon commented on issue #25343: [SPARK-28582][PYTHON] Fix flaky test DaemonTests.do_termination_test which fail on Python 3.7 URL: https://github.com/apache/spark/pull/25343#issuecomment-517884393 FYI, I am manually running the test against branch-2.4 in my local to double check since 2.4.x release is coming. Sorry for this trouble, @dongjoon-hyun. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on issue #25343: [SPARK-28582][PYTHON] Fix flaky test DaemonTests.do_termination_test which fail on Python 3.7
dongjoon-hyun commented on issue #25343: [SPARK-28582][PYTHON] Fix flaky test DaemonTests.do_termination_test which fail on Python 3.7 URL: https://github.com/apache/spark/pull/25343#issuecomment-517884306 Thank you for quick fix~ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #25304: [SPARK-28570][CORE][SHUFFLE] Make UnsafeShuffleWriter use the new API.
AmplabJenkins commented on issue #25304: [SPARK-28570][CORE][SHUFFLE] Make UnsafeShuffleWriter use the new API. URL: https://github.com/apache/spark/pull/25304#issuecomment-517884122 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #25304: [SPARK-28570][CORE][SHUFFLE] Make UnsafeShuffleWriter use the new API.
AmplabJenkins removed a comment on issue #25304: [SPARK-28570][CORE][SHUFFLE] Make UnsafeShuffleWriter use the new API. URL: https://github.com/apache/spark/pull/25304#issuecomment-517884124 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/108587/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #25304: [SPARK-28570][CORE][SHUFFLE] Make UnsafeShuffleWriter use the new API.
AmplabJenkins commented on issue #25304: [SPARK-28570][CORE][SHUFFLE] Make UnsafeShuffleWriter use the new API. URL: https://github.com/apache/spark/pull/25304#issuecomment-517884124 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/108587/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #25304: [SPARK-28570][CORE][SHUFFLE] Make UnsafeShuffleWriter use the new API.
AmplabJenkins removed a comment on issue #25304: [SPARK-28570][CORE][SHUFFLE] Make UnsafeShuffleWriter use the new API. URL: https://github.com/apache/spark/pull/25304#issuecomment-517884122 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #25304: [SPARK-28570][CORE][SHUFFLE] Make UnsafeShuffleWriter use the new API.
SparkQA removed a comment on issue #25304: [SPARK-28570][CORE][SHUFFLE] Make UnsafeShuffleWriter use the new API. URL: https://github.com/apache/spark/pull/25304#issuecomment-517868754 **[Test build #108587 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/108587/testReport)** for PR 25304 at commit [`fc798b7`](https://github.com/apache/spark/commit/fc798b7fbbe921470fc4425ac4b913ae14b48e46). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #25304: [SPARK-28570][CORE][SHUFFLE] Make UnsafeShuffleWriter use the new API.
SparkQA commented on issue #25304: [SPARK-28570][CORE][SHUFFLE] Make UnsafeShuffleWriter use the new API. URL: https://github.com/apache/spark/pull/25304#issuecomment-517884000 **[Test build #108587 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/108587/testReport)** for PR 25304 at commit [`fc798b7`](https://github.com/apache/spark/commit/fc798b7fbbe921470fc4425ac4b913ae14b48e46). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] HyukjinKwon closed pull request #25343: [SPARK-28582][PYTHON] Fix flaky test DaemonTests.do_termination_test which fail on Python 3.7
HyukjinKwon closed pull request #25343: [SPARK-28582][PYTHON] Fix flaky test DaemonTests.do_termination_test which fail on Python 3.7 URL: https://github.com/apache/spark/pull/25343 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #25339: [SPARK-28606][INFRA] Update CRAN key to recover docker image generation
AmplabJenkins commented on issue #25339: [SPARK-28606][INFRA] Update CRAN key to recover docker image generation URL: https://github.com/apache/spark/pull/25339#issuecomment-517883724 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/108586/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #25339: [SPARK-28606][INFRA] Update CRAN key to recover docker image generation
AmplabJenkins commented on issue #25339: [SPARK-28606][INFRA] Update CRAN key to recover docker image generation URL: https://github.com/apache/spark/pull/25339#issuecomment-517883721 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #25339: [SPARK-28606][INFRA] Update CRAN key to recover docker image generation
AmplabJenkins removed a comment on issue #25339: [SPARK-28606][INFRA] Update CRAN key to recover docker image generation URL: https://github.com/apache/spark/pull/25339#issuecomment-517883724 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/108586/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #25339: [SPARK-28606][INFRA] Update CRAN key to recover docker image generation
AmplabJenkins removed a comment on issue #25339: [SPARK-28606][INFRA] Update CRAN key to recover docker image generation URL: https://github.com/apache/spark/pull/25339#issuecomment-517883721 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #25339: [SPARK-28606][INFRA] Update CRAN key to recover docker image generation
SparkQA commented on issue #25339: [SPARK-28606][INFRA] Update CRAN key to recover docker image generation URL: https://github.com/apache/spark/pull/25339#issuecomment-517883603 **[Test build #108586 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/108586/testReport)** for PR 25339 at commit [`ba390bd`](https://github.com/apache/spark/commit/ba390bd463d3504759792f5d2549bbaff0017dc6). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #25339: [SPARK-28606][INFRA] Update CRAN key to recover docker image generation
SparkQA removed a comment on issue #25339: [SPARK-28606][INFRA] Update CRAN key to recover docker image generation URL: https://github.com/apache/spark/pull/25339#issuecomment-517866189 **[Test build #108586 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/108586/testReport)** for PR 25339 at commit [`ba390bd`](https://github.com/apache/spark/commit/ba390bd463d3504759792f5d2549bbaff0017dc6). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on issue #25343: [SPARK-28582][PYTHON] Fix flaky test DaemonTests.do_termination_test which fail on Python 3.7
HyukjinKwon commented on issue #25343: [SPARK-28582][PYTHON] Fix flaky test DaemonTests.do_termination_test which fail on Python 3.7 URL: https://github.com/apache/spark/pull/25343#issuecomment-517883532 cc @dongjoon-hyun This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on issue #25343: [SPARK-28582][PYTHON] Fix flaky test DaemonTests.do_termination_test which fail on Python 3.7
HyukjinKwon commented on issue #25343: [SPARK-28582][PYTHON] Fix flaky test DaemonTests.do_termination_test which fail on Python 3.7 URL: https://github.com/apache/spark/pull/25343#issuecomment-517883544 Merged to master, branch-2.4 and branch-2.3. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
