[GitHub] spark pull request: [SPARK-15375][SQL][Streaming] Add ConsoleSink ...
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/13162 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-15375][SQL][Streaming] Add ConsoleSink ...
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/13162#issuecomment-220489803 Merging in master/2.0. Thanks @jerryshao! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-15375][SQL][Streaming] Add ConsoleSink ...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/13162#issuecomment-219990713 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-15375][SQL][Streaming] Add ConsoleSink ...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/13162#issuecomment-219990714 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/58766/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-15375][SQL][Streaming] Add ConsoleSink ...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/13162#issuecomment-219990499 **[Test build #58766 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/58766/consoleFull)** for PR 13162 at commit [`3c04342`](https://github.com/apache/spark/commit/3c04342267d7b79937f9e25034c6ff616c9b5672). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-15375][SQL][Streaming] Add ConsoleSink ...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/13162#issuecomment-219989695 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-15375][SQL][Streaming] Add ConsoleSink ...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/13162#issuecomment-219989697 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/58765/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-15375][SQL][Streaming] Add ConsoleSink ...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/13162#issuecomment-219989489 **[Test build #58765 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/58765/consoleFull)** for PR 13162 at commit [`77c5985`](https://github.com/apache/spark/commit/77c5985ddf3044c5146857be600d47b90f9ab9ab). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-15375][SQL][Streaming] Add ConsoleSink ...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/13162#issuecomment-219972084 **[Test build #58766 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/58766/consoleFull)** for PR 13162 at commit [`3c04342`](https://github.com/apache/spark/commit/3c04342267d7b79937f9e25034c6ff616c9b5672). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-15375][SQL][Streaming] Add ConsoleSink ...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/13162#issuecomment-219970908 **[Test build #58765 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/58765/consoleFull)** for PR 13162 at commit [`77c5985`](https://github.com/apache/spark/commit/77c5985ddf3044c5146857be600d47b90f9ab9ab). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-15375][SQL][Streaming] Add ConsoleSink ...
Github user jerryshao commented on the pull request: https://github.com/apache/spark/pull/13162#issuecomment-219954515 Sure, let me change the code. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-15375][SQL][Streaming] Add ConsoleSink ...
Github user marmbrus commented on the pull request: https://github.com/apache/spark/pull/13162#issuecomment-219943826 I agree this would be useful for debugging. Can we plug it in as a `StreamSinkProvider` instead of hard coding an `if` into the `DataSource` resolver? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-15375][SQL][Streaming] Add ConsoleSink ...
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/13162#issuecomment-219942612 Seem like a good idea to me. cc @marmbrus --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-15375][SQL][Streaming] Add ConsoleSink ...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/13162#issuecomment-219922682 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-15375][SQL][Streaming] Add ConsoleSink ...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/13162#issuecomment-219922683 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/58738/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-15375][SQL][Streaming] Add ConsoleSink ...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/13162#issuecomment-219922548 **[Test build #58738 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/58738/consoleFull)** for PR 13162 at commit [`235f5d2`](https://github.com/apache/spark/commit/235f5d28c0d53697c12b647df94fa9c7670d8faa). * This patch passes all tests. * This patch merges cleanly. * This patch adds the following public classes _(experimental)_: * `class ConsoleSink(options: Map[String, String]) extends Sink with Logging ` --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-15375][SQL][Streaming] Add ConsoleSink ...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/13162#issuecomment-219914340 **[Test build #58738 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/58738/consoleFull)** for PR 13162 at commit [`235f5d2`](https://github.com/apache/spark/commit/235f5d28c0d53697c12b647df94fa9c7670d8faa). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-15375][SQL][Streaming] Add ConsoleSink ...
GitHub user jerryshao opened a pull request:
https://github.com/apache/spark/pull/13162
[SPARK-15375][SQL][Streaming] Add ConsoleSink to structure streaming
## What changes were proposed in this pull request?
Add ConsoleSink to structure streaming, user could use it to display
dataframes on the console (useful for debugging and demostrating), similar to
the functionality of `DStream#print`, to use it:
```
val query = result.write
.format("console")
.trigger(ProcessingTime("2 seconds"))
.startStream()
```
## How was this patch tested?
local verified.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/jerryshao/apache-spark SPARK-15375
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/13162.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #13162
commit 235f5d28c0d53697c12b647df94fa9c7670d8faa
Author: jerryshao
Date: 2016-05-18T02:51:39Z
Add ConsoleSink to structure streaming
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
