GitHub user HeartSaVioR opened a pull request:
https://github.com/apache/spark/pull/21222
[SPARK-24161][SS] Enable debug package feature on structured streaming
## What changes were proposed in this pull request?
Currently, debug package has a implicit class "DebugQuery" which matches
Dataset to provide debug features on Dataset class. It doesn't work with
structured streaming: it requires query is already started, and the information
can be retrieved from StreamingQuery, not Dataset. I guess that's why "explain"
had to be placed to StreamingQuery whereas it already exists on Dataset.
This patch adds a new implicit class "DebugStreamQuery" which matches
StreamingQuery to provide similar debug features on StreamingQuery class.
## How was this patch tested?
Added relevant unit tests.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/HeartSaVioR/spark SPARK-24161
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/21222.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #21222
----
commit c1ad1c557e6165455457adb6f148d6d9616548a1
Author: Jungtaek Lim <kabhwan@...>
Date: 2018-05-03T02:26:48Z
SPARK-24161 Enable debug package feature on structured streaming
* added implicit class which adds debug features for StreamingQuery
* added unit tests for new functionalities
----
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
