[
https://issues.apache.org/jira/browse/SPARK-7111?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14521078#comment-14521078
]
Tathagata Das commented on SPARK-7111:
--------------------------------------
[~jerryshao] Since you are implementing this new design (thanks!), I am
updating the JIRA title accordingly.
> Add a tracker to track the direct (receiver-less) streams
> ---------------------------------------------------------
>
> Key: SPARK-7111
> URL: https://issues.apache.org/jira/browse/SPARK-7111
> Project: Spark
> Issue Type: Improvement
> Components: Streaming
> Reporter: Saisai Shao
>
> Currently for receiver-based input streams, Spark Streaming offers
> ReceiverTracker and ReceivedBlockTracker to track the status of receivers as
> well as block information. Also this status and block information can be
> retrieved from StreamingListener to expose to the users.
> But for direct-based (receiver-less) input streams, Current Spark Streaming
> lacks such mechanism to track the registered direct streams, also lacks the
> way to track the processed number of data for direct-based input streams.
> Here propose a mechanism to track the register direct stream, also expose the
> processing statistics to the BatchInfo and StreamingListener.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
