[ https://issues.apache.org/jira/browse/SPARK-7111?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Tathagata Das updated SPARK-7111: --------------------------------- Assignee: Saisai Shao > Exposing of input data rates of non-receiver streams like Kafka Direct stream > ----------------------------------------------------------------------------- > > Key: SPARK-7111 > URL: https://issues.apache.org/jira/browse/SPARK-7111 > Project: Spark > Issue Type: Improvement > Components: Streaming > Reporter: Saisai Shao > Assignee: Saisai Shao > > Currently for receiver-based input streams, Spark Streaming offers > ReceiverTracker and ReceivedBlockTracker to track the status of receivers as > well as block information. Also this status and block information can be > retrieved from StreamingListener to expose to the users. > But for direct-based (receiver-less) input streams, Current Spark Streaming > lacks such mechanism to track the registered direct streams, also lacks the > way to track the processed number of data for direct-based input streams. > Here propose a mechanism to track the register direct stream, also expose the > processing statistics to the BatchInfo and StreamingListener. -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org