Rodrigo Boavida created SPARK-12178:
---------------------------------------
Summary: Expose reporting of StreamInputInfo for custom made
streams
Key: SPARK-12178
URL: https://issues.apache.org/jira/browse/SPARK-12178
Project: Spark
Issue Type: Improvement
Components: Streaming
Reporter: Rodrigo Boavida
Priority: Minor
For custom made direct streams, the Spark Streaming context needs to be
informed of the RDD count per batch execution. This is not exposed by the
InputDStream abstract class.
The suggestion is to create a method in the InputDStream class that reports to
the streaming context and make that available to child classes of InputDStream.
Signature example:
def reportInfo(validTime : org.apache.spark.streaming.Time, inputInfo :
org.apache.spark.streaming.scheduler.StreamInputInfo)
I have already done this on my own private branch. I can merge that change in
if approval is given.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
