Unfortunately, there is no API support for this right now. You could implement it yourself by implementing your own receiver and controlling the rate at which objects are "received". If you are using any of the standard receivers (Flume, Kafka, etc.), I recommended looking at the source code of the corresponding receiver and making your own version of
Alternatively, there is a open JIRA<https://issues.apache.org/jira/browse/SPARK-1341>about this implementing this functionality. You could give it a shot at implementing this in a generic that it can be used for all receivers ;) On Tue, May 20, 2014 at 8:31 PM, Francis.Hu <francis...@reachjunction.com>wrote: > sparkers, > > > > Is there a better way to control memory usage when streaming input's speed > is faster than the speed of handled by spark streaming ? > > > > Thanks, > > Francis.Hu >
