Have you seen http://stackoverflow.com/questions/29051579/pausing-throttling-spark-spark-streaming-application ?
Cheers On Wed, May 27, 2015 at 4:11 PM, dgoldenberg <dgoldenberg...@gmail.com> wrote: > Hi, > > With the no receivers approach to streaming from Kafka, is there a way to > set something like spark.streaming.receiver.maxRate so as not to overwhelm > the Spark consumers? > > What would be some of the ways to throttle the streamed messages so that > the > consumers don't run out of memory? > > > > > > -- > View this message in context: > http://apache-spark-user-list.1001560.n3.nabble.com/Spark-Streaming-from-Kafka-no-receivers-and-spark-streaming-receiver-maxRate-tp23061.html > Sent from the Apache Spark User List mailing list archive at Nabble.com. > > --------------------------------------------------------------------- > To unsubscribe, e-mail: user-unsubscr...@spark.apache.org > For additional commands, e-mail: user-h...@spark.apache.org > >
