Re: Back-pressure to Spark Kafka Streaming?

If you want finer-grained max rate setting, SPARK-17510 got merged a while ago. There's also SPARK-18580 which might help address the issue of starting backpressure rate for the first batch. On Mon, Dec 5, 2016 at 4:18 PM, Liren Ding wrote: > Hey all, > > Does

Re: Back-pressure to Spark Kafka Streaming?

tartin.com/ From: Liren Ding <sky.gonna.bri...@gmail.com> Sent: 05 December 2016 22:18 To: d...@spark.apache.org; user@spark.apache.org Subject: Back-pressure to Spark Kafka Streaming? Hey all, Does backressure actually work on spark kafka streaming? According to the latest spark st

Back-pressure to Spark Kafka Streaming?

Hey all, Does backressure actually work on spark kafka streaming? According to the latest spark streaming document: *http://spark.apache.org/docs/latest/streaming-programming-guide.html * "*In Spark 1.5, we have introduced a