[ https://issues.apache.org/jira/browse/SPARK-17510?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Shixiong Zhu resolved SPARK-17510. ---------------------------------- Resolution: Fixed Assignee: Cody Koeninger Fix Version/s: 2.1.0 > Set Streaming MaxRate Independently For Multiple Streams > -------------------------------------------------------- > > Key: SPARK-17510 > URL: https://issues.apache.org/jira/browse/SPARK-17510 > Project: Spark > Issue Type: Improvement > Components: DStreams > Affects Versions: 2.0.0 > Reporter: Jeff Nadler > Assignee: Cody Koeninger > Fix For: 2.1.0 > > > We use multiple DStreams coming from different Kafka topics in a Streaming > application. > Some settings like maxrate and backpressure enabled/disabled would be better > passed as config to KafkaUtils.createStream and > KafkaUtils.createDirectStream, instead of setting them in SparkConf. > Being able to set a different maxrate for different streams is an important > requirement for us; we currently work-around the problem by using one > receiver-based stream and one direct stream. > We would like to be able to turn on backpressure for only one of the streams > as well. -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org