Saisai Shao created SPARK-5718: ---------------------------------- Summary: Change to native offset management for ReliableKafkaReceiver Key: SPARK-5718 URL: https://issues.apache.org/jira/browse/SPARK-5718 Project: Spark Issue Type: Improvement Components: Streaming Affects Versions: 1.3.0 Reporter: Saisai Shao
Kafka 0.8.2 supports native offsets management instead of ZK, this will get better performance, for now in ReliableKafkaReceiver, we rely on ZK to manage the offsets, this potentially will be a bottleneck if the injection rate is high (once per 200ms by default), so here in order to get better performance as well as keeping consistent with Kafka, add native offset management for ReliableKafkaReceiver. -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org