darion yaphet created SPARK-21233:
-------------------------------------
Summary: Support pluggable offset storage
Key: SPARK-21233
URL: https://issues.apache.org/jira/browse/SPARK-21233
Project: Spark
Issue Type: New Feature
Components: DStreams
Affects Versions: 2.1.1, 2.0.2
Reporter: darion yaphet
Currently we using *ZooKeeper* to save the *Kafka Commit Offset* , when there
are a lot of streaming program running in the cluster the ZooKeeper Cluster's
loading is very high . Maybe Zookeeper is not very suitable to save offset
periodicity.
This issue is wish to support a pluggable offset storage to avoid save it in
the zookeeper
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
