Eren Avsarogullari created BAHIR-135:
----------------------------------------
Summary: Add Spark Streaming Hazelcast Extension
Key: BAHIR-135
URL: https://issues.apache.org/jira/browse/BAHIR-135
Project: Bahir
Issue Type: New Feature
Components: Spark Streaming Connectors
Reporter: Eren Avsarogullari
I would like to propose Spark Streaming Hazelcast extension.
Hazelcast is an in-memory data grid(IMDG) solution under Apache 2 License and
provides distributed data structures such as distributed map, list, set, queue
(etc). When a new entry is _added_, _updated_, _removed_ or _evicted_, a new
event is fired by Hazelcast. This flow is almost same for above all distributed
data structures. This extension aims to subscribe these distributed events via
Hazelcast Event Listeners and create a DStream in the light of distributed data
structure changes. This extension supports Distributed Map, List, Set, Queue,
Topic, MultiMap and Replicated Map.
Please find the following documentation for further details.
*Proposal:*
[https://docs.google.com/document/d/1YN_9u72Wv699g8ivM3c8K_zZUbUl73JtquWy-g71Tm4/edit?usp=sharing]
Also repo is ready for review. It covers implementation, full unit test
coverage and examples as well.
*Repo:* [https://github.com/erenavsarogullari/bahir/tree/Hazelcast_Streaming]
This extension can be useful for both Spark and Hazelcast communities to listen
these Hazelcast events & analyze them and transform the events payloads via
Spark.
Please let me know if you need further details and all feedbacks are welcome in
advance.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
