duheng created COMDEV-349:
-----------------------------
Summary: RocketMQ Connect Elasticsearch
Key: COMDEV-349
URL: https://issues.apache.org/jira/browse/COMDEV-349
Project: Community Development
Issue Type: Wish
Components: GSoC/Mentoring ideas
Reporter: duheng
h3. Content
The Elasticsearch sink connector allows moving data from Apache RocketMQ to
Elasticsearch 6.x, and 7.x. It writes data from a topic in Apache RocketMQ to
an index in Elasticsearch and all data for a topic have the same type.
[Elasticsearch|https://www.elastic.co/] is often used for text queries,
analytics and as an key-value store (use cases). The connector covers both the
analytics and key-value store use cases.
For the analytics use case, each message is in RocketMQ is treated as an event
and the connector uses topic+message queue+offset as a unique identifier for
events, which then converted to unique documents in Elasticsearch. For the
key-value store use case, it supports using keys from RocketMQ messages as
document ids in Elasticsearch and provides configurations ensuring that updates
to a key are written to Elasticsearch in order.
So, in this project, you need to implement a sink connector based on
OpenMessaging connect API, and will executed on RocketMQ connect runtime.
h3. You should learn before applying for this topic
[Elasticsearch|https://www.elastic.co/]/[Apache
RocketMQ|https://rocketmq.apache.org/]/[Apache RocketMQ
Connect|https://github.com/apache/rocketmq-externals/tree/master/rocketmq-connect]/
[OpenMessaging Connect
API|https://github.com/openmessaging/openmessaging-connect]
h3. Mentor
[[email protected]|mailto:[email protected]],
[[email protected]|mailto:[email protected]]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
