[ https://issues.apache.org/jira/browse/GEARPUMP-167?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16004425#comment-16004425 ]
Karol Brejna commented on GEARPUMP-167: --------------------------------------- [~yanghua] I started developing PoC on the following branch: https://github.com/karol-brejna-i/incubator-gearpump/tree/GEARPUMP-167-elasticsearch-sink My idea was to have a simple sink that would just accept a JSON document to be stored in elasticsearch (not an object that would need to be serialized to JSON). I anticipated two cases for the incoming message: * Tuple2[String, String] - id, body - message gets stored under given id, so it's actually an upsert * String - body - message is stored under new id * A sequence of any of above I even did some short "demo" that had: * small app subscribing to Twitter and putting messages to Kafka * a DAG that consumed messages from Kafka, did wordcount, stored data to elasticsearch * Kibana visualization The code was written in Java. I planned to convert it to Scala but got distracted by the last release... [~yanghua] I'd be glad to hear your idea on this. Maybe we could cooperate on this feature. > Create Elasticsearch sink and example > ------------------------------------- > > Key: GEARPUMP-167 > URL: https://issues.apache.org/jira/browse/GEARPUMP-167 > Project: Apache Gearpump > Issue Type: Task > Components: connectors > Reporter: Karol Brejna > Assignee: Karol Brejna > Labels: feature > Original Estimate: 18h > Remaining Estimate: 18h > > Create a simple Elasticsearch sink that enables indexing messages as > Elasticsearch docs. > Write an example demonstrating usage of the sink > Write a doc on setting up local development env (installing and confifuring > elastic, etc.) for the users of the sink -- This message was sent by Atlassian JIRA (v6.3.15#6346)