[
https://issues.apache.org/jira/browse/FLUME-2230?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17614526#comment-17614526
]
Ralph Goers commented on FLUME-2230:
------------------------------------
The Elasticsearch sink is no longer supported in Flume. It has been moved to
its own Flume Search Git repo. If it is revived it will almost certainly
require significant modifications so all existing ES issues are being closed.
> Configuring elasticsearch-sink hostNames parameter in AWS
> ---------------------------------------------------------
>
> Key: FLUME-2230
> URL: https://issues.apache.org/jira/browse/FLUME-2230
> Project: Flume
> Issue Type: Question
> Environment: AWS, centos 6
> Reporter: Nikolaos Tsipas
> Priority: Major
> Labels: aws, elasticsearch, sink
>
> Hello,
> We are using flume elasticsearch-sink in AWS and for the {{hostNames}}
> parameter of the sink we use the A record of an internal ELB (Elastic Load
> Balancer). When we do a {{nslookup}} on the load balancer's hostname we get
> back a list of node IPs (in our case we have 3 elasticsearch nodes).
> *config example*
> {code}
> a1.sinks.elasticsearch-sink.hostNames =
> componen-1RSEO3YX5OD3Z-729046292.eu-west-1.elb.amazonaws.com
> {code}
> This config works fine as long as the IPs of the elasticsearch nodes remain
> the same. If we restart one of our elasticsearch nodes, a new IP is assigned
> to it and flume stops being able to communicate with that node.
> In the source code of the elasticsearch-sink I can see that a list of
> {{InetSocketTransportAddress}} objects is created and this is probably the
> reason why flume stops working when we have an IP change and starts working
> only after a restart of the flume-ng-agent service.
> *ElasticSearchSink.java*
> {code}serverAddresses[i] = new InetSocketTransportAddress(host, port);{code}
> +Questions+:
> * Which is the suggested configuration for our case? Should we use static IPs
> for our elasticsearch nodes and then use a comma separated list of these IPs
> in flume configuration?
> * Would be possible to use the A record of the ELB in such a way that flume
> would always hit the A record to get one of the available IP addresses? Does
> this sound feasible and worth spending some time on submitting a patch?
> Regards,
> Nick
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
