[
https://issues.apache.org/jira/browse/FLUME-2230?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ralph Goers resolved FLUME-2230.
--------------------------------
Resolution: Won't Fix
> Configuring elasticsearch-sink hostNames parameter in AWS
> ---------------------------------------------------------
>
> Key: FLUME-2230
> URL: https://issues.apache.org/jira/browse/FLUME-2230
> Project: Flume
> Issue Type: Question
> Environment: AWS, centos 6
> Reporter: Nikolaos Tsipas
> Priority: Major
> Labels: aws, elasticsearch, sink
>
> Hello,
> We are using flume elasticsearch-sink in AWS and for the {{hostNames}}
> parameter of the sink we use the A record of an internal ELB (Elastic Load
> Balancer). When we do a {{nslookup}} on the load balancer's hostname we get
> back a list of node IPs (in our case we have 3 elasticsearch nodes).
> *config example*
> {code}
> a1.sinks.elasticsearch-sink.hostNames =
> componen-1RSEO3YX5OD3Z-729046292.eu-west-1.elb.amazonaws.com
> {code}
> This config works fine as long as the IPs of the elasticsearch nodes remain
> the same. If we restart one of our elasticsearch nodes, a new IP is assigned
> to it and flume stops being able to communicate with that node.
> In the source code of the elasticsearch-sink I can see that a list of
> {{InetSocketTransportAddress}} objects is created and this is probably the
> reason why flume stops working when we have an IP change and starts working
> only after a restart of the flume-ng-agent service.
> *ElasticSearchSink.java*
> {code}serverAddresses[i] = new InetSocketTransportAddress(host, port);{code}
> +Questions+:
> * Which is the suggested configuration for our case? Should we use static IPs
> for our elasticsearch nodes and then use a comma separated list of these IPs
> in flume configuration?
> * Would be possible to use the A record of the ELB in such a way that flume
> would always hit the A record to get one of the available IP addresses? Does
> this sound feasible and worth spending some time on submitting a patch?
> Regards,
> Nick
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
