Downgraded to Elasticsearch 1.7.5 and still get the same error :
RROR
org.apache.flume.lifecycle.LifecycleSupervisor
Unable to start SinkRunner: {
policy:org.apache.flume.sink.DefaultSinkProcessor@730d4fcf counterGroup:{
name:null counters:{} } } - Exception follows.
java.lang.NoSuchMethodError:
org.elasticsearch.common.transport.InetSocketTransportAddress.<init>(Ljava/lang/String;I)V
at
org.apache.flume.sink.elasticsearch.client.ElasticSearchTransportClient.configureHostnames(ElasticSearchTransportClient.java:143)
at
org.apache.flume.sink.elasticsearch.client.ElasticSearchTransportClient.<init>(ElasticSearchTransportClient.java:77)
at
org.apache.flume.sink.elasticsearch.client.ElasticSearchClientFactory.getClient(ElasticSearchClientFactory.java:48)
at
org.apache.flume.sink.elasticsearch.ElasticSearchSink.start(ElasticSearchSink.java:357)
at
org.apache.flume.sink.DefaultSinkProcessor.start(DefaultSinkProcessor.java:46)
at org.apache.flume.SinkRunner.start(SinkRunner.java:79)
at
org.apache.flume.lifecycle.LifecycleSupervisor$MonitorRunnable.run(LifecycleSupervisor.java:251)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
From: iain wright [mailto:[email protected]]
Sent: Monday, May 02, 2016 2:52 PM
To: [email protected]
Cc: Mabrito,Greg
Subject: Re: Flume - elasticsearch sink fails
Hi,
These look relevant to your goals:
https://github.com/elastic/elasticsearch/issues/14187<https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_elastic_elasticsearch_issues_14187&d=CwMFaQ&c=RI9dKKMRNVHr9NFa7OQiQw&r=dUN85GiSQZVDs0gTK4x1mSiAdXTZ-7F0KzGt2fcse38&m=31csM049OHrrKBoryULOX8hgtSb6qxzPl-6Z_c_-N0A&s=p6q-rN0ma3I-5GVFu0e_CASXMGrDeforSajvFopvNic&e=>
https://github.com/lucidfrontier45/ElasticsearchSink2<https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_lucidfrontier45_ElasticsearchSink2&d=CwMFaQ&c=RI9dKKMRNVHr9NFa7OQiQw&r=dUN85GiSQZVDs0gTK4x1mSiAdXTZ-7F0KzGt2fcse38&m=31csM049OHrrKBoryULOX8hgtSb6qxzPl-6Z_c_-N0A&s=RSyBce-WtursnyucSVeewjWRYtSDC1dkIbXOpE3boRc&e=>
HTH,
--
Iain Wright
This email message is confidential, intended only for the recipient(s) named
above and may contain information that is privileged, exempt from disclosure
under applicable law. If you are not the intended recipient, do not disclose or
disseminate the message to anyone except the intended recipient. If you have
received this message in error, or are not the named recipient(s), please
immediately notify the sender by return email, and delete all copies of this
message.
On Mon, May 2, 2016 at 12:44 PM, Arif,Mubaraka
<[email protected]<mailto:[email protected]>> wrote:
Hello,
Trying to flume tweets from Tibco-JMS source into elastic search sink.
Running Elasticsearch on our edge nodes independent of Hadoop.
Running cloudera Hadoop 5.4.8
On running the error is :
May 2, 2:15:14.047 PM
INFO
org.apache.flume.instrumentation.MonitoredCounterGroup
Component type: SINK, name: k1 started
May 2, 2:15:14.050 PM
WARN
org.apache.flume.sink.elasticsearch.client.ElasticSearchTransportClient
[arcapl0021707:9200]
May 2, 2:15:14.052 PM
ERROR
org.apache.flume.lifecycle.LifecycleSupervisor
Unable to start SinkRunner: {
policy:org.apache.flume.sink.DefaultSinkProcessor@50482278 counterGroup:{
name:null counters:{} } } - Exception follows.
java.lang.NoSuchMethodError:
org.elasticsearch.common.transport.InetSocketTransportAddress.<init>(Ljava/lang/String;I)V
at
org.apache.flume.sink.elasticsearch.client.ElasticSearchTransportClient.configureHostnames(ElasticSearchTransportClient.java:143)
at
org.apache.flume.sink.elasticsearch.client.ElasticSearchTransportClient.<init>(ElasticSearchTransportClient.java:77)
at
org.apache.flume.sink.elasticsearch.client.ElasticSearchClientFactory.getClient(ElasticSearchClientFactory.java:48)
at
org.apache.flume.sink.elasticsearch.ElasticSearchSink.start(ElasticSearchSink.java:357)
at
org.apache.flume.sink.DefaultSinkProcessor.start(DefaultSinkProcessor.java:46)
at org.apache.flume.SinkRunner.start(SinkRunner.java:79)
at
org.apache.flume.lifecycle.LifecycleSupervisor$MonitorRunnable.run(LifecycleSupervisor.java:251)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Flume configuration for ElasticSearch is :
#confgiure elastic search sink
test.sinks.k1.type = elasticsearch
test.sinks.k1.hostNames =
172.17.43.207:9200<https://urldefense.proofpoint.com/v2/url?u=http-3A__172.17.43.207-3A9200&d=CwMFaQ&c=RI9dKKMRNVHr9NFa7OQiQw&r=dUN85GiSQZVDs0gTK4x1mSiAdXTZ-7F0KzGt2fcse38&m=31csM049OHrrKBoryULOX8hgtSb6qxzPl-6Z_c_-N0A&s=Ofhp-XmxFUHyihMfLqzDo19Z1xzqkXNLV0T4NQnGy2Q&e=>
test.sinks.k1.indexName = TwitterComp_index
test.sinks.k1.indexType = TwitterComp_type
test.sinks.k1.clusterName = twittersearch
test.sinks.k1.batchSize = 500
test.sinks.k1.ttl = 5d
test.sinks.k1.serializer =
org.apache.flume.sink.elasticsearch.ElasticSearchDynamicSerializer
Any help is greatly appreciated.
Thanks,
Muby
