Hi Flumers
I’m new to Flume so please go easy on me, but this is a placeholder for those
that follow.
I’m seeing the following Exception when configuring a http source for Flume
v1.6.0, pointing at an externally hosted web API I know I can connect to, and
GET/POST to otherwise
15/11/12 21:23:34 ERROR lifecycle.LifecycleSupervisor: Unable to start
EventDrivenSourceRunner: {
source:org.apache.flume.source.http.HTTPSource{name:<source-name>,state:IDLE} }
- Exception follows.
java.lang.IllegalStateException: Running HTTP Server found in source:
<source-name> before I started one.Will not attempt to start. “<--Exception
text!!!”
at
com.google.common.base.Preconditions.checkState(Preconditions.java:145)
at org.apache.flume.source.http.HTTPSource.start(HTTPSource.java:176)
at
org.apache.flume.source.EventDrivenSourceRunner.start(EventDrivenSourceRunner.java:44)
at
org.apache.flume.lifecycle.LifecycleSupervisor$MonitorRunnable.run(LifecycleSupervisor.java:251)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Here are my config lines for the Flume source:
flume-ng agent
-Dflume.root.logger=WARN,console
--conf-file
<Flume-Agent>.sources.<source-name>.type = http
<Flume-Agent>.sources.<source-name>.port = 443
<Flume-Agent>.sources.<source-name>.bind = https://<url<https://%3curl> to a
perfectly healthy web API> <-- serving JSON data
<Flume-Agent>.sources.<source-name>.handler =
org.apache.flume.source.http.JSONHandler
<Flume-Agent>.sources.<source-name>.selector.type = replicating <-- multiple
sinks
Now I’ve seen others have had to recompile the Flume Jar to resolve this but is
that really what’s going on here?
https://community.cloudera.com/t5/Data-Ingestion-Integration/Flume-TwitterSource-language-filter/td-p/23519
https://github.com/cloudera/cdh-twitter-example/issues/10
Apart from the Exceptions, what other logging will help diagnose this?
The Collinson Group Limited; Registered number: 2577557, Registered in England
& Wales; Registered Office: Cutlers Exchange, 123 Houndsditch, London, EC3A 7BU.
This e-mail may contain privileged and confidential information and/or
copyright material and is intended for the use of the addressee only. If you
receive this e-mail by mistake please advise the sender immediately by using
the reply facility in your e-mail software and delete this e-mail from your
computer system. You may not deliver, copy or disclose its contents to anyone
else. Any unauthorised use may be unlawful. Any views expressed in this e-mail
are those of the individual sender and may not necessarily reflect the views of
The Collinson Group Ltd and/or its subsidiaries or any other associated company
(collectively “Collinson Group”).
As communications via the Internet are not secure Collinson Group cannot accept
any liability if this e-mail is accessed by third parties during the course of
transmission or is modified or amended in any way following despatch. Collinson
Group cannot guarantee that any attachment to this email does not contain a
virus, therefore it is strongly recommended that you carry out your own virus
check before opening any attachment, as we cannot accept liability for any
damage sustained as a result of software virus infection. Senders of messages
shall be taken to consent to the monitoring and recording of e-mails addressed
to members of the Company.
