HTTP Source does not pull anything from an address, instead it binds to the address specified in the config. It can listen on that port and then accept data from applications.
Thanks, Hari Shreedharan > On Nov 12, 2015, at 1:59 PM, Timothy Garza <[email protected]> > wrote: > > Hi Flumers > > I’m new to Flume so please go easy on me, but this is a placeholder for those > that follow. > > I’m seeing the following Exception when configuring a http source for Flume > v1.6.0, pointing at an externally hosted web API I know I can connect to, and > GET/POST to otherwise > > 15/11/12 21:23:34 ERROR lifecycle.LifecycleSupervisor: Unable to start > EventDrivenSourceRunner: { > source:org.apache.flume.source.http.HTTPSource{name:<source-name>,state:IDLE} > } - Exception follows. > java.lang.IllegalStateException: Running HTTP Server found in source: > <source-name> before I started one.Will not attempt to start. “ßException > text!!!” > at > com.google.common.base.Preconditions.checkState(Preconditions.java:145) > at org.apache.flume.source.http.HTTPSource.start(HTTPSource.java:176) > at > org.apache.flume.source.EventDrivenSourceRunner.start(EventDrivenSourceRunner.java:44) > at > org.apache.flume.lifecycle.LifecycleSupervisor$MonitorRunnable.run(LifecycleSupervisor.java:251) > at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) > at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304) > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178) > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > at java.lang.Thread.run(Thread.java:745) > > Here are my config lines for the Flume source: > flume-ng agent > -Dflume.root.logger=WARN,console > > --conf-file > <Flume-Agent>.sources.<source-name>.type = http > <Flume-Agent>.sources.<source-name>.port = 443 > <Flume-Agent>.sources.<source-name>.bind = https://<url <https://%3curl/> to > a perfectly healthy web API> ß serving JSON data > <Flume-Agent>.sources.<source-name>.handler = > org.apache.flume.source.http.JSONHandler > <Flume-Agent>.sources.<source-name>.selector.type = replicating ß multiple > sinks > > Now I’ve seen others have had to recompile the Flume Jar to resolve this but > is that really what’s going on here? > https://community.cloudera.com/t5/Data-Ingestion-Integration/Flume-TwitterSource-language-filter/td-p/23519 > > <https://community.cloudera.com/t5/Data-Ingestion-Integration/Flume-TwitterSource-language-filter/td-p/23519> > https://github.com/cloudera/cdh-twitter-example/issues/10 > <https://github.com/cloudera/cdh-twitter-example/issues/10> > > Apart from the Exceptions, what other logging will help diagnose this? > > > > The Collinson Group Limited; Registered number: 2577557, Registered in > England & Wales; Registered Office: Cutlers Exchange, 123 Houndsditch, > London, EC3A 7BU. > > > This e-mail may contain privileged and confidential information and/or > copyright material and is intended for the use of the addressee only. If you > receive this e-mail by mistake please advise the sender immediately by using > the reply facility in your e-mail software and delete this e-mail from your > computer system. You may not deliver, copy or disclose its contents to anyone > else. Any unauthorised use may be unlawful. Any views expressed in this > e-mail are those of the individual sender and may not necessarily reflect the > views of The Collinson Group Ltd and/or its subsidiaries or any other > associated company (collectively “Collinson Group”). > > As communications via the Internet are not secure Collinson Group cannot > accept any liability if this e-mail is accessed by third parties during the > course of transmission or is modified or amended in any way following > despatch. Collinson Group cannot guarantee that any attachment to this email > does not contain a virus, therefore it is strongly recommended that you carry > out your own virus check before opening any attachment, as we cannot accept > liability for any damage sustained as a result of software virus infection. > Senders of messages shall be taken to consent to the monitoring and recording > of e-mails addressed to members of the Company.
