I am new to spark. I am trying to do the following.
Netcat-->Flume-->Spark streaming(process Flume Data)-->HDFS.
My flume config file has following set up.
Source = netcat
Sink=avrosink.
Spark Streaming code:
I am able to print data from flume to the monitor. But I am struggling to
create a fil
What is the error you are getting when you say "??I was trying to write the
data to hdfs..but it fails…"
TD
On Thu, Jul 10, 2014 at 1:36 PM, Sundaram, Muthu X. <
muthu.x.sundaram@sabre.com> wrote:
> I am new to spark. I am trying to do the following.
>
> NetcatàFlumeàSpark streaming(process
g
Cc: u...@spark.incubator.apache.org
Subject: Re: writing FLume data to HDFS
What is the error you are getting when you say "??I was trying to write the
data to hdfs..but it fails…"
TD
On Thu, Jul 10, 2014 at 1:36 PM, Sundaram, Muthu X.
mailto:muthu.x.sundaram@sabre.com>
t;??I was trying to write the data to hdfs..but
> it fails…
>
>
>
>
>
> *From:* Tathagata Das [mailto:tathagata.das1...@gmail.com]
> *Sent:* Friday, July 11, 2014 1:43 PM
> *To:* user@spark.apache.org
> *Cc:* u...@spark.incubator.apache.org
> *Subject:* Re:
, July 14, 2014 2:23 PM
To: user@spark.apache.org
Cc: u...@spark.incubator.apache.org
Subject: Re: writing FLume data to HDFS
Stepping a bit back, if you just want to write flume data to HDFS, you can use
flume's HDFS sink for that.
Trying to do this using Spark Streaming and SparkFlumeEve
