Hi all, I have a coded a custom receiver which receives kafka messages. These
Kafka messages have FTP server credentials in them. The receiver then opens the
message and uses the ftp credentials in it to connect to the ftp server. It
then streams this huge text file (3.3G) . Finally this
Here is the error
yarn.ApplicationMaster: Final app status: FAILED, exitCode: 15, (reason: User
class threw exception: Log directory
hdfs://Sandbox/user/spark/applicationHistory/application_1438113296105_0302
already exists!)
I am using cloudera 5.3.2 with Spark 1.2.0
Any help is appreciated.
...@cloudera.com
To: Varadhan, Jawahar varad...@yahoo.com
Cc: d...@spark.apache.org d...@spark.apache.org
Sent: Friday, August 14, 2015 3:23 PM
Subject: Re: Setting up Spark/flume/? to Ingest 10TB from FTP
Why do you need to use Spark or Flume for this?
You can just use curl and hdfs:
curl
