am trying to store data to MySQL using DBStorage in pig script. when I run
the the script ,I am getting error unable to read the input file. Whereas
when I try to dump to store the same data file its working fine. Sample
pigscript :
%default DATABASE_HOST 'localhost';%default DATABASE_NAME
'Test';%default DATABASE_USER 'username';%default DATABASE_PASS
'password';%default DATABASE_DRIVER 'com.mysql.jdbc.Driver';%default
DATABASE_TYPE 'mysql';
A = LOAD '/tmp/TestDivya/Pig/PigSQLTest.txt' using PigStorage() as
(name: chararray);
STORE A into 'test' using
org.apache.pig.piggybank.storage.DBStorage('$DATABASE_DRIVER',
'jdbc:$DATABASE_TYPE://$DATABASE_HOST/$DATABASE_NAME',
'$DATABASE_USER', '$DATABASE_PASS', 'INSERT INTO Test(name) VALUES
(?)');
HadoopVersion PigVersion UserId StartedAt FinishedAt
Features2.7.1.2.3.4.0-3485 0.15.0.2.3.4.0-3485 hdfs
2016-01-21 01:34:552016-01-21 01:35:07 UNKNOWN
Failed!
Failed Jobs:
JobId Alias Feature Message Outputs
job_1453263223178_0042 A MAP_ONLY Message: Job failed!
hdfs://ip-172-31-29-201.ap-southeast-1.compute.internal:8020/user/hdfs/test,
Input(s):
Failed to read data from "/tmp/TestDivya/Pig/PigSQLTest.txt"
Output(s):
Failed to produce result in
"hdfs://ip-xxx-xx-xx-xxx.ap-xxxxxx-1.compute.internal:8020/user/hdfs/test"
Counters:
Total records written : 0
Total bytes written : 0
Spillable Memory Manager spill count : 0
Total bags proactively spilled: 0
Total records proactively spilled: 0
Job DAG:
job_1453263223178_0042
My cluster is set up on EC2 having Hortonworks HDP 2.3.4 and using the HDP
MYSQL database for storing the pig output.
Would really appreciate the pointers.
Thanks,
