Lohit Giri created PIG-3553:
-------------------------------
Summary: HadoopJobHistoryLoader fails to load job history on
hadoop v 1.2
Key: PIG-3553
URL: https://issues.apache.org/jira/browse/PIG-3553
Project: Pig
Issue Type: Bug
Components: piggybank
Affects Versions: 0.10.0
Reporter: Lohit Giri
while following the example on pig wiki -
a = load '/mapred/history/done' using HadoopJobHistoryLoader() as (j:map[],
m:map[], r:map[]);
b = foreach a generate (Chararray) j#'STATUS' as status, j#'PIG_SCRIPT_ID' as
id, j#'USER' as user, j#'JOBNAME' as script_name, j#'JOBID' as job;
c = filter b by status != 'SUCCESS';
dump c;
when tried to parse /mapred/history logs, it fails to parse the job conf logs
because in hadoop 1 the logs are stored as
job_201310301419_0001_conf.xml
rathar than as per the code says line number 203 in hadoopjobhistoryloader.java
Path parent = full.getParent();
String jobXml = jobDetails[0] + "_" + jobDetails[1] + "_"
+ jobDetails[2] + "_" + jobDetails[3] + "_"
+ jobDetails[4] + "_conf.xml";
Path p = new Path(parent, jobXml);
attaching the patch
--
This message was sent by Atlassian JIRA
(v6.1#6144)
