Change LoadMetadata and StoreMetadata to use Job instead of Configuraiton and
add a cleanupOnFailure method to StoreFuncInterface
---------------------------------------------------------------------------------------------------------------------------------
Key: PIG-1265
URL: https://issues.apache.org/jira/browse/PIG-1265
Project: Pig
Issue Type: Bug
Affects Versions: 0.7.0
Reporter: Pradeep Kamath
Assignee: Pradeep Kamath
Fix For: 0.7.0
Speaking to the hadoop team folks, the direction in hadoop is to use Job
instead of Configuration - for example InputFormat/OutputFormat implementations
use Job to store input/output location. So pig should also do the same in
LoadMetadata and StoreMetadata to be closer to hadoop.
Currently when a job fails, pig assumes the output locations (corresponding to
the stores in the job) are hdfs locations and attempts to delete them. Since
output locations could be non hdfs locations, this cleanup should be delegated
to the StoreFuncInterface implementation - hence a new method -
cleanupOnFailure() should be introduced in StoreFuncInterface and a default
implementation should be provided in the StoreFunc abstract class which checks
if the location exists on hdfs and deletes it if so.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
