[ https://issues.apache.org/jira/browse/PIG-3617?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Cheolsoo Park updated PIG-3617: ------------------------------- Attachment: PIG-3617-2.patch [~nezihyigitbasi], thank you very much for the patch. It looks good, but can I make a minor suggestion? How about using Utils.hasFileSystemImpl() instead of checking whether a path starts with "hbase"? I think this approach is better because then other non-filesystem storages such as accumulo will also be protected from the same problem. PIG-3617-2.patch implements what I describe here. I can commit it if you agree. Thanks! > problem with temp file deletion in MAPREDUCE operator > ----------------------------------------------------- > > Key: PIG-3617 > URL: https://issues.apache.org/jira/browse/PIG-3617 > Project: Pig > Issue Type: Bug > Affects Versions: 0.12.0 > Reporter: Nezih Yigitbasi > Fix For: 0.13.0 > > Attachments: PIG-3617-2.patch, PIG-3617.patch > > > Hi all, > When I run a native MR job with the MAPREDUCE keyword and store the > intermediate data in HBase with: > stored = MAPREDUCE 'my.jar' > STORE x INTO 'hbase://temp_table' > USING > org.apache.pig.backend.hadoop.hbase.HBaseStorage('hbase_schema') > .... and the rest ....; > Pig tries to delete the temp files, which in this case has an HBase path, and > fails with the exception: > Caused by: java.lang.IllegalArgumentException: java.net.URISyntaxException: > Relative path in absolute URI: file:hbase:/temp_table > at org.apache.hadoop.fs.Path.initialize(Path.java:148) > at org.apache.hadoop.fs.Path.<init>(Path.java:126) > at > org.apache.pig.backend.hadoop.datastorage.HDataStorage.isContainer(HDataStorage.java:197) > at > org.apache.pig.backend.hadoop.datastorage.HDataStorage.asElement(HDataStorage.java:128) > at org.apache.pig.impl.io.FileLocalizer.delete(FileLocalizer.java:415) > at > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher.launchPig(MapReduceLauncher.java:419) > at org.apache.pig.PigServer.launchPlan(PigServer.java:1322) -- This message was sent by Atlassian JIRA (v6.1.5#6160)