Akihiro Matsukawa created PIG-3552:
--------------------------------------
Summary: UriUtil used by reducer estimator should support viewfs
Key: PIG-3552
URL: https://issues.apache.org/jira/browse/PIG-3552
Project: Pig
Issue Type: Bug
Components: impl
Environment: Pig 0.11+
Reporter: Akihiro Matsukawa
Assignee: Akihiro Matsukawa
On Hadoop2 running viewfs throws error
{code}
2013-10-30 02:12:46,916 [main] INFO
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler
- Using reducer estimator:
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.InputSizeReducerEstimator
2013-10-30 02:12:46,917 [main] INFO
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.InputSizeReducerEstimator
- BytesPerReducer=1610612736 maxReducers=999 totalInputFileSize=-1
2013-10-30 02:12:46,917 [main] INFO
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler
- Could not estimate number of reducers and no requested or default parallelism
set. Defaulting to 1 reducer.
{code}
This is caused by the fact that UriUtil.isHDFSFileOrLocalOrS3N only supports
files that start with "/", "hdfs:", "file:", "s3n:" but not "viewfs:".
Trivial fix, patch attached.
Testing: ant test-commit passes.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
