[
https://issues.apache.org/jira/browse/PIG-3552?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Akihiro Matsukawa updated PIG-3552:
-----------------------------------
Attachment: UriUtil_enable_viewfs.patch
> UriUtil used by reducer estimator should support viewfs
> -------------------------------------------------------
>
> Key: PIG-3552
> URL: https://issues.apache.org/jira/browse/PIG-3552
> Project: Pig
> Issue Type: Bug
> Components: impl
> Environment: Pig 0.11+
> Reporter: Akihiro Matsukawa
> Assignee: Akihiro Matsukawa
> Attachments: UriUtil_enable_viewfs.patch
>
>
> On Hadoop2 running viewfs throws error
> {code}
> 2013-10-30 02:12:46,916 [main] INFO
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler
> - Using reducer estimator:
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.InputSizeReducerEstimator
> 2013-10-30 02:12:46,917 [main] INFO
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.InputSizeReducerEstimator
> - BytesPerReducer=1610612736 maxReducers=999 totalInputFileSize=-1
> 2013-10-30 02:12:46,917 [main] INFO
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler
> - Could not estimate number of reducers and no requested or default
> parallelism set. Defaulting to 1 reducer.
> {code}
> This is caused by the fact that UriUtil.isHDFSFileOrLocalOrS3N only supports
> files that start with "/", "hdfs:", "file:", "s3n:" but not "viewfs:".
> Trivial fix, patch attached.
> Testing: ant test-commit passes.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
