Lizhao.Du created PIG-3904:
------------------------------
Summary: Pig support windows i18n
Key: PIG-3904
URL: https://issues.apache.org/jira/browse/PIG-3904
Project: Pig
Issue Type: Improvement
Components: impl
Affects Versions: 0.9.2, 0.9.3
Environment: Windows 7(de_DE/fr_FR/zh_CN)
Reporter: Lizhao.Du
Fix For: 0.9.3
Utilize Pig run a pig script in Windows (de_DE), it is failed.
The error message showed "Input path does not exist:
hdfs://10.141.73.10:8020/tmp/测试/pwInput", but /tmp/测试/pwInput is exist
in fact. Because of encoding of hadoop adoption is UTF-8. When encoding of
client OS pig situated is different with it, hadoop will unrecognize there
characters.
Log message as below:
==============================================
ERROR Spring Shell
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.Launcher - Backend
error message during job submission
org.apache.pig.backend.executionengine.ExecException: ERROR 2118: Input path
does not exist: hdfs://10.141.73.10:8020/tmp/测试/pwInput
at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigInputFormat.getSplits(PigInputFormat.java:282)
at
org.apache.hadoop.mapred.JobClient.writeNewSplits(JobClient.java:1054)
at org.apache.hadoop.mapred.JobClient.writeSplits(JobClient.java:1071)
at org.apache.hadoop.mapred.JobClient.access$700(JobClient.java:179)
at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:983)
at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:936)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Unknown Source)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1190)
at
org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:936)
at org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:910)
at org.apache.hadoop.mapred.jobcontrol.Job.submit(Job.java:378)
at
org.apache.hadoop.mapred.jobcontrol.JobControl.startReadyJobs(JobControl.java:247)
at
org.apache.hadoop.mapred.jobcontrol.JobControl.run(JobControl.java:279)
at java.lang.Thread.run(Unknown Source)
Caused by: org.apache.hadoop.mapreduce.lib.input.InvalidInputException: Input
path does not exist: hdfs://10.141.73.10:8020/tmp/测试/pwInput
at
org.apache.hadoop.mapreduce.lib.input.FileInputFormat.listStatus(FileInputFormat.java:235)
at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigTextInputFormat.listStatus(PigTextInputFormat.java:36)
at
org.apache.hadoop.mapreduce.lib.input.FileInputFormat.getSplits(FileInputFormat.java:252)
at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigInputFormat.getSplits(PigInputFormat.java:270)
... 14 more
--
This message was sent by Atlassian JIRA
(v6.2#6252)
