[ https://issues.apache.org/jira/browse/PIG-1917?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13019054#comment-13019054 ]
Richard Ding commented on PIG-1917: ----------------------------------- +1 > NativeMapReduce does not Allow Configuration Parameters containing Spaces > -------------------------------------------------------------------------- > > Key: PIG-1917 > URL: https://issues.apache.org/jira/browse/PIG-1917 > Project: Pig > Issue Type: Bug > Reporter: Mitesh Singh Jat > Assignee: Thejas M Nair > Fix For: 0.9.0 > > Attachments: PIG-1917.1.patch > > > In Pig, NativeMapReduce does not allow Configuration parameters which contain > spaces. > Considering a simple pig script with _wordcount_ as Native M-R job. > {code:title=nativeMR.pig|borderStyle=solid} > A = load 'input/WordCountInput.txt'; > B = MAPREDUCE 'hadoop-examples.jar' Store A into 'inputDir' Load 'outputDir' > as (word:chararray, count:int) `wordcount -Dmapred.job.map.memory.mb=3072 > -Dmapred.child.java.opts="-Xmx1536m -Xss128k" inputDir outputDir`; > store B into 'output/WordCount' using PigStorage; > {code} > The above pig script fails while launching native MAPREDUCE job. Because > space after _-Dmapred.child.java.opts="-Xmx1536m_ makes > _-Xss128k"_ as the first argument to _wordcount_ > Even with other example M-R program, the parameter _mapred.child.java.opts_ > was assigned _"-Xmx1536m_ in Jobconf. > Physical plan, Logical plan, and M-R plan of pig show double quotes. > For example, here is the corresponding M-R Plan for the native M-R > "MAPREDUCE" statement. > {code} > MapReduce node scope-13 > hadoop jar hadoop-examples.jar wordcount -Dmapred.job.map.memory.mb=3072 > -Dmapred.child.java.opts="-Xmx1536m -Xss128k" inputDir outputDir > -------- > {code} > On executing above M-R job after copying 'input/WordCountInput.txt' into > 'inputDir/', as Hadoop Job, this ran successfully. > {code} > $ *hadoop jar hadoop-examples.jar wordcount -Dmapred.job.map.memory.mb=3072 > -Dmapred.child.java.opts="-Xmx1536m -Xss128k" inputDir outputDir* > 11/03/18 14:23:29 INFO input.FileInputFormat: Total input paths to process : 1 > 11/03/18 14:23:29 INFO mapred.JobClient: Running job: job_201103181353_0007 > 11/03/18 14:23:30 INFO mapred.JobClient: map 0% reduce 0% > 11/03/18 14:23:37 INFO mapred.JobClient: map 100% reduce 0% > 11/03/18 14:23:49 INFO mapred.JobClient: map 100% reduce 100% > 11/03/18 14:23:51 INFO mapred.JobClient: Job complete: job_201103181353_0007 > ... > ... > {code} > Whereas, when _-Dmapred.child.java.opts=-Xmx1536m_ was used, the > *nativeMR.pig* executed successfully. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira