[jira] [Commented] (MAPREDUCE-4847) Command Parsing in Hadoop Streaming
[ https://issues.apache.org/jira/browse/MAPREDUCE-4847?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13528823#comment-13528823 ] Peng Lei commented on MAPREDUCE-4847: - Hi Asokan, Thank you. I'm new to the community, who are the hadoop-streaming contributors? don't you? How to assign this issue to hadoop-streaming owners. -Peng > Command Parsing in Hadoop Streaming > --- > > Key: MAPREDUCE-4847 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-4847 > Project: Hadoop Map/Reduce > Issue Type: Improvement > Components: contrib/streaming >Reporter: Peng Lei > Labels: features > Original Estimate: 4h > Remaining Estimate: 4h > > Hadoop streaming parse the mapper and reducer commands by itself, this is not > a good choice, when I write a complex mapper/reducer script inline, such as > 'perl -ne ...', it don't work. > An alternative way is to send the command to the shell, simply create new > process(sh -c "command_and_args"), this not also simplize the streaming code, > but also improve its capability! -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4847) Command Parsing in Hadoop Streaming
[ https://issues.apache.org/jira/browse/MAPREDUCE-4847?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13511031#comment-13511031 ] Peng Lei commented on MAPREDUCE-4847: - Thank you for your comment! I have put the command in a script file as a workaround, it works. But in this case, the command is not too complex to write a dedicate script file, and on fly script generating is a bit tricky(at least for maintainer). It seems hadoop can't run on windows without cygwin. Another solution may be: add a new option to instruct streaming to use an alternative command invoker, such as: -command_invoker "sh -c" This could solve the issue and didn't break the existing hadoop-streaming application. -Peng > Command Parsing in Hadoop Streaming > --- > > Key: MAPREDUCE-4847 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-4847 > Project: Hadoop Map/Reduce > Issue Type: Improvement > Components: contrib/streaming >Reporter: Peng Lei > Labels: features > Original Estimate: 4h > Remaining Estimate: 4h > > Hadoop streaming parse the mapper and reducer commands by itself, this is not > a good choice, when I write a complex mapper/reducer script inline, such as > 'perl -ne ...', it don't work. > An alternative way is to send the command to the shell, simply create new > process(sh -c "command_and_args"), this not also simplize the streaming code, > but also improve its capability! -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (MAPREDUCE-4847) Command Parsing in Hadoop Streaming
Peng Lei created MAPREDUCE-4847: --- Summary: Command Parsing in Hadoop Streaming Key: MAPREDUCE-4847 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4847 Project: Hadoop Map/Reduce Issue Type: Improvement Components: contrib/streaming Reporter: Peng Lei Hadoop streaming parse the mapper and reducer commands by itself, this is not a good choice, when I write a complex mapper/reducer script inline, such as 'perl -ne ...', it don't work. An alternative way is to send the command to the shell, simply create new process(sh -c "command_and_args"), this not also simplize the streaming code, but also improve its capability! -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
