I am using streaming with perl, and I want to get jobconf variable values. As
many tutorials say they are in environment, but I can not get them.
For example, in reducer:
while (<STDIN>){
my $part = $ENV{"mapred.task.partition"};
print ("$part\n");
}
It turns out that $ENV{"mapred.task.partition"} is not defined.
HOWEVER, I can get myself defined variable value. For example:
$HADOOP_HOME/bin/hadoop \
jar $HADOOP_HOME/hadoop-streaming.jar \
-input file1 \
-output myOutputDir \
-mapper mapper \
-reducer reducer \
-jobcont arg=test
In reducer:
while (<STDIN>){
my $part2 = $ENV{"arg"};
print ("$part2\n");
}
It works.
Anybody knows why is that? How to get jobconf variables in streaming? Thanks
lot!
