Hi Steve,
The variables are transformed before going to the mappers.
mapred.task.partition turns into $MAPRED_TASK_PARTITION to be more unix-y
-Todd
On Fri, May 15, 2009 at 4:52 PM, Steve Gao <steve....@yahoo.com> wrote:
> I am using streaming with perl, and I want to get jobconf variable values.
> As many tutorials say they are in environment, but I can not get them.
>
> For example, in reducer:
> while (<STDIN>){
> my $part = $ENV{"mapred.task.partition"};
> print ("$part\n");
> }
>
> It turns out that $ENV{"mapred.task.partition"} is not defined.
>
> HOWEVER, I can get myself defined variable value. For example:
>
> $HADOOP_HOME/bin/hadoop \
> jar $HADOOP_HOME/hadoop-streaming.jar \
> -input file1 \
> -output myOutputDir \
> -mapper mapper \
> -reducer reducer \
> -jobcont arg=test
>
> In reducer:
>
> while (<STDIN>){
>
> my $part2 = $ENV{"arg"};
>
> print ("$part2\n");
>
> }
>
>
> It works.
>
> Anybody knows why is that? How to get jobconf variables in streaming?
> Thanks lot!
>
>
>
>
