Thanks Gopal!
So what do you suggest to get unique-id for mapper task with tez execution
engine?
conf.get("*mapreduce.task.**partition*");
Is this correct?
Regards,
Amey
On Wed, Dec 23, 2015 at 10:58 AM, Gopal Vijayaraghavan <gop...@apache.org>
wrote:
> Hi,
>
> (x-posts to bcc:)
>
> On 12/22/15, 9:19 PM, "Amey Barve" <ameybarv...@gmail.com> wrote:
>
> >conf.get("mapreduce.task.id");
> >
> >Now I want to run same hive queries with tez engine and I want to know
> >what
> >should be my unique-id. Is there any property from configuration or other
> >that can give me unique-id from the mapper task?
> >
> >I am using conf.get("mapreduce.task.partition") property, Is that
> >guranteed
> >to give unique - id for tez?
>
> That's populated in Tez in pretty much the exact format used by MRv2.
>
> https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hi
> ve/ql/exec/tez/TezProcessor.java#L112
>
>
> But beware, each mapper & reducer think it's the only one of its kind out
> there, so there are collisions between DAG stages for the unique name.
>
> Cheers,
> Gopal
>
>
>
