Charan Hebri created MAPREDUCE-7062:
---------------------------------------
Summary: MR job tags not compatible with YARN ATSv2 flow names,
flow run ids and flow versions
Key: MAPREDUCE-7062
URL: https://issues.apache.org/jira/browse/MAPREDUCE-7062
Project: Hadoop Map/Reduce
Issue Type: Bug
Reporter: Charan Hebri
When applications are submitted to YARN, tags are generated in the format
TIMELINE_FLOW_NAME_TAG:\{flow_name},TIMELINE_FLOW_VERSION_TAG:\{flow_version},
TIMELINE_FLOW_RUN_ID_TAG:\{flow_run_id}
However, MR applications don't follow this format and the tags submitted via
the property mapreduce.job.tags are of the format,
{flow_name},\{flow_version},\{flow_run_id}
Due to this, YARN falls back to default values for flow name, flow version and
flow run id which in turn are used in ATSv2.
There are 2 approaches that could be taken to make MR tags compatible with
ATSv2,
Fix in the MR code
-------------------------
Prefix any tags specified with the ones needed by the YARN Timeline Service v2.
But MR is legacy code and hence these changes could affect how users are using
these tags.
Add a note in mapred-default.xml
--------------------------------------------
Add notes in the property name, mapreduce.job.tags mentioning that for purposes
of ATSv2, prefixes need to be added to the tag names.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: mapreduce-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-dev-h...@hadoop.apache.org
