Abhishek Jain created GOBBLIN-2166:
--------------------------------------
Summary: GoT must fill in info required for `RMAppSummaryEvent`
fields - azkabanexecid, azkabanprojectname, azkabanflowid, azkabanjobid
Key: GOBBLIN-2166
URL: https://issues.apache.org/jira/browse/GOBBLIN-2166
Project: Apache Gobblin
Issue Type: Bug
Reporter: Abhishek Jain
right now, it's not possible to find `RMAppSummaryEvent`s by any of the above
named fields even though the `GaaS-Gobblin-Temporal-Azkaban` project is used by
GoT execs
because `azkabanprojectname` is not populated in events for any GoT execution
(the way it IS for GoMR executions), the only way to locate
`RMAppSummaryEvent`s for GoT executions is `appid`.
*why does this matter?*
a significant consequence of missing these fields is it thwarts joining
`GaaSJobObservabilityEvent`s to `RMAppSummaryEvent`s. this severely
complicates analysis, because the GaaS obs. event does NOT contain the YARN
appid, only the AZ flow ID.
since there is clearly an AZ execution involved, the solution is for GoT to set
whatever props are required on the YARN app side, so YARN will emit
fully-populated `RMAppSummaryEvent`s, with all of their `azkaban*` fields set.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
