Daniel Dai created PIG-4714:
-------------------------------
Summary: Improve logging across multiple components with callerId
Key: PIG-4714
URL: https://issues.apache.org/jira/browse/PIG-4714
Project: Pig
Issue Type: Improvement
Components: impl
Reporter: Daniel Dai
Assignee: Daniel Dai
Fix For: 0.16.0
The idea is to add callerId to every component, so we can track the chain of
application which cause the underlining operation. A typical chain is
Oozie->Pig->Tez->Hdfs. With proper callerId logging, we can trace Hdfs
operation back to Oozie workflow which triggers it.
The protocol we decided is every component log its immediate callerId.
For Pig, this includes passing Pig script ID to underlining components as
callerId using component specific API, log callerId of Pig and store it on ATS.
More specific, it includes:
1. Generate a CallerId for each Pig script, pass it to Hdfs/Yarn/MapReduce/Tez
which Pig invokes
2. Pig caller passes "pig.log.trace.id" to Pig, Pig will publish it to ATS
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
