[
https://issues.apache.org/jira/browse/TEZ-4039?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17800336#comment-17800336
]
Ayush Saxena commented on TEZ-4039:
-----------------------------------
For me this doesn't work with hive master & puts up a warn log for every query
{noformat}
2023-12-23T03:13:08,920 WARN [DAGAppMaster Thread] util.LoggingUtils: Cannot
set log4j global MDC, mdcContext won't be applied to log4j's MDC class
java.lang.NoSuchFieldException: tlm
at java.lang.Class.getDeclaredField(Class.java:2070) ~[?:1.8.0_342]
at org.apache.tez.util.LoggingUtils.setupLog4j(LoggingUtils.java:113)
~[tez-common-0.10.3-SNAPSHOT.jar:0.10.3-SNAPSHOT]
at org.apache.tez.dag.app.DAGAppMaster.<init>(DAGAppMaster.java:353)
~[tez-dag-0.10.3-SNAPSHOT.jar:0.10.3-SNAPSHOT]
at
org.apache.tez.client.LocalClient.createDAGAppMaster(LocalClient.java:403)
~[tez-dag-0.10.3-SNAPSHOT.jar:0.10.3-SNAPSHOT]
at org.apache.tez.client.LocalClient$1.run(LocalClient.java:357)
~[tez-dag-0.10.3-SNAPSHOT.jar:0.10.3-SNAPSHOT]
at java.lang.Thread.run(Thread.java:750) ~[?:1.8.0_342]
2023-12-23T03:13:08,921 INFO [DAGAppMaster Thread] app.DAGAppMaster: Created
DAGAppMaster for application appattempt_1703281387528_0001_000000,
versionInfo=[ component=tez-dag, version=0.10.3-SNAPSHOT,
revision=0c5cf688690bacc662a015b1bff0ecb556314b90,
SCM-URL=scm:git:https://gitbox.apache.org/repos/asf/tez.git,
buildTime=2023-12-22T19:59:29Z, buildUser=ayushsaxena,
buildJavaVersion=1.8.0_342 ]
{noformat}
We should remove the trace I believe & change it to info I believe, else it
will be logging this unnecessarily, mostly it is due to log4j version of hive
which doesn't have this {{tlm}} field
> Tez should inject dag id, query id into MDC
> -------------------------------------------
>
> Key: TEZ-4039
> URL: https://issues.apache.org/jira/browse/TEZ-4039
> Project: Apache Tez
> Issue Type: Bug
> Reporter: Prasanth Jayachandran
> Assignee: László Bodor
> Priority: Major
> Fix For: 0.10.3
>
> Attachments: TEZ-4039.01.patch, TEZ-4039.02.patch, TEZ-4039.03.patch,
> TEZ-4039.04.patch, TEZ-4039.05.patch
>
> Time Spent: 4h
> Remaining Estimate: 0h
>
> Tez currently uses CallableWithNdc to store thread specific context. It
> should also inject the context into MDC so that pattern layout can dump the
> contexts from MDC (with NDC it is not possible to read the context in pattern
> lyaout).
> Hive for example, sets queryId in the MDC and pattern layout prints the
> queryId
>
> {code:java}
> %d{ISO8601} %-5p [%t (%X{queryId})] %c{2}: %m%n
> {code}
> Llap sets dagId, fragmentId and queryId into MDC which is used for queryId
> based routing of logging.
> Similarly, Tez AM should set dagId and queryId (if available) into MDC.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
