Svend Vanderveken created FLINK-22407:
-----------------------------------------
Summary: upgrade log4j to 2.14
Key: FLINK-22407
URL: https://issues.apache.org/jira/browse/FLINK-22407
Project: Flink
Issue Type: Improvement
Reporter: Svend Vanderveken
Flink is currently [relying on log4j
2.12.1|https://github.com/apache/flink/blob/master/pom.xml#L110]
.Unfortunately, this vesrion has a bug related to json layout that prevents a
user from adding additional log fields, as reported here
https://issues.apache.org/jira/browse/LOG4J2-2652 as well as here:
[https://stackoverflow.com/questions/57003440/why-is-log4j2-jsonlayout-keyvaluepair-printing-empty-logevent-messages]
The problem is fixed in Log4j 2.13.1.
Is there a good reason to keep Log4j 2.12.1, or can we upgrade?
As an illustration, the presence of {{additional1}} in the snippet below:
{code:java}
rootLogger.level = INFO
rootLogger.appenderRef.console.ref = LogConsole
appender.console.name = LogConsole
appender.console.type = CONSOLE
appender.console.layout.type = JsonLayout
appender.console.layout.complete = false
appender.console.layout.compact = true
appender.console.layout.eventEol = true
appender.console.layout.properties = true
appender.console.layout.includeStacktrace=true
appender.console.layout.stacktraceAsString=true
appender.console.layout.additional1.type=KeyValuePair
appender.console.layout.additional1.key=timestamp
appender.console.layout.additional1.value=$${date:yyyy-MM-dd'T'HH:mm:ss.SSSZ}
{code}
leads to missing fields in the resulting logs, e.g.:
{code:java}
{"logEvent":"Recover all persisted job
graphs.","timestamp":"2021-04-21T16:50:31.722+0000"}
{"logEvent":"Successfully recovered 0 persisted job
graphs.","timestamp":"2021-04-21T16:50:31.723+0000"}
{"logEvent":"Starting the
SlotManager.","timestamp":"2021-04-21T16:50:31.732+0000"}
{"logEvent":"Starting RPC endpoint for
org.apache.flink.runtime.dispatcher.StandaloneDispatcher at
akka://flink/user/rpc/dispatcher_1
.","timestamp":"2021-04-21T16:50:31.822+0000"}
{code}
Removing the {{additional1}} resolves the issue and yield json logs containing
all expected fields:
{code:java}
{"thread":"cluster-io-thread-1","level":"INFO","loggerName":"org.apache.flink.runtime.dispatcher.runner.SessionDispatcherLeaderProcess","message":"Successfully
recovered 0 persisted job
graphs.","endOfBatch":false,"loggerFqcn":"org.apache.logging.slf4j.Log4jLogger","instant":{"epochSecond":1619080838,"nanoOfSecond":216868000},"threadId":48,"contextMap":{},"threadPriority":5}
{"thread":"flink-akka.actor.default-dispatcher-3","level":"INFO","loggerName":"org.apache.flink.runtime.resourcemanager.slotmanager.SlotManagerImpl","message":"Starting
the
SlotManager.","endOfBatch":false,"loggerFqcn":"org.apache.logging.slf4j.Log4jLogger","instant":{"epochSecond":1619080838,"nanoOfSecond":313130000},"threadId":19,"contextMap":{},"threadPriority":5}
{"thread":"cluster-io-thread-1","level":"INFO","loggerName":"org.apache.flink.runtime.rpc.akka.AkkaRpcService","message":"Starting
RPC endpoint for org.apache.flink.runtime.dispatcher.StandaloneDispatcher at
akka://flink/user/rpc/dispatcher_1
.","endOfBatch":false,"loggerFqcn":"org.apache.logging.slf4j.Log4jLogger","instant":{"epochSecond":1619080838,"nanoOfSecond":408714000},"threadId":48,"contextMap":{},"threadPriority":5}
{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
