[
https://issues.apache.org/jira/browse/KAFKA-3816?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15323651#comment-15323651
]
Ewen Cheslack-Postava commented on KAFKA-3816:
----------------------------------------------
This would be awesome, and I'm hoping this can also help us with
framework-level debugging too -- one of the really tricky things to understand
in logs is which AbstractCoordinator messages were coming from a consumer vs
distributed herder.
One thing we might want to do in conjunction with this is to allow the
framework to allocate threads for connectors and set this up. Background
threads that monitor sources are a particular case that we don't provide any
help to connector developers for and would mean we might be missing this info
in those threads.
For the naming -- the scope makes sense but we want a reliable way to extract
this from the connector without it having to provide extra info if possible.
Basing this on class name could work. My guess is that 3 parameters should be
sufficient (and if not, we probably need to do some refactoring to simplify
that code), but we'll just have to try it to find out.
> Provide more context in Kafka Connect log messages
> --------------------------------------------------
>
> Key: KAFKA-3816
> URL: https://issues.apache.org/jira/browse/KAFKA-3816
> Project: Kafka
> Issue Type: Improvement
> Components: KafkaConnect
> Affects Versions: 0.9.0.1
> Reporter: Randall Hauch
> Assignee: Ewen Cheslack-Postava
>
> Currently it is relatively difficult to correlate individual log messages
> with the various threads and activities that are going on within a Kafka
> Connect worker, let along a cluster of workers. Log messages should provide
> more context to make it easier and to allow log scraping tools to coalesce
> related log messages.
> One simple way to do this is by using _mapped diagnostic contexts_, or MDC.
> This is supported by the SLF4J API, and by the Logback and Log4J logging
> frameworks.
> Basically, the framework would be changed so that each thread is configured
> with one or more MDC parameters using the
> {{org.slf4j.MDC.put(String,String)}} method in SLF4J. Once that thread is
> configured, all log messages made using that thread have that context. The
> logs can then be configured to use those parameters.
> It would be ideal to define a convention for connectors and the Kafka Connect
> framework. A single set of MDC parameters means that the logging framework
> can use the specific parameters on its message formats.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
