Hi Viktor, Thank you for your comments. I agree that this change is low-risk - the default false feature flag shouldn't cause any problems to existing users.
As for the rejected alternative of adding an attribute to the MDC - some of the internal worker classes (such as the different WorkerTask subclasses) have a toString implementation which is used in the log message. When those toString calls are used in logging, we don't always have a logging context with MDC attributes. In my current PR, I changed those toString methods in a backward compatible way, so if the feature flag is set to false, the method will return the same string as it used to. Even if we went with the extra MDC attribute, we would still probably need an extra config to make the toString methods backward compatible. Because of this, it might be easier to have a single flag to control the full feature. Daniel Viktor Somogyi-Vass <viktor.somo...@cloudera.com.invalid> ezt írta (időpont: 2023. ápr. 21., P, 15:07): > Hi Daniel, > > I think this is a useful addition, it helps resolving issues and > escalations, and improves overall traceability. > Changing the logging context may imply the risk of making certain log > parsers unable to work on new logs. As I see we by default disable this > feature which solves this problem, however I also think that by disabling > it by default it isn't much of a help because users may not know about this > configuration and would not benefit from these when they face problems. So > overall I'd like to go with default=true and wanted to put this out here > for the community to discuss whether it's a problem. > Also, what was the reasoning behind rejecting the second alternative? As I > see that would be a viable option and maybe a bit more idiomatic to the > logging framework. > > A minor note: please update the JIRA link in the KIP to point to the right > one. > > Best, > Viktor > > On Thu, Apr 13, 2023 at 2:19 PM Dániel Urbán <urb.dani...@gmail.com> > wrote: > > > Hi everyone, > > > > I would like to bump this thread. I think this would be very useful for > any > > MM2 users, as the current logs with certain architectures (e.g. fan-out) > > are impossible to decipher. > > I already submitted a PR to demonstrate the proposed solution: > > https://github.com/apache/kafka/pull/13475 > > > > Thanks for your comments in advance, > > Daniel > > > > Dániel Urbán <urb.dani...@gmail.com> ezt írta (időpont: 2023. márc. 30., > > Cs, 18:24): > > > > > Hello everyone, > > > > > > I would like to kick off a discussion about KIP-916: > > > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-916%3A+MM2+distributed+mode+flow+log+context > > > > > > The KIP aims to enhance the diagnostic information for MM2 distributed > > > mode. MM2 relies on multiple Connect worker instances nested in a > single > > > process. In Connect, Connector names are guaranteed to be unique in a > > > single process, but in MM2, this is not true. Because of this, the > > > diagnostics provided by Connect (client.ids, log context) do not ensure > > > that logs are distinguishable for different flows (Connect workers) > > inside > > > an MM2 process. > > > > > > Thanks for all you input in advance, > > > Daniel > > > > > >
