Hey Daniel,
Thanks for the KIP! The current logging behavior seems like a
usability nightmare. I've only debugged single-worker JVM deployments,
and I see now that I took the uniqueness of a connector name for
granted.
I had a few questions about the change:
1. The MDC context change and the
If there are no further comments, I will kick off a vote soon for the KIP.
Dániel Urbán ezt írta (időpont: 2023. jún. 12., H,
11:27):
> Updated the KIP with a few example log lines before/after the change.
> Daniel
>
> Dániel Urbán ezt írta (időpont: 2023. jún. 7.,
> Sze, 13:59):
>
>> Hi
Updated the KIP with a few example log lines before/after the change.
Daniel
Dániel Urbán ezt írta (időpont: 2023. jún. 7., Sze,
13:59):
> Hi Chris,
>
> Thank you for your comments! I updated the KIP. I still need to add the
> example before/after log lines, will do that soon, but I addressed
Hi Chris,
Thank you for your comments! I updated the KIP. I still need to add the
example before/after log lines, will do that soon, but I addressed all the
other points.
1. Added more details on thread renaming under Public Interfaces, removed
pseudo code.
2. Removed the stale header -
Hi Daniel,
Thanks for the updates! A few more thoughts:
1. The "Proposed Changes" section seems a bit like a pseudocode
implementation of the KIP. We don't really need this level of detail; most
of the time, we're just looking for implementation details that don't
directly affect the user-facing
I updated the KIP accordingly. Tried to come up with more generic terms in
the Connect code instead of referring to "flow" anywhere.
Daniel
Dániel Urbán ezt írta (időpont: 2023. jún. 5., H,
14:49):
> Hi Chris,
>
> Thank you for your comments!
>
> I agree that the toString based logging is not
Hi Chris,
Thank you for your comments!
I agree that the toString based logging is not ideal, and I believe all
occurrences are within a proper logging context, so they can be ignored.
If thread names can be changed unconditionally, I agree, using a new MDC
key is the ideal solution.
Will update
Hi Daniel,
Are there any cases of Object::toString being used by internal classes for
logging context that can't be covered by MDC contexts? For example,
anything logged by WorkerSinkTask (or any WorkerTask subclass) already has
the MDC set for the task [1]. Since the Object::toString-prefixed
Hi Chris,
Thank you for your comments!
1. This approach is OK for me. I thought that the "sample" configs in the
repo do not fall into the same category as the default of a new config.
Adding a commented line instead should be ok, and the future opt-out change
sounds good to me.
2. Besides the
Hi Daniel,
Thanks for the KIP! A few thoughts:
1. This change is similar to when we added connector contexts to Connect in
KIPs 449 [1] and 721 [2]. I wonder if we can follow a similar approach
(combining both KIPs' changes into this single one), by doing the following:
- Disabling the changes
Hello everyone,
I would like to bump this thread, pretty straightforward KIP, and helps a
lot with MM2 dedicated mode diagnostics.
Thanks in advance,
Daniel
Dániel Urbán ezt írta (időpont: 2023. máj. 4., Cs,
14:08):
> Hi Viktor,
>
> Thank you for your comments. I agree that this change is
Hi Viktor,
Thank you for your comments. I agree that this change is low-risk - the
default false feature flag shouldn't cause any problems to existing users.
As for the rejected alternative of adding an attribute to the MDC - some of
the internal worker classes (such as the different WorkerTask
Hi Daniel,
I think this is a useful addition, it helps resolving issues and
escalations, and improves overall traceability.
Changing the logging context may imply the risk of making certain log
parsers unable to work on new logs. As I see we by default disable this
feature which solves this
Hi everyone,
I would like to bump this thread. I think this would be very useful for any
MM2 users, as the current logs with certain architectures (e.g. fan-out)
are impossible to decipher.
I already submitted a PR to demonstrate the proposed solution:
https://github.com/apache/kafka/pull/13475
Hello everyone,
I would like to kick off a discussion about KIP-916:
https://cwiki.apache.org/confluence/display/KAFKA/KIP-916%3A+MM2+distributed+mode+flow+log+context
The KIP aims to enhance the diagnostic information for MM2 distributed
mode. MM2 relies on multiple Connect worker instances
15 matches
Mail list logo
