[jira] [Created] (KAFKA-10873) Inaccurate "Ignore stop request for unowned connector" log messages
Chris Egerton created KAFKA-10873:
-
Summary: Inaccurate "Ignore stop request for unowned connector"
log messages
Key: KAFKA-10873
URL: https://issues.apache.org/jira/browse/KAFKA-10873
Project: Kafka
Issue Type: Bug
Components: KafkaConnect
Reporter: Chris Egerton
If a connector fails during startup, it will never be added to the worker's
internal list of running connectors (see
[https://github.com/apache/kafka/blob/87260a33b01590d0f73577840422e24fa589bed0/connect/runtime/src/main/java/org/apache/kafka/connect/runtime/Worker.java#L298|https://github.com/apache/kafka/blob/87260a33b01590d0f73577840422e24fa589bed0/connect/runtime/src/main/java/org/apache/kafka/connect/runtime/Worker.java#L298).]),
and the same happens for tasks (see
[https://github.com/apache/kafka/blob/87260a33b01590d0f73577840422e24fa589bed0/connect/runtime/src/main/java/org/apache/kafka/connect/runtime/Worker.java#L570]).
This leads to the following {{WARN}}-level log messages when that
connector/task is scheduled to be stopped by the worker:
* [Ignoring stop request for unowned connector
|https://github.com/apache/kafka/blob/87260a33b01590d0f73577840422e24fa589bed0/connect/runtime/src/main/java/org/apache/kafka/connect/runtime/Worker.java#L390]
* [Ignoring await stop request for non-present connector
|https://github.com/apache/kafka/blob/87260a33b01590d0f73577840422e24fa589bed0/connect/runtime/src/main/java/org/apache/kafka/connect/runtime/Worker.java#L415]
* [Ignoring stop request for unowned task
|https://github.com/apache/kafka/blob/87260a33b01590d0f73577840422e24fa589bed0/connect/runtime/src/main/java/org/apache/kafka/connect/runtime/Worker.java#L832]
* [Ignoring await stop request for non-present task
|https://github.com/apache/kafka/blob/87260a33b01590d0f73577840422e24fa589bed0/connect/runtime/src/main/java/org/apache/kafka/connect/runtime/Worker.java#L862]
If the connector/task failed on startup, there should already be log messages
detailing the failure and its cause; there is no need to emit warning messages
about not stopping that connector when it is scheduled for shutdown. Even
worse, emitting these messages may cause users to believe that their cluster is
experiencing a rebalancing bug that is somehow causing connectors/tasks that
are not assigned to a worker to be revoked from it.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
Re: [DISCUSS] KIP-696: Update Streams FSM to clarify ERROR state meaning
Thanks for the comets Matthias. I respond inline below. Thanks, walker On Sat, Dec 19, 2020 at 11:35 AM Matthias J. Sax wrote: > Overall LGTM. > > A few minor comments: > > > The SHUTDOWN_CLIENT option in the Streams Uncaught Exception Handler > should leave the client state in ERROR instead of NOT_RUNNING > > and > > > In order to be consistent, SHUTDOWN_CLIENT will leave the client state > in ERROR instead of NOT_RUNNING > > Both should also apply to SHUTDOWN_APPLICATION? If not, why? > I totally agree. This is already the behavior of SHUTDOWN_APPLICATION we are just bringing SHUTDOWN_CLIENT to match. > > > > Close() called on ERROR or PENDING_ERROR will be idempotent > > Should we replae `idempotent` with `a no-op`, because the difference is > that `close()` would normally transit to NOT_RUNNING? > That is fine with me. > > The Jira links to 3 tickets, but uses different markup. That is > confusing. Also, I actually believe that 6520 is unrelated? > I'll fix the mark up, I think that this kip changes how that 6520 is handled but it probably doesn't need to be on the list. I have added a comment on the ticket to notify anyone that related behavior has changed > > > -Matthias > > > On 12/10/20 1:52 AM, Bruno Cadonna wrote: > > Thanks for the KIP, Walker! > > > > The KIP looks good to me. I have just a minor comment about the KIP > > document. > > > > You talk about SHUTDOWN_CLIENT in the KIP, but never explain that it is > > a possible action that can be taken in the Streams uncaught exception > > handler. Could you please clarify that? > > > > Best, > > Bruno > > > > On 09.12.20 19:04, Walker Carlson wrote: > >> Thanks for the comments. If there are no further concerns I would like > to > >> call for a vote on KIP-696 to clarify and clean up the Streams State > >> Machine. > >> > >> walker > >> > >> On Wed, Dec 9, 2020 at 8:50 AM John Roesler > wrote: > >> > >>> Thanks, Walker! > >>> > >>> Your proposal looks good to me. > >>> > >>> -John > >>> > >>> On Tue, 2020-12-08 at 18:29 -0800, Walker Carlson wrote: > Thanks for the feedback Guozhang! > > I clarified some of the points in the Proposed Changes section so > >>> hopefully > it will be more clear what is going on now. I also agree with your > suggestion about the possible call to close() on ERROR so I added this > line. > "Close() called on ERROR will be idempotent and not throw an > exception, > >>> but > we will log a warning." > > I have linked those tickets and I will leave a comment trying to > explain > how these changes will affect their issue. > > walker > > On Tue, Dec 8, 2020 at 4:57 PM Guozhang Wang > wrote: > > > Hello Walker, > > > > Thanks for the KIP! Overall it looks reasonable to me. Just a few > > minor > > comments for the wiki page itself: > > > > 1) Could you clarify the conditions when RUNNING / REBALANCING -> > > PENDING_ERROR will happen; and when PENDING_ERROR -> ERROR will > > happen. > > E.g. when I read "Streams will only reach ERROR state in the event of > >>> an > > exceptional failure in which the `StreamsUncaughtExceptionHandler` > >>> chose to > > either shutdown the application or the client." I thought the first > > transition would happen before the handler, and the second transition > >>> would > > happen immediately after the handler returns "shutdown client" or > >>> "shutdown > > application", until I read the last statement regarding > >>> "SHUTDOWN_CLIENT". > > > > 2) A compatibility issue: today it is possible that users would call > > Streams APIs like shutdown in the global state transition listener. > > And > > it's common to try shutting down the application automatically when > > transiting to ERROR (assuming it was not a terminating state). I > think > >>> we > > could consider making this call a no-op and log a warning. > > > > 3) Could you link the following JIRAs in the "JIRA" field? > > > > https://issues.apache.org/jira/browse/KAFKA-10555 > > https://issues.apache.org/jira/browse/KAFKA-9638 > > https://issues.apache.org/jira/browse/KAFKA-6520 > > > > And maybe we can also left a comment on those tickets explaining what > >>> would > > happen to tackle the issues after this KIP. > > > > > > Guozhang > > > > > > On Tue, Dec 8, 2020 at 12:16 PM Walker Carlson < > wcarl...@confluent.io> > > wrote: > > > >> Hello all, > >> > >> I'd like to propose KIP-696 to clarify the meaning of ERROR state in > >>> the > >> KafkaStreams Client State Machine. This will update the States to be > >> consistent with changes in KIP-671 and KIP-663. > >> > >> Here are the details: https://cwiki.apache.org/confluence/x/lCvZCQ > >> > >> Thanks, > >> Walker > >> > > > > > > -- > > -- Guozhang > > > >>> >
[jira] [Created] (KAFKA-10872) Log broker configuration prefixed with "listener.name.*"
Badai Aqrandista created KAFKA-10872:
Summary: Log broker configuration prefixed with "listener.name.*"
Key: KAFKA-10872
URL: https://issues.apache.org/jira/browse/KAFKA-10872
Project: Kafka
Issue Type: Improvement
Reporter: Badai Aqrandista
When configuring broker listeners with "listener.name.*" prefix, it is very
hard to verify in the log if we are passing the correct value or if we're
missing any values.
Can we log these configuration at INFO level? For example:
{code:java}
listener.name.internal.ssl.truststore.location
listener.name.internal.ssl.truststore.password
listener.name.internal.ssl.keystore.location
listener.name.internal.ssl.keystore.password
{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
Re: Kafka JIRA ticket assign access request
Done, thanks for your contribution! From: Avijit Chakraborty Sent: Sunday, December 20, 2020 8:47 PM To: dev@kafka.apache.org Subject: Kafka JIRA ticket assign access request Hello Team, I am a newbie in Kafka. So I would like to contribute to the Kafka project. Could you please provide the necessary access in JIRA to assign unassigned tickets to me? My userName is: *aviprogrammer* Cheers, Avijit Chakraborty
[jira] [Created] (KAFKA-10871) StreamTask shouldn't take WallClockTime as input parameter in process method
Rohit Deshpande created KAFKA-10871: --- Summary: StreamTask shouldn't take WallClockTime as input parameter in process method Key: KAFKA-10871 URL: https://issues.apache.org/jira/browse/KAFKA-10871 Project: Kafka Issue Type: Bug Components: streams Affects Versions: 2.7.0 Reporter: Rohit Deshpande Assignee: Rohit Deshpande While working on https://issues.apache.org/jira/browse/KAFKA-10062 I realized process method in StreamTask is taking wallClockTime as input parameter which is redundant as StreamTask already contains time(https://github.com/apache/kafka/blob/2.5.0/streams/src/main/java/org/apache/kafka/streams/processor/internals/StreamTask.java#L75) field which represents wallClockTime. In process method (https://github.com/apache/kafka/blob/2.7/streams/src/main/java/org/apache/kafka/streams/processor/internals/StreamTask.java#L664), wallClockTime can be passed from StreamTask's time field itself. This method was changed as part of pr: https://github.com/apache/kafka/pull/7997. As part of https://issues.apache.org/jira/browse/KAFKA-10062, I believe wallClockTime need not be stored in ProcessorContext(https://github.com/apache/kafka/blob/trunk/streams/src/main/java/org/apache/kafka/streams/processor/internals/AbstractProcessorContext.java#L48) but should be fetched from StreamTask's time field. Reference pr: https://github.com/apache/kafka/pull/9744 -- This message was sent by Atlassian Jira (v8.3.4#803005)
Kafka JIRA ticket assign access request
Hello Team, I am a newbie in Kafka. So I would like to contribute to the Kafka project. Could you please provide the necessary access in JIRA to assign unassigned tickets to me? My userName is: *aviprogrammer* Cheers, Avijit Chakraborty
