Thanks for the KIP Jose! Adding this ability makes sense to me. Just a few
quick wonderments:
1. Do we need this value to be of the order of `ms`. Is it better off being
tunable to a minute granularity?
2. Somewhat related to 1 - Do we need to make any tweaks to the way we
cleanup the metadata
[
https://issues.apache.org/jira/browse/KAFKA-14275?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Niket Goel resolved KAFKA-14275.
Fix Version/s: 3.3
Reviewer: Jason Gustafson
Resolution: Fixed
> KR
Niket Goel created KAFKA-14275:
--
Summary: KRaft Controllers should crash after failing to apply any
metadata record
Key: KAFKA-14275
URL: https://issues.apache.org/jira/browse/KAFKA-14275
Project
[
https://issues.apache.org/jira/browse/KAFKA-13888?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Niket Goel resolved KAFKA-13888.
Resolution: Fixed
> KIP-836: Addition of Information in DescribeQuorumResponse about Voter
[
https://issues.apache.org/jira/browse/KAFKA-14114?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Niket Goel resolved KAFKA-14114.
Resolution: Fixed
[https://github.com/apache/kafka/commit
(hit send too soon)
KIP --
https://cwiki.apache.org/confluence/display/KAFKA/KIP-859%3A+Add+Metadata+Log+Processing+Error+Related+Metrics
JIRA -- https://issues.apache.org/jira/browse/KAFKA-14114
On Thu, Aug 4, 2022 at 2:33 PM Niket Goel wrote:
> Hey Jose,
>
> I would like to request a
.
Thanks
Niket Goel
On Thu, Aug 4, 2022 at 8:58 AM José Armando García Sancio
wrote:
> On Thu, Aug 4, 2022 at 8:37 AM Justine Olshan
> wrote:
> >
> > Hey Jose.
> > I found a gap in handling ISR changes in ZK mode. We just need to prevent
> > brokers that are offline
Thanks everyone for the feedback and votes. I have three +1s (David, Colin,
Jose).
Closing this vote now.
On Thu, Aug 4, 2022 at 2:09 PM José Armando García Sancio
wrote:
> Thanks for the improvement. LGTM. +1 (binding).
>
> --
> -José
>
- Niket
Hey Jose,
> How about the inactive controller? Are inactive controllers going to
update this metric when they encounter an error when replaying a
record?
Yes, this metric will be reported for both active and inactive controllers.
The Inactive controllers will update this metric when they
ntroller metadata error metric.
>
> How do you feel about:
>
> ForceRenounceCount =>
> kafka.controller:type=KafkaController,name=MetadataErrorCount
> publisher-error-count => metadata-load-error-count
> listener-batch-load-error-count => metadata-apply-error-count
combined mode -- you get reduced fault isolation in exchange for the lower
> overhead of one fewer JVM process).
>
> best,
> Colin
>
>
> On Mon, Aug 1, 2022, at 18:05, David Arthur wrote:
>> Thanks, Niket.
>>
>> +1 binding from me
>>
>> -David
>>
he cumulative number of publisher errors since the broker started?
> 2) Same question for "listener-batch-load-error-count"
> 3) Will ForceRenounceCount be zero for non-leader controllers? Or will this
> value remain between elections and only get reset to zero upon a restart
>
Hi all,
I would like to start a vote on KIP-859 which adds some new metrics to KRaft to
allow for better visibility into log processing errors.
KIP
—ttps://cwiki.apache.org/confluence/display/KAFKA/KIP-859%3A+Add+Metadata+Log+Processing+Error+Related+Metrics
ge? Or will this value be
> the cumulative number of publisher errors since the broker started?
> 2) Same question for "listener-batch-load-error-count"
> 3) Will ForceRenounceCount be zero for non-leader controllers? Or will this
> value remain between elections and only get
Hi all,
I would like to start a discussion on adding some new metrics to KRaft to allow
for better visibility into log processing errors.
KIP URL:
https://cwiki.apache.org/confluence/display/KAFKA/KIP-859%3A+Add+Metadata+Log+Processing+Error+Related+Metrics
Thanks!
Niket
Niket Goel created KAFKA-14114:
--
Summary: KIP-859: Add Metadata Log Processing Error Related Metrics
Key: KAFKA-14114
URL: https://issues.apache.org/jira/browse/KAFKA-14114
Project: Kafka
Issue
[
https://issues.apache.org/jira/browse/KAFKA-13806?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Niket Goel resolved KAFKA-13806.
Fix Version/s: 3.3
Resolution: Duplicate
> Check CRC when reading snapsh
Niket Goel created KAFKA-14104:
--
Summary: Perform CRC validation on KRaft Batch Records and
Snapshots
Key: KAFKA-14104
URL: https://issues.apache.org/jira/browse/KAFKA-14104
Project: Kafka
Hey Jose,
Thanks for the KIP. This is a good improvement and will make the KRaft
implementation much more robust in the face of failures and generally make it
more flexible for users.
I did a first pass through the KIP and here are some comments (some of these
might just be a little
Niket Goel created KAFKA-14035:
--
Summary: QuorumController handleRenounce throws NPE
Key: KAFKA-14035
URL: https://issues.apache.org/jira/browse/KAFKA-14035
Project: Kafka
Issue Type: Bug
pic name from QuorumInfo.
> 2) Making ReplicaState public APIs more aligned with existing ones.
> 3) Renaming from DescribeQuorumResult to DescribeMetadataQuorumResult.
>
> Is that all? If yes, I'm +1 on the changes.
>
>
> Guozhang
>
> On Mon, Jun 13, 2022 at 2:0
Niket Goel created KAFKA-13986:
--
Summary: DescribeQuorum does not return the observers (brokers)
for the Metadata log
Key: KAFKA-13986
URL: https://issues.apache.org/jira/browse/KAFKA-13986
Project
, 2022 at 11:52 AM Niket Goel
> wrote:
>>
>> Thanks everyone for the feedback on this. KIP-836 was approved with
>> 3 binding votes from Luke, David Arthur and David Jacot.
>> - Niket
>>
>>> On May 20, 2022, at 11:44 PM, David Jacot wrote:
>>>
>
Thanks everyone for the feedback on this. KIP-836 was approved with
3 binding votes from Luke, David Arthur and David Jacot.
- Niket
> On May 20, 2022, at 11:44 PM, David Jacot wrote:
>
> +1 (binding)
>
> Thanks for the KIP!
>
> David
>
> Le ven. 20 mai 2022 à 21:08, David Arthur a écrit :
So Ideally the LastCatchUpTimeMs is supposed to be a proxy for lag. We could
report this field as lag, but I am personally not in favor of doing that as the
LastCaughtUpTimeMs is just an approximation of lag, but not actual lag.
I guess it depends on the experience we want through the tool. Do
. You only add an Handle in AdminClient, can you also add an
> `kafka-metadata-quorum.sh` tool to help this?
>
>
> > On May 17, 2022, at 9:50 AM, Niket Goel
> wrote:
> >
> > Thanks for the call out David. We will populate these fields for the
> > Observers as well.
ell?
>
> Thanks!
> David
>
>
> On Mon, May 16, 2022 at 1:46 PM Niket Goel
> wrote:
>
> > Hi all,
> >
> > Thank you for the feedback on this. I have started a voting thread for
> > this KIP here:
> > https://lists.apache.org/thread/bkb7gsbxpljh5
Hi all,
Thank you for the feedback on this. I have started a voting thread for this KIP
here:
https://lists.apache.org/thread/bkb7gsbxpljh5qh014ztffq7bldjrb2x
Thanks
Niket Goel
From: Niket Goel
Date: Thursday, May 12, 2022 at 5:25 PM
To: dev@kafka.apache.org
Subject: Re: [DISCUSS] KIP-836
Hi all,
I would like to start a vote for KIP-836:
https://cwiki.apache.org/confluence/display/KAFKA/KIP-836%3A+Addition+of+Information+in+DescribeQuorumResponse+about+Voter+Lag
Thanks
Niket Goel
Appreciate the careful review Jose.!
Ack on 1 and 2. Will fix.
For number 3 (and I am using [1] as a reference for this discussion), I
think the correct language to use would be:
"Whenever a new fetch request
comes in the replica's last caught up time is updated to the time of
this fetch
er voters, and then what would it send in the
> response? If this is possible then we should be explicit about what
> is to be sent in this case.
>
> Ron
>
> On Thu, May 12, 2022 at 12:54 PM Niket Goel
> wrote:
> >
> > Thanks for the suggestion Colin.
> >
> >
the wall clock time at which the leader made the latest fetch. If we
> take both timestamps directly from the metadata log, we know they'll be
> comparable even in the presence of clock skew. And we know because of
> KIP-835 that the metadata log won't go quiet for prolonged per
lish this.
> >
> > --
> > Thanks
> > Ziming
> >
> > > On May 10, 2022, at 1:33 AM, Niket Goel
> > wrote:
> > >
> > > Hi all,
> > >
> > > I created a KIP to add some more information to
> > `DesscribeQuorumRe
Thanks for your feedback,
Niket Goel
Niket Goel created KAFKA-13888:
--
Summary: KIP-836: Addition of Information in
DescribeQuorumResponse about Voter Lag
Key: KAFKA-13888
URL: https://issues.apache.org/jira/browse/KAFKA-13888
Project
ss, or even
point me towards more suggested reading if I am missing things. :)
Thanks
Niket Goel
[1] https://issues.apache.org/jira/browse/KAFKA-2260
<https://issues.apache.org/jira/browse/KAFKA-2260>
[2] https://wepay.github.io/waltz/docs/introduction
<https://wepay.github.io/waltz
[
https://issues.apache.org/jira/browse/KAFKA-13189?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Niket Goel resolved KAFKA-13189.
Resolution: Fixed
> Revisit the CreateTopic API behavior when number of Unfenced Brok
Niket Goel created KAFKA-13190:
--
Summary: Revisit the CreateTopic API behavior when number of
Unfenced Brokers available for placement is less than min_isr
Key: KAFKA-13190
URL: https://issues.apache.org/jira/browse
Niket Goel created KAFKA-13189:
--
Summary: Revisit the CreateTopic API behavior when number of
Unfenced Brokers available for placement is less than min_isr
Key: KAFKA-13189
URL: https://issues.apache.org/jira/browse
[
https://issues.apache.org/jira/browse/KAFKA-13015?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Niket Goel resolved KAFKA-13015.
Fix Version/s: 3.0.0
Reviewer: Jose Armando Garcia Sancio
Resolution: Fixed
Niket Goel created KAFKA-13110:
--
Summary: Review and Optimize KIP-500 related Settings
Key: KAFKA-13110
URL: https://issues.apache.org/jira/browse/KAFKA-13110
Project: Kafka
Issue Type
Niket Goel created KAFKA-13025:
--
Summary: Add more information to the Snapshot header and footer
Key: KAFKA-13025
URL: https://issues.apache.org/jira/browse/KAFKA-13025
Project: Kafka
Issue
Hi kafka-dev team,
Please add me as a contributor to the Kafka JIRA and Confluence wiki. My JIRA
username is 'Niket Goel' and my Confluent ID is ngoel.
Thanks
Niket Goel
Niket Goel created KAFKA-13016:
--
Summary: Interpret snapshot header version to correctly parse the
snapshot
Key: KAFKA-13016
URL: https://issues.apache.org/jira/browse/KAFKA-13016
Project: Kafka
Niket Goel created KAFKA-13015:
--
Summary: Create System tests for Metadata Snapshots
Key: KAFKA-13015
URL: https://issues.apache.org/jira/browse/KAFKA-13015
Project: Kafka
Issue Type: Sub-task
Niket Goel created KAFKA-12997:
--
Summary: Expose log record append time to the controller/broker
Key: KAFKA-12997
URL: https://issues.apache.org/jira/browse/KAFKA-12997
Project: Kafka
Issue
Niket Goel created KAFKA-12952:
--
Summary: Metadata Snapshot File Header
Key: KAFKA-12952
URL: https://issues.apache.org/jira/browse/KAFKA-12952
Project: Kafka
Issue Type: Sub-task
47 matches
Mail list logo
