[jira] [Resolved] (KAFKA-17044) Connector deletion can lead to resource leak during a long running connector startup

running connector > startup > > > Key: KAFKA-17044 > URL: https://issues.apache.org/jira/browse/KAFKA-17044 > Project: Kafka > Issue Ty

[jira] [Reopened] (KAFKA-16935) Automatically wait for cluster startup in embedded Connect integration tests

[ https://issues.apache.org/jira/browse/KAFKA-16935?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Egerton reopened KAFKA-16935: --- > Automatically wait for cluster startup in embedded Connect integration te

[jira] [Resolved] (KAFKA-16935) Automatically wait for cluster startup in embedded Connect integration tests

[ https://issues.apache.org/jira/browse/KAFKA-16935?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Egerton resolved KAFKA-16935. --- Resolution: Fixed > Automatically wait for cluster startup in embedded Connect integrat

[jira] [Created] (KAFKA-16943) Synchronously verify Connect worker startup failure in InternalTopicsIntegrationTest

Chris Egerton created KAFKA-16943: - Summary: Synchronously verify Connect worker startup failure in InternalTopicsIntegrationTest Key: KAFKA-16943 URL: https://issues.apache.org/jira/browse/KAFKA-16943

[jira] [Created] (KAFKA-16935) Automatically wait for cluster startup in embedded Connect integration tests

Chris Egerton created KAFKA-16935: - Summary: Automatically wait for cluster startup in embedded Connect integration tests Key: KAFKA-16935 URL: https://issues.apache.org/jira/browse/KAFKA-16935

[jira] [Resolved] (KAFKA-16814) KRaft broker cannot startup when `partition.metadata` is missing

[ https://issues.apache.org/jira/browse/KAFKA-16814?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Luke Chen resolved KAFKA-16814. --- Resolution: Fixed > KRaft broker cannot startup when `partition.metadata` is miss

[jira] [Resolved] (KAFKA-6208) Reduce startup time for Kafka Connect workers

=service_load on 3.6+, see KAFKA-14627/KIP-898 for more details. > Reduce startup time for Kafka Connect workers > - > > Key: KAFKA-6208 > URL: https://issues.apache.org/jira/browse/KAFKA-6208 >

[jira] [Created] (KAFKA-16814) KRaft broker cannot startup when `partition.metadata` is missing

Luke Chen created KAFKA-16814: - Summary: KRaft broker cannot startup when `partition.metadata` is missing Key: KAFKA-16814 URL: https://issues.apache.org/jira/browse/KAFKA-16814 Project: Kafka

[jira] [Created] (KAFKA-16553) log controller configs when startup

Chia-Ping Tsai created KAFKA-16553: -- Summary: log controller configs when startup Key: KAFKA-16553 URL: https://issues.apache.org/jira/browse/KAFKA-16553 Project: Kafka Issue Type

[jira] [Resolved] (KAFKA-16126) Kcontroller dynamic configurations may fail to apply at startup

[ https://issues.apache.org/jira/browse/KAFKA-16126?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Divij Vaidya resolved KAFKA-16126. -- Resolution: Fixed > Kcontroller dynamic configurations may fail to apply at star

Re: [DISCUSS] Uninitialized metadata cache during broker startup

I up PR 15441 with the immediate fix for the KafkaAdminClient to help get this conversation started. I would appreciate some feedback on that. Thanks, Chris On Fri, Feb 23, 2024 at 3:41 PM Chris Wildman wrote: > Hi All, > > I recently discovered the

[DISCUSS] Uninitialized metadata cache during broker startup

Hi All, I recently discovered the race condition where Kafka clients may request metadata from brokers who have not yet received a snapshot of the cluster metadata. At scale with ZK managed clusters we see this happen frequently when brokers are restarted, possibly due to the admin client's prefer

[jira] [Created] (KAFKA-16126) Kcontroller dynamic configurations may fail to apply at startup

Colin McCabe created KAFKA-16126: Summary: Kcontroller dynamic configurations may fail to apply at startup Key: KAFKA-16126 URL: https://issues.apache.org/jira/browse/KAFKA-16126 Project: Kafka

[jira] [Resolved] (KAFKA-15633) Bug: Generated Persistent Directory IDs are overwritten on startup.

on startup. > --- > > Key: KAFKA-15633 > URL: https://issues.apache.org/jira/browse/KAFKA-15633 > Project: Kafka > Issue Type: Sub-task > Components: jb

[jira] [Created] (KAFKA-15804) Broker leaks ServerSocketChannel when exception is thrown from ZkConfigManager during startup

Greg Harris created KAFKA-15804: --- Summary: Broker leaks ServerSocketChannel when exception is thrown from ZkConfigManager during startup Key: KAFKA-15804 URL: https://issues.apache.org/jira/browse/KAFKA-15804

[jira] [Created] (KAFKA-15633) Bug: Generated Persistent Directory IDs are overwritten on startup.

Proven Provenzano created KAFKA-15633: - Summary: Bug: Generated Persistent Directory IDs are overwritten on startup. Key: KAFKA-15633 URL: https://issues.apache.org/jira/browse/KAFKA-15633

[jira] [Resolved] (KAFKA-15157) Print startup time for RemoteIndexCache

[ https://issues.apache.org/jira/browse/KAFKA-15157?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Divij Vaidya resolved KAFKA-15157. -- Resolution: Fixed > Print startup time for RemoteIndexCa

[jira] [Created] (KAFKA-15161) InvalidReplicationFactorException at connect startup

Viktor Somogyi-Vass created KAFKA-15161: --- Summary: InvalidReplicationFactorException at connect startup Key: KAFKA-15161 URL: https://issues.apache.org/jira/browse/KAFKA-15161 Project: Kafka

[jira] [Created] (KAFKA-15157) Print startup time for RemoteIndexCache

Divij Vaidya created KAFKA-15157: Summary: Print startup time for RemoteIndexCache Key: KAFKA-15157 URL: https://issues.apache.org/jira/browse/KAFKA-15157 Project: Kafka Issue Type: Sub-task

[jira] [Created] (KAFKA-15086) The unreasonable segment size setting of the internal topics in MM2 may cause the worker startup time to be too long

hudeqi created KAFKA-15086: -- Summary: The unreasonable segment size setting of the internal topics in MM2 may cause the worker startup time to be too long Key: KAFKA-15086 URL: https://issues.apache.org/jira/browse

[jira] [Resolved] (KAFKA-14962) Whitespace in ACL configuration causes Kafka startup to fail

fka startup to fail > > > Key: KAFKA-14962 > URL: https://issues.apache.org/jira/browse/KAFKA-14962 > Project: Kafka > Issue Type: Bug >Reporter: Divij Vaid

[jira] [Created] (KAFKA-14962) Whitespace in ACL configuration causes Kafka startup to fail

Divij Vaidya created KAFKA-14962: Summary: Whitespace in ACL configuration causes Kafka startup to fail Key: KAFKA-14962 URL: https://issues.apache.org/jira/browse/KAFKA-14962 Project: Kafka

[jira] [Created] (KAFKA-14773) Make MirrorMaker startup synchronous

Divij Vaidya created KAFKA-14773: Summary: Make MirrorMaker startup synchronous Key: KAFKA-14773 URL: https://issues.apache.org/jira/browse/KAFKA-14773 Project: Kafka Issue Type: Improvement

[jira] [Resolved] (KAFKA-14713) Kafka Streams global table startup takes too long

[ https://issues.apache.org/jira/browse/KAFKA-14713?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax resolved KAFKA-14713. - Resolution: Fixed > Kafka Streams global table startup takes too l

[jira] [Reopened] (KAFKA-14713) Kafka Streams global table startup takes too long

[ https://issues.apache.org/jira/browse/KAFKA-14713?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax reopened KAFKA-14713: - > Kafka Streams global table startup takes too l

[jira] [Resolved] (KAFKA-14621) Don't startup in migration mode if an authorizer is enabled

  > Don't startup in migration mode if an authorizer is enabled > --- > > Key: KAFKA-14621 > URL: https://issues.apache.org/jira/browse/KAFKA-14621 > Project: Kafka >

[jira] [Resolved] (KAFKA-14713) Kafka Streams global table startup takes too long

[ https://issues.apache.org/jira/browse/KAFKA-14713?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tamas resolved KAFKA-14713. --- Fix Version/s: 3.4.0 Resolution: Invalid > Kafka Streams global table startup takes too l

[jira] [Created] (KAFKA-14713) Kafka Streams global table startup takes too long

Tamas created KAFKA-14713: - Summary: Kafka Streams global table startup takes too long Key: KAFKA-14713 URL: https://issues.apache.org/jira/browse/KAFKA-14713 Project: Kafka Issue Type: Bug

[jira] [Created] (KAFKA-14649) Failures instantiating Connect plugins crash worker on startup

Greg Harris created KAFKA-14649: --- Summary: Failures instantiating Connect plugins crash worker on startup Key: KAFKA-14649 URL: https://issues.apache.org/jira/browse/KAFKA-14649 Project: Kafka

[jira] [Created] (KAFKA-14621) Don't startup in migration mode if an authorizer is enabled

David Arthur created KAFKA-14621: Summary: Don't startup in migration mode if an authorizer is enabled Key: KAFKA-14621 URL: https://issues.apache.org/jira/browse/KAFKA-14621 Project:

Kafka log segments recovery on startup

Hello, Can anyone please point to any docs re how replica recovery works when broker is started?  The documentation has information about broker rebuilding based on data on data dir- but what happens if data is completely lost? Would an empty replica be created based on zookeeper state- and if y

[jira] [Resolved] (KAFKA-13586) ConfigExceptions thrown by FileConfigProvider during connector/task startup crash worker

ask startup > crash worker > > > Key: KAFKA-13586 > URL: https://issues.apache.org/jira/browse/KAFKA-13586 > Project: Kafka > Issue Type: B

[jira] [Created] (KAFKA-14330) The Kafka Connection Is High During Service Startup

flybird created KAFKA-14330: --- Summary: The Kafka Connection Is High During Service Startup Key: KAFKA-14330 URL: https://issues.apache.org/jira/browse/KAFKA-14330 Project: Kafka Issue Type

[jira] [Resolved] (KAFKA-14270) Kafka Streams logs exception on startup

[ https://issues.apache.org/jira/browse/KAFKA-14270?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bruno Cadonna resolved KAFKA-14270. --- Resolution: Fixed > Kafka Streams logs exception on star

[jira] [Created] (KAFKA-14270) Kafka Streams logs exception on startup

Oliver Eikemeier created KAFKA-14270: Summary: Kafka Streams logs exception on startup Key: KAFKA-14270 URL: https://issues.apache.org/jira/browse/KAFKA-14270 Project: Kafka Issue Type

[jira] [Created] (KAFKA-14258) Add ducktape or junit test verifying that brokers can reload snapshots after startup

Colin McCabe created KAFKA-14258: Summary: Add ducktape or junit test verifying that brokers can reload snapshots after startup Key: KAFKA-14258 URL: https://issues.apache.org/jira/browse/KAFKA-14258

[jira] [Resolved] (KAFKA-14197) Kraft broker fails to startup after topic creation failure

[ https://issues.apache.org/jira/browse/KAFKA-14197?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin McCabe resolved KAFKA-14197. -- Resolution: Duplicate > Kraft broker fails to startup after topic creation fail

[jira] [Created] (KAFKA-14197) Kraft broker fails to startup after topic creation failure

Luke Chen created KAFKA-14197: - Summary: Kraft broker fails to startup after topic creation failure Key: KAFKA-14197 URL: https://issues.apache.org/jira/browse/KAFKA-14197 Project: Kafka Issue

[jira] [Created] (KAFKA-13586) ConfigExceptions thrown by FileConfigProvider during connector/task startup crash worker

Greg Harris created KAFKA-13586: --- Summary: ConfigExceptions thrown by FileConfigProvider during connector/task startup crash worker Key: KAFKA-13586 URL: https://issues.apache.org/jira/browse/KAFKA-13586

[jira] [Created] (KAFKA-13454) kafka has duplicate configuration information during startup

zhangzhanchang created KAFKA-13454: -- Summary: kafka has duplicate configuration information during startup Key: KAFKA-13454 URL: https://issues.apache.org/jira/browse/KAFKA-13454 Project: Kafka

[jira] [Created] (KAFKA-13232) UnknownTopicOrPartitionException for loading topic during broker startup.

Pranciskus Gecas created KAFKA-13232: Summary: UnknownTopicOrPartitionException for loading topic during broker startup. Key: KAFKA-13232 URL: https://issues.apache.org/jira/browse/KAFKA-13232

[jira] [Created] (KAFKA-13151) KRaft does not support Policies (e.g. AlterConfigPolicy). The server should fail startup if any are configured.

Ryan Dielhenn created KAFKA-13151: - Summary: KRaft does not support Policies (e.g. AlterConfigPolicy). The server should fail startup if any are configured. Key: KAFKA-13151 URL: https://issues.apache.org/jira

[jira] [Resolved] (KAFKA-13119) Validate the KRaft controllerListener config on startup

[ https://issues.apache.org/jira/browse/KAFKA-13119?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin McCabe resolved KAFKA-13119. -- Resolution: Fixed > Validate the KRaft controllerListener config on star

[jira] [Created] (KAFKA-13119) Validate the KRaft controllerListener config on startup

Colin McCabe created KAFKA-13119: Summary: Validate the KRaft controllerListener config on startup Key: KAFKA-13119 URL: https://issues.apache.org/jira/browse/KAFKA-13119 Project: Kafka

[jira] [Resolved] (KAFKA-12520) Producer state is needlessly rebuilt on startup

the PR to trunk > Producer state is needlessly rebuilt on startup > --- > > Key: KAFKA-12520 > URL: https://issues.apache.org/jira/browse/KAFKA-12520 > Project: Kafka >

[jira] [Created] (KAFKA-12520) Producer state is needlessly rebuilt on startup

Dhruvil Shah created KAFKA-12520: Summary: Producer state is needlessly rebuilt on startup Key: KAFKA-12520 URL: https://issues.apache.org/jira/browse/KAFKA-12520 Project: Kafka Issue Type

[jira] [Created] (KAFKA-10207) Untrimmed Index files cause premature log segment deletions on startup

Johnny Malizia created KAFKA-10207: -- Summary: Untrimmed Index files cause premature log segment deletions on startup Key: KAFKA-10207 URL: https://issues.apache.org/jira/browse/KAFKA-10207 Project

[jira] [Resolved] (KAFKA-9216) Enforce connect internal topic configuration at startup

the `cleanup.policy` topic setting on Connect's three internal topics, and cherry-picked it to the `2.6` (for upcoming 2.6.0). However, merging to earlier branches requires too many changes in integration tests. > Enforce connect internal topic configuration at

[jira] [Reopened] (KAFKA-9216) Enforce connect internal topic configuration at startup

al topic configuration at startup > --- > > Key: KAFKA-9216 > URL: https://issues.apache.org/jira/browse/KAFKA-9216 > Project: Kafka > Issue Type: Improvement > Componen

[jira] [Resolved] (KAFKA-9216) Enforce connect internal topic configuration at startup

[ https://issues.apache.org/jira/browse/KAFKA-9216?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Konstantine Karantasis resolved KAFKA-9216. --- Resolution: Fixed > Enforce connect internal topic configuration at star

[jira] [Resolved] (KAFKA-9901) TimeoutError: Never saw message indicating StreamsTest finished startup on ducker@ducker07

[ https://issues.apache.org/jira/browse/KAFKA-9901?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jiamei xie resolved KAFKA-9901. --- Resolution: Duplicate > TimeoutError: Never saw message indicating StreamsTest finished startup

[jira] [Created] (KAFKA-9912) Kafka deletes full log on startup written 30mins ago - wrongly claims retention time 7d breach

Igor Baltiyskiy created KAFKA-9912: -- Summary: Kafka deletes full log on startup written 30mins ago - wrongly claims retention time 7d breach Key: KAFKA-9912 URL: https://issues.apache.org/jira/browse/KAFKA-9912

[jira] [Created] (KAFKA-9901) TimeoutError: Never saw message indicating StreamsTest finished startup on ducker@ducker07

jiamei xie created KAFKA-9901: - Summary: TimeoutError: Never saw message indicating StreamsTest finished startup on ducker@ducker07 Key: KAFKA-9901 URL: https://issues.apache.org/jira/browse/KAFKA-9901

[jira] [Created] (KAFKA-9887) failed-task-count JMX metric not updated if task fails during startup

Chris Egerton created KAFKA-9887: Summary: failed-task-count JMX metric not updated if task fails during startup Key: KAFKA-9887 URL: https://issues.apache.org/jira/browse/KAFKA-9887 Project: Kafka

[jira] [Created] (KAFKA-9216) Enforce connect internal topic configuration at startup

Randall Hauch created KAFKA-9216: Summary: Enforce connect internal topic configuration at startup Key: KAFKA-9216 URL: https://issues.apache.org/jira/browse/KAFKA-9216 Project: Kafka Issue

[jira] [Resolved] (KAFKA-1111) Broker prematurely accepts TopicMetadataRequests on startup

still an issue we can always reopen this. > Broker prematurely accepts TopicMetadataRequests on startup > --- > > Key: KAFKA- > URL: https://issues.apache.org/jira/browse/KAFKA- >

[jira] [Created] (KAFKA-8731) InMemorySessionStore throws NullPointerException on startup

Jonathan Gordon created KAFKA-8731: -- Summary: InMemorySessionStore throws NullPointerException on startup Key: KAFKA-8731 URL: https://issues.apache.org/jira/browse/KAFKA-8731 Project: Kafka

Re: Broker Startup Time (Prolonged URPs)

Hi All,  Does anyone have any thoughts on this? Many Thanks,  Jamie -Original Message- From: Jamie To: dev ; users Sent: Mon, 8 Jul 2019 18:38 Subject: Broker Startup Time (Prolonged URPs) Hi All, I'm currently doing some work into investigating why our Kafka deployment

Broker Startup Time (Prolonged URPs)

tions are replicas. What I noticed in the logs was that after the server had completed startup was around 30-45 minutes of the messages similar to the below in the logs:    INFO [Log partition=topic-100-6, dir=/d/d4/kafka-data/kafka-data] Truncating to offset 170 (kafka.log.Log)    INFO [Log p

[jira] [Resolved] (KAFKA-8564) NullPointerException when loading logs at startup

> NullPointerException when loading logs at startup > - > > Key: KAFKA-8564 > URL: https://issues.apache.org/jira/browse/KAFKA-8564 > Project: Kafka > Issue Type: Bug >

[jira] [Created] (KAFKA-8564) NullPointerException when loading logs at startup

Mickael Maison created KAFKA-8564: - Summary: NullPointerException when loading logs at startup Key: KAFKA-8564 URL: https://issues.apache.org/jira/browse/KAFKA-8564 Project: Kafka Issue Type

[jira] [Created] (KAFKA-8480) Clients may fetch incomplete set of topic partitions during cluster startup

Anna Povzner created KAFKA-8480: --- Summary: Clients may fetch incomplete set of topic partitions during cluster startup Key: KAFKA-8480 URL: https://issues.apache.org/jira/browse/KAFKA-8480 Project

[jira] [Resolved] (KAFKA-8066) ReplicaFetcherThread fails to startup because of failing to register the metric.

2.0.2 > ReplicaFetcherThread fails to startup because of failing to register the > metric. > > > Key: KAFKA-8066 > URL: https://issues.apache.org/jira

[jira] [Created] (KAFKA-8066) ReplicaFetcherThread fails to startup because of failing to register the metric.

Zhanxiang (Patrick) Huang created KAFKA-8066: Summary: ReplicaFetcherThread fails to startup because of failing to register the metric. Key: KAFKA-8066 URL: https://issues.apache.org/jira/browse/KAFKA

[jira] [Created] (KAFKA-7924) Kafka broker does not list nodes first ~ 30s after startup

Gert van Dijk created KAFKA-7924: Summary: Kafka broker does not list nodes first ~ 30s after startup Key: KAFKA-7924 URL: https://issues.apache.org/jira/browse/KAFKA-7924 Project: Kafka

[jira] [Created] (KAFKA-7468) KafkaController no need to startup the inner kafkaScheducer when auto.leader.rebalance.enable is false

ZhangShuai created KAFKA-7468: - Summary: KafkaController no need to startup the inner kafkaScheducer when auto.leader.rebalance.enable is false Key: KAFKA-7468 URL: https://issues.apache.org/jira/browse/KAFKA-7468

Could not find or load main class kafka.Kafka (Eclipse Startup)

Hello, I completed set up according to cwiki instruction - https://cwiki.apache.org/confluence/display/KAFKA/Developer+Setup I have also built all the sources and jars using `gradlew jarAll` After that, I created a Scala application configuration in Eclipse to debug. I provided the sources to lo

Re: [DISCUSS] KIP-263: Allow broker to skip sanity check of inactive segments on broker startup

skip sanity check of inactive segments on broker startup Here is some information related to the KIP. Previously we thought we can ignore the sanity check by default to speedup broker startup and there is no need for extra configuration. However based on the code and some previous experiment resu

Re: [DISCUSS] KIP-263: Allow broker to skip sanity check of inactive segments on broker startup

Here is some information related to the KIP. Previously we thought we can ignore the sanity check by default to speedup broker startup and there is no need for extra configuration. However based on the code and some previous experiment result it is not clear whether it will work. When

[jira] [Resolved] (KAFKA-5037) Infinite loop if all input topics are unknown at startup

://github.com/apache/kafka/pull/5322] > Infinite loop if all input topics are unknown at startup > > > Key: KAFKA-5037 > URL: https://issues.apache.org/jira/browse/KAFKA-5037 >

[jira] [Resolved] (KAFKA-4122) Consumer startup swallows DNS resolution exception and infinitely retries

[ https://issues.apache.org/jira/browse/KAFKA-4122?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Manikumar resolved KAFKA-4122. -- Resolution: Duplicate Fixed under KAFKA-7111. > Consumer startup swallows DNS resolution except

Re: [DISCUSS] KIP-263: Allow broker to skip sanity check of inactive segments on broker startup

in my opinion. We > could handle the missing index case at startup. I think we could have > missing index files only when users are upgrading from a version that did > not have a particular type of index to a version that does, or if the > operator physically deleted these files. Because

Re: [DISCUSS] KIP-263: Allow broker to skip sanity check of inactive segments on broker startup

+1 to what Jason said. We need a better long-term strategy for dealing with corrupted log and index data, but the sanity checks we have do not guarantee much in this regard. For now, we could do away with these index sanity checks in my opinion. We could handle the missing index case at startup

Re: [DISCUSS] KIP-263: Allow broker to skip sanity check of inactive segments on broker startup

very rare, e.g. it happens only when there is > disk error or when there is human error. I think we should probably still handle the case when an index file is missing during startup? But considering how weak the sanity check is, it seems fine to skip it. Also, could we just make this change

Re: [DISCUSS] KIP-263: Allow broker to skip sanity check of inactive segments on broker startup

t;> first >> place. We detect garbage at the end of the index file, but that's about >> it. >> Is there any reason to think that corruption is more likely to occur there >> or any other reason to think this check is still beneficial for flushed >> data? I assu

Re: [DISCUSS] KIP-263: Allow broker to skip sanity check of inactive segments on broker startup

ost is adding up as the number of partitions grows. How much > does startup time improve if we skip the sanity check for data earlier than > the recovery point? Does the lazy loading itself give some additional > benefit beyond skipping the sanity check? As Jay mentions above, the sanity &

Re: [DISCUSS] KIP-263: Allow broker to skip sanity check of inactive segments on broker startup

other reason to think this check is still beneficial for flushed data? I assume we did the check because we presumed it was cheap, but perhaps the cost is adding up as the number of partitions grows. How much does startup time improve if we skip the sanity check for data earlier than the recovery point?

[jira] [Created] (KAFKA-7076) Broker startup could be inefficient when using old message format

Dhruvil Shah created KAFKA-7076: --- Summary: Broker startup could be inefficient when using old message format Key: KAFKA-7076 URL: https://issues.apache.org/jira/browse/KAFKA-7076 Project: Kafka

[Kafka tests] "TimeoutError: MiniKdc didn't finish startup"

Hello, Folks anybody have fix the problem "TimeoutError: MiniKdc didn't finish startup" on the kafka sanity tests in ../kafka-1.0.1.redhat-7/tests/kafkatest/sanity_checks/test_console_consumer.py? Also interesting who works with kafka tests and how better do debug the tests ? BR

[jira] [Created] (KAFKA-6757) Log runtime exception in case of server startup errors

Enrico Olivelli created KAFKA-6757: -- Summary: Log runtime exception in case of server startup errors Key: KAFKA-6757 URL: https://issues.apache.org/jira/browse/KAFKA-6757 Project: Kafka

[jira] [Resolved] (KAFKA-6724) ConsumerPerformance resets offsets on every startup

ery startup > --- > > Key: KAFKA-6724 > URL: https://issues.apache.org/jira/browse/KAFKA-6724 > Project: Kafka > Issue Type: Bug > Components: core, tools >

[jira] [Created] (KAFKA-6724) ConsumerPerformance resets offsets on every startup

Alex Dunayevsky created KAFKA-6724: -- Summary: ConsumerPerformance resets offsets on every startup Key: KAFKA-6724 URL: https://issues.apache.org/jira/browse/KAFKA-6724 Project: Kafka Issue

Re: [DISCUSS] KIP-263: Allow broker to skip sanity check of inactive segments on broker startup

file is corrupted, i.e. if two messages with the same offset are appended to the disk (KAFKA-6488). The sanity check on broker startup is a bit different since it deals with the corruption of index files (e.g. offset index, time index and snapshot files) instead of the log data. In this case if index

Re: [DISCUSS] KIP-263: Allow broker to skip sanity check of inactive segments on broker startup

lean > broker shutdown. And in that case if we delay the sanity check then we may > delay the log recovery. The main goal of this KIP is to optimize the sanity > check related work so that it does not delay the broker startup much. > > The KIP mentioned that the sanity check is done

Re: [DISCUSS] KIP-263: Allow broker to skip sanity check of inactive segments on broker startup

this KIP is to optimize the sanity check related work so that it does not delay the broker startup much. The KIP mentioned that the sanity check is done using log recovery background thread. The name "recovery" is mentioned mainly because the background thread number is determined

Re: [DISCUSS] KIP-263: Allow broker to skip sanity check of inactive segments on broker startup

Clarificarion, it's a by name parameter so nothing happens unless the first parameter evaluates to false. Ismael On Mon, 19 Mar 2018, 19:21 Jay Kreps, wrote: > Optimizing startup seems really valuable but I'm a little confused by this. > > There are two different things

Re: [DISCUSS] KIP-263: Allow broker to skip sanity check of inactive segments on broker startup

Optimizing startup seems really valuable but I'm a little confused by this. There are two different things: 1. Recovery 2. Sanity check The terminology we're using is a bit mixed here. Recovery means checksumming the log segments and rebuilding the index on a hard crash. This only

[jira] [Resolved] (KAFKA-3978) Cannot truncate to a negative offset (-1) exception at broker startup

(-1) exception at broker startup > - > > Key: KAFKA-3978 > URL: https://issues.apache.org/jira/browse/KAFKA-3978 > Project: Kafka > Issue Type: Bug >Af

Re: [DISCUSS] KIP-263: Allow broker to skip sanity check of inactive segments on broker startup

. > > best, > Colin > > > On Tue, Feb 27, 2018, at 23:57, Stephane Maarek wrote: > > This is great and definitely needed. I'm not exactly sure of what goes in > > the process of checking log files at startup, but is there something like > > signature checks

Re: [DISCUSS] KIP-263: Allow broker to skip sanity check of inactive segments on broker startup

m not exactly sure of what goes in > the process of checking log files at startup, but is there something like > signature checks of files (especially closed, immutable ones) that can be > saved on disk and checked against at startup ? Wouldn't that help speed up > boot time, for

Re: [DISCUSS] KIP-263: Allow broker to skip sanity check of inactive segments on broker startup

This is great and definitely needed. I'm not exactly sure of what goes in the process of checking log files at startup, but is there something like signature checks of files (especially closed, immutable ones) that can be saved on disk and checked against at startup ? Wouldn't that hel

[jira] [Created] (KAFKA-6597) Issues with Zookeeper and Kafka startup in Windows environment

Alex Dunayevsky created KAFKA-6597: -- Summary: Issues with Zookeeper and Kafka startup in Windows environment Key: KAFKA-6597 URL: https://issues.apache.org/jira/browse/KAFKA-6597 Project: Kafka

[DISCUSS] KIP-263: Allow broker to skip sanity check of inactive segments on broker startup

Hi all, I have created KIP-263: Allow broker to skip sanity check of inactive segments on broker startup. See https://cwiki.apache.org/confluence/display/KAFKA/KIP-263%3A+Allow+broker+to+skip+sanity+check+of+inactive+segments+on+broker+startup . This KIP provides a way to significantly reduce

[jira] [Created] (KAFKA-6208) Reduce startup time for Kafka Connect workers

Randall Hauch created KAFKA-6208: Summary: Reduce startup time for Kafka Connect workers Key: KAFKA-6208 URL: https://issues.apache.org/jira/browse/KAFKA-6208 Project: Kafka Issue Type

[GitHub] kafka pull request #4182: KAFKA-6163 fail fast on startup

GitHub user xvrl opened a pull request: https://github.com/apache/kafka/pull/4182 KAFKA-6163 fail fast on startup skip loading remaining logs if we encounter an unrecoverable error on startup @hachikuji @ijuma You can merge this pull request into a Git repository by

[jira] [Created] (KAFKA-6163) Broker should fail fast on startup if an error occurs while loading logs

Xavier Léauté created KAFKA-6163: Summary: Broker should fail fast on startup if an error occurs while loading logs Key: KAFKA-6163 URL: https://issues.apache.org/jira/browse/KAFKA-6163 Project

[GitHub] kafka pull request #3782: KAFKA-5829; Speedup broker startup after unclean s...

Github user lindong28 closed the pull request at: https://github.com/apache/kafka/pull/3782 ---

[jira] [Resolved] (KAFKA-6008) Kafka Connect: Unsanitized workerID causes exception during startup

/apache/kafka/pull/4012] > Kafka Connect: Unsanitized workerID causes exception during startup > --- > > Key: KAFKA-6008 > URL: https://issues.apache.org/jira/browse/KAFKA-6008 >

[jira] [Created] (KAFKA-6008) Kafka Connect: Unsanitized workerID causes exception during startup

Jakub Scholz created KAFKA-6008: --- Summary: Kafka Connect: Unsanitized workerID causes exception during startup Key: KAFKA-6008 URL: https://issues.apache.org/jira/browse/KAFKA-6008 Project: Kafka

Speeding up Kafka Start cold startup time

Hi, We have our Kafka Brokers in AWS backed by st1 EBS drives. These are optimised for throughput On a kafka restart (warm restart), if some of the appropriate data is in the pagecache, everything goes well and the broker boots in just a few seconds. (1000 partitions) On a cold start (new

  1   2   3   4   5   6   7   >