Re: Getting replicas back in sync

We're seeing the same behaviour today on our cluster. It is not like a single broker went out of the cluster, rather a few partitions seem lazy on every broker. On Fri, Sep 12, 2014 at 9:31 PM, Cory Watson wrote: > I noticed this morning that a few of our partitions do not have their full > comp

Re: Migrating data from old brokers to new borkers question

If you are also planning on a version upgrade as part of this, it might be safer to create the new cluster separately and use the mirror maker to copy data over. On Aug 26, 2014 6:29 AM, "Marcin Michalski" wrote: > Hi, I would like to migrate my Kafka setup from old servers to new servers. > Let

Re: Delete message after consuming it

Kafka is a log and not a queue. The client is remembering a position in the log rather than working with individual messages. On Fri, Aug 1, 2014 at 4:02 PM, anand jain wrote: > I want to delete the message from a Kafka broker after consuming it(Java > consumer). How can I do that? > -- “ T

Re: Serious Bug? Segment getting deleted as soon as it is rolled over

bug is that if a topic is on all brokers, then the znode added to /config/changes will never be removed. That also means that config changes to a topic that is on all brokers are safe. On Fri, Jul 25, 2014 at 11:07 AM, Kashyap Paidimarri wrote: > Attached a transcript that explains what I&#

Re: Serious Bug? Segment getting deleted as soon as it is rolled over

Attached a transcript that explains what I'm seeing On Fri, Jul 25, 2014 at 10:52 AM, Kashyap Paidimarri wrote: > No, we haven't configured that. We have a few hundred topics but this > seems to be the only one affected (I did a quick check, not thorough). > > The relev

Re: Serious Bug? Segment getting deleted as soon as it is rolled over

al.ms=1000 log.retention.hours=168 log.segment.bytes=536870912 log.cleanup.interval.mins=1 log.retention.hours=336 On Fri, Jul 25, 2014 at 10:11 AM, Jun Rao wrote: > Have you configured log.retention.bytes? > > Thanks, > > Jun > > > On Thu, Jul 24, 2014 at 10:04 AM

Re: Serious Bug? Segment getting deleted as soon as it is rolled over

Relevant parameters from server.properties log.dir=/var/lib/fk-3p-kafka/logs log.flush.interval.messages=1 log.flush.interval.ms=1000 log.retention.hours=168 log.segment.bytes=536870912 log.cleanup.interval.mins=1 log.retention.hours=336 On Thu, Jul 24, 2014 at 10:34 PM, Kashyap

Serious Bug? Segment getting deleted as soon as it is rolled over

We just noticed that one of our topics has been horribly misbehaving. *retention.ms * for the topic is set to 120960 ms However, segments are getting schedule for deletetion as soon as a new one is rolled over. And naturally consumers are running into a kafka.common.Offse