Re: Getting replicas back in sync

2014-09-12 Thread Kashyap Paidimarri
We're seeing the same behaviour today on our cluster. It is not like a single broker went out of the cluster, rather a few partitions seem lazy on every broker. On Fri, Sep 12, 2014 at 9:31 PM, Cory Watson gp...@keen.io wrote: I noticed this morning that a few of our partitions do not have

Re: Migrating data from old brokers to new borkers question

2014-08-25 Thread Kashyap Paidimarri
If you are also planning on a version upgrade as part of this, it might be safer to create the new cluster separately and use the mirror maker to copy data over. On Aug 26, 2014 6:29 AM, Marcin Michalski mmichal...@tagged.com wrote: Hi, I would like to migrate my Kafka setup from old servers to

Re: Delete message after consuming it

2014-08-01 Thread Kashyap Paidimarri
Kafka is a log and not a queue. The client is remembering a position in the log rather than working with individual messages. On Fri, Aug 1, 2014 at 4:02 PM, anand jain anandjain1...@gmail.com wrote: I want to delete the message from a Kafka broker after consuming it(Java consumer). How can I

Re: Serious Bug? Segment getting deleted as soon as it is rolled over

2014-07-25 Thread Kashyap Paidimarri
is that if a topic is on all brokers, then the znode added to /config/changes will never be removed. That also means that config changes to a topic that is on all brokers are safe. On Fri, Jul 25, 2014 at 11:07 AM, Kashyap Paidimarri kashy...@gmail.com wrote: Attached a transcript that explains what I'm

Re: Serious Bug? Segment getting deleted as soon as it is rolled over

2014-07-24 Thread Kashyap Paidimarri
Relevant parameters from server.properties log.dir=/var/lib/fk-3p-kafka/logs log.flush.interval.messages=1 log.flush.interval.ms=1000 log.retention.hours=168 log.segment.bytes=536870912 log.cleanup.interval.mins=1 log.retention.hours=336 On Thu, Jul 24, 2014 at 10:34 PM, Kashyap

Re: Serious Bug? Segment getting deleted as soon as it is rolled over

2014-07-24 Thread Kashyap Paidimarri
=1000 log.retention.hours=168 log.segment.bytes=536870912 log.cleanup.interval.mins=1 log.retention.hours=336 On Fri, Jul 25, 2014 at 10:11 AM, Jun Rao jun...@gmail.com wrote: Have you configured log.retention.bytes? Thanks, Jun On Thu, Jul 24, 2014 at 10:04 AM, Kashyap Paidimarri kashy