Re: Problem with node after restart no partitions?

2015-04-02 Thread Gwen Shapira
wow, thats scary for sure. Just to be clear - all you did is restart *one* broker in the cluster? everything else was ok before the restart? and that was controlled shutdown? Gwen On Wed, Apr 1, 2015 at 11:54 AM, Thunder Stumpges wrote: > Well it appears we lost all the data on the one node ag

Re: Per-topic retention.bytes uses kilobytes not bytes?

2015-04-02 Thread Harsha
Hi Willy,           retention.bytes used to check if log  in total exceeded and take the diff between log.size and retention bytes and only delete those log segment files that exceeds this diff. So its rounded off to the segment.size and it does check bytes not in kbs. Since your log.segment.siz

Per-topic retention.bytes uses kilobytes not bytes?

2015-04-02 Thread Willy Hoang
Hello, I’ve been having trouble using the retention.bytes per-topic configuration (using Kafka version 0.8.2.1). I had the same issue that users described in these two threads where logs were growing to sizes larger than retention.bytes. I couldn’t find an explanation to explain the issue in ei

Re: reusing node ids

2015-04-02 Thread Gwen Shapira
On Thu, Apr 2, 2015 at 12:19 PM, Wes Chow wrote: > > How reusable are node ids? Specifically: > > 1. Node goes down and loses all data in its local Kafka logs. I bring it > back up, same hostname, same IP, same node id, but with empty logs. Does it > properly sync with the replicas and continue o

reusing node ids

2015-04-02 Thread Wes Chow
How reusable are node ids? Specifically: 1. Node goes down and loses all data in its local Kafka logs. I bring it back up, same hostname, same IP, same node id, but with empty logs. Does it properly sync with the replicas and continue operation? 2. Node goes down and I replace it with a new

Kafka meetup at ApacheCon in Apr

2015-04-02 Thread Jun Rao
Hi, A few of us will be at ApacheCon in April. We are organizing a Kafka meetup Tuesday evening ( http://events.linuxfoundation.org/events/apachecon-north-america/extend-the-experience/meetups). Every attendee of ApacheCon is welcome to join the meetup. We also plan to have a a few short presentat

Re: Kafka Consumer Offset

2015-04-02 Thread Mayuresh Gharat
You can use it. Are you facing any problems using it? On Thu, Apr 2, 2015 at 6:53 AM, Amreen Khan wrote: > Is KafkaConsumerOffsetChecker still in development for 0.8.2? > > Amreen Khan -- -Regards, Mayuresh R. Gharat (862) 250-7125

Kafka Consumer Offset

2015-04-02 Thread Amreen Khan
Is KafkaConsumerOffsetChecker still in development for 0.8.2? Amreen Khan