Re: consumer_offsets partition skew and possibly ignored retention

2016-11-02 Thread James Brown
Jeff: This was with 0.9.0.1. It has not recurred since upgrading to 0.10.1.0. On Fri, Oct 28, 2016 at 9:28 PM, Jeff Widman wrote: > James, > What version did you experience the problem with? > > On Oct 28, 2016 6:26 PM, "James Brown" wrote: > > > I was having this problem with one of my __consu

Re: consumer_offsets partition skew and possibly ignored retention

2016-11-02 Thread Chi Hoang
I tried running reassignment on the topic, but that didn't help. I had to restart the broker for it to release the file handlers, then manually delete. On Fri, Oct 28, 2016 at 6:25 PM, James Brown wrote: > I was having this problem with one of my __consumer_offsets partitions; I > used reassign

Re: consumer_offsets partition skew and possibly ignored retention

2016-10-28 Thread Jeff Widman
James, What version did you experience the problem with? On Oct 28, 2016 6:26 PM, "James Brown" wrote: > I was having this problem with one of my __consumer_offsets partitions; I > used reassignment to move the large partition onto a different set of > machines (which forced the cleaner to run t

Re: consumer_offsets partition skew and possibly ignored retention

2016-10-28 Thread James Brown
I was having this problem with one of my __consumer_offsets partitions; I used reassignment to move the large partition onto a different set of machines (which forced the cleaner to run through them again) and after the new machines finished replicating, the partition was back down from 41GB to a n

consumer_offsets partition skew and possibly ignored retention

2016-10-28 Thread Chi Hoang
Hi, We have a 3-node cluster that is running 0.9.0.1, and recently saw that the "__consumer_offsets" topic on one of the nodes seems really skewed with disk usage that looks like: 73G ./__consumer_offsets-10 0 ./__consumer_offsets-7 0 ./__consumer_offsets-4 0 ./__consumer_off