Also note that there were a number of bugs fixed in the log cleaner thread between 0.8 and the latest release. I wouldn't be comfortable relying on kafka committed offsets on a version under 0.10 for a new production system, and would carefully consider an upgrade all the way to the latest release.
On Thu, Jul 14, 2016 at 1:37 PM, Todd Palino <tpal...@gmail.com> wrote: > It's safe to move the partitions of the offsets topic around. You'll move > the consumer coordinators as you do, however, so the one thing you want to > make sure of, especially running an older version, is that log compaction > is working on your brokers and those partitions have been compacted. The > coordinator needs to bootstrap the topic, and if log compaction is broken > that can take a very long time. During that time, it will return errors to > consumers for offset operations, and that can cause offset resets. > > -Todd > > On Thursday, July 14, 2016, Anderson Goulart <anderson.goul...@boxever.com > > > wrote: > > > Hi, > > > > I am running kafka 0.8.2.1 under aws instances with multiple availability > > zones. As we want a rack aware partition replication, we have our own > > partition layout distribution, to make sure all partitions are well > > balanced between nodes, leaders and availability zones. > > > > The problem arises with __consumer_offsets internal topic. In our current > > environment it has 50 partitions and are all under the same AZ, with an > > unbalanced leader (all wrong!) > > > > The question is: should I manually change its partition layout > > distribution as I do for the other topics? Is it safe to reassign the new > > layout for this internal topic, using kafka-reassign-partitions.sh? > > > > > > Thanks, Anderson > > > > > -- > *Todd Palino* > Staff Site Reliability Engineer > Data Infrastructure Streaming > > > > linkedin.com/in/toddpalino >
