RE: Experiences testing new producer performance across multiple threads/producer counts

Hi Guozhang/Jay/Becket, Thanks for the responses. Regarding my point on performance dropping when the number of partitions was increased, that surprised me too as on another cluster I had done just this to help with the issue of lots of ISR churn and it had been a straight win. I mentioned in

Re: Partitions reassignment failed due to Partition reassignment currently in progress for Map()

Hi Clark, Thank you for your reply! I do see that znode under /admin: root> ./opt/cloudera/parcels/CDH-5.1.3-1.cdh5.1.3.p0.12/lib/zookeeper/bin/zkCli.sh -server myhost.mydomain.com:2181 ls /admin Connecting to myhost.mydomain.com:2181 WATCHER:: WatchedEvent state:SyncConnected type:None path:n

Re: Partitions reassignment failed due to Partition reassignment currently in progress for Map()

The reassign_partitions znode is the important one. Please paste the contents of it. That node should only exist while there is a reassignment in progress. You can probably fix this up by forcing a new controller to come online by deleting /controller -Clark Sent from my iPhone > On May 17,

Re: Partitions reassignment failed due to Partition reassignment currently in progress for Map()

Sure thing :) Hopefully I did this right. Somewhat of a Zookeeper noob. root> ./opt/cloudera/parcels/CDH-5.1.3-1.cdh5.1.3.p0.12/lib/zookeeper/bin/zkCli.sh -server myhost.mydomain.com :2181 ls /admin/reassign_partitions Connecting to myhost.mydomain.com

Re: Partitions reassignment failed due to Partition reassignment currently in progress for Map()

Do a get /admin/reassign_partitions -Clark Sent from my iPhone > On May 17, 2015, at 10:20 AM, Chris Neal wrote: > > Sure thing :) > Hopefully I did this right. Somewhat of a Zookeeper noob. > > root> > ./opt/cloudera/parcels/CDH-5.1.3-1.cdh5.1.3.p0.12/lib/zookeeper/bin/zkCli.sh > -server my

Re: Partitions reassignment failed due to Partition reassignment currently in progress for Map()

Sorry :) Here you go: root> ./opt/cloudera/parcels/CDH-5.1.3-1.cdh5.1.3.p0.12/lib/zookeeper/bin/zkCli.sh -server myhost.mydomain.com :2181 get /admin/reassign_partitions Connecting to myhost.mydomain.com :2181 WATCHER:: WatchedE

Re: Partitions reassignment failed due to Partition reassignment currently in progress for Map()

Yeah it's empty. So I would try and force a new controller election to see if that cleans up the znode. Delete /controller If that doesn't work you will have to delete the reassign_partitions znode manually -Clark Sent from my iPhone > On May 17, 2015, at 11:16 AM, Chris Neal wrote: > > S

Re: Partitions reassignment failed due to Partition reassignment currently in progress for Map()

Thank you again. :) I ran the delete, but I'm not sure if it did anything: root> ./opt/cloudera/parcels/CDH-5.1.3-1.cdh5.1.3.p0.12/lib/zookeeper/bin/zkCli.sh -server myhost.mydomain.com:2181 delete /controller Connecting to myhost.mydomain.com:2181 WATCHER:: WatchedEvent state:SyncConnected ty

Re: Partitions reassignment failed due to Partition reassignment currently in progress for Map()

Ok, I might have lied. Timezone got me. I do see this in the Kafka logs, so maybe it did work? [2015-05-17 14:36:57,924] INFO 0 successfully elected as leader (kafka.server.ZookeeperLeaderElector) [2015-05-17 14:36:57,924] INFO 0 successfully elected as leader (kafka.server.ZookeeperLeaderElecto

Re: Partitions reassignment failed due to Partition reassignment currently in progress for Map()

Gah. Sorry for the spamming replies. The reassign_partitions znode is also still there, and still emptyFor what that's worth. :) Back to thinking that deleting the reassign_partitions znode is the right way to go. Chris On Sun, May 17, 2015 at 1:46 PM, Chris Neal wrote: > Ok, I might have

Re: Partitions reassignment failed due to Partition reassignment currently in progress for Map()

No problem. Delete the reassign_partitions znode. Then do the controller again. The controller znode gets instantly re-created but with a different broker as the controller. If you do a get /controller you can see the values change -Clark Sent from my iPhone > On May 17, 2015, at 11:48 AM, C

Re: Partitions reassignment failed due to Partition reassignment currently in progress for Map()

Done, and things look clean in Zookeeper now. I'm guessing that it is safe for me to try to increase the replication count on this topic again? :) Any idea what might have happened to put me in this state? I had successfully increased probably 10 or so topics, but then I started getting this mess

Re: Partitions reassignment failed due to Partition reassignment currently in progress for Map()

Yeah you can start using the reassignment tool again. It's likely some issue with the controller that put you in this state. Without gathering the logs it will be hard to tell. -Clark Sent from my iPhone > On May 17, 2015, at 3:42 PM, Chris Neal wrote: > > Done, and things look clean in Zook