I recently moved my 0.8.0 cluster to a set of entirely new brokers, and was surprised to find that the producers did not update their list of brokers to remove the brokers that were no longer in the cluster. That is, I had brokers 1,2,3,4,5, added brokers 6,7,8,910, waited a day, and stopped brokers 1,2,3,4,5. After stopping the original brokers, the producers continued trying to fetch metadata from them, even though they no longer appeared the topic metadata. It looks like it was basically the same issue described in http://mail-archives.apache.org/mod_mbox/kafka-users/201402.mbox/%3ccadpdzrksmjcpr0-l9txxoegthbjtptamh_fupq3ranwytkp...@mail.gmail.com%3E .
The documentation for "metadata.broker.list" says "This is for bootstrapping and the producer will only use it for getting metadata". I, and apparently others, have interpreted the "this is for bootstrapping" part to mean that it is only used for the initial metadata fetch, and not as the sole list of brokers to use to fetch metadata. Does it make sense to change the documentation to something like "The list of brokers used to fetch metadata. These brokers must always be available"? Am I understanding metadata.broker.list correctly? Is it necessary to load balance these brokers or otherwise make sure it does not refer directly to any brokers that could be removed? Thanks, Ryan
