Viktor Somogyi-Vass created KAFKA-9166:
------------------------------------------

             Summary: Implement MetadataFetch API
                 Key: KAFKA-9166
                 URL: https://issues.apache.org/jira/browse/KAFKA-9166
             Project: Kafka
          Issue Type: Sub-task
          Components: controller
            Reporter: Viktor Somogyi-Vass


Instead of the controller pushing out updates to the other brokers, those 
brokers will fetch updates from the active controller via the new MetadataFetch 
API.

A MetadataFetch is similar to a fetch request.  Just like with a fetch request, 
the broker will track the offset of the last updates it fetched, and only 
request newer updates from the active controller. 

The broker will persist the metadata it fetched to disk.  This will allow the 
broker to start up very quickly, even if there are hundreds of thousands or 
even millions of partitions.  (Note that since this persistence is an 
optimization, we can leave it out of the first version, if it makes development 
easier.)

Most of the time, the broker should only need to fetch the deltas, not the full 
state.  However, if the broker is too far behind the active controller, or if 
the broker has no cached metadata at all, the controller will send a full 
metadata image rather than a series of deltas.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Reply via email to