+1 (binding) On Fri, Jun 24, 2016 at 7:16 PM, Onur Karaman <okara...@linkedin.com.invalid > wrote:
> +1 > > On Fri, Jun 24, 2016 at 9:52 AM, Jay Kreps <j...@confluent.io> wrote: > > > +1 > > > > On Thu, Jun 23, 2016 at 8:32 PM, Grant Henke <ghe...@cloudera.com> > wrote: > > > > > I would like to initiate the voting process for the "KIP-4 Delete > Topics > > > Schema changes". This is not a vote for all of KIP-4, but specifically > > for > > > the delete topics changes. I have included the exact changes below for > > > clarity: > > > > > > > > Delete Topics Request (KAFKA-2946 > > > > <https://issues.apache.org/jira/browse/KAFKA-2946>) > > > > > > > > DeleteTopics Request (Version: 0) => [topics] timeout > > > > topics => STRING > > > > timeout => INT32 > > > > > > > > DeleteTopicsRequest is a batch request to initiate topic deletion. > > > > > > > > Request semantics: > > > > > > > > 1. Must be sent to the controller broker > > > > 2. If there are multiple instructions for the same topic in one > > > > request the extra request will be ingnored > > > > - This is because the list of topics is modeled server side as > a > > > set > > > > - Multiple deletes results in the same end goal, so handling > this > > > > error for the user should be okay > > > > 3. When requesting to delete a topic that does not exist, a an > > > > InvalidTopic error will be returned for that topic. > > > > 4. When requesting to delete a topic that is already marked for > > > > deletion, the request will wait up to the timeout until the delete > > is > > > > "complete" and return as usual. > > > > - This is to avoid errors due to concurrent delete requests. > The > > > > end result is the same, the topic is deleted. > > > > 5. The principal must be authorized to the "Delete" Operation on > the > > > > "Topic" resource to delete the topic. > > > > - Unauthorized requests will receive a > > TopicAuthorizationException > > > > if they are authorized to the "Describe" Operation on the > "Topic" > > > resource > > > > - Otherwise they will receive an InvalidTopicException as if > the > > > > topic does not exist. > > > > 6. Setting a timeout > 0 will allow the request to block until > > the > > > > delete is "complete" on the controller node. > > > > - Complete means the local topic metadata cache no longer > > contains > > > > the topic > > > > - The topic metadata is updated when the controller sends > out > > > > update metadata requests to the brokers > > > > - If a timeout error occurs, the topic could still be deleted > > > > successfully at a later time. Its up to the client to query for > > > the state > > > > at that point. > > > > 7. Setting a timeout <= 0 will validate arguments and trigger the > > > > delete topics and return immediately. > > > > - This is essentially the fully asynchronous mode we have in > the > > > > Zookeeper tools today. > > > > - The error code in the response will either contain an > argument > > > > validation exception or a timeout exception. If you receive a > > > timeout > > > > exception, because you asked for 0 timeout, you can assume the > > > message was > > > > valid and the topic deletion was triggered. > > > > 8. The request is not transactional. > > > > 1. If an error occurs on one topic, the others could still be > > > > deleted. > > > > 2. Errors are reported independently. > > > > > > > > QA: > > > > > > > > - Why is DeleteTopicsRequest a batch request? > > > > - Scenarios where tools or admins want to delete many topics > > should > > > > be able to with fewer requests > > > > - Example: Removing all cluster topics > > > > - What happens if some topics error immediately? Will it > > > > return immediately? > > > > - The request will block until all topics have either been > > deleted, > > > > errors, or the timeout has been hit > > > > - There is no "short circuiting" where 1 error stops the other > > > > topics from being deleted > > > > - Why have a timeout at all? Deletes could take a while? > > > > - True some deletes may take a while or never finish, however > > some > > > > admin tools may want extended blocking regardless. > > > > - If you don't want any blocking setting a timeout of 0 works. > > > > - Future changes may make deletes much faster. See the Follow > Up > > > > Changes > > > > < > > > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-4+-+Command+line+and+centralized+administrative+operations#KIP-4-Commandlineandcentralizedadministrativeoperations-follow-up-changes > > > > > > section > > > > above. > > > > - Why implement "partial blocking" instead of fully async or fully > > > > consistent? > > > > - See Cluster Consistent Blocking > > > > < > > > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-4+-+Command+line+and+centralized+administrative+operations#KIP-4-Commandlineandcentralizedadministrativeoperations-cluster-consistent-blocking > > > > > > > > below > > > > - Why require the request to go to the controller? > > > > - The controller is responsible for the cluster metadata and > its > > > > propagation > > > > - See Request Forwarding > > > > < > > > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-4+-+Command+line+and+centralized+administrative+operations#KIP-4-Commandlineandcentralizedadministrativeoperations-request > > > > > > > > below > > > > > > > > Delete Topics Response > > > > > > > > DeleteTopics Response (Version: 0) => [topic_error_codes] > > > > topic_error_codes => topic error_code > > > > topic => STRING > > > > error_code => INT16 > > > > > > > > DeleteTopicsResponse contains a map between topic and topic creation > > > > result error code (see New Protocol Errors > > > > < > > > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-4+-+Command+line+and+centralized+administrative+operations#KIP-4-Commandlineandcentralizedadministrativeoperations-NewProtocolErrors > > > > > > > > ). > > > > > > > > Response semantics: > > > > > > > > 1. When a request hits the timeout, the topics that are not > > "complete" > > > > will have the TimeoutException error code. > > > > - The topics that did complete successfully with have no error. > > > > > > > > > > > The KIP is available here for reference (linked to the Create Topics > > schema > > > section): > > > * > > > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-4+-+Command+line+and+centralized+administrative+operations#KIP-4-Commandlineandcentralizedadministrativeoperations-DeleteTopicsRequest(KAFKA-2946) > > > < > > > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-4+-+Command+line+and+centralized+administrative+operations#KIP-4-Commandlineandcentralizedadministrativeoperations-DeleteTopicsRequest(KAFKA-2946) > > > >* > > > > > > A sample patch is on github: > > > https://github.com/granthenke/kafka/tree/delete-wire-new > > > Note: This branch and patch is based on the CreateTopic > request/response > > > PR. I will open a PR once that patch is complete. > > > > > > Here is a link to the past discussion on the mailing list: > > > > > > * > > > > > > http://search-hadoop.com/m/uyzND1fokOBn6uNd2&subj=+DISCUSS+KIP+4+Delete+Topic+Schema > > > < > > > > > > http://search-hadoop.com/m/uyzND1fokOBn6uNd2&subj=+DISCUSS+KIP+4+Delete+Topic+Schema > > > >* > > > > > > Thank you, > > > Grant > > > -- > > > Grant Henke > > > Software Engineer | Cloudera > > > gr...@cloudera.com | twitter.com/gchenke | linkedin.com/in/granthenke > > > > > >
