I have update the patch and wiki based on the feedback in the discussion thread. The only change is that instead of logging and disconnecting in the case of invalid messages (duplicate topics or both arguments) we now return and InvalidRequest error back to the client for that topic.
I would like to restart the vote now including that change. If you have already voted, please revote in this thread. Thank you, Grant On Sun, Jun 19, 2016 at 8:57 PM, Ewen Cheslack-Postava <e...@confluent.io> wrote: > Don't necessarily want to add noise here, but I'm -1 based on the > disconnect part. See discussion in other thread. (I'm +1 otherwise, and > happy to have my vote applied assuming we clean up that one issue.) > > -Ewen > > On Thu, Jun 16, 2016 at 6:05 PM, Harsha <ka...@harsha.io> wrote: > > > +1 (binding) > > Thanks, > > Harsha > > > > On Thu, Jun 16, 2016, at 04:15 PM, Guozhang Wang wrote: > > > +1. > > > > > > On Thu, Jun 16, 2016 at 3:47 PM, Ismael Juma <ism...@juma.me.uk> > wrote: > > > > > > > +1 (binding) > > > > > > > > On Thu, Jun 16, 2016 at 11:50 PM, Grant Henke <ghe...@cloudera.com> > > wrote: > > > > > > > > > I would like to initiate the voting process for the "KIP-4 Create > > Topics > > > > > Schema changes". This is not a vote for all of KIP-4, but > > specifically > > > > for > > > > > the create topics changes. I have included the exact changes below > > for > > > > > clarity: > > > > > > > > > > > > Create Topics Request (KAFKA-2945 > > > > > > <https://issues.apache.org/jira/browse/KAFKA-2945>) > > > > > > > > > > > > CreateTopics Request (Version: 0) => [create_topic_requests] > > timeout > > > > > > create_topic_requests => topic num_partitions > replication_factor > > > > > [replica_assignment] [configs] > > > > > > topic => STRING > > > > > > num_partitions => INT32 > > > > > > replication_factor => INT16 > > > > > > replica_assignment => partition_id [replicas] > > > > > > partition_id => INT32 > > > > > > replicas => INT32 > > > > > > configs => config_key config_value > > > > > > config_key => STRING > > > > > > config_value => STRING > > > > > > timeout => INT32 > > > > > > > > > > > > CreateTopicsRequest is a batch request to initiate topic creation > > with > > > > > > either predefined or automatic replica assignment and optionally > > topic > > > > > > configuration. > > > > > > > > > > > > Request semantics: > > > > > > > > > > > > 1. Must be sent to the controller broker > > > > > > 2. If there are multiple instructions for the same topic in > one > > > > > > request an InvalidRequestException will be logged on the > broker > > and > > > > > the > > > > > > client will be disconnected. > > > > > > - This is because the list of topics is modeled server side > > as a > > > > > > map with TopicName as the key > > > > > > 3. The principal must be authorized to the "Create" Operation > > on the > > > > > > "Cluster" resource to create topics. > > > > > > - Unauthorized requests will receive a > > > > > ClusterAuthorizationException > > > > > > 4. > > > > > > > > > > > > Only one from ReplicaAssignment or (num_partitions + > > > > > replication_factor > > > > > > ), can be defined in one instruction. > > > > > > - If both parameters are specified an InvalidRequestException > > will > > > > be > > > > > > logged on the broker and the client will be disconnected. > > > > > > - In the case ReplicaAssignment is defined number of > > partitions > > > > and > > > > > > replicas will be calculated from the supplied > > replica_assignment. > > > > > > - In the case of defined (num_partitions + > > replication_factor) > > > > > > replica assignment will be automatically generated by the > > server. > > > > > > - One or the other must be defined. The existing broker > side > > auto > > > > > > create defaults will not be used > > > > > > (default.replication.factor, num.partitions). The client > > > > > implementation can > > > > > > have defaults for these options when generating the > messages. > > > > > > - The first replica in [replicas] is assumed to be the > > preferred > > > > > > leader. This matches current behavior elsewhere. > > > > > > 5. Setting a timeout > 0 will allow the request to block until > > the > > > > > > topic metadata is "complete" on the controller node. > > > > > > - Complete means the local topic metadata cache been > > completely > > > > > > populated and all partitions have leaders > > > > > > - The topic metadata is updated when the controller > sends > > out > > > > > > update metadata requests to the brokers > > > > > > - If a timeout error occurs, the topic could still be > created > > > > > > successfully at a later time. Its up to the client to query > > for > > > > > the state > > > > > > at that point. > > > > > > 6. Setting a timeout <= 0 will validate arguments and trigger > > the > > > > > > create topics and return immediately. > > > > > > - This is essentially the fully asynchronous mode we have > in > > the > > > > > > Zookeeper tools today. > > > > > > - The error code in the response will either contain an > > argument > > > > > > validation exception or a timeout exception. If you > receive a > > > > > timeout > > > > > > exception, because you asked for 0 timeout, you can assume > > the > > > > > message was > > > > > > valid and the topic creation was triggered. > > > > > > 7. The request is not transactional. > > > > > > 1. If an error occurs on one topic, the others could still > be > > > > > > created. > > > > > > 2. Errors are reported independently. > > > > > > > > > > > > QA: > > > > > > > > > > > > - Why is CreateTopicsRequest a batch request? > > > > > > - Scenarios where tools or admins want to create many > topics > > > > should > > > > > > be able to with fewer requests > > > > > > - Example: MirrorMaker may want to create the topics > > downstream > > > > > > - What happens if some topics error immediately? Will it > > > > > > return immediately? > > > > > > - The request will block until all topics have either been > > > > created, > > > > > > errors, or the timeout has been hit > > > > > > - There is no "short circuiting" where 1 error stops the > > other > > > > > > topics from being created > > > > > > - Why implement "partial blocking" instead of fully async or > > fully > > > > > > consistent? > > > > > > - See Cluster Consistent Blocking > > > > > > < > > > > > > > > > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-4+-+Command+line+and+centralized+administrative+operations#KIP-4-Commandlineandcentralizedadministrativeoperations-cluster-consistent-blocking > > > > > > > > > > > > below > > > > > > - Why require the request to go to the controller? > > > > > > - The controller is responsible for the cluster metadata > and > > its > > > > > > propagation > > > > > > - See Request Forwarding > > > > > > < > > > > > > > > > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-4+-+Command+line+and+centralized+administrative+operations#KIP-4-Commandlineandcentralizedadministrativeoperations-request > > > > > > > > > > > > below > > > > > > > > > > > > Create Topics Response > > > > > > > > > > > > > > > > > > > > > > > > CreateTopics Response (Version: 0) => [topic_error_codes] > > > > > > topic_error_codes => topic error_code > > > > > > topic => STRING > > > > > > error_code => INT16 > > > > > > > > > > > > CreateTopicsResponse contains a map between topic and topic > > creation > > > > > > result error code (see New Protocol Errors > > > > > > < > > > > > > > > > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-4+-+Command+line+and+centralized+administrative+operations#KIP-4-Commandlineandcentralizedadministrativeoperations-NewProtocolErrors > > > > > > > > > > > > ). > > > > > > > > > > > > > > > > The KIP is available here for reference (linked to the Create > Topics > > > > schema > > > > > section): > > > > > * > > > > > > > > > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-4+-+Command+line+and+centralized+administrative+operations#KIP-4-Commandlineandcentralizedadministrativeoperations-CreateTopicsRequest(KAFKA-2945) > > > > > < > > > > > > > > > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-4+-+Command+line+and+centralized+administrative+operations#KIP-4-Commandlineandcentralizedadministrativeoperations-CreateTopicsRequest(KAFKA-2945) > > > > > >* > > > > > > > > > > A pull request is available implementing the proposed changes here: > > > > > https://github.com/apache/kafka/pull/1489 > > > > > > > > > > Here is a link to the past discussion on the mailing list: > > > > > * > > > > > > > > > > > > http://search-hadoop.com/m/uyzND1rfG6v1oixmZ&subj=+DISCUSS+KIP+4+Create+Topic+Schema > > > > > < > > > > > > > > > > > > http://search-hadoop.com/m/uyzND1rfG6v1oixmZ&subj=+DISCUSS+KIP+4+Create+Topic+Schema > > > > > >* > > > > > > > > > > Thank you, > > > > > Grant > > > > > -- > > > > > Grant Henke > > > > > Software Engineer | Cloudera > > > > > gr...@cloudera.com | twitter.com/gchenke | > > linkedin.com/in/granthenke > > > > > > > > > > > > > > > > > > > > > -- > > > -- Guozhang > > > > > > -- > Thanks, > Ewen > -- Grant Henke Software Engineer | Cloudera gr...@cloudera.com | twitter.com/gchenke | linkedin.com/in/granthenke
