+1 (binding) On Mon, Jun 20, 2016 at 8:27 PM, Dana Powers <dana.pow...@gmail.com> wrote:
> +1 -- thanks for the update > > On Mon, Jun 20, 2016 at 10:49 AM, Grant Henke <ghe...@cloudera.com> wrote: > > I have update the patch and wiki based on the feedback in the discussion > > thread. The only change is that instead of logging and disconnecting in > the > > case of invalid messages (duplicate topics or both arguments) we now > return > > and InvalidRequest error back to the client for that topic. > > > > I would like to restart the vote now including that change. If you have > > already voted, please revote in this thread. > > > > Thank you, > > Grant > > > > On Sun, Jun 19, 2016 at 8:57 PM, Ewen Cheslack-Postava < > e...@confluent.io> > > wrote: > > > >> Don't necessarily want to add noise here, but I'm -1 based on the > >> disconnect part. See discussion in other thread. (I'm +1 otherwise, and > >> happy to have my vote applied assuming we clean up that one issue.) > >> > >> -Ewen > >> > >> On Thu, Jun 16, 2016 at 6:05 PM, Harsha <ka...@harsha.io> wrote: > >> > >> > +1 (binding) > >> > Thanks, > >> > Harsha > >> > > >> > On Thu, Jun 16, 2016, at 04:15 PM, Guozhang Wang wrote: > >> > > +1. > >> > > > >> > > On Thu, Jun 16, 2016 at 3:47 PM, Ismael Juma <ism...@juma.me.uk> > >> wrote: > >> > > > >> > > > +1 (binding) > >> > > > > >> > > > On Thu, Jun 16, 2016 at 11:50 PM, Grant Henke < > ghe...@cloudera.com> > >> > wrote: > >> > > > > >> > > > > I would like to initiate the voting process for the "KIP-4 > Create > >> > Topics > >> > > > > Schema changes". This is not a vote for all of KIP-4, but > >> > specifically > >> > > > for > >> > > > > the create topics changes. I have included the exact changes > below > >> > for > >> > > > > clarity: > >> > > > > > > >> > > > > > Create Topics Request (KAFKA-2945 > >> > > > > > <https://issues.apache.org/jira/browse/KAFKA-2945>) > >> > > > > > > >> > > > > > CreateTopics Request (Version: 0) => [create_topic_requests] > >> > timeout > >> > > > > > create_topic_requests => topic num_partitions > >> replication_factor > >> > > > > [replica_assignment] [configs] > >> > > > > > topic => STRING > >> > > > > > num_partitions => INT32 > >> > > > > > replication_factor => INT16 > >> > > > > > replica_assignment => partition_id [replicas] > >> > > > > > partition_id => INT32 > >> > > > > > replicas => INT32 > >> > > > > > configs => config_key config_value > >> > > > > > config_key => STRING > >> > > > > > config_value => STRING > >> > > > > > timeout => INT32 > >> > > > > > > >> > > > > > CreateTopicsRequest is a batch request to initiate topic > creation > >> > with > >> > > > > > either predefined or automatic replica assignment and > optionally > >> > topic > >> > > > > > configuration. > >> > > > > > > >> > > > > > Request semantics: > >> > > > > > > >> > > > > > 1. Must be sent to the controller broker > >> > > > > > 2. If there are multiple instructions for the same topic in > >> one > >> > > > > > request an InvalidRequestException will be logged on the > >> broker > >> > and > >> > > > > the > >> > > > > > client will be disconnected. > >> > > > > > - This is because the list of topics is modeled server > side > >> > as a > >> > > > > > map with TopicName as the key > >> > > > > > 3. The principal must be authorized to the "Create" > Operation > >> > on the > >> > > > > > "Cluster" resource to create topics. > >> > > > > > - Unauthorized requests will receive a > >> > > > > ClusterAuthorizationException > >> > > > > > 4. > >> > > > > > > >> > > > > > Only one from ReplicaAssignment or (num_partitions + > >> > > > > replication_factor > >> > > > > > ), can be defined in one instruction. > >> > > > > > - If both parameters are specified an > InvalidRequestException > >> > will > >> > > > be > >> > > > > > logged on the broker and the client will be > disconnected. > >> > > > > > - In the case ReplicaAssignment is defined number of > >> > partitions > >> > > > and > >> > > > > > replicas will be calculated from the supplied > >> > replica_assignment. > >> > > > > > - In the case of defined (num_partitions + > >> > replication_factor) > >> > > > > > replica assignment will be automatically generated by > the > >> > server. > >> > > > > > - One or the other must be defined. The existing broker > >> side > >> > auto > >> > > > > > create defaults will not be used > >> > > > > > (default.replication.factor, num.partitions). The client > >> > > > > implementation can > >> > > > > > have defaults for these options when generating the > >> messages. > >> > > > > > - The first replica in [replicas] is assumed to be the > >> > preferred > >> > > > > > leader. This matches current behavior elsewhere. > >> > > > > > 5. Setting a timeout > 0 will allow the request to block > until > >> > the > >> > > > > > topic metadata is "complete" on the controller node. > >> > > > > > - Complete means the local topic metadata cache been > >> > completely > >> > > > > > populated and all partitions have leaders > >> > > > > > - The topic metadata is updated when the controller > >> sends > >> > out > >> > > > > > update metadata requests to the brokers > >> > > > > > - If a timeout error occurs, the topic could still be > >> created > >> > > > > > successfully at a later time. Its up to the client to > query > >> > for > >> > > > > the state > >> > > > > > at that point. > >> > > > > > 6. Setting a timeout <= 0 will validate arguments and > trigger > >> > the > >> > > > > > create topics and return immediately. > >> > > > > > - This is essentially the fully asynchronous mode we > have > >> in > >> > the > >> > > > > > Zookeeper tools today. > >> > > > > > - The error code in the response will either contain an > >> > argument > >> > > > > > validation exception or a timeout exception. If you > >> receive a > >> > > > > timeout > >> > > > > > exception, because you asked for 0 timeout, you can > assume > >> > the > >> > > > > message was > >> > > > > > valid and the topic creation was triggered. > >> > > > > > 7. The request is not transactional. > >> > > > > > 1. If an error occurs on one topic, the others could > still > >> be > >> > > > > > created. > >> > > > > > 2. Errors are reported independently. > >> > > > > > > >> > > > > > QA: > >> > > > > > > >> > > > > > - Why is CreateTopicsRequest a batch request? > >> > > > > > - Scenarios where tools or admins want to create many > >> topics > >> > > > should > >> > > > > > be able to with fewer requests > >> > > > > > - Example: MirrorMaker may want to create the topics > >> > downstream > >> > > > > > - What happens if some topics error immediately? Will it > >> > > > > > return immediately? > >> > > > > > - The request will block until all topics have either > been > >> > > > created, > >> > > > > > errors, or the timeout has been hit > >> > > > > > - There is no "short circuiting" where 1 error stops the > >> > other > >> > > > > > topics from being created > >> > > > > > - Why implement "partial blocking" instead of fully async > or > >> > fully > >> > > > > > consistent? > >> > > > > > - See Cluster Consistent Blocking > >> > > > > > < > >> > > > > > >> > > > > >> > > >> > https://cwiki.apache.org/confluence/display/KAFKA/KIP-4+-+Command+line+and+centralized+administrative+operations#KIP-4-Commandlineandcentralizedadministrativeoperations-cluster-consistent-blocking > >> > > > > > > >> > > > > > below > >> > > > > > - Why require the request to go to the controller? > >> > > > > > - The controller is responsible for the cluster metadata > >> and > >> > its > >> > > > > > propagation > >> > > > > > - See Request Forwarding > >> > > > > > < > >> > > > > > >> > > > > >> > > >> > https://cwiki.apache.org/confluence/display/KAFKA/KIP-4+-+Command+line+and+centralized+administrative+operations#KIP-4-Commandlineandcentralizedadministrativeoperations-request > >> > > > > > > >> > > > > > below > >> > > > > > > >> > > > > > Create Topics Response > >> > > > > > > >> > > > > > > >> > > > > > > >> > > > > > CreateTopics Response (Version: 0) => [topic_error_codes] > >> > > > > > topic_error_codes => topic error_code > >> > > > > > topic => STRING > >> > > > > > error_code => INT16 > >> > > > > > > >> > > > > > CreateTopicsResponse contains a map between topic and topic > >> > creation > >> > > > > > result error code (see New Protocol Errors > >> > > > > > < > >> > > > > > >> > > > > >> > > >> > https://cwiki.apache.org/confluence/display/KAFKA/KIP-4+-+Command+line+and+centralized+administrative+operations#KIP-4-Commandlineandcentralizedadministrativeoperations-NewProtocolErrors > >> > > > > > > >> > > > > > ). > >> > > > > > > >> > > > > > >> > > > > The KIP is available here for reference (linked to the Create > >> Topics > >> > > > schema > >> > > > > section): > >> > > > > * > >> > > > > > >> > > > > >> > > >> > https://cwiki.apache.org/confluence/display/KAFKA/KIP-4+-+Command+line+and+centralized+administrative+operations#KIP-4-Commandlineandcentralizedadministrativeoperations-CreateTopicsRequest(KAFKA-2945) > >> > > > > < > >> > > > > > >> > > > > >> > > >> > https://cwiki.apache.org/confluence/display/KAFKA/KIP-4+-+Command+line+and+centralized+administrative+operations#KIP-4-Commandlineandcentralizedadministrativeoperations-CreateTopicsRequest(KAFKA-2945) > >> > > > > >* > >> > > > > > >> > > > > A pull request is available implementing the proposed changes > here: > >> > > > > https://github.com/apache/kafka/pull/1489 > >> > > > > > >> > > > > Here is a link to the past discussion on the mailing list: > >> > > > > * > >> > > > > > >> > > > > >> > > >> > http://search-hadoop.com/m/uyzND1rfG6v1oixmZ&subj=+DISCUSS+KIP+4+Create+Topic+Schema > >> > > > > < > >> > > > > > >> > > > > >> > > >> > http://search-hadoop.com/m/uyzND1rfG6v1oixmZ&subj=+DISCUSS+KIP+4+Create+Topic+Schema > >> > > > > >* > >> > > > > > >> > > > > Thank you, > >> > > > > Grant > >> > > > > -- > >> > > > > Grant Henke > >> > > > > Software Engineer | Cloudera > >> > > > > gr...@cloudera.com | twitter.com/gchenke | > >> > linkedin.com/in/granthenke > >> > > > > > >> > > > > >> > > > >> > > > >> > > > >> > > -- > >> > > -- Guozhang > >> > > >> > >> > >> > >> -- > >> Thanks, > >> Ewen > >> > > > > > > > > -- > > Grant Henke > > Software Engineer | Cloudera > > gr...@cloudera.com | twitter.com/gchenke | linkedin.com/in/granthenke >
