+1 (binding)
On Mon, Jun 20, 2016 at 12:13 PM, Tom Crayford <tcrayf...@heroku.com> wrote: > +1 (non-binding) > > On Mon, Jun 20, 2016 at 8:07 PM, Harsha <ka...@harsha.io> wrote: > >> +1 (binding) >> -Harsha >> >> On Mon, Jun 20, 2016, at 11:33 AM, Ismael Juma wrote: >> > +1 (binding) >> > >> > On Mon, Jun 20, 2016 at 8:27 PM, Dana Powers <dana.pow...@gmail.com> >> > wrote: >> > >> > > +1 -- thanks for the update >> > > >> > > On Mon, Jun 20, 2016 at 10:49 AM, Grant Henke <ghe...@cloudera.com> >> wrote: >> > > > I have update the patch and wiki based on the feedback in the >> discussion >> > > > thread. The only change is that instead of logging and disconnecting >> in >> > > the >> > > > case of invalid messages (duplicate topics or both arguments) we now >> > > return >> > > > and InvalidRequest error back to the client for that topic. >> > > > >> > > > I would like to restart the vote now including that change. If you >> have >> > > > already voted, please revote in this thread. >> > > > >> > > > Thank you, >> > > > Grant >> > > > >> > > > On Sun, Jun 19, 2016 at 8:57 PM, Ewen Cheslack-Postava < >> > > e...@confluent.io> >> > > > wrote: >> > > > >> > > >> Don't necessarily want to add noise here, but I'm -1 based on the >> > > >> disconnect part. See discussion in other thread. (I'm +1 otherwise, >> and >> > > >> happy to have my vote applied assuming we clean up that one issue.) >> > > >> >> > > >> -Ewen >> > > >> >> > > >> On Thu, Jun 16, 2016 at 6:05 PM, Harsha <ka...@harsha.io> wrote: >> > > >> >> > > >> > +1 (binding) >> > > >> > Thanks, >> > > >> > Harsha >> > > >> > >> > > >> > On Thu, Jun 16, 2016, at 04:15 PM, Guozhang Wang wrote: >> > > >> > > +1. >> > > >> > > >> > > >> > > On Thu, Jun 16, 2016 at 3:47 PM, Ismael Juma <ism...@juma.me.uk >> > >> > > >> wrote: >> > > >> > > >> > > >> > > > +1 (binding) >> > > >> > > > >> > > >> > > > On Thu, Jun 16, 2016 at 11:50 PM, Grant Henke < >> > > ghe...@cloudera.com> >> > > >> > wrote: >> > > >> > > > >> > > >> > > > > I would like to initiate the voting process for the "KIP-4 >> > > Create >> > > >> > Topics >> > > >> > > > > Schema changes". This is not a vote for all of KIP-4, but >> > > >> > specifically >> > > >> > > > for >> > > >> > > > > the create topics changes. I have included the exact changes >> > > below >> > > >> > for >> > > >> > > > > clarity: >> > > >> > > > > > >> > > >> > > > > > Create Topics Request (KAFKA-2945 >> > > >> > > > > > <https://issues.apache.org/jira/browse/KAFKA-2945>) >> > > >> > > > > > >> > > >> > > > > > CreateTopics Request (Version: 0) => >> [create_topic_requests] >> > > >> > timeout >> > > >> > > > > > create_topic_requests => topic num_partitions >> > > >> replication_factor >> > > >> > > > > [replica_assignment] [configs] >> > > >> > > > > > topic => STRING >> > > >> > > > > > num_partitions => INT32 >> > > >> > > > > > replication_factor => INT16 >> > > >> > > > > > replica_assignment => partition_id [replicas] >> > > >> > > > > > partition_id => INT32 >> > > >> > > > > > replicas => INT32 >> > > >> > > > > > configs => config_key config_value >> > > >> > > > > > config_key => STRING >> > > >> > > > > > config_value => STRING >> > > >> > > > > > timeout => INT32 >> > > >> > > > > > >> > > >> > > > > > CreateTopicsRequest is a batch request to initiate topic >> > > creation >> > > >> > with >> > > >> > > > > > either predefined or automatic replica assignment and >> > > optionally >> > > >> > topic >> > > >> > > > > > configuration. >> > > >> > > > > > >> > > >> > > > > > Request semantics: >> > > >> > > > > > >> > > >> > > > > > 1. Must be sent to the controller broker >> > > >> > > > > > 2. If there are multiple instructions for the same >> topic in >> > > >> one >> > > >> > > > > > request an InvalidRequestException will be logged on >> the >> > > >> broker >> > > >> > and >> > > >> > > > > the >> > > >> > > > > > client will be disconnected. >> > > >> > > > > > - This is because the list of topics is modeled >> server >> > > side >> > > >> > as a >> > > >> > > > > > map with TopicName as the key >> > > >> > > > > > 3. The principal must be authorized to the "Create" >> > > Operation >> > > >> > on the >> > > >> > > > > > "Cluster" resource to create topics. >> > > >> > > > > > - Unauthorized requests will receive a >> > > >> > > > > ClusterAuthorizationException >> > > >> > > > > > 4. >> > > >> > > > > > >> > > >> > > > > > Only one from ReplicaAssignment or (num_partitions + >> > > >> > > > > replication_factor >> > > >> > > > > > ), can be defined in one instruction. >> > > >> > > > > > - If both parameters are specified an >> > > InvalidRequestException >> > > >> > will >> > > >> > > > be >> > > >> > > > > > logged on the broker and the client will be >> > > disconnected. >> > > >> > > > > > - In the case ReplicaAssignment is defined number of >> > > >> > partitions >> > > >> > > > and >> > > >> > > > > > replicas will be calculated from the supplied >> > > >> > replica_assignment. >> > > >> > > > > > - In the case of defined (num_partitions + >> > > >> > replication_factor) >> > > >> > > > > > replica assignment will be automatically generated >> by >> > > the >> > > >> > server. >> > > >> > > > > > - One or the other must be defined. The existing >> broker >> > > >> side >> > > >> > auto >> > > >> > > > > > create defaults will not be used >> > > >> > > > > > (default.replication.factor, num.partitions). The >> client >> > > >> > > > > implementation can >> > > >> > > > > > have defaults for these options when generating the >> > > >> messages. >> > > >> > > > > > - The first replica in [replicas] is assumed to be >> the >> > > >> > preferred >> > > >> > > > > > leader. This matches current behavior elsewhere. >> > > >> > > > > > 5. Setting a timeout > 0 will allow the request to >> block >> > > until >> > > >> > the >> > > >> > > > > > topic metadata is "complete" on the controller node. >> > > >> > > > > > - Complete means the local topic metadata cache been >> > > >> > completely >> > > >> > > > > > populated and all partitions have leaders >> > > >> > > > > > - The topic metadata is updated when the >> controller >> > > >> sends >> > > >> > out >> > > >> > > > > > update metadata requests to the brokers >> > > >> > > > > > - If a timeout error occurs, the topic could still >> be >> > > >> created >> > > >> > > > > > successfully at a later time. Its up to the client >> to >> > > query >> > > >> > for >> > > >> > > > > the state >> > > >> > > > > > at that point. >> > > >> > > > > > 6. Setting a timeout <= 0 will validate arguments and >> > > trigger >> > > >> > the >> > > >> > > > > > create topics and return immediately. >> > > >> > > > > > - This is essentially the fully asynchronous mode we >> > > have >> > > >> in >> > > >> > the >> > > >> > > > > > Zookeeper tools today. >> > > >> > > > > > - The error code in the response will either >> contain an >> > > >> > argument >> > > >> > > > > > validation exception or a timeout exception. If you >> > > >> receive a >> > > >> > > > > timeout >> > > >> > > > > > exception, because you asked for 0 timeout, you can >> > > assume >> > > >> > the >> > > >> > > > > message was >> > > >> > > > > > valid and the topic creation was triggered. >> > > >> > > > > > 7. The request is not transactional. >> > > >> > > > > > 1. If an error occurs on one topic, the others could >> > > still >> > > >> be >> > > >> > > > > > created. >> > > >> > > > > > 2. Errors are reported independently. >> > > >> > > > > > >> > > >> > > > > > QA: >> > > >> > > > > > >> > > >> > > > > > - Why is CreateTopicsRequest a batch request? >> > > >> > > > > > - Scenarios where tools or admins want to create >> many >> > > >> topics >> > > >> > > > should >> > > >> > > > > > be able to with fewer requests >> > > >> > > > > > - Example: MirrorMaker may want to create the topics >> > > >> > downstream >> > > >> > > > > > - What happens if some topics error immediately? Will >> it >> > > >> > > > > > return immediately? >> > > >> > > > > > - The request will block until all topics have >> either >> > > been >> > > >> > > > created, >> > > >> > > > > > errors, or the timeout has been hit >> > > >> > > > > > - There is no "short circuiting" where 1 error >> stops the >> > > >> > other >> > > >> > > > > > topics from being created >> > > >> > > > > > - Why implement "partial blocking" instead of fully >> async >> > > or >> > > >> > fully >> > > >> > > > > > consistent? >> > > >> > > > > > - See Cluster Consistent Blocking >> > > >> > > > > > < >> > > >> > > > > >> > > >> > > > >> > > >> > >> > > >> >> > > >> https://cwiki.apache.org/confluence/display/KAFKA/KIP-4+-+Command+line+and+centralized+administrative+operations#KIP-4-Commandlineandcentralizedadministrativeoperations-cluster-consistent-blocking >> > > >> > > > > > >> > > >> > > > > > below >> > > >> > > > > > - Why require the request to go to the controller? >> > > >> > > > > > - The controller is responsible for the cluster >> metadata >> > > >> and >> > > >> > its >> > > >> > > > > > propagation >> > > >> > > > > > - See Request Forwarding >> > > >> > > > > > < >> > > >> > > > > >> > > >> > > > >> > > >> > >> > > >> >> > > >> https://cwiki.apache.org/confluence/display/KAFKA/KIP-4+-+Command+line+and+centralized+administrative+operations#KIP-4-Commandlineandcentralizedadministrativeoperations-request >> > > >> > > > > > >> > > >> > > > > > below >> > > >> > > > > > >> > > >> > > > > > Create Topics Response >> > > >> > > > > > >> > > >> > > > > > >> > > >> > > > > > >> > > >> > > > > > CreateTopics Response (Version: 0) => [topic_error_codes] >> > > >> > > > > > topic_error_codes => topic error_code >> > > >> > > > > > topic => STRING >> > > >> > > > > > error_code => INT16 >> > > >> > > > > > >> > > >> > > > > > CreateTopicsResponse contains a map between topic and >> topic >> > > >> > creation >> > > >> > > > > > result error code (see New Protocol Errors >> > > >> > > > > > < >> > > >> > > > > >> > > >> > > > >> > > >> > >> > > >> >> > > >> https://cwiki.apache.org/confluence/display/KAFKA/KIP-4+-+Command+line+and+centralized+administrative+operations#KIP-4-Commandlineandcentralizedadministrativeoperations-NewProtocolErrors >> > > >> > > > > > >> > > >> > > > > > ). >> > > >> > > > > > >> > > >> > > > > >> > > >> > > > > The KIP is available here for reference (linked to the >> Create >> > > >> Topics >> > > >> > > > schema >> > > >> > > > > section): >> > > >> > > > > * >> > > >> > > > > >> > > >> > > > >> > > >> > >> > > >> >> > > >> https://cwiki.apache.org/confluence/display/KAFKA/KIP-4+-+Command+line+and+centralized+administrative+operations#KIP-4-Commandlineandcentralizedadministrativeoperations-CreateTopicsRequest(KAFKA-2945) >> > > >> > > > > < >> > > >> > > > > >> > > >> > > > >> > > >> > >> > > >> >> > > >> https://cwiki.apache.org/confluence/display/KAFKA/KIP-4+-+Command+line+and+centralized+administrative+operations#KIP-4-Commandlineandcentralizedadministrativeoperations-CreateTopicsRequest(KAFKA-2945) >> > > >> > > > > >* >> > > >> > > > > >> > > >> > > > > A pull request is available implementing the proposed >> changes >> > > here: >> > > >> > > > > https://github.com/apache/kafka/pull/1489 >> > > >> > > > > >> > > >> > > > > Here is a link to the past discussion on the mailing list: >> > > >> > > > > * >> > > >> > > > > >> > > >> > > > >> > > >> > >> > > >> >> > > >> http://search-hadoop.com/m/uyzND1rfG6v1oixmZ&subj=+DISCUSS+KIP+4+Create+Topic+Schema >> > > >> > > > > < >> > > >> > > > > >> > > >> > > > >> > > >> > >> > > >> >> > > >> http://search-hadoop.com/m/uyzND1rfG6v1oixmZ&subj=+DISCUSS+KIP+4+Create+Topic+Schema >> > > >> > > > > >* >> > > >> > > > > >> > > >> > > > > Thank you, >> > > >> > > > > Grant >> > > >> > > > > -- >> > > >> > > > > Grant Henke >> > > >> > > > > Software Engineer | Cloudera >> > > >> > > > > gr...@cloudera.com | twitter.com/gchenke | >> > > >> > linkedin.com/in/granthenke >> > > >> > > > > >> > > >> > > > >> > > >> > > >> > > >> > > >> > > >> > > >> > > >> > > -- >> > > >> > > -- Guozhang >> > > >> > >> > > >> >> > > >> >> > > >> >> > > >> -- >> > > >> Thanks, >> > > >> Ewen >> > > >> >> > > > >> > > > >> > > > >> > > > -- >> > > > Grant Henke >> > > > Software Engineer | Cloudera >> > > > gr...@cloudera.com | twitter.com/gchenke | >> linkedin.com/in/granthenke >> > > >>
