Re: Review Request 27799: New consumer

2015-01-29 Thread Jay Kreps 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/27799/
---

(Updated Jan. 29, 2015, 11:20 a.m.)


Review request for kafka.


Bugs: KAFKA-1760
https://issues.apache.org/jira/browse/KAFKA-1760


Repository: kafka


Description (updated)
---

KAFKA-1760: New consumer.


Diffs (updated)
-

  build.gradle a980f61def59ae59dbd5e58050285a801b21674f 
  clients/src/main/java/org/apache/kafka/clients/ClientRequest.java 
d32c319d8ee4c46dad309ea54b136ea9798e2fd7 
  clients/src/main/java/org/apache/kafka/clients/ClusterConnectionStates.java 
8aece7e81a804b177a6f2c12e2dc6c89c1613262 
  clients/src/main/java/org/apache/kafka/clients/CommonClientConfigs.java 
PRE-CREATION 
  clients/src/main/java/org/apache/kafka/clients/ConnectionState.java 
ab7e3220f9b76b92ef981d695299656f041ad5ed 
  clients/src/main/java/org/apache/kafka/clients/KafkaClient.java 
397695568d3fd8e835d8f923a89b3b00c96d0ead 
  clients/src/main/java/org/apache/kafka/clients/NetworkClient.java 
6746275d0b2596cd6ff7ce464a3a8225ad75ef00 
  clients/src/main/java/org/apache/kafka/clients/NodeConnectionState.java 
752a979ea0b8bde7ff6d2e1a23bf54052305d841 
  clients/src/main/java/org/apache/kafka/clients/consumer/CommitType.java 
PRE-CREATION 
  clients/src/main/java/org/apache/kafka/clients/consumer/Consumer.java 
c0c636b3e1ba213033db6d23655032c9bbd5e378 
  clients/src/main/java/org/apache/kafka/clients/consumer/ConsumerConfig.java 
57c1807ccba9f264186f83e91f37c34b959c8060 
  
clients/src/main/java/org/apache/kafka/clients/consumer/ConsumerRebalanceCallback.java
 e4cf7d1cfa01c2844b53213a7b539cdcbcbeaa3a 
  clients/src/main/java/org/apache/kafka/clients/consumer/ConsumerRecord.java 
16af70a5de52cca786fdea147a6a639b7dc4a311 
  clients/src/main/java/org/apache/kafka/clients/consumer/ConsumerRecords.java 
bdf4b26942d5a8c8a9503e05908e9a9eff6228a7 
  clients/src/main/java/org/apache/kafka/clients/consumer/KafkaConsumer.java 
76efc216c9e6c3ab084461d792877092a189ad0f 
  clients/src/main/java/org/apache/kafka/clients/consumer/MockConsumer.java 
fa88ac1a8b19b4294f211c4467fe68c7707ddbae 
  
clients/src/main/java/org/apache/kafka/clients/consumer/NoOffsetForPartitionException.java
 PRE-CREATION 
  clients/src/main/java/org/apache/kafka/clients/consumer/OffsetMetadata.java 
ea423ad15eebd262d20d5ec05d592cc115229177 
  
clients/src/main/java/org/apache/kafka/clients/consumer/internals/Heartbeat.java
 PRE-CREATION 
  
clients/src/main/java/org/apache/kafka/clients/consumer/internals/NoOpConsumerRebalanceCallback.java
 PRE-CREATION 
  
clients/src/main/java/org/apache/kafka/clients/consumer/internals/SubscriptionState.java
 PRE-CREATION 
  clients/src/main/java/org/apache/kafka/clients/producer/KafkaProducer.java 
fc71710dd5997576d3841a1c3b0f7e19a8c9698e 
  clients/src/main/java/org/apache/kafka/clients/producer/MockProducer.java 
904976fadf0610982958628eaee810b60a98d725 
  clients/src/main/java/org/apache/kafka/clients/producer/ProducerConfig.java 
8b3e565edd1ae04d8d34bd9f1a41e9fa8c880a75 
  
clients/src/main/java/org/apache/kafka/clients/producer/internals/Metadata.java 
dcf46581b912cfb1b5c8d4cbc293d2d1444b7740 
  
clients/src/main/java/org/apache/kafka/clients/producer/internals/Partitioner.java
 483899d2e69b33655d0e08949f5f64af2519660a 
  clients/src/main/java/org/apache/kafka/clients/producer/internals/Sender.java 
ccc03d8447ebba40131a70e16969686ac4aab58a 
  clients/src/main/java/org/apache/kafka/common/Cluster.java 
d3299b944062d96852452de455902659ad8af757 
  clients/src/main/java/org/apache/kafka/common/PartitionInfo.java 
b15aa2c3ef2d7c4b24618ff42fd4da324237a813 
  clients/src/main/java/org/apache/kafka/common/config/ConfigDef.java 
98cb79b701918eca3f6ca9823b6c7b7c97b3ecec 
  clients/src/main/java/org/apache/kafka/common/errors/ApiException.java 
7c948b166a8ac07616809f260754116ae7764973 
  clients/src/main/java/org/apache/kafka/common/network/Selectable.java 
b68bbf00ab8eba6c5867d346c91188142593ca6e 
  clients/src/main/java/org/apache/kafka/common/network/Selector.java 
74d695ba39de44b6a3d15340ec0114bc4fce2ba2 
  clients/src/main/java/org/apache/kafka/common/protocol/Errors.java 
3316b6a1098311b8603a4a5893bf57b75d2e43cb 
  clients/src/main/java/org/apache/kafka/common/protocol/types/Struct.java 
121e880a941fcd3e6392859edba11a94236494cc 
  clients/src/main/java/org/apache/kafka/common/record/LogEntry.java 
e4d688cbe0c61b74ea15fc8dd3b634f9e5ee9b83 
  clients/src/main/java/org/apache/kafka/common/record/MemoryRecords.java 
040e5b91005edb8f015afdfa76fd94e0bf3cb4ca 
  
clients/src/main/java/org/apache/kafka/common/requests/ConsumerMetadataRequest.java
 99b52c23d639df010bf2affc0f79d1c6e16ed67c 
  
clients/src/main/java/org/apache/kafka/common/requests/ConsumerMetadataResponse.java
 8b8f591c4b2802a9cbbe34746c0b3ca4a64a8681 
  clients/src/main/java/org/apache/kafka/common/requests/FetchRequest.java

Re: Review Request 27799: New consumer

2015-01-29 Thread Jay Kreps 


 On Jan. 20, 2015, 8:03 a.m., Guozhang Wang wrote:
  clients/src/main/java/org/apache/kafka/clients/KafkaClient.java, lines 30-33
  https://reviews.apache.org/r/27799/diff/3/?file=824651#file824651line30
 
  Wondering why we make newline for @param but keep the same line for 
  @return?

Yeah that is annoying auto-formatting from my IDE, will fix.


 On Jan. 20, 2015, 8:03 a.m., Guozhang Wang wrote:
  clients/src/main/java/org/apache/kafka/clients/NetworkClient.java, line 167
  https://reviews.apache.org/r/27799/diff/3/?file=824652#file824652line167
 
  Is this function really private? If yes we do not need keep the javadoc 
  for it.

Well we actually have javadoc'd everythin in that class.


 On Jan. 20, 2015, 8:03 a.m., Guozhang Wang wrote:
  clients/src/main/java/org/apache/kafka/clients/consumer/Consumer.java, 
  lines 37-44
  https://reviews.apache.org/r/27799/diff/3/?file=824655#file824655line37
 
  The javadocs here are a little confusing: users looking for its 
  function APIs need to look into KafkaConsumer, an implementation of the 
  interface Consumer.

The motivation here was that the description of behavior is actually very 
specific to the KafkaConsumer implementation (e.g. details of communication to 
the server and so on). I didn't want to duplicate the docs either. Actually the 
interface is not really meant for use, we don't want a bunch of 
implementations, people should either use our mock or the real class. Mocks for 
our class will likely break as we add methods.


 On Jan. 20, 2015, 8:03 a.m., Guozhang Wang wrote:
  clients/src/main/java/org/apache/kafka/clients/consumer/ConsumerConfig.java,
   line 114
  https://reviews.apache.org/r/27799/diff/3/?file=824656#file824656line114
 
  Should this inherit from CommonClientConfig?

Well, actually ConsumerConfig and ProducerConfig are public and 
CommonClientConfig is private...I just wanted a way to share those variables.


 On Jan. 20, 2015, 8:03 a.m., Guozhang Wang wrote:
  clients/src/main/java/org/apache/kafka/clients/consumer/ConsumerConfig.java,
   lines 185-191
  https://reviews.apache.org/r/27799/diff/3/?file=824656#file824656line185
 
  Could these two be moved to CommonClientConfig?

Actually it's subtly different since it is key.serializer in the producer and 
key.deserializer in the consumer.


 On Jan. 20, 2015, 8:03 a.m., Guozhang Wang wrote:
  clients/src/main/java/org/apache/kafka/clients/consumer/KafkaConsumer.java, 
  line 845
  https://reviews.apache.org/r/27799/diff/3/?file=824660#file824660line845
 
  Maybe add some comments on why consumerId / generationId are 
  initialized as such and when they will be updated and used.

This is actually not handled yet as we need the consumer co-ordinator to 
maintain and update these variables first.


 On Jan. 20, 2015, 8:03 a.m., Guozhang Wang wrote:
  clients/src/main/java/org/apache/kafka/clients/consumer/KafkaConsumer.java, 
  line 917
  https://reviews.apache.org/r/27799/diff/3/?file=824660#file824660line917
 
  There is a potential risk that a topic is deleted and the consumer 
  unscribes to it, but not removing it from its metadata topic list, causing 
  the underlying network client to keep refreshing metadata.

I think you are flagging that we never remove topics from our metadata fetch 
list. This is true. But I don't think this will cause metadata refreshes to 
occur will it?

Another question here is just what the behavior should be for a topic that is 
deleted. There are obviously a bunch of states the consumer could be in when 
the topic gets deleted so we should ideally be sure we handle them all. I 
haven't thought a lot about this.


 On Jan. 20, 2015, 8:03 a.m., Guozhang Wang wrote:
  clients/src/main/java/org/apache/kafka/clients/consumer/KafkaConsumer.java, 
  lines 968-973
  https://reviews.apache.org/r/27799/diff/3/?file=824660#file824660line968
 
  Should the ordering of these two else if be swapped?

I don't think so. I think the idea is that if the coordinator has hit the 
failure detection criteria you want to mark it dead and continue. You don't to 
keep trying to heartbeat to it (since you have already failed multiple times).


 On Jan. 20, 2015, 8:03 a.m., Guozhang Wang wrote:
  clients/src/main/java/org/apache/kafka/clients/consumer/KafkaConsumer.java, 
  line 1075
  https://reviews.apache.org/r/27799/diff/3/?file=824660#file824660line1075
 
  Do we need to back off here?

Good call.


 On Jan. 20, 2015, 8:03 a.m., Guozhang Wang wrote:
  clients/src/main/java/org/apache/kafka/clients/consumer/KafkaConsumer.java, 
  line 1343
  https://reviews.apache.org/r/27799/diff/3/?file=824660#file824660line1343
 
  Should we change the this.client.inFlightRequestCount ..  condition 
  to just node.ready()?

Actually the criteria here is stricter. We need to issue successive fetch 
requests at offset X, X', X'', etc. So we need to prevent ourselves from 
issuing fetch request for offset X' until X has completed. So if

Re: Review Request 27799: New consumer

2015-01-29 Thread Jay Kreps 


 On Jan. 27, 2015, 10:25 a.m., Onur Karaman wrote:
  clients/src/main/java/org/apache/kafka/clients/consumer/ConsumerConfig.java,
   line 205
  https://reviews.apache.org/r/27799/diff/7/?file=832174#file832174line205
 
  You left a blah.

This one is actually intentional. We haven't implemented the server assignment 
which would determine the name that the client should send. I wanted to leave 
this something made up so we would remember to change it when we have that in 
place.


 On Jan. 27, 2015, 10:25 a.m., Onur Karaman wrote:
  clients/src/main/java/org/apache/kafka/clients/consumer/KafkaConsumer.java, 
  lines 163-176
  https://reviews.apache.org/r/27799/diff/7/?file=832178#file832178line163
 
  Was this switch to html codes intentional?

Yeah my IDE seems to do that. Kind of annoying.


- Jay


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/27799/#review69737
---


On Jan. 23, 2015, 9:15 p.m., Jay Kreps wrote:
 
 ---
 This is an automatically generated e-mail. To reply, visit:
 https://reviews.apache.org/r/27799/
 ---
 
 (Updated Jan. 23, 2015, 9:15 p.m.)
 
 
 Review request for kafka.
 
 
 Bugs: KAFKA-1760
 https://issues.apache.org/jira/browse/KAFKA-1760
 
 
 Repository: kafka
 
 
 Description
 ---
 
 New consumer.
 
 
 Diffs
 -
 
   build.gradle 1cbab29ce83e20dae0561b51eed6fdb86d522f28 
   clients/src/main/java/org/apache/kafka/clients/ClientRequest.java 
 d32c319d8ee4c46dad309ea54b136ea9798e2fd7 
   clients/src/main/java/org/apache/kafka/clients/ClusterConnectionStates.java 
 8aece7e81a804b177a6f2c12e2dc6c89c1613262 
   clients/src/main/java/org/apache/kafka/clients/CommonClientConfigs.java 
 PRE-CREATION 
   clients/src/main/java/org/apache/kafka/clients/ConnectionState.java 
 ab7e3220f9b76b92ef981d695299656f041ad5ed 
   clients/src/main/java/org/apache/kafka/clients/KafkaClient.java 
 397695568d3fd8e835d8f923a89b3b00c96d0ead 
   clients/src/main/java/org/apache/kafka/clients/NetworkClient.java 
 6746275d0b2596cd6ff7ce464a3a8225ad75ef00 
   clients/src/main/java/org/apache/kafka/clients/NodeConnectionState.java 
 752a979ea0b8bde7ff6d2e1a23bf54052305d841 
   clients/src/main/java/org/apache/kafka/clients/consumer/CommitType.java 
 PRE-CREATION 
   clients/src/main/java/org/apache/kafka/clients/consumer/Consumer.java 
 c0c636b3e1ba213033db6d23655032c9bbd5e378 
   clients/src/main/java/org/apache/kafka/clients/consumer/ConsumerConfig.java 
 57c1807ccba9f264186f83e91f37c34b959c8060 
   
 clients/src/main/java/org/apache/kafka/clients/consumer/ConsumerRebalanceCallback.java
  e4cf7d1cfa01c2844b53213a7b539cdcbcbeaa3a 
   clients/src/main/java/org/apache/kafka/clients/consumer/ConsumerRecord.java 
 16af70a5de52cca786fdea147a6a639b7dc4a311 
   
 clients/src/main/java/org/apache/kafka/clients/consumer/ConsumerRecords.java 
 bdf4b26942d5a8c8a9503e05908e9a9eff6228a7 
   clients/src/main/java/org/apache/kafka/clients/consumer/KafkaConsumer.java 
 76efc216c9e6c3ab084461d792877092a189ad0f 
   clients/src/main/java/org/apache/kafka/clients/consumer/MockConsumer.java 
 fa88ac1a8b19b4294f211c4467fe68c7707ddbae 
   
 clients/src/main/java/org/apache/kafka/clients/consumer/NoOffsetForPartitionException.java
  PRE-CREATION 
   clients/src/main/java/org/apache/kafka/clients/consumer/OffsetMetadata.java 
 ea423ad15eebd262d20d5ec05d592cc115229177 
   
 clients/src/main/java/org/apache/kafka/clients/consumer/internals/Heartbeat.java
  PRE-CREATION 
   
 clients/src/main/java/org/apache/kafka/clients/consumer/internals/NoOpConsumerRebalanceCallback.java
  PRE-CREATION 
   
 clients/src/main/java/org/apache/kafka/clients/consumer/internals/SubscriptionState.java
  PRE-CREATION 
   clients/src/main/java/org/apache/kafka/clients/producer/KafkaProducer.java 
 fc71710dd5997576d3841a1c3b0f7e19a8c9698e 
   clients/src/main/java/org/apache/kafka/clients/producer/MockProducer.java 
 904976fadf0610982958628eaee810b60a98d725 
   clients/src/main/java/org/apache/kafka/clients/producer/ProducerConfig.java 
 8b3e565edd1ae04d8d34bd9f1a41e9fa8c880a75 
   
 clients/src/main/java/org/apache/kafka/clients/producer/internals/Metadata.java
  dcf46581b912cfb1b5c8d4cbc293d2d1444b7740 
   
 clients/src/main/java/org/apache/kafka/clients/producer/internals/Partitioner.java
  483899d2e69b33655d0e08949f5f64af2519660a 
   
 clients/src/main/java/org/apache/kafka/clients/producer/internals/Sender.java 
 ccc03d8447ebba40131a70e16969686ac4aab58a 
   clients/src/main/java/org/apache/kafka/common/Cluster.java 
 d3299b944062d96852452de455902659ad8af757 
   clients/src/main/java/org/apache/kafka/common/PartitionInfo.java 
 b15aa2c3ef2d7c4b24618ff42fd4da324237a813 
   clients/src/main/java/org/apache/kafka/common/config/ConfigDef.java

Re: Review Request 27799: New consumer

2015-01-29 Thread Jay Kreps 


 On Jan. 28, 2015, 1:34 a.m., Guozhang Wang wrote:
  patch -p1  patch-file does not do the renaming of 
  RequestCompletionHandler.java so I have to do that manually (weird), but 
  other than that, build / test LGTM. 
  
  It seems some of previous comments are not addressed yet. For exmaple ones 
  from Jan. 20, the METADATA_MAX_AGE configs, etc. Could you double check to 
  see if they are valid and then I think we can check it in.

Oops, I forgot to hit publish on some of the replies.


- Jay


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/27799/#review69936
---


On Jan. 23, 2015, 9:15 p.m., Jay Kreps wrote:
 
 ---
 This is an automatically generated e-mail. To reply, visit:
 https://reviews.apache.org/r/27799/
 ---
 
 (Updated Jan. 23, 2015, 9:15 p.m.)
 
 
 Review request for kafka.
 
 
 Bugs: KAFKA-1760
 https://issues.apache.org/jira/browse/KAFKA-1760
 
 
 Repository: kafka
 
 
 Description
 ---
 
 New consumer.
 
 
 Diffs
 -
 
   build.gradle 1cbab29ce83e20dae0561b51eed6fdb86d522f28 
   clients/src/main/java/org/apache/kafka/clients/ClientRequest.java 
 d32c319d8ee4c46dad309ea54b136ea9798e2fd7 
   clients/src/main/java/org/apache/kafka/clients/ClusterConnectionStates.java 
 8aece7e81a804b177a6f2c12e2dc6c89c1613262 
   clients/src/main/java/org/apache/kafka/clients/CommonClientConfigs.java 
 PRE-CREATION 
   clients/src/main/java/org/apache/kafka/clients/ConnectionState.java 
 ab7e3220f9b76b92ef981d695299656f041ad5ed 
   clients/src/main/java/org/apache/kafka/clients/KafkaClient.java 
 397695568d3fd8e835d8f923a89b3b00c96d0ead 
   clients/src/main/java/org/apache/kafka/clients/NetworkClient.java 
 6746275d0b2596cd6ff7ce464a3a8225ad75ef00 
   clients/src/main/java/org/apache/kafka/clients/NodeConnectionState.java 
 752a979ea0b8bde7ff6d2e1a23bf54052305d841 
   clients/src/main/java/org/apache/kafka/clients/consumer/CommitType.java 
 PRE-CREATION 
   clients/src/main/java/org/apache/kafka/clients/consumer/Consumer.java 
 c0c636b3e1ba213033db6d23655032c9bbd5e378 
   clients/src/main/java/org/apache/kafka/clients/consumer/ConsumerConfig.java 
 57c1807ccba9f264186f83e91f37c34b959c8060 
   
 clients/src/main/java/org/apache/kafka/clients/consumer/ConsumerRebalanceCallback.java
  e4cf7d1cfa01c2844b53213a7b539cdcbcbeaa3a 
   clients/src/main/java/org/apache/kafka/clients/consumer/ConsumerRecord.java 
 16af70a5de52cca786fdea147a6a639b7dc4a311 
   
 clients/src/main/java/org/apache/kafka/clients/consumer/ConsumerRecords.java 
 bdf4b26942d5a8c8a9503e05908e9a9eff6228a7 
   clients/src/main/java/org/apache/kafka/clients/consumer/KafkaConsumer.java 
 76efc216c9e6c3ab084461d792877092a189ad0f 
   clients/src/main/java/org/apache/kafka/clients/consumer/MockConsumer.java 
 fa88ac1a8b19b4294f211c4467fe68c7707ddbae 
   
 clients/src/main/java/org/apache/kafka/clients/consumer/NoOffsetForPartitionException.java
  PRE-CREATION 
   clients/src/main/java/org/apache/kafka/clients/consumer/OffsetMetadata.java 
 ea423ad15eebd262d20d5ec05d592cc115229177 
   
 clients/src/main/java/org/apache/kafka/clients/consumer/internals/Heartbeat.java
  PRE-CREATION 
   
 clients/src/main/java/org/apache/kafka/clients/consumer/internals/NoOpConsumerRebalanceCallback.java
  PRE-CREATION 
   
 clients/src/main/java/org/apache/kafka/clients/consumer/internals/SubscriptionState.java
  PRE-CREATION 
   clients/src/main/java/org/apache/kafka/clients/producer/KafkaProducer.java 
 fc71710dd5997576d3841a1c3b0f7e19a8c9698e 
   clients/src/main/java/org/apache/kafka/clients/producer/MockProducer.java 
 904976fadf0610982958628eaee810b60a98d725 
   clients/src/main/java/org/apache/kafka/clients/producer/ProducerConfig.java 
 8b3e565edd1ae04d8d34bd9f1a41e9fa8c880a75 
   
 clients/src/main/java/org/apache/kafka/clients/producer/internals/Metadata.java
  dcf46581b912cfb1b5c8d4cbc293d2d1444b7740 
   
 clients/src/main/java/org/apache/kafka/clients/producer/internals/Partitioner.java
  483899d2e69b33655d0e08949f5f64af2519660a 
   
 clients/src/main/java/org/apache/kafka/clients/producer/internals/Sender.java 
 ccc03d8447ebba40131a70e16969686ac4aab58a 
   clients/src/main/java/org/apache/kafka/common/Cluster.java 
 d3299b944062d96852452de455902659ad8af757 
   clients/src/main/java/org/apache/kafka/common/PartitionInfo.java 
 b15aa2c3ef2d7c4b24618ff42fd4da324237a813 
   clients/src/main/java/org/apache/kafka/common/config/ConfigDef.java 
 98cb79b701918eca3f6ca9823b6c7b7c97b3ecec 
   clients/src/main/java/org/apache/kafka/common/errors/ApiException.java 
 7c948b166a8ac07616809f260754116ae7764973 
   clients/src/main/java/org/apache/kafka/common/network/Selectable.java 
 b68bbf00ab8eba6c5867d346c91188142593ca6e 
   clients/src/main/java/org/apache/kafka/common/network/Selector.java

Re: Review Request 27799: New consumer

2015-01-29 Thread Jay Kreps 


 On Jan. 22, 2015, 5:35 p.m., Aditya Auradkar wrote:
  clients/src/main/java/org/apache/kafka/clients/RequestCompletionHandler.java,
   line 21
  https://reviews.apache.org/r/27799/diff/4/?file=828377#file828377line21
 
  nit. Can we remove the public from the interface methods?
 
 Jay Kreps wrote:
 Can you explain...?
 
 Aditya Auradkar wrote:
 I gather all interface methods are implicitly public.. so that should be 
 unnecessary.
 
 Aditya Auradkar wrote:
 http://docs.oracle.com/javase/specs/jls/se7/html/jls-9.html#jls-9.4

Oh interesting. Did not know that. I suppose we should probably either leave it 
or remove then all then.


- Jay


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/27799/#review69204
---


On Jan. 23, 2015, 9:15 p.m., Jay Kreps wrote:
 
 ---
 This is an automatically generated e-mail. To reply, visit:
 https://reviews.apache.org/r/27799/
 ---
 
 (Updated Jan. 23, 2015, 9:15 p.m.)
 
 
 Review request for kafka.
 
 
 Bugs: KAFKA-1760
 https://issues.apache.org/jira/browse/KAFKA-1760
 
 
 Repository: kafka
 
 
 Description
 ---
 
 New consumer.
 
 
 Diffs
 -
 
   build.gradle 1cbab29ce83e20dae0561b51eed6fdb86d522f28 
   clients/src/main/java/org/apache/kafka/clients/ClientRequest.java 
 d32c319d8ee4c46dad309ea54b136ea9798e2fd7 
   clients/src/main/java/org/apache/kafka/clients/ClusterConnectionStates.java 
 8aece7e81a804b177a6f2c12e2dc6c89c1613262 
   clients/src/main/java/org/apache/kafka/clients/CommonClientConfigs.java 
 PRE-CREATION 
   clients/src/main/java/org/apache/kafka/clients/ConnectionState.java 
 ab7e3220f9b76b92ef981d695299656f041ad5ed 
   clients/src/main/java/org/apache/kafka/clients/KafkaClient.java 
 397695568d3fd8e835d8f923a89b3b00c96d0ead 
   clients/src/main/java/org/apache/kafka/clients/NetworkClient.java 
 6746275d0b2596cd6ff7ce464a3a8225ad75ef00 
   clients/src/main/java/org/apache/kafka/clients/NodeConnectionState.java 
 752a979ea0b8bde7ff6d2e1a23bf54052305d841 
   clients/src/main/java/org/apache/kafka/clients/consumer/CommitType.java 
 PRE-CREATION 
   clients/src/main/java/org/apache/kafka/clients/consumer/Consumer.java 
 c0c636b3e1ba213033db6d23655032c9bbd5e378 
   clients/src/main/java/org/apache/kafka/clients/consumer/ConsumerConfig.java 
 57c1807ccba9f264186f83e91f37c34b959c8060 
   
 clients/src/main/java/org/apache/kafka/clients/consumer/ConsumerRebalanceCallback.java
  e4cf7d1cfa01c2844b53213a7b539cdcbcbeaa3a 
   clients/src/main/java/org/apache/kafka/clients/consumer/ConsumerRecord.java 
 16af70a5de52cca786fdea147a6a639b7dc4a311 
   
 clients/src/main/java/org/apache/kafka/clients/consumer/ConsumerRecords.java 
 bdf4b26942d5a8c8a9503e05908e9a9eff6228a7 
   clients/src/main/java/org/apache/kafka/clients/consumer/KafkaConsumer.java 
 76efc216c9e6c3ab084461d792877092a189ad0f 
   clients/src/main/java/org/apache/kafka/clients/consumer/MockConsumer.java 
 fa88ac1a8b19b4294f211c4467fe68c7707ddbae 
   
 clients/src/main/java/org/apache/kafka/clients/consumer/NoOffsetForPartitionException.java
  PRE-CREATION 
   clients/src/main/java/org/apache/kafka/clients/consumer/OffsetMetadata.java 
 ea423ad15eebd262d20d5ec05d592cc115229177 
   
 clients/src/main/java/org/apache/kafka/clients/consumer/internals/Heartbeat.java
  PRE-CREATION 
   
 clients/src/main/java/org/apache/kafka/clients/consumer/internals/NoOpConsumerRebalanceCallback.java
  PRE-CREATION 
   
 clients/src/main/java/org/apache/kafka/clients/consumer/internals/SubscriptionState.java
  PRE-CREATION 
   clients/src/main/java/org/apache/kafka/clients/producer/KafkaProducer.java 
 fc71710dd5997576d3841a1c3b0f7e19a8c9698e 
   clients/src/main/java/org/apache/kafka/clients/producer/MockProducer.java 
 904976fadf0610982958628eaee810b60a98d725 
   clients/src/main/java/org/apache/kafka/clients/producer/ProducerConfig.java 
 8b3e565edd1ae04d8d34bd9f1a41e9fa8c880a75 
   
 clients/src/main/java/org/apache/kafka/clients/producer/internals/Metadata.java
  dcf46581b912cfb1b5c8d4cbc293d2d1444b7740 
   
 clients/src/main/java/org/apache/kafka/clients/producer/internals/Partitioner.java
  483899d2e69b33655d0e08949f5f64af2519660a 
   
 clients/src/main/java/org/apache/kafka/clients/producer/internals/Sender.java 
 ccc03d8447ebba40131a70e16969686ac4aab58a 
   clients/src/main/java/org/apache/kafka/common/Cluster.java 
 d3299b944062d96852452de455902659ad8af757 
   clients/src/main/java/org/apache/kafka/common/PartitionInfo.java 
 b15aa2c3ef2d7c4b24618ff42fd4da324237a813 
   clients/src/main/java/org/apache/kafka/common/config/ConfigDef.java 
 98cb79b701918eca3f6ca9823b6c7b7c97b3ecec 
   clients/src/main/java/org/apache/kafka/common/errors/ApiException.java 
 7c948b166a8ac07616809f260754116ae7764973

Re: Review Request 27799: New consumer

2015-01-29 Thread Guozhang Wang 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/27799/#review70291
---

Ship it!


- Guozhang Wang


On Jan. 29, 2015, 11:20 a.m., Jay Kreps wrote:
 
 ---
 This is an automatically generated e-mail. To reply, visit:
 https://reviews.apache.org/r/27799/
 ---
 
 (Updated Jan. 29, 2015, 11:20 a.m.)
 
 
 Review request for kafka.
 
 
 Bugs: KAFKA-1760
 https://issues.apache.org/jira/browse/KAFKA-1760
 
 
 Repository: kafka
 
 
 Description
 ---
 
 KAFKA-1760: New consumer.
 
 
 Diffs
 -
 
   build.gradle a980f61def59ae59dbd5e58050285a801b21674f 
   clients/src/main/java/org/apache/kafka/clients/ClientRequest.java 
 d32c319d8ee4c46dad309ea54b136ea9798e2fd7 
   clients/src/main/java/org/apache/kafka/clients/ClusterConnectionStates.java 
 8aece7e81a804b177a6f2c12e2dc6c89c1613262 
   clients/src/main/java/org/apache/kafka/clients/CommonClientConfigs.java 
 PRE-CREATION 
   clients/src/main/java/org/apache/kafka/clients/ConnectionState.java 
 ab7e3220f9b76b92ef981d695299656f041ad5ed 
   clients/src/main/java/org/apache/kafka/clients/KafkaClient.java 
 397695568d3fd8e835d8f923a89b3b00c96d0ead 
   clients/src/main/java/org/apache/kafka/clients/NetworkClient.java 
 6746275d0b2596cd6ff7ce464a3a8225ad75ef00 
   clients/src/main/java/org/apache/kafka/clients/NodeConnectionState.java 
 752a979ea0b8bde7ff6d2e1a23bf54052305d841 
   clients/src/main/java/org/apache/kafka/clients/consumer/CommitType.java 
 PRE-CREATION 
   clients/src/main/java/org/apache/kafka/clients/consumer/Consumer.java 
 c0c636b3e1ba213033db6d23655032c9bbd5e378 
   clients/src/main/java/org/apache/kafka/clients/consumer/ConsumerConfig.java 
 57c1807ccba9f264186f83e91f37c34b959c8060 
   
 clients/src/main/java/org/apache/kafka/clients/consumer/ConsumerRebalanceCallback.java
  e4cf7d1cfa01c2844b53213a7b539cdcbcbeaa3a 
   clients/src/main/java/org/apache/kafka/clients/consumer/ConsumerRecord.java 
 16af70a5de52cca786fdea147a6a639b7dc4a311 
   
 clients/src/main/java/org/apache/kafka/clients/consumer/ConsumerRecords.java 
 bdf4b26942d5a8c8a9503e05908e9a9eff6228a7 
   clients/src/main/java/org/apache/kafka/clients/consumer/KafkaConsumer.java 
 76efc216c9e6c3ab084461d792877092a189ad0f 
   clients/src/main/java/org/apache/kafka/clients/consumer/MockConsumer.java 
 fa88ac1a8b19b4294f211c4467fe68c7707ddbae 
   
 clients/src/main/java/org/apache/kafka/clients/consumer/NoOffsetForPartitionException.java
  PRE-CREATION 
   clients/src/main/java/org/apache/kafka/clients/consumer/OffsetMetadata.java 
 ea423ad15eebd262d20d5ec05d592cc115229177 
   
 clients/src/main/java/org/apache/kafka/clients/consumer/internals/Heartbeat.java
  PRE-CREATION 
   
 clients/src/main/java/org/apache/kafka/clients/consumer/internals/NoOpConsumerRebalanceCallback.java
  PRE-CREATION 
   
 clients/src/main/java/org/apache/kafka/clients/consumer/internals/SubscriptionState.java
  PRE-CREATION 
   clients/src/main/java/org/apache/kafka/clients/producer/KafkaProducer.java 
 fc71710dd5997576d3841a1c3b0f7e19a8c9698e 
   clients/src/main/java/org/apache/kafka/clients/producer/MockProducer.java 
 904976fadf0610982958628eaee810b60a98d725 
   clients/src/main/java/org/apache/kafka/clients/producer/ProducerConfig.java 
 8b3e565edd1ae04d8d34bd9f1a41e9fa8c880a75 
   
 clients/src/main/java/org/apache/kafka/clients/producer/internals/Metadata.java
  dcf46581b912cfb1b5c8d4cbc293d2d1444b7740 
   
 clients/src/main/java/org/apache/kafka/clients/producer/internals/Partitioner.java
  483899d2e69b33655d0e08949f5f64af2519660a 
   
 clients/src/main/java/org/apache/kafka/clients/producer/internals/Sender.java 
 ccc03d8447ebba40131a70e16969686ac4aab58a 
   clients/src/main/java/org/apache/kafka/common/Cluster.java 
 d3299b944062d96852452de455902659ad8af757 
   clients/src/main/java/org/apache/kafka/common/PartitionInfo.java 
 b15aa2c3ef2d7c4b24618ff42fd4da324237a813 
   clients/src/main/java/org/apache/kafka/common/config/ConfigDef.java 
 98cb79b701918eca3f6ca9823b6c7b7c97b3ecec 
   clients/src/main/java/org/apache/kafka/common/errors/ApiException.java 
 7c948b166a8ac07616809f260754116ae7764973 
   clients/src/main/java/org/apache/kafka/common/network/Selectable.java 
 b68bbf00ab8eba6c5867d346c91188142593ca6e 
   clients/src/main/java/org/apache/kafka/common/network/Selector.java 
 74d695ba39de44b6a3d15340ec0114bc4fce2ba2 
   clients/src/main/java/org/apache/kafka/common/protocol/Errors.java 
 3316b6a1098311b8603a4a5893bf57b75d2e43cb 
   clients/src/main/java/org/apache/kafka/common/protocol/types/Struct.java 
 121e880a941fcd3e6392859edba11a94236494cc 
   clients/src/main/java/org/apache/kafka/common/record/LogEntry.java 
 e4d688cbe0c61b74ea15fc8dd3b634f9e5ee9b83

Re: Review Request 27799: New consumer

2015-01-27 Thread Guozhang Wang 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/27799/#review69936
---


patch -p1  patch-file does not do the renaming of 
RequestCompletionHandler.java so I have to do that manually (weird), but other 
than that, build / test LGTM. 

It seems some of previous comments are not addressed yet. For exmaple ones from 
Jan. 20, the METADATA_MAX_AGE configs, etc. Could you double check to see if 
they are valid and then I think we can check it in.

- Guozhang Wang


On Jan. 23, 2015, 9:15 p.m., Jay Kreps wrote:
 
 ---
 This is an automatically generated e-mail. To reply, visit:
 https://reviews.apache.org/r/27799/
 ---
 
 (Updated Jan. 23, 2015, 9:15 p.m.)
 
 
 Review request for kafka.
 
 
 Bugs: KAFKA-1760
 https://issues.apache.org/jira/browse/KAFKA-1760
 
 
 Repository: kafka
 
 
 Description
 ---
 
 New consumer.
 
 
 Diffs
 -
 
   build.gradle 1cbab29ce83e20dae0561b51eed6fdb86d522f28 
   clients/src/main/java/org/apache/kafka/clients/ClientRequest.java 
 d32c319d8ee4c46dad309ea54b136ea9798e2fd7 
   clients/src/main/java/org/apache/kafka/clients/ClusterConnectionStates.java 
 8aece7e81a804b177a6f2c12e2dc6c89c1613262 
   clients/src/main/java/org/apache/kafka/clients/CommonClientConfigs.java 
 PRE-CREATION 
   clients/src/main/java/org/apache/kafka/clients/ConnectionState.java 
 ab7e3220f9b76b92ef981d695299656f041ad5ed 
   clients/src/main/java/org/apache/kafka/clients/KafkaClient.java 
 397695568d3fd8e835d8f923a89b3b00c96d0ead 
   clients/src/main/java/org/apache/kafka/clients/NetworkClient.java 
 6746275d0b2596cd6ff7ce464a3a8225ad75ef00 
   clients/src/main/java/org/apache/kafka/clients/NodeConnectionState.java 
 752a979ea0b8bde7ff6d2e1a23bf54052305d841 
   clients/src/main/java/org/apache/kafka/clients/consumer/CommitType.java 
 PRE-CREATION 
   clients/src/main/java/org/apache/kafka/clients/consumer/Consumer.java 
 c0c636b3e1ba213033db6d23655032c9bbd5e378 
   clients/src/main/java/org/apache/kafka/clients/consumer/ConsumerConfig.java 
 57c1807ccba9f264186f83e91f37c34b959c8060 
   
 clients/src/main/java/org/apache/kafka/clients/consumer/ConsumerRebalanceCallback.java
  e4cf7d1cfa01c2844b53213a7b539cdcbcbeaa3a 
   clients/src/main/java/org/apache/kafka/clients/consumer/ConsumerRecord.java 
 16af70a5de52cca786fdea147a6a639b7dc4a311 
   
 clients/src/main/java/org/apache/kafka/clients/consumer/ConsumerRecords.java 
 bdf4b26942d5a8c8a9503e05908e9a9eff6228a7 
   clients/src/main/java/org/apache/kafka/clients/consumer/KafkaConsumer.java 
 76efc216c9e6c3ab084461d792877092a189ad0f 
   clients/src/main/java/org/apache/kafka/clients/consumer/MockConsumer.java 
 fa88ac1a8b19b4294f211c4467fe68c7707ddbae 
   
 clients/src/main/java/org/apache/kafka/clients/consumer/NoOffsetForPartitionException.java
  PRE-CREATION 
   clients/src/main/java/org/apache/kafka/clients/consumer/OffsetMetadata.java 
 ea423ad15eebd262d20d5ec05d592cc115229177 
   
 clients/src/main/java/org/apache/kafka/clients/consumer/internals/Heartbeat.java
  PRE-CREATION 
   
 clients/src/main/java/org/apache/kafka/clients/consumer/internals/NoOpConsumerRebalanceCallback.java
  PRE-CREATION 
   
 clients/src/main/java/org/apache/kafka/clients/consumer/internals/SubscriptionState.java
  PRE-CREATION 
   clients/src/main/java/org/apache/kafka/clients/producer/KafkaProducer.java 
 fc71710dd5997576d3841a1c3b0f7e19a8c9698e 
   clients/src/main/java/org/apache/kafka/clients/producer/MockProducer.java 
 904976fadf0610982958628eaee810b60a98d725 
   clients/src/main/java/org/apache/kafka/clients/producer/ProducerConfig.java 
 8b3e565edd1ae04d8d34bd9f1a41e9fa8c880a75 
   
 clients/src/main/java/org/apache/kafka/clients/producer/internals/Metadata.java
  dcf46581b912cfb1b5c8d4cbc293d2d1444b7740 
   
 clients/src/main/java/org/apache/kafka/clients/producer/internals/Partitioner.java
  483899d2e69b33655d0e08949f5f64af2519660a 
   
 clients/src/main/java/org/apache/kafka/clients/producer/internals/Sender.java 
 ccc03d8447ebba40131a70e16969686ac4aab58a 
   clients/src/main/java/org/apache/kafka/common/Cluster.java 
 d3299b944062d96852452de455902659ad8af757 
   clients/src/main/java/org/apache/kafka/common/PartitionInfo.java 
 b15aa2c3ef2d7c4b24618ff42fd4da324237a813 
   clients/src/main/java/org/apache/kafka/common/config/ConfigDef.java 
 98cb79b701918eca3f6ca9823b6c7b7c97b3ecec 
   clients/src/main/java/org/apache/kafka/common/errors/ApiException.java 
 7c948b166a8ac07616809f260754116ae7764973 
   clients/src/main/java/org/apache/kafka/common/network/Selectable.java 
 b68bbf00ab8eba6c5867d346c91188142593ca6e 
   clients/src/main/java/org/apache/kafka/common/network/Selector.java 
 74d695ba39de44b6a3d15340ec0114bc4fce2ba2

Re: Review Request 27799: New consumer

2015-01-27 Thread Onur Karaman 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/27799/#review69737
---


minor nitpicks


clients/src/main/java/org/apache/kafka/clients/consumer/ConsumerConfig.java
https://reviews.apache.org/r/27799/#comment114499

You left a blah.



clients/src/main/java/org/apache/kafka/clients/consumer/ConsumerConfig.java
https://reviews.apache.org/r/27799/#comment114498

RECONNECT_BACKOFF_MS_DOC



clients/src/main/java/org/apache/kafka/clients/consumer/KafkaConsumer.java
https://reviews.apache.org/r/27799/#comment114578

Was this switch to html codes intentional?



clients/src/main/java/org/apache/kafka/clients/consumer/KafkaConsumer.java
https://reviews.apache.org/r/27799/#comment114579

Should this be time.milliseconds() ?


- Onur Karaman


On Jan. 23, 2015, 9:15 p.m., Jay Kreps wrote:
 
 ---
 This is an automatically generated e-mail. To reply, visit:
 https://reviews.apache.org/r/27799/
 ---
 
 (Updated Jan. 23, 2015, 9:15 p.m.)
 
 
 Review request for kafka.
 
 
 Bugs: KAFKA-1760
 https://issues.apache.org/jira/browse/KAFKA-1760
 
 
 Repository: kafka
 
 
 Description
 ---
 
 New consumer.
 
 
 Diffs
 -
 
   build.gradle 1cbab29ce83e20dae0561b51eed6fdb86d522f28 
   clients/src/main/java/org/apache/kafka/clients/ClientRequest.java 
 d32c319d8ee4c46dad309ea54b136ea9798e2fd7 
   clients/src/main/java/org/apache/kafka/clients/ClusterConnectionStates.java 
 8aece7e81a804b177a6f2c12e2dc6c89c1613262 
   clients/src/main/java/org/apache/kafka/clients/CommonClientConfigs.java 
 PRE-CREATION 
   clients/src/main/java/org/apache/kafka/clients/ConnectionState.java 
 ab7e3220f9b76b92ef981d695299656f041ad5ed 
   clients/src/main/java/org/apache/kafka/clients/KafkaClient.java 
 397695568d3fd8e835d8f923a89b3b00c96d0ead 
   clients/src/main/java/org/apache/kafka/clients/NetworkClient.java 
 6746275d0b2596cd6ff7ce464a3a8225ad75ef00 
   clients/src/main/java/org/apache/kafka/clients/NodeConnectionState.java 
 752a979ea0b8bde7ff6d2e1a23bf54052305d841 
   clients/src/main/java/org/apache/kafka/clients/consumer/CommitType.java 
 PRE-CREATION 
   clients/src/main/java/org/apache/kafka/clients/consumer/Consumer.java 
 c0c636b3e1ba213033db6d23655032c9bbd5e378 
   clients/src/main/java/org/apache/kafka/clients/consumer/ConsumerConfig.java 
 57c1807ccba9f264186f83e91f37c34b959c8060 
   
 clients/src/main/java/org/apache/kafka/clients/consumer/ConsumerRebalanceCallback.java
  e4cf7d1cfa01c2844b53213a7b539cdcbcbeaa3a 
   clients/src/main/java/org/apache/kafka/clients/consumer/ConsumerRecord.java 
 16af70a5de52cca786fdea147a6a639b7dc4a311 
   
 clients/src/main/java/org/apache/kafka/clients/consumer/ConsumerRecords.java 
 bdf4b26942d5a8c8a9503e05908e9a9eff6228a7 
   clients/src/main/java/org/apache/kafka/clients/consumer/KafkaConsumer.java 
 76efc216c9e6c3ab084461d792877092a189ad0f 
   clients/src/main/java/org/apache/kafka/clients/consumer/MockConsumer.java 
 fa88ac1a8b19b4294f211c4467fe68c7707ddbae 
   
 clients/src/main/java/org/apache/kafka/clients/consumer/NoOffsetForPartitionException.java
  PRE-CREATION 
   clients/src/main/java/org/apache/kafka/clients/consumer/OffsetMetadata.java 
 ea423ad15eebd262d20d5ec05d592cc115229177 
   
 clients/src/main/java/org/apache/kafka/clients/consumer/internals/Heartbeat.java
  PRE-CREATION 
   
 clients/src/main/java/org/apache/kafka/clients/consumer/internals/NoOpConsumerRebalanceCallback.java
  PRE-CREATION 
   
 clients/src/main/java/org/apache/kafka/clients/consumer/internals/SubscriptionState.java
  PRE-CREATION 
   clients/src/main/java/org/apache/kafka/clients/producer/KafkaProducer.java 
 fc71710dd5997576d3841a1c3b0f7e19a8c9698e 
   clients/src/main/java/org/apache/kafka/clients/producer/MockProducer.java 
 904976fadf0610982958628eaee810b60a98d725 
   clients/src/main/java/org/apache/kafka/clients/producer/ProducerConfig.java 
 8b3e565edd1ae04d8d34bd9f1a41e9fa8c880a75 
   
 clients/src/main/java/org/apache/kafka/clients/producer/internals/Metadata.java
  dcf46581b912cfb1b5c8d4cbc293d2d1444b7740 
   
 clients/src/main/java/org/apache/kafka/clients/producer/internals/Partitioner.java
  483899d2e69b33655d0e08949f5f64af2519660a 
   
 clients/src/main/java/org/apache/kafka/clients/producer/internals/Sender.java 
 ccc03d8447ebba40131a70e16969686ac4aab58a 
   clients/src/main/java/org/apache/kafka/common/Cluster.java 
 d3299b944062d96852452de455902659ad8af757 
   clients/src/main/java/org/apache/kafka/common/PartitionInfo.java 
 b15aa2c3ef2d7c4b24618ff42fd4da324237a813 
   clients/src/main/java/org/apache/kafka/common/config/ConfigDef.java 
 98cb79b701918eca3f6ca9823b6c7b7c97b3ecec 
   clients/src/main/java/org/apache/kafka/common/errors/ApiException.java 
 7c948b166a8ac07616809f260754116ae7764973

Re: Review Request 27799: New consumer

2015-01-23 Thread Guozhang Wang 


 On Jan. 22, 2015, 7:10 p.m., Guozhang Wang wrote:
  core/src/test/scala/integration/kafka/api/ConsumerTest.scala, line 64
  https://reviews.apache.org/r/27799/diff/5/?file=830064#file830064line64
 
  Shall we add the @Test label just in case?
 
 Jay Kreps wrote:
 No I don't think so. I think the point you made in the other JIRA was 
 that @Test only works for junit 4, junit 3 requires the name test. So adding 
 Junit4 annotations to junit 3 tests which has no effect will be super 
 confusing, right?

The other JIRA is for @expected label not being honored in junit3. My point 
here is that although currently test prefix works for the current version it 
may be most safe to add the label @Test which will be honored by all versions.


 On Jan. 22, 2015, 7:10 p.m., Guozhang Wang wrote:
  core/src/test/scala/integration/kafka/api/ConsumerTest.scala, lines 158-180
  https://reviews.apache.org/r/27799/diff/5/?file=830064#file830064line158
 
  I think a more comprehensive test will be running the producer / 
  consumer in background threads while the main thread will just iterate over 
  killing / restarting brokers, as with this we are assured at least enough 
  iterations will be executed before all produced records get consumed.
 
 Jay Kreps wrote:
 Hmm, not sure I agree. Those threaded tests are insane to read and debug. 
 I really really think we need to focus on maintainability/debugability of 
 tests.
 
 I actually think this catches all the cases, right? You don't actually 
 care that the disconnect and execution happens in parallel or not.
 
 Also not sure that helps the number of iterations which is really 
 determined by the number of messages. That is parameterized already.

My past experience in ProducerFailureHandling test cases is that some bugs may 
be hidden when we kill brokers synchronously within the iteration, but maybe 
there are no such cases in consumer tests as it is single threaded and the 
connection will only be checked upon each poll().


 On Jan. 22, 2015, 7:10 p.m., Guozhang Wang wrote:
  core/src/test/scala/integration/kafka/api/ConsumerTest.scala, lines 268-270
  https://reviews.apache.org/r/27799/diff/5/?file=830064#file830064line268
 
  If there is not enough records then this will be blocked forever, so 
  shall we add a timeout config and fail the test if timeout is reached?
 
 Jay Kreps wrote:
 Yeah but I mean we have this all over where the failure case is to hang, 
 right? It isn't ideal but the fix tends to be pretty convoluted, and it is 
 always clear the problem since the test that hangs is the failing test.

The problem is that when we do ./gradlew test, only the finished (passed or 
failed) tests will print on stdout, and hence when it hanges we will only know 
which test class is problematic but cannot nail down to the specific test case 
unless we add -i. This process has been very cumbersome for at least myself in 
the past, and I think it is better we fail the test clearly instead of let it 
hang for detection.


- Guozhang


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/27799/#review68947
---


On Jan. 23, 2015, 4:22 a.m., Jay Kreps wrote:
 
 ---
 This is an automatically generated e-mail. To reply, visit:
 https://reviews.apache.org/r/27799/
 ---
 
 (Updated Jan. 23, 2015, 4:22 a.m.)
 
 
 Review request for kafka.
 
 
 Bugs: KAFKA-1760
 https://issues.apache.org/jira/browse/KAFKA-1760
 
 
 Repository: kafka
 
 
 Description
 ---
 
 New consumer.
 
 
 Diffs
 -
 
   build.gradle 1cbab29ce83e20dae0561b51eed6fdb86d522f28 
   clients/src/main/java/org/apache/kafka/clients/ClientRequest.java 
 d32c319d8ee4c46dad309ea54b136ea9798e2fd7 
   clients/src/main/java/org/apache/kafka/clients/ClusterConnectionStates.java 
 8aece7e81a804b177a6f2c12e2dc6c89c1613262 
   clients/src/main/java/org/apache/kafka/clients/CommonClientConfigs.java 
 PRE-CREATION 
   clients/src/main/java/org/apache/kafka/clients/ConnectionState.java 
 ab7e3220f9b76b92ef981d695299656f041ad5ed 
   clients/src/main/java/org/apache/kafka/clients/KafkaClient.java 
 397695568d3fd8e835d8f923a89b3b00c96d0ead 
   clients/src/main/java/org/apache/kafka/clients/NetworkClient.java 
 6746275d0b2596cd6ff7ce464a3a8225ad75ef00 
   
 clients/src/main/java/org/apache/kafka/clients/RequestCompletionHandler.java 
 PRE-CREATION 
   clients/src/main/java/org/apache/kafka/clients/consumer/CommitType.java 
 PRE-CREATION 
   clients/src/main/java/org/apache/kafka/clients/consumer/Consumer.java 
 c0c636b3e1ba213033db6d23655032c9bbd5e378 
   clients/src/main/java/org/apache/kafka/clients/consumer/ConsumerConfig.java 
 57c1807ccba9f264186f83e91f37c34b959c8060

Re: Review Request 27799: New consumer

2015-01-23 Thread Jay Kreps 


 On Jan. 23, 2015, 8:57 a.m., Ewen Cheslack-Postava wrote:
  clients/src/main/java/org/apache/kafka/clients/NetworkClient.java, line 247
  https://reviews.apache.org/r/27799/diff/6/?file=831485#file831485line247
 
  I think these methods need to have timeouts on them. They get called 
  via synchronized methods in KafkaConsumer and KafkaConsumer.close() is also 
  synchronized, so an attempt to shutdown the consumer could be blocked 
  indefinitely by a call to completeAll().

Yeah I agree. I think this has the same issue the producer does where it is 
vulnerable to silent failures blocking for whatever the socket retransmit time 
or whatever it is hits. I think it would be worth turning this into a follow-up 
issue though as there are actually a number of similar problems in our infinite 
retry strategy. Likely the best thing to do is introduce a new config with a 
longish default setting and bound both these calls as well as all retry loops 
with that time in a comprehensive way. Filed KAFKA-1894.


 On Jan. 23, 2015, 8:57 a.m., Ewen Cheslack-Postava wrote:
  clients/src/main/java/org/apache/kafka/clients/consumer/KafkaConsumer.java, 
  line 777
  https://reviews.apache.org/r/27799/diff/6/?file=831493#file831493line777
 
  Does this give the right metric names? Looks different than the one in 
  KafkaProducer (which doesn't have a trailing dot.

Nice catch.


 On Jan. 23, 2015, 8:57 a.m., Ewen Cheslack-Postava wrote:
  clients/src/main/java/org/apache/kafka/clients/consumer/KafkaConsumer.java, 
  line 1036
  https://reviews.apache.org/r/27799/diff/6/?file=831493#file831493line1036
 
  Won't this cause busy looping during network failures (and maybe due to 
  other errors returned in the OffsetCommitResponse)?

I think I fixed that in one of the updates--this uses the same backoff logic on 
retry now.


 On Jan. 23, 2015, 8:57 a.m., Ewen Cheslack-Postava wrote:
  clients/src/main/java/org/apache/kafka/clients/consumer/KafkaConsumer.java, 
  line 1195
  https://reviews.apache.org/r/27799/diff/6/?file=831493#file831493line1195
 
  Timeout? Need to eventually allow things like close() requests to 
  process.

Agreed, I'm going to lump that in with the other infinite loop issues.


 On Jan. 23, 2015, 8:57 a.m., Ewen Cheslack-Postava wrote:
  clients/src/main/java/org/apache/kafka/clients/consumer/KafkaConsumer.java, 
  line 1526
  https://reviews.apache.org/r/27799/diff/6/?file=831493#file831493line1526
 
  I'm confused what's going on here -- why Integer.MIN_VALUE? And how 
  does this end up working with the rest of the code since this result node 
  is used for consumerCoordinator and other code compares node IDs?

Yeah this is actually a bit of a hack and I think with a little effort could be 
removed.

We need to maintain a seperate connection for interaction with the co-ordinator 
so that important co-ordinator requests don't queue up behind slow fetch 
requests. But currently NetworkClient is a bit dumb and just maintains a single 
connection per node id. So giving a bogus node id just ensures another 
connection. I'll file another ticket on this.


 On Jan. 23, 2015, 8:57 a.m., Ewen Cheslack-Postava wrote:
  core/src/main/scala/kafka/tools/ConsumerPerformance.scala, line 21
  https://reviews.apache.org/r/27799/diff/6/?file=831557#file831557line21
 
  Lots of unused import cleanup here.

Yeah I use Eclipse which doesn't show those for scala :-(


- Jay


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/27799/#review69355
---


On Jan. 23, 2015, 4:22 a.m., Jay Kreps wrote:
 
 ---
 This is an automatically generated e-mail. To reply, visit:
 https://reviews.apache.org/r/27799/
 ---
 
 (Updated Jan. 23, 2015, 4:22 a.m.)
 
 
 Review request for kafka.
 
 
 Bugs: KAFKA-1760
 https://issues.apache.org/jira/browse/KAFKA-1760
 
 
 Repository: kafka
 
 
 Description
 ---
 
 New consumer.
 
 
 Diffs
 -
 
   build.gradle 1cbab29ce83e20dae0561b51eed6fdb86d522f28 
   clients/src/main/java/org/apache/kafka/clients/ClientRequest.java 
 d32c319d8ee4c46dad309ea54b136ea9798e2fd7 
   clients/src/main/java/org/apache/kafka/clients/ClusterConnectionStates.java 
 8aece7e81a804b177a6f2c12e2dc6c89c1613262 
   clients/src/main/java/org/apache/kafka/clients/CommonClientConfigs.java 
 PRE-CREATION 
   clients/src/main/java/org/apache/kafka/clients/ConnectionState.java 
 ab7e3220f9b76b92ef981d695299656f041ad5ed 
   clients/src/main/java/org/apache/kafka/clients/KafkaClient.java 
 397695568d3fd8e835d8f923a89b3b00c96d0ead 
   clients/src/main/java/org/apache/kafka/clients/NetworkClient.java 
 6746275d0b2596cd6ff7ce464a3a8225ad75ef00 
   
 clients/src/main/java/org/apache/kafka/clients/RequestCompletionHandler.java 
 PRE-CREATION

Re: Review Request 27799: New consumer

2015-01-23 Thread Jay Kreps 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/27799/#review69453
---



core/src/test/scala/integration/kafka/api/ConsumerTest.scala
https://reviews.apache.org/r/27799/#comment114158

I tried this and I don't think that is correct.

@Test was introduced in junit4. Adding @Test to methods in ConsumerTest 
which extends stuff based on junit3 and changing their name to something that 
doesn't begin with testXXX causes them not to run.

So the annotations have no effect, but look like they should, which is what 
I think is confusing.

Here is what I think we could agree on: let's get off junit3 entirely, I 
think the mixture of both is error prone.



core/src/test/scala/integration/kafka/api/ConsumerTest.scala
https://reviews.apache.org/r/27799/#comment114165

Yeah it might be good to consider doing a multithreaded test with multiple 
consumers and producers once we have the group management stuff.

I guess the argument for why this would be different from the producer is 
that the consumer itself is single-threaded so there is no background work 
happening.

The real weakness I felt in these tests is that they only cover graceful 
shutdown. Actual machine death would I think expose a ton of issues. But we can 
probably do this in the system tests...



core/src/test/scala/integration/kafka/api/ConsumerTest.scala
https://reviews.apache.org/r/27799/#comment114169

Can't you just run it in the ide and immediately debug and see what is 
happening? That also takes only a few secs whereas the full test run is slow

Anyhow I'll fix this case but I think we have this issue in about 1000 
places and fixing it is quite ugly (see the fixed code I'm about to post) and 
obscures what is going on. Also fixing it with a timeout is far worse since 
then you have a timing assumption that will randomly fail.


- Jay Kreps


On Jan. 23, 2015, 4:22 a.m., Jay Kreps wrote:
 
 ---
 This is an automatically generated e-mail. To reply, visit:
 https://reviews.apache.org/r/27799/
 ---
 
 (Updated Jan. 23, 2015, 4:22 a.m.)
 
 
 Review request for kafka.
 
 
 Bugs: KAFKA-1760
 https://issues.apache.org/jira/browse/KAFKA-1760
 
 
 Repository: kafka
 
 
 Description
 ---
 
 New consumer.
 
 
 Diffs
 -
 
   build.gradle 1cbab29ce83e20dae0561b51eed6fdb86d522f28 
   clients/src/main/java/org/apache/kafka/clients/ClientRequest.java 
 d32c319d8ee4c46dad309ea54b136ea9798e2fd7 
   clients/src/main/java/org/apache/kafka/clients/ClusterConnectionStates.java 
 8aece7e81a804b177a6f2c12e2dc6c89c1613262 
   clients/src/main/java/org/apache/kafka/clients/CommonClientConfigs.java 
 PRE-CREATION 
   clients/src/main/java/org/apache/kafka/clients/ConnectionState.java 
 ab7e3220f9b76b92ef981d695299656f041ad5ed 
   clients/src/main/java/org/apache/kafka/clients/KafkaClient.java 
 397695568d3fd8e835d8f923a89b3b00c96d0ead 
   clients/src/main/java/org/apache/kafka/clients/NetworkClient.java 
 6746275d0b2596cd6ff7ce464a3a8225ad75ef00 
   
 clients/src/main/java/org/apache/kafka/clients/RequestCompletionHandler.java 
 PRE-CREATION 
   clients/src/main/java/org/apache/kafka/clients/consumer/CommitType.java 
 PRE-CREATION 
   clients/src/main/java/org/apache/kafka/clients/consumer/Consumer.java 
 c0c636b3e1ba213033db6d23655032c9bbd5e378 
   clients/src/main/java/org/apache/kafka/clients/consumer/ConsumerConfig.java 
 57c1807ccba9f264186f83e91f37c34b959c8060 
   
 clients/src/main/java/org/apache/kafka/clients/consumer/ConsumerRebalanceCallback.java
  e4cf7d1cfa01c2844b53213a7b539cdcbcbeaa3a 
   clients/src/main/java/org/apache/kafka/clients/consumer/ConsumerRecord.java 
 16af70a5de52cca786fdea147a6a639b7dc4a311 
   
 clients/src/main/java/org/apache/kafka/clients/consumer/ConsumerRecords.java 
 bdf4b26942d5a8c8a9503e05908e9a9eff6228a7 
   clients/src/main/java/org/apache/kafka/clients/consumer/KafkaConsumer.java 
 76efc216c9e6c3ab084461d792877092a189ad0f 
   clients/src/main/java/org/apache/kafka/clients/consumer/MockConsumer.java 
 fa88ac1a8b19b4294f211c4467fe68c7707ddbae 
   
 clients/src/main/java/org/apache/kafka/clients/consumer/NoOffsetForPartitionException.java
  PRE-CREATION 
   clients/src/main/java/org/apache/kafka/clients/consumer/OffsetMetadata.java 
 ea423ad15eebd262d20d5ec05d592cc115229177 
   
 clients/src/main/java/org/apache/kafka/clients/consumer/internals/Heartbeat.java
  PRE-CREATION 
   
 clients/src/main/java/org/apache/kafka/clients/consumer/internals/NoOpConsumerRebalanceCallback.java
  PRE-CREATION 
   
 clients/src/main/java/org/apache/kafka/clients/consumer/internals/SubscriptionState.java
  PRE-CREATION 
   clients/src/main/java/org/apache/kafka/clients/producer/KafkaProducer.java

Re: Review Request 27799: New consumer

2015-01-23 Thread Jay Kreps 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/27799/
---

(Updated Jan. 23, 2015, 9:15 p.m.)


Review request for kafka.


Bugs: KAFKA-1760
https://issues.apache.org/jira/browse/KAFKA-1760


Repository: kafka


Description (updated)
---

New consumer.

Addressed comments from Ewen and Guozhang. Rebased.


Diffs
-

  build.gradle 1cbab29ce83e20dae0561b51eed6fdb86d522f28 
  clients/src/main/java/org/apache/kafka/clients/ClientRequest.java 
d32c319d8ee4c46dad309ea54b136ea9798e2fd7 
  clients/src/main/java/org/apache/kafka/clients/ClusterConnectionStates.java 
8aece7e81a804b177a6f2c12e2dc6c89c1613262 
  clients/src/main/java/org/apache/kafka/clients/CommonClientConfigs.java 
PRE-CREATION 
  clients/src/main/java/org/apache/kafka/clients/ConnectionState.java 
ab7e3220f9b76b92ef981d695299656f041ad5ed 
  clients/src/main/java/org/apache/kafka/clients/KafkaClient.java 
397695568d3fd8e835d8f923a89b3b00c96d0ead 
  clients/src/main/java/org/apache/kafka/clients/NetworkClient.java 
6746275d0b2596cd6ff7ce464a3a8225ad75ef00 
  clients/src/main/java/org/apache/kafka/clients/NodeConnectionState.java 
752a979ea0b8bde7ff6d2e1a23bf54052305d841 
  clients/src/main/java/org/apache/kafka/clients/consumer/CommitType.java 
PRE-CREATION 
  clients/src/main/java/org/apache/kafka/clients/consumer/Consumer.java 
c0c636b3e1ba213033db6d23655032c9bbd5e378 
  clients/src/main/java/org/apache/kafka/clients/consumer/ConsumerConfig.java 
57c1807ccba9f264186f83e91f37c34b959c8060 
  
clients/src/main/java/org/apache/kafka/clients/consumer/ConsumerRebalanceCallback.java
 e4cf7d1cfa01c2844b53213a7b539cdcbcbeaa3a 
  clients/src/main/java/org/apache/kafka/clients/consumer/ConsumerRecord.java 
16af70a5de52cca786fdea147a6a639b7dc4a311 
  clients/src/main/java/org/apache/kafka/clients/consumer/ConsumerRecords.java 
bdf4b26942d5a8c8a9503e05908e9a9eff6228a7 
  clients/src/main/java/org/apache/kafka/clients/consumer/KafkaConsumer.java 
76efc216c9e6c3ab084461d792877092a189ad0f 
  clients/src/main/java/org/apache/kafka/clients/consumer/MockConsumer.java 
fa88ac1a8b19b4294f211c4467fe68c7707ddbae 
  
clients/src/main/java/org/apache/kafka/clients/consumer/NoOffsetForPartitionException.java
 PRE-CREATION 
  clients/src/main/java/org/apache/kafka/clients/consumer/OffsetMetadata.java 
ea423ad15eebd262d20d5ec05d592cc115229177 
  
clients/src/main/java/org/apache/kafka/clients/consumer/internals/Heartbeat.java
 PRE-CREATION 
  
clients/src/main/java/org/apache/kafka/clients/consumer/internals/NoOpConsumerRebalanceCallback.java
 PRE-CREATION 
  
clients/src/main/java/org/apache/kafka/clients/consumer/internals/SubscriptionState.java
 PRE-CREATION 
  clients/src/main/java/org/apache/kafka/clients/producer/KafkaProducer.java 
fc71710dd5997576d3841a1c3b0f7e19a8c9698e 
  clients/src/main/java/org/apache/kafka/clients/producer/MockProducer.java 
904976fadf0610982958628eaee810b60a98d725 
  clients/src/main/java/org/apache/kafka/clients/producer/ProducerConfig.java 
8b3e565edd1ae04d8d34bd9f1a41e9fa8c880a75 
  
clients/src/main/java/org/apache/kafka/clients/producer/internals/Metadata.java 
dcf46581b912cfb1b5c8d4cbc293d2d1444b7740 
  
clients/src/main/java/org/apache/kafka/clients/producer/internals/Partitioner.java
 483899d2e69b33655d0e08949f5f64af2519660a 
  clients/src/main/java/org/apache/kafka/clients/producer/internals/Sender.java 
ccc03d8447ebba40131a70e16969686ac4aab58a 
  clients/src/main/java/org/apache/kafka/common/Cluster.java 
d3299b944062d96852452de455902659ad8af757 
  clients/src/main/java/org/apache/kafka/common/PartitionInfo.java 
b15aa2c3ef2d7c4b24618ff42fd4da324237a813 
  clients/src/main/java/org/apache/kafka/common/config/ConfigDef.java 
98cb79b701918eca3f6ca9823b6c7b7c97b3ecec 
  clients/src/main/java/org/apache/kafka/common/errors/ApiException.java 
7c948b166a8ac07616809f260754116ae7764973 
  clients/src/main/java/org/apache/kafka/common/network/Selectable.java 
b68bbf00ab8eba6c5867d346c91188142593ca6e 
  clients/src/main/java/org/apache/kafka/common/network/Selector.java 
74d695ba39de44b6a3d15340ec0114bc4fce2ba2 
  clients/src/main/java/org/apache/kafka/common/protocol/Errors.java 
3316b6a1098311b8603a4a5893bf57b75d2e43cb 
  clients/src/main/java/org/apache/kafka/common/protocol/types/Struct.java 
121e880a941fcd3e6392859edba11a94236494cc 
  clients/src/main/java/org/apache/kafka/common/record/LogEntry.java 
e4d688cbe0c61b74ea15fc8dd3b634f9e5ee9b83 
  clients/src/main/java/org/apache/kafka/common/record/MemoryRecords.java 
040e5b91005edb8f015afdfa76fd94e0bf3cb4ca 
  
clients/src/main/java/org/apache/kafka/common/requests/ConsumerMetadataRequest.java
 99b52c23d639df010bf2affc0f79d1c6e16ed67c 
  
clients/src/main/java/org/apache/kafka/common/requests/ConsumerMetadataResponse.java
 8b8f591c4b2802a9cbbe34746c0b3ca4a64a8681

Re: Review Request 27799: New consumer

2015-01-23 Thread Jay Kreps 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/27799/
---

(Updated Jan. 23, 2015, 9:15 p.m.)


Review request for kafka.


Changes
---

Addressed comments from Ewen and Guozhang. Rebased.


Bugs: KAFKA-1760
https://issues.apache.org/jira/browse/KAFKA-1760


Repository: kafka


Description (updated)
---

New consumer.


Diffs
-

  build.gradle 1cbab29ce83e20dae0561b51eed6fdb86d522f28 
  clients/src/main/java/org/apache/kafka/clients/ClientRequest.java 
d32c319d8ee4c46dad309ea54b136ea9798e2fd7 
  clients/src/main/java/org/apache/kafka/clients/ClusterConnectionStates.java 
8aece7e81a804b177a6f2c12e2dc6c89c1613262 
  clients/src/main/java/org/apache/kafka/clients/CommonClientConfigs.java 
PRE-CREATION 
  clients/src/main/java/org/apache/kafka/clients/ConnectionState.java 
ab7e3220f9b76b92ef981d695299656f041ad5ed 
  clients/src/main/java/org/apache/kafka/clients/KafkaClient.java 
397695568d3fd8e835d8f923a89b3b00c96d0ead 
  clients/src/main/java/org/apache/kafka/clients/NetworkClient.java 
6746275d0b2596cd6ff7ce464a3a8225ad75ef00 
  clients/src/main/java/org/apache/kafka/clients/NodeConnectionState.java 
752a979ea0b8bde7ff6d2e1a23bf54052305d841 
  clients/src/main/java/org/apache/kafka/clients/consumer/CommitType.java 
PRE-CREATION 
  clients/src/main/java/org/apache/kafka/clients/consumer/Consumer.java 
c0c636b3e1ba213033db6d23655032c9bbd5e378 
  clients/src/main/java/org/apache/kafka/clients/consumer/ConsumerConfig.java 
57c1807ccba9f264186f83e91f37c34b959c8060 
  
clients/src/main/java/org/apache/kafka/clients/consumer/ConsumerRebalanceCallback.java
 e4cf7d1cfa01c2844b53213a7b539cdcbcbeaa3a 
  clients/src/main/java/org/apache/kafka/clients/consumer/ConsumerRecord.java 
16af70a5de52cca786fdea147a6a639b7dc4a311 
  clients/src/main/java/org/apache/kafka/clients/consumer/ConsumerRecords.java 
bdf4b26942d5a8c8a9503e05908e9a9eff6228a7 
  clients/src/main/java/org/apache/kafka/clients/consumer/KafkaConsumer.java 
76efc216c9e6c3ab084461d792877092a189ad0f 
  clients/src/main/java/org/apache/kafka/clients/consumer/MockConsumer.java 
fa88ac1a8b19b4294f211c4467fe68c7707ddbae 
  
clients/src/main/java/org/apache/kafka/clients/consumer/NoOffsetForPartitionException.java
 PRE-CREATION 
  clients/src/main/java/org/apache/kafka/clients/consumer/OffsetMetadata.java 
ea423ad15eebd262d20d5ec05d592cc115229177 
  
clients/src/main/java/org/apache/kafka/clients/consumer/internals/Heartbeat.java
 PRE-CREATION 
  
clients/src/main/java/org/apache/kafka/clients/consumer/internals/NoOpConsumerRebalanceCallback.java
 PRE-CREATION 
  
clients/src/main/java/org/apache/kafka/clients/consumer/internals/SubscriptionState.java
 PRE-CREATION 
  clients/src/main/java/org/apache/kafka/clients/producer/KafkaProducer.java 
fc71710dd5997576d3841a1c3b0f7e19a8c9698e 
  clients/src/main/java/org/apache/kafka/clients/producer/MockProducer.java 
904976fadf0610982958628eaee810b60a98d725 
  clients/src/main/java/org/apache/kafka/clients/producer/ProducerConfig.java 
8b3e565edd1ae04d8d34bd9f1a41e9fa8c880a75 
  
clients/src/main/java/org/apache/kafka/clients/producer/internals/Metadata.java 
dcf46581b912cfb1b5c8d4cbc293d2d1444b7740 
  
clients/src/main/java/org/apache/kafka/clients/producer/internals/Partitioner.java
 483899d2e69b33655d0e08949f5f64af2519660a 
  clients/src/main/java/org/apache/kafka/clients/producer/internals/Sender.java 
ccc03d8447ebba40131a70e16969686ac4aab58a 
  clients/src/main/java/org/apache/kafka/common/Cluster.java 
d3299b944062d96852452de455902659ad8af757 
  clients/src/main/java/org/apache/kafka/common/PartitionInfo.java 
b15aa2c3ef2d7c4b24618ff42fd4da324237a813 
  clients/src/main/java/org/apache/kafka/common/config/ConfigDef.java 
98cb79b701918eca3f6ca9823b6c7b7c97b3ecec 
  clients/src/main/java/org/apache/kafka/common/errors/ApiException.java 
7c948b166a8ac07616809f260754116ae7764973 
  clients/src/main/java/org/apache/kafka/common/network/Selectable.java 
b68bbf00ab8eba6c5867d346c91188142593ca6e 
  clients/src/main/java/org/apache/kafka/common/network/Selector.java 
74d695ba39de44b6a3d15340ec0114bc4fce2ba2 
  clients/src/main/java/org/apache/kafka/common/protocol/Errors.java 
3316b6a1098311b8603a4a5893bf57b75d2e43cb 
  clients/src/main/java/org/apache/kafka/common/protocol/types/Struct.java 
121e880a941fcd3e6392859edba11a94236494cc 
  clients/src/main/java/org/apache/kafka/common/record/LogEntry.java 
e4d688cbe0c61b74ea15fc8dd3b634f9e5ee9b83 
  clients/src/main/java/org/apache/kafka/common/record/MemoryRecords.java 
040e5b91005edb8f015afdfa76fd94e0bf3cb4ca 
  
clients/src/main/java/org/apache/kafka/common/requests/ConsumerMetadataRequest.java
 99b52c23d639df010bf2affc0f79d1c6e16ed67c 
  
clients/src/main/java/org/apache/kafka/common/requests/ConsumerMetadataResponse.java
 8b8f591c4b2802a9cbbe34746c0b3ca4a64a8681

Re: Review Request 27799: New consumer

2015-01-23 Thread Jay Kreps 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/27799/
---

(Updated Jan. 23, 2015, 9:13 p.m.)


Review request for kafka.


Bugs: KAFKA-1760
https://issues.apache.org/jira/browse/KAFKA-1760


Repository: kafka


Description
---

New consumer.


Diffs (updated)
-

  build.gradle 1cbab29ce83e20dae0561b51eed6fdb86d522f28 
  clients/src/main/java/org/apache/kafka/clients/ClientRequest.java 
d32c319d8ee4c46dad309ea54b136ea9798e2fd7 
  clients/src/main/java/org/apache/kafka/clients/ClusterConnectionStates.java 
8aece7e81a804b177a6f2c12e2dc6c89c1613262 
  clients/src/main/java/org/apache/kafka/clients/CommonClientConfigs.java 
PRE-CREATION 
  clients/src/main/java/org/apache/kafka/clients/ConnectionState.java 
ab7e3220f9b76b92ef981d695299656f041ad5ed 
  clients/src/main/java/org/apache/kafka/clients/KafkaClient.java 
397695568d3fd8e835d8f923a89b3b00c96d0ead 
  clients/src/main/java/org/apache/kafka/clients/NetworkClient.java 
6746275d0b2596cd6ff7ce464a3a8225ad75ef00 
  clients/src/main/java/org/apache/kafka/clients/NodeConnectionState.java 
752a979ea0b8bde7ff6d2e1a23bf54052305d841 
  clients/src/main/java/org/apache/kafka/clients/consumer/CommitType.java 
PRE-CREATION 
  clients/src/main/java/org/apache/kafka/clients/consumer/Consumer.java 
c0c636b3e1ba213033db6d23655032c9bbd5e378 
  clients/src/main/java/org/apache/kafka/clients/consumer/ConsumerConfig.java 
57c1807ccba9f264186f83e91f37c34b959c8060 
  
clients/src/main/java/org/apache/kafka/clients/consumer/ConsumerRebalanceCallback.java
 e4cf7d1cfa01c2844b53213a7b539cdcbcbeaa3a 
  clients/src/main/java/org/apache/kafka/clients/consumer/ConsumerRecord.java 
16af70a5de52cca786fdea147a6a639b7dc4a311 
  clients/src/main/java/org/apache/kafka/clients/consumer/ConsumerRecords.java 
bdf4b26942d5a8c8a9503e05908e9a9eff6228a7 
  clients/src/main/java/org/apache/kafka/clients/consumer/KafkaConsumer.java 
76efc216c9e6c3ab084461d792877092a189ad0f 
  clients/src/main/java/org/apache/kafka/clients/consumer/MockConsumer.java 
fa88ac1a8b19b4294f211c4467fe68c7707ddbae 
  
clients/src/main/java/org/apache/kafka/clients/consumer/NoOffsetForPartitionException.java
 PRE-CREATION 
  clients/src/main/java/org/apache/kafka/clients/consumer/OffsetMetadata.java 
ea423ad15eebd262d20d5ec05d592cc115229177 
  
clients/src/main/java/org/apache/kafka/clients/consumer/internals/Heartbeat.java
 PRE-CREATION 
  
clients/src/main/java/org/apache/kafka/clients/consumer/internals/NoOpConsumerRebalanceCallback.java
 PRE-CREATION 
  
clients/src/main/java/org/apache/kafka/clients/consumer/internals/SubscriptionState.java
 PRE-CREATION 
  clients/src/main/java/org/apache/kafka/clients/producer/KafkaProducer.java 
fc71710dd5997576d3841a1c3b0f7e19a8c9698e 
  clients/src/main/java/org/apache/kafka/clients/producer/MockProducer.java 
904976fadf0610982958628eaee810b60a98d725 
  clients/src/main/java/org/apache/kafka/clients/producer/ProducerConfig.java 
8b3e565edd1ae04d8d34bd9f1a41e9fa8c880a75 
  
clients/src/main/java/org/apache/kafka/clients/producer/internals/Metadata.java 
dcf46581b912cfb1b5c8d4cbc293d2d1444b7740 
  
clients/src/main/java/org/apache/kafka/clients/producer/internals/Partitioner.java
 483899d2e69b33655d0e08949f5f64af2519660a 
  clients/src/main/java/org/apache/kafka/clients/producer/internals/Sender.java 
ccc03d8447ebba40131a70e16969686ac4aab58a 
  clients/src/main/java/org/apache/kafka/common/Cluster.java 
d3299b944062d96852452de455902659ad8af757 
  clients/src/main/java/org/apache/kafka/common/PartitionInfo.java 
b15aa2c3ef2d7c4b24618ff42fd4da324237a813 
  clients/src/main/java/org/apache/kafka/common/config/ConfigDef.java 
98cb79b701918eca3f6ca9823b6c7b7c97b3ecec 
  clients/src/main/java/org/apache/kafka/common/errors/ApiException.java 
7c948b166a8ac07616809f260754116ae7764973 
  clients/src/main/java/org/apache/kafka/common/network/Selectable.java 
b68bbf00ab8eba6c5867d346c91188142593ca6e 
  clients/src/main/java/org/apache/kafka/common/network/Selector.java 
74d695ba39de44b6a3d15340ec0114bc4fce2ba2 
  clients/src/main/java/org/apache/kafka/common/protocol/Errors.java 
3316b6a1098311b8603a4a5893bf57b75d2e43cb 
  clients/src/main/java/org/apache/kafka/common/protocol/types/Struct.java 
121e880a941fcd3e6392859edba11a94236494cc 
  clients/src/main/java/org/apache/kafka/common/record/LogEntry.java 
e4d688cbe0c61b74ea15fc8dd3b634f9e5ee9b83 
  clients/src/main/java/org/apache/kafka/common/record/MemoryRecords.java 
040e5b91005edb8f015afdfa76fd94e0bf3cb4ca 
  
clients/src/main/java/org/apache/kafka/common/requests/ConsumerMetadataRequest.java
 99b52c23d639df010bf2affc0f79d1c6e16ed67c 
  
clients/src/main/java/org/apache/kafka/common/requests/ConsumerMetadataResponse.java
 8b8f591c4b2802a9cbbe34746c0b3ca4a64a8681 
  clients/src/main/java/org/apache/kafka/common/requests/FetchRequest.java

Re: Review Request 27799: New consumer

2015-01-23 Thread Ewen Cheslack-Postava 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/27799/#review69355
---



clients/src/main/java/org/apache/kafka/clients/ClusterConnectionStates.java
https://reviews.apache.org/r/27799/#comment114034

This should probably just return the ConnectionState since that's all it's 
used for. In fact, NodeConnectionState could be made a private nested class 
since ClusterConnectionStates is the only class that uses it.



clients/src/main/java/org/apache/kafka/clients/CommonClientConfigs.java
https://reviews.apache.org/r/27799/#comment114036

METADATA_MAX_AGE_CONFIG/DOC are still duplicated in the ProducerConfig and 
ConsumerConfig



clients/src/main/java/org/apache/kafka/clients/NetworkClient.java
https://reviews.apache.org/r/27799/#comment114037

Parameter now is not used, should be removed.



clients/src/main/java/org/apache/kafka/clients/NetworkClient.java
https://reviews.apache.org/r/27799/#comment114038

I think these methods need to have timeouts on them. They get called via 
synchronized methods in KafkaConsumer and KafkaConsumer.close() is also 
synchronized, so an attempt to shutdown the consumer could be blocked 
indefinitely by a call to completeAll().



clients/src/main/java/org/apache/kafka/clients/NetworkClient.java
https://reviews.apache.org/r/27799/#comment114039

These are just for debugging, but there's a return statement earlier where 
the selected node won't be logged.



clients/src/main/java/org/apache/kafka/clients/consumer/ConsumerRecords.java
https://reviews.apache.org/r/27799/#comment114040

Using ConcatenatedIterable here too would require less copying -- just 
build up a list of iterators rather than copying all the entries into the 
ArrayList



clients/src/main/java/org/apache/kafka/clients/consumer/KafkaConsumer.java
https://reviews.apache.org/r/27799/#comment114043

Does this give the right metric names? Looks different than the one in 
KafkaProducer (which doesn't have a trailing dot.



clients/src/main/java/org/apache/kafka/clients/consumer/KafkaConsumer.java
https://reviews.apache.org/r/27799/#comment114047

Don't need to call time.milliseconds twice here



clients/src/main/java/org/apache/kafka/clients/consumer/KafkaConsumer.java
https://reviews.apache.org/r/27799/#comment114050

Won't this cause busy looping during network failures (and maybe due to 
other errors returned in the OffsetCommitResponse)?



clients/src/main/java/org/apache/kafka/clients/consumer/KafkaConsumer.java
https://reviews.apache.org/r/27799/#comment114055

Timeout? Need to eventually allow things like close() requests to process.



clients/src/main/java/org/apache/kafka/clients/consumer/KafkaConsumer.java
https://reviews.apache.org/r/27799/#comment114063

A bunch of redundant time.milliseconds() calls in this method. A couple are 
necessary with the loops, but a lot could be removed.



clients/src/main/java/org/apache/kafka/clients/consumer/KafkaConsumer.java
https://reviews.apache.org/r/27799/#comment114062

I'm confused what's going on here -- why Integer.MIN_VALUE? And how does 
this end up working with the rest of the code since this result node is used 
for consumerCoordinator and other code compares node IDs?



core/src/main/scala/kafka/tools/ConsumerPerformance.scala
https://reviews.apache.org/r/27799/#comment114069

Lots of unused import cleanup here.


- Ewen Cheslack-Postava


On Jan. 23, 2015, 4:22 a.m., Jay Kreps wrote:
 
 ---
 This is an automatically generated e-mail. To reply, visit:
 https://reviews.apache.org/r/27799/
 ---
 
 (Updated Jan. 23, 2015, 4:22 a.m.)
 
 
 Review request for kafka.
 
 
 Bugs: KAFKA-1760
 https://issues.apache.org/jira/browse/KAFKA-1760
 
 
 Repository: kafka
 
 
 Description
 ---
 
 New consumer.
 
 
 Diffs
 -
 
   build.gradle 1cbab29ce83e20dae0561b51eed6fdb86d522f28 
   clients/src/main/java/org/apache/kafka/clients/ClientRequest.java 
 d32c319d8ee4c46dad309ea54b136ea9798e2fd7 
   clients/src/main/java/org/apache/kafka/clients/ClusterConnectionStates.java 
 8aece7e81a804b177a6f2c12e2dc6c89c1613262 
   clients/src/main/java/org/apache/kafka/clients/CommonClientConfigs.java 
 PRE-CREATION 
   clients/src/main/java/org/apache/kafka/clients/ConnectionState.java 
 ab7e3220f9b76b92ef981d695299656f041ad5ed 
   clients/src/main/java/org/apache/kafka/clients/KafkaClient.java 
 397695568d3fd8e835d8f923a89b3b00c96d0ead 
   clients/src/main/java/org/apache/kafka/clients/NetworkClient.java 
 6746275d0b2596cd6ff7ce464a3a8225ad75ef00 
   
 clients/src/main/java/org/apache/kafka/clients/RequestCompletionHandler.java 
 PRE-CREATION 
   clients/src/main/java/org/apache/kafka/clients/consumer/CommitType.java 
 PRE-CREATION

Re: Review Request 27799: New consumer

2015-01-22 Thread Jay Kreps 


 On Jan. 22, 2015, 7:10 p.m., Guozhang Wang wrote:
  clients/src/main/java/org/apache/kafka/common/network/Selector.java, line 
  198
  https://reviews.apache.org/r/27799/diff/4/?file=828402#file828402line198
 
  Just trying to understand the rationale why we want to special-care for 
  failures in send() call and leave them as disconnected state at the 
  beginning of the next poll() call?

The rationale is that the disconnection can either happen prior to the send 
call or after the send call. But how you handle these are pretty much the same. 
So it is easier to just treat them the same, otherwise you need to have the 
same handling logic in two places, once as a try/catch on send and once to 
handle the disconnection response.


 On Jan. 22, 2015, 7:10 p.m., Guozhang Wang wrote:
  clients/src/main/java/org/apache/kafka/common/utils/Utils.java, lines 
  334-336
  https://reviews.apache.org/r/27799/diff/4/?file=828426#file828426line334
 
  Newline here?

Hmm, I don't think we've been very consistent about that to date. If we want to 
get consistent let's do a clean sweep and fix them all. If I change just these 
two methods they will be inconsistent with everything else in Utils.java, and 
if I change that it will still be inconsistent with much of the other code.


 On Jan. 22, 2015, 7:10 p.m., Guozhang Wang wrote:
  clients/src/test/java/org/apache/kafka/clients/consumer/ConsumerExampleTest.java,
   lines 285-297
  https://reviews.apache.org/r/27799/diff/4/?file=828429#file828429line285
 
  Is this intentional?

No, totally forgot about that, I think these examples should just be deleted.


 On Jan. 22, 2015, 7:10 p.m., Guozhang Wang wrote:
  clients/src/test/java/org/apache/kafka/clients/consumer/internals/SubscriptionStateTest.java,
   lines 45-53
  https://reviews.apache.org/r/27799/diff/4/?file=828431#file828431line45
 
  expected usage may be fragile with junit, see KAFKA-1782 for more 
  details.

Yeah but that is just due to scala stupidity. This is java, so it should work 
fine, right?


 On Jan. 22, 2015, 7:10 p.m., Guozhang Wang wrote:
  core/src/main/scala/kafka/server/KafkaApis.scala, lines 460-461
  https://reviews.apache.org/r/27799/diff/4/?file=828445#file828445line460
 
  Just realized we might have a mis-match in our design while reading 
  this code:
  
  When consumer subscribe (topic-partition), does it still needs to send 
  a join-group request to the coordinator? If not, then we will not be able 
  to detect / exclude the case where consumers within the same group 
  subscribe to both topic-partition and topic and the coordinator will only 
  try to balance with those only subscribing to topics; if yes then the 
  join-group request needs to be modified as it only contain topics field 
  today.

Well the goal of this code was just to give back something to be able to test 
against.

But to answer your question, here is my belief about how it is supposed to work:
1. If you subscribe to topics we check validity on the server side and do all 
the fancy assignment.
2. If you subscribe to particular topic-partitions we don't check anything at 
all. It is up to you. You may have consumers for all partitions, you may not, 
we don't help in any way and there is no join group request. Basically if you 
subscribe to a partitition you are subscribed to that partition, there is 
nothing to check. I think this is actually the right thing as there are many 
possible patterns for partition-level subscription and nothing we can check 
that will be correct or helpful across them.
3. We disallow mixing of partition and topic subscriptions in the same client, 
just for implementation simplicity.

So say you have some code that subscribes to partitions and other code that 
subscribes to topics. The right behavior is that the topic subscribers all 
join the group and divide up ALL the partitions amongst themselves. The code 
subscribing to partitions gets whatever it subscribes to, duplicating the topic 
stuff. In practice this is a mistake that is a bit hard to make, I'd imagine.


 On Jan. 22, 2015, 7:10 p.m., Guozhang Wang wrote:
  core/src/main/scala/kafka/tools/ConsumerPerformance.scala, lines 60-64
  https://reviews.apache.org/r/27799/diff/5/?file=830061#file830061line60
 
  The usage of the new consumer here assumes there is only one broker and 
  hence according to the stub logic it will get all the partitions, which is 
  a bit risky.

That is why you need to implement the co-ordinator :-)


 On Jan. 22, 2015, 7:10 p.m., Guozhang Wang wrote:
  core/src/test/scala/integration/kafka/api/ConsumerTest.scala, line 64
  https://reviews.apache.org/r/27799/diff/5/?file=830064#file830064line64
 
  Shall we add the @Test label just in case?

No I don't think so. I think the point you made in the other JIRA was that 
@Test only works for junit 4, junit 3 requires the name test. So adding Junit4 
annotations to junit 3 tests which has no effect will be super

Re: Review Request 27799: New consumer

2015-01-22 Thread Jay Kreps 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/27799/
---

(Updated Jan. 23, 2015, 4:22 a.m.)


Review request for kafka.


Bugs: KAFKA-1760
https://issues.apache.org/jira/browse/KAFKA-1760


Repository: kafka


Description (updated)
---

New consumer.


Diffs (updated)
-

  build.gradle 1cbab29ce83e20dae0561b51eed6fdb86d522f28 
  clients/src/main/java/org/apache/kafka/clients/ClientRequest.java 
d32c319d8ee4c46dad309ea54b136ea9798e2fd7 
  clients/src/main/java/org/apache/kafka/clients/ClusterConnectionStates.java 
8aece7e81a804b177a6f2c12e2dc6c89c1613262 
  clients/src/main/java/org/apache/kafka/clients/CommonClientConfigs.java 
PRE-CREATION 
  clients/src/main/java/org/apache/kafka/clients/ConnectionState.java 
ab7e3220f9b76b92ef981d695299656f041ad5ed 
  clients/src/main/java/org/apache/kafka/clients/KafkaClient.java 
397695568d3fd8e835d8f923a89b3b00c96d0ead 
  clients/src/main/java/org/apache/kafka/clients/NetworkClient.java 
6746275d0b2596cd6ff7ce464a3a8225ad75ef00 
  clients/src/main/java/org/apache/kafka/clients/RequestCompletionHandler.java 
PRE-CREATION 
  clients/src/main/java/org/apache/kafka/clients/consumer/CommitType.java 
PRE-CREATION 
  clients/src/main/java/org/apache/kafka/clients/consumer/Consumer.java 
c0c636b3e1ba213033db6d23655032c9bbd5e378 
  clients/src/main/java/org/apache/kafka/clients/consumer/ConsumerConfig.java 
57c1807ccba9f264186f83e91f37c34b959c8060 
  
clients/src/main/java/org/apache/kafka/clients/consumer/ConsumerRebalanceCallback.java
 e4cf7d1cfa01c2844b53213a7b539cdcbcbeaa3a 
  clients/src/main/java/org/apache/kafka/clients/consumer/ConsumerRecord.java 
16af70a5de52cca786fdea147a6a639b7dc4a311 
  clients/src/main/java/org/apache/kafka/clients/consumer/ConsumerRecords.java 
bdf4b26942d5a8c8a9503e05908e9a9eff6228a7 
  clients/src/main/java/org/apache/kafka/clients/consumer/KafkaConsumer.java 
76efc216c9e6c3ab084461d792877092a189ad0f 
  clients/src/main/java/org/apache/kafka/clients/consumer/MockConsumer.java 
fa88ac1a8b19b4294f211c4467fe68c7707ddbae 
  
clients/src/main/java/org/apache/kafka/clients/consumer/NoOffsetForPartitionException.java
 PRE-CREATION 
  clients/src/main/java/org/apache/kafka/clients/consumer/OffsetMetadata.java 
ea423ad15eebd262d20d5ec05d592cc115229177 
  
clients/src/main/java/org/apache/kafka/clients/consumer/internals/Heartbeat.java
 PRE-CREATION 
  
clients/src/main/java/org/apache/kafka/clients/consumer/internals/NoOpConsumerRebalanceCallback.java
 PRE-CREATION 
  
clients/src/main/java/org/apache/kafka/clients/consumer/internals/SubscriptionState.java
 PRE-CREATION 
  clients/src/main/java/org/apache/kafka/clients/producer/KafkaProducer.java 
fc71710dd5997576d3841a1c3b0f7e19a8c9698e 
  clients/src/main/java/org/apache/kafka/clients/producer/MockProducer.java 
904976fadf0610982958628eaee810b60a98d725 
  clients/src/main/java/org/apache/kafka/clients/producer/ProducerConfig.java 
8b3e565edd1ae04d8d34bd9f1a41e9fa8c880a75 
  
clients/src/main/java/org/apache/kafka/clients/producer/internals/Metadata.java 
dcf46581b912cfb1b5c8d4cbc293d2d1444b7740 
  
clients/src/main/java/org/apache/kafka/clients/producer/internals/Partitioner.java
 483899d2e69b33655d0e08949f5f64af2519660a 
  clients/src/main/java/org/apache/kafka/clients/producer/internals/Sender.java 
ccc03d8447ebba40131a70e16969686ac4aab58a 
  clients/src/main/java/org/apache/kafka/common/Cluster.java 
d3299b944062d96852452de455902659ad8af757 
  clients/src/main/java/org/apache/kafka/common/PartitionInfo.java 
b15aa2c3ef2d7c4b24618ff42fd4da324237a813 
  clients/src/main/java/org/apache/kafka/common/config/ConfigDef.java 
98cb79b701918eca3f6ca9823b6c7b7c97b3ecec 
  clients/src/main/java/org/apache/kafka/common/errors/ApiException.java 
7c948b166a8ac07616809f260754116ae7764973 
  clients/src/main/java/org/apache/kafka/common/network/Selectable.java 
b68bbf00ab8eba6c5867d346c91188142593ca6e 
  clients/src/main/java/org/apache/kafka/common/network/Selector.java 
74d695ba39de44b6a3d15340ec0114bc4fce2ba2 
  clients/src/main/java/org/apache/kafka/common/protocol/Errors.java 
3316b6a1098311b8603a4a5893bf57b75d2e43cb 
  clients/src/main/java/org/apache/kafka/common/protocol/types/Struct.java 
121e880a941fcd3e6392859edba11a94236494cc 
  clients/src/main/java/org/apache/kafka/common/record/LogEntry.java 
e4d688cbe0c61b74ea15fc8dd3b634f9e5ee9b83 
  clients/src/main/java/org/apache/kafka/common/record/MemoryRecords.java 
040e5b91005edb8f015afdfa76fd94e0bf3cb4ca 
  
clients/src/main/java/org/apache/kafka/common/requests/ConsumerMetadataRequest.java
 99b52c23d639df010bf2affc0f79d1c6e16ed67c 
  
clients/src/main/java/org/apache/kafka/common/requests/ConsumerMetadataResponse.java
 8b8f591c4b2802a9cbbe34746c0b3ca4a64a8681 
  clients/src/main/java/org/apache/kafka/common/requests/FetchRequest.java 
2fc471f64f4352eeb128bbd3941779780076fb8c

Re: Review Request 27799: New consumer

2015-01-22 Thread Jay Kreps 


 On Jan. 22, 2015, 7:10 p.m., Guozhang Wang wrote:
  core/src/test/scala/unit/kafka/utils/TestUtils.scala, lines 733-743
  https://reviews.apache.org/r/27799/diff/5/?file=830071#file830071line733
 
  Is this the same as createTopic in line 172?
 
 Jay Kreps wrote:
 Yeah, weird, I swear I actually didn't add that method. The code style 
 isn't even mine (e.g. I would never never put spaces after the paren in a for 
 loop). Yet there it is. Maybe it came with the patch that added the request 
 that I built off of...dunno. Anyhow it isn't used so I'll delete it.

Nevermind, I see what it was, I moved that out of PrimativeApiTest since it was 
general purpose. But the right thing to do was delete it and just use the 
already existing methods in TestUtils as you point out. done.


- Jay


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/27799/#review68947
---


On Jan. 23, 2015, 4:22 a.m., Jay Kreps wrote:
 
 ---
 This is an automatically generated e-mail. To reply, visit:
 https://reviews.apache.org/r/27799/
 ---
 
 (Updated Jan. 23, 2015, 4:22 a.m.)
 
 
 Review request for kafka.
 
 
 Bugs: KAFKA-1760
 https://issues.apache.org/jira/browse/KAFKA-1760
 
 
 Repository: kafka
 
 
 Description
 ---
 
 New consumer.
 
 
 Diffs
 -
 
   build.gradle 1cbab29ce83e20dae0561b51eed6fdb86d522f28 
   clients/src/main/java/org/apache/kafka/clients/ClientRequest.java 
 d32c319d8ee4c46dad309ea54b136ea9798e2fd7 
   clients/src/main/java/org/apache/kafka/clients/ClusterConnectionStates.java 
 8aece7e81a804b177a6f2c12e2dc6c89c1613262 
   clients/src/main/java/org/apache/kafka/clients/CommonClientConfigs.java 
 PRE-CREATION 
   clients/src/main/java/org/apache/kafka/clients/ConnectionState.java 
 ab7e3220f9b76b92ef981d695299656f041ad5ed 
   clients/src/main/java/org/apache/kafka/clients/KafkaClient.java 
 397695568d3fd8e835d8f923a89b3b00c96d0ead 
   clients/src/main/java/org/apache/kafka/clients/NetworkClient.java 
 6746275d0b2596cd6ff7ce464a3a8225ad75ef00 
   
 clients/src/main/java/org/apache/kafka/clients/RequestCompletionHandler.java 
 PRE-CREATION 
   clients/src/main/java/org/apache/kafka/clients/consumer/CommitType.java 
 PRE-CREATION 
   clients/src/main/java/org/apache/kafka/clients/consumer/Consumer.java 
 c0c636b3e1ba213033db6d23655032c9bbd5e378 
   clients/src/main/java/org/apache/kafka/clients/consumer/ConsumerConfig.java 
 57c1807ccba9f264186f83e91f37c34b959c8060 
   
 clients/src/main/java/org/apache/kafka/clients/consumer/ConsumerRebalanceCallback.java
  e4cf7d1cfa01c2844b53213a7b539cdcbcbeaa3a 
   clients/src/main/java/org/apache/kafka/clients/consumer/ConsumerRecord.java 
 16af70a5de52cca786fdea147a6a639b7dc4a311 
   
 clients/src/main/java/org/apache/kafka/clients/consumer/ConsumerRecords.java 
 bdf4b26942d5a8c8a9503e05908e9a9eff6228a7 
   clients/src/main/java/org/apache/kafka/clients/consumer/KafkaConsumer.java 
 76efc216c9e6c3ab084461d792877092a189ad0f 
   clients/src/main/java/org/apache/kafka/clients/consumer/MockConsumer.java 
 fa88ac1a8b19b4294f211c4467fe68c7707ddbae 
   
 clients/src/main/java/org/apache/kafka/clients/consumer/NoOffsetForPartitionException.java
  PRE-CREATION 
   clients/src/main/java/org/apache/kafka/clients/consumer/OffsetMetadata.java 
 ea423ad15eebd262d20d5ec05d592cc115229177 
   
 clients/src/main/java/org/apache/kafka/clients/consumer/internals/Heartbeat.java
  PRE-CREATION 
   
 clients/src/main/java/org/apache/kafka/clients/consumer/internals/NoOpConsumerRebalanceCallback.java
  PRE-CREATION 
   
 clients/src/main/java/org/apache/kafka/clients/consumer/internals/SubscriptionState.java
  PRE-CREATION 
   clients/src/main/java/org/apache/kafka/clients/producer/KafkaProducer.java 
 fc71710dd5997576d3841a1c3b0f7e19a8c9698e 
   clients/src/main/java/org/apache/kafka/clients/producer/MockProducer.java 
 904976fadf0610982958628eaee810b60a98d725 
   clients/src/main/java/org/apache/kafka/clients/producer/ProducerConfig.java 
 8b3e565edd1ae04d8d34bd9f1a41e9fa8c880a75 
   
 clients/src/main/java/org/apache/kafka/clients/producer/internals/Metadata.java
  dcf46581b912cfb1b5c8d4cbc293d2d1444b7740 
   
 clients/src/main/java/org/apache/kafka/clients/producer/internals/Partitioner.java
  483899d2e69b33655d0e08949f5f64af2519660a 
   
 clients/src/main/java/org/apache/kafka/clients/producer/internals/Sender.java 
 ccc03d8447ebba40131a70e16969686ac4aab58a 
   clients/src/main/java/org/apache/kafka/common/Cluster.java 
 d3299b944062d96852452de455902659ad8af757 
   clients/src/main/java/org/apache/kafka/common/PartitionInfo.java 
 b15aa2c3ef2d7c4b24618ff42fd4da324237a813 
   clients/src/main/java/org/apache/kafka/common/config/ConfigDef.java 
 98cb79b701918eca3f6ca9823b6c7b7c97b3ecec

Re: Review Request 27799: New consumer

2015-01-22 Thread Guozhang Wang 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/27799/#review68947
---



clients/src/main/java/org/apache/kafka/common/network/Selector.java
https://reviews.apache.org/r/27799/#comment113486

Just trying to understand the rationale why we want to special-care for 
failures in send() call and leave them as disconnected state at the beginning 
of the next poll() call?



clients/src/main/java/org/apache/kafka/common/network/Selector.java
https://reviews.apache.org/r/27799/#comment113485

Newline for @param?



clients/src/main/java/org/apache/kafka/common/utils/Utils.java
https://reviews.apache.org/r/27799/#comment113490

Newline here?



clients/src/main/java/org/apache/kafka/common/utils/Utils.java
https://reviews.apache.org/r/27799/#comment113491

Ditto above.



clients/src/test/java/org/apache/kafka/clients/consumer/ConsumerExampleTest.java
https://reviews.apache.org/r/27799/#comment113493

Is this intentional?



clients/src/test/java/org/apache/kafka/clients/consumer/MockConsumerTest.java
https://reviews.apache.org/r/27799/#comment113497

Do we need to commit here? position() call should return 1L anyways as it 
returns the fetch position of the next message right?



clients/src/test/java/org/apache/kafka/clients/consumer/internals/SubscriptionStateTest.java
https://reviews.apache.org/r/27799/#comment113499

expected usage may be fragile with junit, see KAFKA-1782 for more details.



core/src/main/scala/kafka/cluster/Partition.scala
https://reviews.apache.org/r/27799/#comment113509

Good catch. This warning keeps pop-up at server.



core/src/main/scala/kafka/server/KafkaApis.scala
https://reviews.apache.org/r/27799/#comment113524

Just realized we might have a mis-match in our design while reading this 
code:

When consumer subscribe (topic-partition), does it still needs to send a 
join-group request to the coordinator? If not, then we will not be able to 
detect / exclude the case where consumers within the same group subscribe to 
both topic-partition and topic and the coordinator will only try to balance 
with those only subscribing to topics; if yes then the join-group request needs 
to be modified as it only contain topics field today.



core/src/main/scala/kafka/tools/ConsumerPerformance.scala
https://reviews.apache.org/r/27799/#comment113840

The usage of the new consumer here assumes there is only one broker and 
hence according to the stub logic it will get all the partitions, which is a 
bit risky.



core/src/test/scala/integration/kafka/api/ConsumerTest.scala
https://reviews.apache.org/r/27799/#comment113842

Shall we add the @Test label just in case?



core/src/test/scala/integration/kafka/api/ConsumerTest.scala
https://reviews.apache.org/r/27799/#comment113843

I think a more comprehensive test will be running the producer / consumer 
in background threads while the main thread will just iterate over killing / 
restarting brokers, as with this we are assured at least enough iterations will 
be executed before all produced records get consumed.



core/src/test/scala/integration/kafka/api/ConsumerTest.scala
https://reviews.apache.org/r/27799/#comment113844

Same here.



core/src/test/scala/integration/kafka/api/ConsumerTest.scala
https://reviews.apache.org/r/27799/#comment113846

If there is not enough records then this will be blocked forever, so shall 
we add a timeout config and fail the test if timeout is reached?



core/src/test/scala/unit/kafka/utils/TestUtils.scala
https://reviews.apache.org/r/27799/#comment113848

Is this the same as createTopic in line 172?


- Guozhang Wang


On Jan. 22, 2015, 6:06 p.m., Jay Kreps wrote:
 
 ---
 This is an automatically generated e-mail. To reply, visit:
 https://reviews.apache.org/r/27799/
 ---
 
 (Updated Jan. 22, 2015, 6:06 p.m.)
 
 
 Review request for kafka.
 
 
 Bugs: KAFKA-1760
 https://issues.apache.org/jira/browse/KAFKA-1760
 
 
 Repository: kafka
 
 
 Description
 ---
 
 New consumer.
 
 Addressed second round of comments, rebased again.
 
 
 Diffs
 -
 
   build.gradle 1cbab29ce83e20dae0561b51eed6fdb86d522f28 
   clients/src/main/java/org/apache/kafka/clients/ClientRequest.java 
 d32c319d8ee4c46dad309ea54b136ea9798e2fd7 
   clients/src/main/java/org/apache/kafka/clients/ClusterConnectionStates.java 
 8aece7e81a804b177a6f2c12e2dc6c89c1613262 
   clients/src/main/java/org/apache/kafka/clients/CommonClientConfigs.java 
 PRE-CREATION 
   clients/src/main/java/org/apache/kafka/clients/ConnectionState.java 
 ab7e3220f9b76b92ef981d695299656f041ad5ed 
   clients/src/main/java/org/apache/kafka/clients/KafkaClient.java 
 397695568d3fd8e835d8f923a89b3b00c96d0ead

Re: Review Request 27799: New consumer

2015-01-22 Thread Jay Kreps 


 On Jan. 22, 2015, 5:35 p.m., Aditya Auradkar wrote:
  clients/src/main/java/org/apache/kafka/clients/RequestCompletionHandler.java,
   line 21
  https://reviews.apache.org/r/27799/diff/4/?file=828377#file828377line21
 
  nit. Can we remove the public from the interface methods?

Can you explain...?


- Jay


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/27799/#review69204
---


On Jan. 21, 2015, 4:47 p.m., Jay Kreps wrote:
 
 ---
 This is an automatically generated e-mail. To reply, visit:
 https://reviews.apache.org/r/27799/
 ---
 
 (Updated Jan. 21, 2015, 4:47 p.m.)
 
 
 Review request for kafka.
 
 
 Bugs: KAFKA-1760
 https://issues.apache.org/jira/browse/KAFKA-1760
 
 
 Repository: kafka
 
 
 Description
 ---
 
 New consumer.
 
 Addressed the first round of comments.
 
 
 Diffs
 -
 
   build.gradle c9ac43378c3bf5443f0f47c8ba76067237ecb348 
   clients/src/main/java/org/apache/kafka/clients/ClientRequest.java 
 d32c319d8ee4c46dad309ea54b136ea9798e2fd7 
   clients/src/main/java/org/apache/kafka/clients/ClusterConnectionStates.java 
 8aece7e81a804b177a6f2c12e2dc6c89c1613262 
   clients/src/main/java/org/apache/kafka/clients/CommonClientConfigs.java 
 PRE-CREATION 
   clients/src/main/java/org/apache/kafka/clients/ConnectionState.java 
 ab7e3220f9b76b92ef981d695299656f041ad5ed 
   clients/src/main/java/org/apache/kafka/clients/KafkaClient.java 
 397695568d3fd8e835d8f923a89b3b00c96d0ead 
   clients/src/main/java/org/apache/kafka/clients/NetworkClient.java 
 6746275d0b2596cd6ff7ce464a3a8225ad75ef00 
   
 clients/src/main/java/org/apache/kafka/clients/RequestCompletionHandler.java 
 PRE-CREATION 
   clients/src/main/java/org/apache/kafka/clients/consumer/CommitType.java 
 PRE-CREATION 
   clients/src/main/java/org/apache/kafka/clients/consumer/Consumer.java 
 c0c636b3e1ba213033db6d23655032c9bbd5e378 
   clients/src/main/java/org/apache/kafka/clients/consumer/ConsumerConfig.java 
 57c1807ccba9f264186f83e91f37c34b959c8060 
   
 clients/src/main/java/org/apache/kafka/clients/consumer/ConsumerRebalanceCallback.java
  e4cf7d1cfa01c2844b53213a7b539cdcbcbeaa3a 
   clients/src/main/java/org/apache/kafka/clients/consumer/ConsumerRecord.java 
 16af70a5de52cca786fdea147a6a639b7dc4a311 
   
 clients/src/main/java/org/apache/kafka/clients/consumer/ConsumerRecords.java 
 bdf4b26942d5a8c8a9503e05908e9a9eff6228a7 
   clients/src/main/java/org/apache/kafka/clients/consumer/KafkaConsumer.java 
 76efc216c9e6c3ab084461d792877092a189ad0f 
   clients/src/main/java/org/apache/kafka/clients/consumer/MockConsumer.java 
 fa88ac1a8b19b4294f211c4467fe68c7707ddbae 
   
 clients/src/main/java/org/apache/kafka/clients/consumer/NoOffsetForPartitionException.java
  PRE-CREATION 
   clients/src/main/java/org/apache/kafka/clients/consumer/OffsetMetadata.java 
 ea423ad15eebd262d20d5ec05d592cc115229177 
   
 clients/src/main/java/org/apache/kafka/clients/consumer/internals/Heartbeat.java
  PRE-CREATION 
   
 clients/src/main/java/org/apache/kafka/clients/consumer/internals/NoOpConsumerRebalanceCallback.java
  PRE-CREATION 
   
 clients/src/main/java/org/apache/kafka/clients/consumer/internals/SubscriptionState.java
  PRE-CREATION 
   clients/src/main/java/org/apache/kafka/clients/producer/KafkaProducer.java 
 fc71710dd5997576d3841a1c3b0f7e19a8c9698e 
   clients/src/main/java/org/apache/kafka/clients/producer/MockProducer.java 
 904976fadf0610982958628eaee810b60a98d725 
   clients/src/main/java/org/apache/kafka/clients/producer/ProducerConfig.java 
 8b3e565edd1ae04d8d34bd9f1a41e9fa8c880a75 
   
 clients/src/main/java/org/apache/kafka/clients/producer/internals/Metadata.java
  dcf46581b912cfb1b5c8d4cbc293d2d1444b7740 
   
 clients/src/main/java/org/apache/kafka/clients/producer/internals/Partitioner.java
  483899d2e69b33655d0e08949f5f64af2519660a 
   
 clients/src/main/java/org/apache/kafka/clients/producer/internals/Sender.java 
 ccc03d8447ebba40131a70e16969686ac4aab58a 
   clients/src/main/java/org/apache/kafka/common/Cluster.java 
 d3299b944062d96852452de455902659ad8af757 
   clients/src/main/java/org/apache/kafka/common/PartitionInfo.java 
 b15aa2c3ef2d7c4b24618ff42fd4da324237a813 
   clients/src/main/java/org/apache/kafka/common/config/ConfigDef.java 
 98cb79b701918eca3f6ca9823b6c7b7c97b3ecec 
   clients/src/main/java/org/apache/kafka/common/errors/ApiException.java 
 7c948b166a8ac07616809f260754116ae7764973 
   clients/src/main/java/org/apache/kafka/common/network/Selectable.java 
 b68bbf00ab8eba6c5867d346c91188142593ca6e 
   clients/src/main/java/org/apache/kafka/common/network/Selector.java 
 74d695ba39de44b6a3d15340ec0114bc4fce2ba2 
   clients/src/main/java/org/apache/kafka/common/protocol/Errors.java 
 3316b6a1098311b8603a4a5893bf57b75d2e43cb

Re: Review Request 27799: New consumer

2015-01-22 Thread Jay Kreps 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/27799/
---

(Updated Jan. 22, 2015, 6:03 p.m.)


Review request for kafka.


Bugs: KAFKA-1760
https://issues.apache.org/jira/browse/KAFKA-1760


Repository: kafka


Description (updated)
---

New consumer.


Diffs (updated)
-

  build.gradle 1cbab29ce83e20dae0561b51eed6fdb86d522f28 
  clients/src/main/java/org/apache/kafka/clients/ClientRequest.java 
d32c319d8ee4c46dad309ea54b136ea9798e2fd7 
  clients/src/main/java/org/apache/kafka/clients/ClusterConnectionStates.java 
8aece7e81a804b177a6f2c12e2dc6c89c1613262 
  clients/src/main/java/org/apache/kafka/clients/CommonClientConfigs.java 
PRE-CREATION 
  clients/src/main/java/org/apache/kafka/clients/ConnectionState.java 
ab7e3220f9b76b92ef981d695299656f041ad5ed 
  clients/src/main/java/org/apache/kafka/clients/KafkaClient.java 
397695568d3fd8e835d8f923a89b3b00c96d0ead 
  clients/src/main/java/org/apache/kafka/clients/NetworkClient.java 
6746275d0b2596cd6ff7ce464a3a8225ad75ef00 
  clients/src/main/java/org/apache/kafka/clients/RequestCompletionHandler.java 
PRE-CREATION 
  clients/src/main/java/org/apache/kafka/clients/consumer/CommitType.java 
PRE-CREATION 
  clients/src/main/java/org/apache/kafka/clients/consumer/Consumer.java 
c0c636b3e1ba213033db6d23655032c9bbd5e378 
  clients/src/main/java/org/apache/kafka/clients/consumer/ConsumerConfig.java 
57c1807ccba9f264186f83e91f37c34b959c8060 
  
clients/src/main/java/org/apache/kafka/clients/consumer/ConsumerRebalanceCallback.java
 e4cf7d1cfa01c2844b53213a7b539cdcbcbeaa3a 
  clients/src/main/java/org/apache/kafka/clients/consumer/ConsumerRecord.java 
16af70a5de52cca786fdea147a6a639b7dc4a311 
  clients/src/main/java/org/apache/kafka/clients/consumer/ConsumerRecords.java 
bdf4b26942d5a8c8a9503e05908e9a9eff6228a7 
  clients/src/main/java/org/apache/kafka/clients/consumer/KafkaConsumer.java 
76efc216c9e6c3ab084461d792877092a189ad0f 
  clients/src/main/java/org/apache/kafka/clients/consumer/MockConsumer.java 
fa88ac1a8b19b4294f211c4467fe68c7707ddbae 
  
clients/src/main/java/org/apache/kafka/clients/consumer/NoOffsetForPartitionException.java
 PRE-CREATION 
  clients/src/main/java/org/apache/kafka/clients/consumer/OffsetMetadata.java 
ea423ad15eebd262d20d5ec05d592cc115229177 
  
clients/src/main/java/org/apache/kafka/clients/consumer/internals/Heartbeat.java
 PRE-CREATION 
  
clients/src/main/java/org/apache/kafka/clients/consumer/internals/NoOpConsumerRebalanceCallback.java
 PRE-CREATION 
  
clients/src/main/java/org/apache/kafka/clients/consumer/internals/SubscriptionState.java
 PRE-CREATION 
  clients/src/main/java/org/apache/kafka/clients/producer/KafkaProducer.java 
fc71710dd5997576d3841a1c3b0f7e19a8c9698e 
  clients/src/main/java/org/apache/kafka/clients/producer/MockProducer.java 
904976fadf0610982958628eaee810b60a98d725 
  clients/src/main/java/org/apache/kafka/clients/producer/ProducerConfig.java 
8b3e565edd1ae04d8d34bd9f1a41e9fa8c880a75 
  
clients/src/main/java/org/apache/kafka/clients/producer/internals/Metadata.java 
dcf46581b912cfb1b5c8d4cbc293d2d1444b7740 
  
clients/src/main/java/org/apache/kafka/clients/producer/internals/Partitioner.java
 483899d2e69b33655d0e08949f5f64af2519660a 
  clients/src/main/java/org/apache/kafka/clients/producer/internals/Sender.java 
ccc03d8447ebba40131a70e16969686ac4aab58a 
  clients/src/main/java/org/apache/kafka/common/Cluster.java 
d3299b944062d96852452de455902659ad8af757 
  clients/src/main/java/org/apache/kafka/common/PartitionInfo.java 
b15aa2c3ef2d7c4b24618ff42fd4da324237a813 
  clients/src/main/java/org/apache/kafka/common/config/ConfigDef.java 
98cb79b701918eca3f6ca9823b6c7b7c97b3ecec 
  clients/src/main/java/org/apache/kafka/common/errors/ApiException.java 
7c948b166a8ac07616809f260754116ae7764973 
  clients/src/main/java/org/apache/kafka/common/network/Selectable.java 
b68bbf00ab8eba6c5867d346c91188142593ca6e 
  clients/src/main/java/org/apache/kafka/common/network/Selector.java 
74d695ba39de44b6a3d15340ec0114bc4fce2ba2 
  clients/src/main/java/org/apache/kafka/common/protocol/Errors.java 
3316b6a1098311b8603a4a5893bf57b75d2e43cb 
  clients/src/main/java/org/apache/kafka/common/protocol/types/Struct.java 
121e880a941fcd3e6392859edba11a94236494cc 
  clients/src/main/java/org/apache/kafka/common/record/LogEntry.java 
e4d688cbe0c61b74ea15fc8dd3b634f9e5ee9b83 
  clients/src/main/java/org/apache/kafka/common/record/MemoryRecords.java 
040e5b91005edb8f015afdfa76fd94e0bf3cb4ca 
  
clients/src/main/java/org/apache/kafka/common/requests/ConsumerMetadataRequest.java
 99b52c23d639df010bf2affc0f79d1c6e16ed67c 
  
clients/src/main/java/org/apache/kafka/common/requests/ConsumerMetadataResponse.java
 8b8f591c4b2802a9cbbe34746c0b3ca4a64a8681 
  clients/src/main/java/org/apache/kafka/common/requests/FetchRequest.java 
2fc471f64f4352eeb128bbd3941779780076fb8c

Re: Review Request 27799: New consumer

2015-01-22 Thread Jay Kreps 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/27799/
---

(Updated Jan. 22, 2015, 6:06 p.m.)


Review request for kafka.


Bugs: KAFKA-1760
https://issues.apache.org/jira/browse/KAFKA-1760


Repository: kafka


Description (updated)
---

New consumer.

Addressed second round of comments, rebased again.


Diffs
-

  build.gradle 1cbab29ce83e20dae0561b51eed6fdb86d522f28 
  clients/src/main/java/org/apache/kafka/clients/ClientRequest.java 
d32c319d8ee4c46dad309ea54b136ea9798e2fd7 
  clients/src/main/java/org/apache/kafka/clients/ClusterConnectionStates.java 
8aece7e81a804b177a6f2c12e2dc6c89c1613262 
  clients/src/main/java/org/apache/kafka/clients/CommonClientConfigs.java 
PRE-CREATION 
  clients/src/main/java/org/apache/kafka/clients/ConnectionState.java 
ab7e3220f9b76b92ef981d695299656f041ad5ed 
  clients/src/main/java/org/apache/kafka/clients/KafkaClient.java 
397695568d3fd8e835d8f923a89b3b00c96d0ead 
  clients/src/main/java/org/apache/kafka/clients/NetworkClient.java 
6746275d0b2596cd6ff7ce464a3a8225ad75ef00 
  clients/src/main/java/org/apache/kafka/clients/RequestCompletionHandler.java 
PRE-CREATION 
  clients/src/main/java/org/apache/kafka/clients/consumer/CommitType.java 
PRE-CREATION 
  clients/src/main/java/org/apache/kafka/clients/consumer/Consumer.java 
c0c636b3e1ba213033db6d23655032c9bbd5e378 
  clients/src/main/java/org/apache/kafka/clients/consumer/ConsumerConfig.java 
57c1807ccba9f264186f83e91f37c34b959c8060 
  
clients/src/main/java/org/apache/kafka/clients/consumer/ConsumerRebalanceCallback.java
 e4cf7d1cfa01c2844b53213a7b539cdcbcbeaa3a 
  clients/src/main/java/org/apache/kafka/clients/consumer/ConsumerRecord.java 
16af70a5de52cca786fdea147a6a639b7dc4a311 
  clients/src/main/java/org/apache/kafka/clients/consumer/ConsumerRecords.java 
bdf4b26942d5a8c8a9503e05908e9a9eff6228a7 
  clients/src/main/java/org/apache/kafka/clients/consumer/KafkaConsumer.java 
76efc216c9e6c3ab084461d792877092a189ad0f 
  clients/src/main/java/org/apache/kafka/clients/consumer/MockConsumer.java 
fa88ac1a8b19b4294f211c4467fe68c7707ddbae 
  
clients/src/main/java/org/apache/kafka/clients/consumer/NoOffsetForPartitionException.java
 PRE-CREATION 
  clients/src/main/java/org/apache/kafka/clients/consumer/OffsetMetadata.java 
ea423ad15eebd262d20d5ec05d592cc115229177 
  
clients/src/main/java/org/apache/kafka/clients/consumer/internals/Heartbeat.java
 PRE-CREATION 
  
clients/src/main/java/org/apache/kafka/clients/consumer/internals/NoOpConsumerRebalanceCallback.java
 PRE-CREATION 
  
clients/src/main/java/org/apache/kafka/clients/consumer/internals/SubscriptionState.java
 PRE-CREATION 
  clients/src/main/java/org/apache/kafka/clients/producer/KafkaProducer.java 
fc71710dd5997576d3841a1c3b0f7e19a8c9698e 
  clients/src/main/java/org/apache/kafka/clients/producer/MockProducer.java 
904976fadf0610982958628eaee810b60a98d725 
  clients/src/main/java/org/apache/kafka/clients/producer/ProducerConfig.java 
8b3e565edd1ae04d8d34bd9f1a41e9fa8c880a75 
  
clients/src/main/java/org/apache/kafka/clients/producer/internals/Metadata.java 
dcf46581b912cfb1b5c8d4cbc293d2d1444b7740 
  
clients/src/main/java/org/apache/kafka/clients/producer/internals/Partitioner.java
 483899d2e69b33655d0e08949f5f64af2519660a 
  clients/src/main/java/org/apache/kafka/clients/producer/internals/Sender.java 
ccc03d8447ebba40131a70e16969686ac4aab58a 
  clients/src/main/java/org/apache/kafka/common/Cluster.java 
d3299b944062d96852452de455902659ad8af757 
  clients/src/main/java/org/apache/kafka/common/PartitionInfo.java 
b15aa2c3ef2d7c4b24618ff42fd4da324237a813 
  clients/src/main/java/org/apache/kafka/common/config/ConfigDef.java 
98cb79b701918eca3f6ca9823b6c7b7c97b3ecec 
  clients/src/main/java/org/apache/kafka/common/errors/ApiException.java 
7c948b166a8ac07616809f260754116ae7764973 
  clients/src/main/java/org/apache/kafka/common/network/Selectable.java 
b68bbf00ab8eba6c5867d346c91188142593ca6e 
  clients/src/main/java/org/apache/kafka/common/network/Selector.java 
74d695ba39de44b6a3d15340ec0114bc4fce2ba2 
  clients/src/main/java/org/apache/kafka/common/protocol/Errors.java 
3316b6a1098311b8603a4a5893bf57b75d2e43cb 
  clients/src/main/java/org/apache/kafka/common/protocol/types/Struct.java 
121e880a941fcd3e6392859edba11a94236494cc 
  clients/src/main/java/org/apache/kafka/common/record/LogEntry.java 
e4d688cbe0c61b74ea15fc8dd3b634f9e5ee9b83 
  clients/src/main/java/org/apache/kafka/common/record/MemoryRecords.java 
040e5b91005edb8f015afdfa76fd94e0bf3cb4ca 
  
clients/src/main/java/org/apache/kafka/common/requests/ConsumerMetadataRequest.java
 99b52c23d639df010bf2affc0f79d1c6e16ed67c 
  
clients/src/main/java/org/apache/kafka/common/requests/ConsumerMetadataResponse.java
 8b8f591c4b2802a9cbbe34746c0b3ca4a64a8681 
  clients/src/main/java/org/apache/kafka/common/requests/FetchRequest.java

Re: Review Request 27799: New consumer

2015-01-22 Thread Aditya Auradkar 


 On Jan. 22, 2015, 5:35 p.m., Aditya Auradkar wrote:
  clients/src/main/java/org/apache/kafka/clients/RequestCompletionHandler.java,
   line 21
  https://reviews.apache.org/r/27799/diff/4/?file=828377#file828377line21
 
  nit. Can we remove the public from the interface methods?
 
 Jay Kreps wrote:
 Can you explain...?

I gather all interface methods are implicitly public.. so that should be 
unnecessary.


- Aditya


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/27799/#review69204
---


On Jan. 22, 2015, 6:06 p.m., Jay Kreps wrote:
 
 ---
 This is an automatically generated e-mail. To reply, visit:
 https://reviews.apache.org/r/27799/
 ---
 
 (Updated Jan. 22, 2015, 6:06 p.m.)
 
 
 Review request for kafka.
 
 
 Bugs: KAFKA-1760
 https://issues.apache.org/jira/browse/KAFKA-1760
 
 
 Repository: kafka
 
 
 Description
 ---
 
 New consumer.
 
 Addressed second round of comments, rebased again.
 
 
 Diffs
 -
 
   build.gradle 1cbab29ce83e20dae0561b51eed6fdb86d522f28 
   clients/src/main/java/org/apache/kafka/clients/ClientRequest.java 
 d32c319d8ee4c46dad309ea54b136ea9798e2fd7 
   clients/src/main/java/org/apache/kafka/clients/ClusterConnectionStates.java 
 8aece7e81a804b177a6f2c12e2dc6c89c1613262 
   clients/src/main/java/org/apache/kafka/clients/CommonClientConfigs.java 
 PRE-CREATION 
   clients/src/main/java/org/apache/kafka/clients/ConnectionState.java 
 ab7e3220f9b76b92ef981d695299656f041ad5ed 
   clients/src/main/java/org/apache/kafka/clients/KafkaClient.java 
 397695568d3fd8e835d8f923a89b3b00c96d0ead 
   clients/src/main/java/org/apache/kafka/clients/NetworkClient.java 
 6746275d0b2596cd6ff7ce464a3a8225ad75ef00 
   
 clients/src/main/java/org/apache/kafka/clients/RequestCompletionHandler.java 
 PRE-CREATION 
   clients/src/main/java/org/apache/kafka/clients/consumer/CommitType.java 
 PRE-CREATION 
   clients/src/main/java/org/apache/kafka/clients/consumer/Consumer.java 
 c0c636b3e1ba213033db6d23655032c9bbd5e378 
   clients/src/main/java/org/apache/kafka/clients/consumer/ConsumerConfig.java 
 57c1807ccba9f264186f83e91f37c34b959c8060 
   
 clients/src/main/java/org/apache/kafka/clients/consumer/ConsumerRebalanceCallback.java
  e4cf7d1cfa01c2844b53213a7b539cdcbcbeaa3a 
   clients/src/main/java/org/apache/kafka/clients/consumer/ConsumerRecord.java 
 16af70a5de52cca786fdea147a6a639b7dc4a311 
   
 clients/src/main/java/org/apache/kafka/clients/consumer/ConsumerRecords.java 
 bdf4b26942d5a8c8a9503e05908e9a9eff6228a7 
   clients/src/main/java/org/apache/kafka/clients/consumer/KafkaConsumer.java 
 76efc216c9e6c3ab084461d792877092a189ad0f 
   clients/src/main/java/org/apache/kafka/clients/consumer/MockConsumer.java 
 fa88ac1a8b19b4294f211c4467fe68c7707ddbae 
   
 clients/src/main/java/org/apache/kafka/clients/consumer/NoOffsetForPartitionException.java
  PRE-CREATION 
   clients/src/main/java/org/apache/kafka/clients/consumer/OffsetMetadata.java 
 ea423ad15eebd262d20d5ec05d592cc115229177 
   
 clients/src/main/java/org/apache/kafka/clients/consumer/internals/Heartbeat.java
  PRE-CREATION 
   
 clients/src/main/java/org/apache/kafka/clients/consumer/internals/NoOpConsumerRebalanceCallback.java
  PRE-CREATION 
   
 clients/src/main/java/org/apache/kafka/clients/consumer/internals/SubscriptionState.java
  PRE-CREATION 
   clients/src/main/java/org/apache/kafka/clients/producer/KafkaProducer.java 
 fc71710dd5997576d3841a1c3b0f7e19a8c9698e 
   clients/src/main/java/org/apache/kafka/clients/producer/MockProducer.java 
 904976fadf0610982958628eaee810b60a98d725 
   clients/src/main/java/org/apache/kafka/clients/producer/ProducerConfig.java 
 8b3e565edd1ae04d8d34bd9f1a41e9fa8c880a75 
   
 clients/src/main/java/org/apache/kafka/clients/producer/internals/Metadata.java
  dcf46581b912cfb1b5c8d4cbc293d2d1444b7740 
   
 clients/src/main/java/org/apache/kafka/clients/producer/internals/Partitioner.java
  483899d2e69b33655d0e08949f5f64af2519660a 
   
 clients/src/main/java/org/apache/kafka/clients/producer/internals/Sender.java 
 ccc03d8447ebba40131a70e16969686ac4aab58a 
   clients/src/main/java/org/apache/kafka/common/Cluster.java 
 d3299b944062d96852452de455902659ad8af757 
   clients/src/main/java/org/apache/kafka/common/PartitionInfo.java 
 b15aa2c3ef2d7c4b24618ff42fd4da324237a813 
   clients/src/main/java/org/apache/kafka/common/config/ConfigDef.java 
 98cb79b701918eca3f6ca9823b6c7b7c97b3ecec 
   clients/src/main/java/org/apache/kafka/common/errors/ApiException.java 
 7c948b166a8ac07616809f260754116ae7764973 
   clients/src/main/java/org/apache/kafka/common/network/Selectable.java 
 b68bbf00ab8eba6c5867d346c91188142593ca6e 
   clients/src/main/java/org/apache/kafka/common/network/Selector.java

Re: Review Request 27799: New consumer

2015-01-22 Thread Aditya Auradkar 


 On Jan. 22, 2015, 5:35 p.m., Aditya Auradkar wrote:
  clients/src/main/java/org/apache/kafka/clients/RequestCompletionHandler.java,
   line 21
  https://reviews.apache.org/r/27799/diff/4/?file=828377#file828377line21
 
  nit. Can we remove the public from the interface methods?
 
 Jay Kreps wrote:
 Can you explain...?
 
 Aditya Auradkar wrote:
 I gather all interface methods are implicitly public.. so that should be 
 unnecessary.

http://docs.oracle.com/javase/specs/jls/se7/html/jls-9.html#jls-9.4


- Aditya


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/27799/#review69204
---


On Jan. 22, 2015, 6:06 p.m., Jay Kreps wrote:
 
 ---
 This is an automatically generated e-mail. To reply, visit:
 https://reviews.apache.org/r/27799/
 ---
 
 (Updated Jan. 22, 2015, 6:06 p.m.)
 
 
 Review request for kafka.
 
 
 Bugs: KAFKA-1760
 https://issues.apache.org/jira/browse/KAFKA-1760
 
 
 Repository: kafka
 
 
 Description
 ---
 
 New consumer.
 
 Addressed second round of comments, rebased again.
 
 
 Diffs
 -
 
   build.gradle 1cbab29ce83e20dae0561b51eed6fdb86d522f28 
   clients/src/main/java/org/apache/kafka/clients/ClientRequest.java 
 d32c319d8ee4c46dad309ea54b136ea9798e2fd7 
   clients/src/main/java/org/apache/kafka/clients/ClusterConnectionStates.java 
 8aece7e81a804b177a6f2c12e2dc6c89c1613262 
   clients/src/main/java/org/apache/kafka/clients/CommonClientConfigs.java 
 PRE-CREATION 
   clients/src/main/java/org/apache/kafka/clients/ConnectionState.java 
 ab7e3220f9b76b92ef981d695299656f041ad5ed 
   clients/src/main/java/org/apache/kafka/clients/KafkaClient.java 
 397695568d3fd8e835d8f923a89b3b00c96d0ead 
   clients/src/main/java/org/apache/kafka/clients/NetworkClient.java 
 6746275d0b2596cd6ff7ce464a3a8225ad75ef00 
   
 clients/src/main/java/org/apache/kafka/clients/RequestCompletionHandler.java 
 PRE-CREATION 
   clients/src/main/java/org/apache/kafka/clients/consumer/CommitType.java 
 PRE-CREATION 
   clients/src/main/java/org/apache/kafka/clients/consumer/Consumer.java 
 c0c636b3e1ba213033db6d23655032c9bbd5e378 
   clients/src/main/java/org/apache/kafka/clients/consumer/ConsumerConfig.java 
 57c1807ccba9f264186f83e91f37c34b959c8060 
   
 clients/src/main/java/org/apache/kafka/clients/consumer/ConsumerRebalanceCallback.java
  e4cf7d1cfa01c2844b53213a7b539cdcbcbeaa3a 
   clients/src/main/java/org/apache/kafka/clients/consumer/ConsumerRecord.java 
 16af70a5de52cca786fdea147a6a639b7dc4a311 
   
 clients/src/main/java/org/apache/kafka/clients/consumer/ConsumerRecords.java 
 bdf4b26942d5a8c8a9503e05908e9a9eff6228a7 
   clients/src/main/java/org/apache/kafka/clients/consumer/KafkaConsumer.java 
 76efc216c9e6c3ab084461d792877092a189ad0f 
   clients/src/main/java/org/apache/kafka/clients/consumer/MockConsumer.java 
 fa88ac1a8b19b4294f211c4467fe68c7707ddbae 
   
 clients/src/main/java/org/apache/kafka/clients/consumer/NoOffsetForPartitionException.java
  PRE-CREATION 
   clients/src/main/java/org/apache/kafka/clients/consumer/OffsetMetadata.java 
 ea423ad15eebd262d20d5ec05d592cc115229177 
   
 clients/src/main/java/org/apache/kafka/clients/consumer/internals/Heartbeat.java
  PRE-CREATION 
   
 clients/src/main/java/org/apache/kafka/clients/consumer/internals/NoOpConsumerRebalanceCallback.java
  PRE-CREATION 
   
 clients/src/main/java/org/apache/kafka/clients/consumer/internals/SubscriptionState.java
  PRE-CREATION 
   clients/src/main/java/org/apache/kafka/clients/producer/KafkaProducer.java 
 fc71710dd5997576d3841a1c3b0f7e19a8c9698e 
   clients/src/main/java/org/apache/kafka/clients/producer/MockProducer.java 
 904976fadf0610982958628eaee810b60a98d725 
   clients/src/main/java/org/apache/kafka/clients/producer/ProducerConfig.java 
 8b3e565edd1ae04d8d34bd9f1a41e9fa8c880a75 
   
 clients/src/main/java/org/apache/kafka/clients/producer/internals/Metadata.java
  dcf46581b912cfb1b5c8d4cbc293d2d1444b7740 
   
 clients/src/main/java/org/apache/kafka/clients/producer/internals/Partitioner.java
  483899d2e69b33655d0e08949f5f64af2519660a 
   
 clients/src/main/java/org/apache/kafka/clients/producer/internals/Sender.java 
 ccc03d8447ebba40131a70e16969686ac4aab58a 
   clients/src/main/java/org/apache/kafka/common/Cluster.java 
 d3299b944062d96852452de455902659ad8af757 
   clients/src/main/java/org/apache/kafka/common/PartitionInfo.java 
 b15aa2c3ef2d7c4b24618ff42fd4da324237a813 
   clients/src/main/java/org/apache/kafka/common/config/ConfigDef.java 
 98cb79b701918eca3f6ca9823b6c7b7c97b3ecec 
   clients/src/main/java/org/apache/kafka/common/errors/ApiException.java 
 7c948b166a8ac07616809f260754116ae7764973 
   clients/src/main/java/org/apache/kafka/common/network/Selectable.java 
 b68bbf00ab8eba6c5867d346c91188142593ca6e

Re: Review Request 27799: New consumer

2015-01-22 Thread Aditya Auradkar 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/27799/#review69204
---



clients/src/main/java/org/apache/kafka/clients/CommonClientConfigs.java
https://reviews.apache.org/r/27799/#comment113817

Can we remove these trailing spaces from a few files?



clients/src/main/java/org/apache/kafka/clients/RequestCompletionHandler.java
https://reviews.apache.org/r/27799/#comment113818

nit. Can we remove the public from the interface methods?


- Aditya Auradkar


On Jan. 21, 2015, 4:47 p.m., Jay Kreps wrote:
 
 ---
 This is an automatically generated e-mail. To reply, visit:
 https://reviews.apache.org/r/27799/
 ---
 
 (Updated Jan. 21, 2015, 4:47 p.m.)
 
 
 Review request for kafka.
 
 
 Bugs: KAFKA-1760
 https://issues.apache.org/jira/browse/KAFKA-1760
 
 
 Repository: kafka
 
 
 Description
 ---
 
 New consumer.
 
 Addressed the first round of comments.
 
 
 Diffs
 -
 
   build.gradle c9ac43378c3bf5443f0f47c8ba76067237ecb348 
   clients/src/main/java/org/apache/kafka/clients/ClientRequest.java 
 d32c319d8ee4c46dad309ea54b136ea9798e2fd7 
   clients/src/main/java/org/apache/kafka/clients/ClusterConnectionStates.java 
 8aece7e81a804b177a6f2c12e2dc6c89c1613262 
   clients/src/main/java/org/apache/kafka/clients/CommonClientConfigs.java 
 PRE-CREATION 
   clients/src/main/java/org/apache/kafka/clients/ConnectionState.java 
 ab7e3220f9b76b92ef981d695299656f041ad5ed 
   clients/src/main/java/org/apache/kafka/clients/KafkaClient.java 
 397695568d3fd8e835d8f923a89b3b00c96d0ead 
   clients/src/main/java/org/apache/kafka/clients/NetworkClient.java 
 6746275d0b2596cd6ff7ce464a3a8225ad75ef00 
   
 clients/src/main/java/org/apache/kafka/clients/RequestCompletionHandler.java 
 PRE-CREATION 
   clients/src/main/java/org/apache/kafka/clients/consumer/CommitType.java 
 PRE-CREATION 
   clients/src/main/java/org/apache/kafka/clients/consumer/Consumer.java 
 c0c636b3e1ba213033db6d23655032c9bbd5e378 
   clients/src/main/java/org/apache/kafka/clients/consumer/ConsumerConfig.java 
 57c1807ccba9f264186f83e91f37c34b959c8060 
   
 clients/src/main/java/org/apache/kafka/clients/consumer/ConsumerRebalanceCallback.java
  e4cf7d1cfa01c2844b53213a7b539cdcbcbeaa3a 
   clients/src/main/java/org/apache/kafka/clients/consumer/ConsumerRecord.java 
 16af70a5de52cca786fdea147a6a639b7dc4a311 
   
 clients/src/main/java/org/apache/kafka/clients/consumer/ConsumerRecords.java 
 bdf4b26942d5a8c8a9503e05908e9a9eff6228a7 
   clients/src/main/java/org/apache/kafka/clients/consumer/KafkaConsumer.java 
 76efc216c9e6c3ab084461d792877092a189ad0f 
   clients/src/main/java/org/apache/kafka/clients/consumer/MockConsumer.java 
 fa88ac1a8b19b4294f211c4467fe68c7707ddbae 
   
 clients/src/main/java/org/apache/kafka/clients/consumer/NoOffsetForPartitionException.java
  PRE-CREATION 
   clients/src/main/java/org/apache/kafka/clients/consumer/OffsetMetadata.java 
 ea423ad15eebd262d20d5ec05d592cc115229177 
   
 clients/src/main/java/org/apache/kafka/clients/consumer/internals/Heartbeat.java
  PRE-CREATION 
   
 clients/src/main/java/org/apache/kafka/clients/consumer/internals/NoOpConsumerRebalanceCallback.java
  PRE-CREATION 
   
 clients/src/main/java/org/apache/kafka/clients/consumer/internals/SubscriptionState.java
  PRE-CREATION 
   clients/src/main/java/org/apache/kafka/clients/producer/KafkaProducer.java 
 fc71710dd5997576d3841a1c3b0f7e19a8c9698e 
   clients/src/main/java/org/apache/kafka/clients/producer/MockProducer.java 
 904976fadf0610982958628eaee810b60a98d725 
   clients/src/main/java/org/apache/kafka/clients/producer/ProducerConfig.java 
 8b3e565edd1ae04d8d34bd9f1a41e9fa8c880a75 
   
 clients/src/main/java/org/apache/kafka/clients/producer/internals/Metadata.java
  dcf46581b912cfb1b5c8d4cbc293d2d1444b7740 
   
 clients/src/main/java/org/apache/kafka/clients/producer/internals/Partitioner.java
  483899d2e69b33655d0e08949f5f64af2519660a 
   
 clients/src/main/java/org/apache/kafka/clients/producer/internals/Sender.java 
 ccc03d8447ebba40131a70e16969686ac4aab58a 
   clients/src/main/java/org/apache/kafka/common/Cluster.java 
 d3299b944062d96852452de455902659ad8af757 
   clients/src/main/java/org/apache/kafka/common/PartitionInfo.java 
 b15aa2c3ef2d7c4b24618ff42fd4da324237a813 
   clients/src/main/java/org/apache/kafka/common/config/ConfigDef.java 
 98cb79b701918eca3f6ca9823b6c7b7c97b3ecec 
   clients/src/main/java/org/apache/kafka/common/errors/ApiException.java 
 7c948b166a8ac07616809f260754116ae7764973 
   clients/src/main/java/org/apache/kafka/common/network/Selectable.java 
 b68bbf00ab8eba6c5867d346c91188142593ca6e 
   clients/src/main/java/org/apache/kafka/common/network/Selector.java 
 74d695ba39de44b6a3d15340ec0114bc4fce2ba2

Re: Review Request 27799: New consumer

2015-01-21 Thread Jay Kreps 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/27799/
---

(Updated Jan. 21, 2015, 4:42 p.m.)


Review request for kafka.


Summary (updated)
-

New consumer


Bugs: KAFKA-1760
https://issues.apache.org/jira/browse/KAFKA-1760


Repository: kafka


Description
---

New consumer.


Diffs (updated)
-

  build.gradle c9ac43378c3bf5443f0f47c8ba76067237ecb348 
  clients/src/main/java/org/apache/kafka/clients/ClientRequest.java 
d32c319d8ee4c46dad309ea54b136ea9798e2fd7 
  clients/src/main/java/org/apache/kafka/clients/ClusterConnectionStates.java 
8aece7e81a804b177a6f2c12e2dc6c89c1613262 
  clients/src/main/java/org/apache/kafka/clients/CommonClientConfigs.java 
PRE-CREATION 
  clients/src/main/java/org/apache/kafka/clients/ConnectionState.java 
ab7e3220f9b76b92ef981d695299656f041ad5ed 
  clients/src/main/java/org/apache/kafka/clients/KafkaClient.java 
397695568d3fd8e835d8f923a89b3b00c96d0ead 
  clients/src/main/java/org/apache/kafka/clients/NetworkClient.java 
6746275d0b2596cd6ff7ce464a3a8225ad75ef00 
  clients/src/main/java/org/apache/kafka/clients/RequestCompletionHandler.java 
PRE-CREATION 
  clients/src/main/java/org/apache/kafka/clients/consumer/CommitType.java 
PRE-CREATION 
  clients/src/main/java/org/apache/kafka/clients/consumer/Consumer.java 
c0c636b3e1ba213033db6d23655032c9bbd5e378 
  clients/src/main/java/org/apache/kafka/clients/consumer/ConsumerConfig.java 
57c1807ccba9f264186f83e91f37c34b959c8060 
  
clients/src/main/java/org/apache/kafka/clients/consumer/ConsumerRebalanceCallback.java
 e4cf7d1cfa01c2844b53213a7b539cdcbcbeaa3a 
  clients/src/main/java/org/apache/kafka/clients/consumer/ConsumerRecord.java 
16af70a5de52cca786fdea147a6a639b7dc4a311 
  clients/src/main/java/org/apache/kafka/clients/consumer/ConsumerRecords.java 
bdf4b26942d5a8c8a9503e05908e9a9eff6228a7 
  clients/src/main/java/org/apache/kafka/clients/consumer/KafkaConsumer.java 
76efc216c9e6c3ab084461d792877092a189ad0f 
  clients/src/main/java/org/apache/kafka/clients/consumer/MockConsumer.java 
fa88ac1a8b19b4294f211c4467fe68c7707ddbae 
  
clients/src/main/java/org/apache/kafka/clients/consumer/NoOffsetForPartitionException.java
 PRE-CREATION 
  clients/src/main/java/org/apache/kafka/clients/consumer/OffsetMetadata.java 
ea423ad15eebd262d20d5ec05d592cc115229177 
  
clients/src/main/java/org/apache/kafka/clients/consumer/internals/Heartbeat.java
 PRE-CREATION 
  
clients/src/main/java/org/apache/kafka/clients/consumer/internals/NoOpConsumerRebalanceCallback.java
 PRE-CREATION 
  
clients/src/main/java/org/apache/kafka/clients/consumer/internals/SubscriptionState.java
 PRE-CREATION 
  clients/src/main/java/org/apache/kafka/clients/producer/KafkaProducer.java 
fc71710dd5997576d3841a1c3b0f7e19a8c9698e 
  clients/src/main/java/org/apache/kafka/clients/producer/MockProducer.java 
904976fadf0610982958628eaee810b60a98d725 
  clients/src/main/java/org/apache/kafka/clients/producer/ProducerConfig.java 
8b3e565edd1ae04d8d34bd9f1a41e9fa8c880a75 
  
clients/src/main/java/org/apache/kafka/clients/producer/internals/Metadata.java 
dcf46581b912cfb1b5c8d4cbc293d2d1444b7740 
  
clients/src/main/java/org/apache/kafka/clients/producer/internals/Partitioner.java
 483899d2e69b33655d0e08949f5f64af2519660a 
  clients/src/main/java/org/apache/kafka/clients/producer/internals/Sender.java 
ccc03d8447ebba40131a70e16969686ac4aab58a 
  clients/src/main/java/org/apache/kafka/common/Cluster.java 
d3299b944062d96852452de455902659ad8af757 
  clients/src/main/java/org/apache/kafka/common/PartitionInfo.java 
b15aa2c3ef2d7c4b24618ff42fd4da324237a813 
  clients/src/main/java/org/apache/kafka/common/config/ConfigDef.java 
98cb79b701918eca3f6ca9823b6c7b7c97b3ecec 
  clients/src/main/java/org/apache/kafka/common/errors/ApiException.java 
7c948b166a8ac07616809f260754116ae7764973 
  clients/src/main/java/org/apache/kafka/common/network/Selectable.java 
b68bbf00ab8eba6c5867d346c91188142593ca6e 
  clients/src/main/java/org/apache/kafka/common/network/Selector.java 
74d695ba39de44b6a3d15340ec0114bc4fce2ba2 
  clients/src/main/java/org/apache/kafka/common/protocol/Errors.java 
3316b6a1098311b8603a4a5893bf57b75d2e43cb 
  clients/src/main/java/org/apache/kafka/common/protocol/types/Struct.java 
121e880a941fcd3e6392859edba11a94236494cc 
  clients/src/main/java/org/apache/kafka/common/record/LogEntry.java 
e4d688cbe0c61b74ea15fc8dd3b634f9e5ee9b83 
  clients/src/main/java/org/apache/kafka/common/record/MemoryRecords.java 
040e5b91005edb8f015afdfa76fd94e0bf3cb4ca 
  
clients/src/main/java/org/apache/kafka/common/requests/ConsumerMetadataRequest.java
 99b52c23d639df010bf2affc0f79d1c6e16ed67c 
  
clients/src/main/java/org/apache/kafka/common/requests/ConsumerMetadataResponse.java
 8b8f591c4b2802a9cbbe34746c0b3ca4a64a8681 
  clients/src/main/java/org/apache/kafka/common/requests/FetchRequest.java

Re: Review Request 27799: New consumer

2015-01-21 Thread Jay Kreps 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/27799/
---

(Updated Jan. 21, 2015, 4:47 p.m.)


Review request for kafka.


Bugs: KAFKA-1760
https://issues.apache.org/jira/browse/KAFKA-1760


Repository: kafka


Description (updated)
---

New consumer.

Addressed the first round of comments.


Diffs
-

  build.gradle c9ac43378c3bf5443f0f47c8ba76067237ecb348 
  clients/src/main/java/org/apache/kafka/clients/ClientRequest.java 
d32c319d8ee4c46dad309ea54b136ea9798e2fd7 
  clients/src/main/java/org/apache/kafka/clients/ClusterConnectionStates.java 
8aece7e81a804b177a6f2c12e2dc6c89c1613262 
  clients/src/main/java/org/apache/kafka/clients/CommonClientConfigs.java 
PRE-CREATION 
  clients/src/main/java/org/apache/kafka/clients/ConnectionState.java 
ab7e3220f9b76b92ef981d695299656f041ad5ed 
  clients/src/main/java/org/apache/kafka/clients/KafkaClient.java 
397695568d3fd8e835d8f923a89b3b00c96d0ead 
  clients/src/main/java/org/apache/kafka/clients/NetworkClient.java 
6746275d0b2596cd6ff7ce464a3a8225ad75ef00 
  clients/src/main/java/org/apache/kafka/clients/RequestCompletionHandler.java 
PRE-CREATION 
  clients/src/main/java/org/apache/kafka/clients/consumer/CommitType.java 
PRE-CREATION 
  clients/src/main/java/org/apache/kafka/clients/consumer/Consumer.java 
c0c636b3e1ba213033db6d23655032c9bbd5e378 
  clients/src/main/java/org/apache/kafka/clients/consumer/ConsumerConfig.java 
57c1807ccba9f264186f83e91f37c34b959c8060 
  
clients/src/main/java/org/apache/kafka/clients/consumer/ConsumerRebalanceCallback.java
 e4cf7d1cfa01c2844b53213a7b539cdcbcbeaa3a 
  clients/src/main/java/org/apache/kafka/clients/consumer/ConsumerRecord.java 
16af70a5de52cca786fdea147a6a639b7dc4a311 
  clients/src/main/java/org/apache/kafka/clients/consumer/ConsumerRecords.java 
bdf4b26942d5a8c8a9503e05908e9a9eff6228a7 
  clients/src/main/java/org/apache/kafka/clients/consumer/KafkaConsumer.java 
76efc216c9e6c3ab084461d792877092a189ad0f 
  clients/src/main/java/org/apache/kafka/clients/consumer/MockConsumer.java 
fa88ac1a8b19b4294f211c4467fe68c7707ddbae 
  
clients/src/main/java/org/apache/kafka/clients/consumer/NoOffsetForPartitionException.java
 PRE-CREATION 
  clients/src/main/java/org/apache/kafka/clients/consumer/OffsetMetadata.java 
ea423ad15eebd262d20d5ec05d592cc115229177 
  
clients/src/main/java/org/apache/kafka/clients/consumer/internals/Heartbeat.java
 PRE-CREATION 
  
clients/src/main/java/org/apache/kafka/clients/consumer/internals/NoOpConsumerRebalanceCallback.java
 PRE-CREATION 
  
clients/src/main/java/org/apache/kafka/clients/consumer/internals/SubscriptionState.java
 PRE-CREATION 
  clients/src/main/java/org/apache/kafka/clients/producer/KafkaProducer.java 
fc71710dd5997576d3841a1c3b0f7e19a8c9698e 
  clients/src/main/java/org/apache/kafka/clients/producer/MockProducer.java 
904976fadf0610982958628eaee810b60a98d725 
  clients/src/main/java/org/apache/kafka/clients/producer/ProducerConfig.java 
8b3e565edd1ae04d8d34bd9f1a41e9fa8c880a75 
  
clients/src/main/java/org/apache/kafka/clients/producer/internals/Metadata.java 
dcf46581b912cfb1b5c8d4cbc293d2d1444b7740 
  
clients/src/main/java/org/apache/kafka/clients/producer/internals/Partitioner.java
 483899d2e69b33655d0e08949f5f64af2519660a 
  clients/src/main/java/org/apache/kafka/clients/producer/internals/Sender.java 
ccc03d8447ebba40131a70e16969686ac4aab58a 
  clients/src/main/java/org/apache/kafka/common/Cluster.java 
d3299b944062d96852452de455902659ad8af757 
  clients/src/main/java/org/apache/kafka/common/PartitionInfo.java 
b15aa2c3ef2d7c4b24618ff42fd4da324237a813 
  clients/src/main/java/org/apache/kafka/common/config/ConfigDef.java 
98cb79b701918eca3f6ca9823b6c7b7c97b3ecec 
  clients/src/main/java/org/apache/kafka/common/errors/ApiException.java 
7c948b166a8ac07616809f260754116ae7764973 
  clients/src/main/java/org/apache/kafka/common/network/Selectable.java 
b68bbf00ab8eba6c5867d346c91188142593ca6e 
  clients/src/main/java/org/apache/kafka/common/network/Selector.java 
74d695ba39de44b6a3d15340ec0114bc4fce2ba2 
  clients/src/main/java/org/apache/kafka/common/protocol/Errors.java 
3316b6a1098311b8603a4a5893bf57b75d2e43cb 
  clients/src/main/java/org/apache/kafka/common/protocol/types/Struct.java 
121e880a941fcd3e6392859edba11a94236494cc 
  clients/src/main/java/org/apache/kafka/common/record/LogEntry.java 
e4d688cbe0c61b74ea15fc8dd3b634f9e5ee9b83 
  clients/src/main/java/org/apache/kafka/common/record/MemoryRecords.java 
040e5b91005edb8f015afdfa76fd94e0bf3cb4ca 
  
clients/src/main/java/org/apache/kafka/common/requests/ConsumerMetadataRequest.java
 99b52c23d639df010bf2affc0f79d1c6e16ed67c 
  
clients/src/main/java/org/apache/kafka/common/requests/ConsumerMetadataResponse.java
 8b8f591c4b2802a9cbbe34746c0b3ca4a64a8681 
  clients/src/main/java/org/apache/kafka/common/requests/FetchRequest.java

Re: Review Request 27799: New consumer

2015-01-21 Thread Jaikiran Pai 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/27799/#review69121
---



clients/src/main/java/org/apache/kafka/clients/NetworkClient.java
https://reviews.apache.org/r/27799/#comment113741

Hi Jay,

I think doing this unmuteAll in a finally block might be a good idea, since 
that way we don't end up with a muted selected when/if something goes wrong 
during that polling.


- Jaikiran Pai


On Jan. 21, 2015, 4:47 p.m., Jay Kreps wrote:
 
 ---
 This is an automatically generated e-mail. To reply, visit:
 https://reviews.apache.org/r/27799/
 ---
 
 (Updated Jan. 21, 2015, 4:47 p.m.)
 
 
 Review request for kafka.
 
 
 Bugs: KAFKA-1760
 https://issues.apache.org/jira/browse/KAFKA-1760
 
 
 Repository: kafka
 
 
 Description
 ---
 
 New consumer.
 
 Addressed the first round of comments.
 
 
 Diffs
 -
 
   build.gradle c9ac43378c3bf5443f0f47c8ba76067237ecb348 
   clients/src/main/java/org/apache/kafka/clients/ClientRequest.java 
 d32c319d8ee4c46dad309ea54b136ea9798e2fd7 
   clients/src/main/java/org/apache/kafka/clients/ClusterConnectionStates.java 
 8aece7e81a804b177a6f2c12e2dc6c89c1613262 
   clients/src/main/java/org/apache/kafka/clients/CommonClientConfigs.java 
 PRE-CREATION 
   clients/src/main/java/org/apache/kafka/clients/ConnectionState.java 
 ab7e3220f9b76b92ef981d695299656f041ad5ed 
   clients/src/main/java/org/apache/kafka/clients/KafkaClient.java 
 397695568d3fd8e835d8f923a89b3b00c96d0ead 
   clients/src/main/java/org/apache/kafka/clients/NetworkClient.java 
 6746275d0b2596cd6ff7ce464a3a8225ad75ef00 
   
 clients/src/main/java/org/apache/kafka/clients/RequestCompletionHandler.java 
 PRE-CREATION 
   clients/src/main/java/org/apache/kafka/clients/consumer/CommitType.java 
 PRE-CREATION 
   clients/src/main/java/org/apache/kafka/clients/consumer/Consumer.java 
 c0c636b3e1ba213033db6d23655032c9bbd5e378 
   clients/src/main/java/org/apache/kafka/clients/consumer/ConsumerConfig.java 
 57c1807ccba9f264186f83e91f37c34b959c8060 
   
 clients/src/main/java/org/apache/kafka/clients/consumer/ConsumerRebalanceCallback.java
  e4cf7d1cfa01c2844b53213a7b539cdcbcbeaa3a 
   clients/src/main/java/org/apache/kafka/clients/consumer/ConsumerRecord.java 
 16af70a5de52cca786fdea147a6a639b7dc4a311 
   
 clients/src/main/java/org/apache/kafka/clients/consumer/ConsumerRecords.java 
 bdf4b26942d5a8c8a9503e05908e9a9eff6228a7 
   clients/src/main/java/org/apache/kafka/clients/consumer/KafkaConsumer.java 
 76efc216c9e6c3ab084461d792877092a189ad0f 
   clients/src/main/java/org/apache/kafka/clients/consumer/MockConsumer.java 
 fa88ac1a8b19b4294f211c4467fe68c7707ddbae 
   
 clients/src/main/java/org/apache/kafka/clients/consumer/NoOffsetForPartitionException.java
  PRE-CREATION 
   clients/src/main/java/org/apache/kafka/clients/consumer/OffsetMetadata.java 
 ea423ad15eebd262d20d5ec05d592cc115229177 
   
 clients/src/main/java/org/apache/kafka/clients/consumer/internals/Heartbeat.java
  PRE-CREATION 
   
 clients/src/main/java/org/apache/kafka/clients/consumer/internals/NoOpConsumerRebalanceCallback.java
  PRE-CREATION 
   
 clients/src/main/java/org/apache/kafka/clients/consumer/internals/SubscriptionState.java
  PRE-CREATION 
   clients/src/main/java/org/apache/kafka/clients/producer/KafkaProducer.java 
 fc71710dd5997576d3841a1c3b0f7e19a8c9698e 
   clients/src/main/java/org/apache/kafka/clients/producer/MockProducer.java 
 904976fadf0610982958628eaee810b60a98d725 
   clients/src/main/java/org/apache/kafka/clients/producer/ProducerConfig.java 
 8b3e565edd1ae04d8d34bd9f1a41e9fa8c880a75 
   
 clients/src/main/java/org/apache/kafka/clients/producer/internals/Metadata.java
  dcf46581b912cfb1b5c8d4cbc293d2d1444b7740 
   
 clients/src/main/java/org/apache/kafka/clients/producer/internals/Partitioner.java
  483899d2e69b33655d0e08949f5f64af2519660a 
   
 clients/src/main/java/org/apache/kafka/clients/producer/internals/Sender.java 
 ccc03d8447ebba40131a70e16969686ac4aab58a 
   clients/src/main/java/org/apache/kafka/common/Cluster.java 
 d3299b944062d96852452de455902659ad8af757 
   clients/src/main/java/org/apache/kafka/common/PartitionInfo.java 
 b15aa2c3ef2d7c4b24618ff42fd4da324237a813 
   clients/src/main/java/org/apache/kafka/common/config/ConfigDef.java 
 98cb79b701918eca3f6ca9823b6c7b7c97b3ecec 
   clients/src/main/java/org/apache/kafka/common/errors/ApiException.java 
 7c948b166a8ac07616809f260754116ae7764973 
   clients/src/main/java/org/apache/kafka/common/network/Selectable.java 
 b68bbf00ab8eba6c5867d346c91188142593ca6e 
   clients/src/main/java/org/apache/kafka/common/network/Selector.java 
 74d695ba39de44b6a3d15340ec0114bc4fce2ba2 
   clients/src/main/java/org/apache/kafka/common/protocol/Errors.java

Re: Review Request 27799: New consumer

2015-01-21 Thread Jaikiran Pai 


 On Jan. 22, 2015, 3:14 a.m., Jaikiran Pai wrote:
  clients/src/main/java/org/apache/kafka/clients/NetworkClient.java, line 253
  https://reviews.apache.org/r/27799/diff/4/?file=828376#file828376line253
 
  Hi Jay,
  
  I think doing this unmuteAll in a finally block might be a good idea, 
  since that way we don't end up with a muted selected when/if something goes 
  wrong during that polling.

Typo in my previous comment. Should have been ... since that way we don't end 
up with a muted selector


- Jaikiran


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/27799/#review69121
---


On Jan. 21, 2015, 4:47 p.m., Jay Kreps wrote:
 
 ---
 This is an automatically generated e-mail. To reply, visit:
 https://reviews.apache.org/r/27799/
 ---
 
 (Updated Jan. 21, 2015, 4:47 p.m.)
 
 
 Review request for kafka.
 
 
 Bugs: KAFKA-1760
 https://issues.apache.org/jira/browse/KAFKA-1760
 
 
 Repository: kafka
 
 
 Description
 ---
 
 New consumer.
 
 Addressed the first round of comments.
 
 
 Diffs
 -
 
   build.gradle c9ac43378c3bf5443f0f47c8ba76067237ecb348 
   clients/src/main/java/org/apache/kafka/clients/ClientRequest.java 
 d32c319d8ee4c46dad309ea54b136ea9798e2fd7 
   clients/src/main/java/org/apache/kafka/clients/ClusterConnectionStates.java 
 8aece7e81a804b177a6f2c12e2dc6c89c1613262 
   clients/src/main/java/org/apache/kafka/clients/CommonClientConfigs.java 
 PRE-CREATION 
   clients/src/main/java/org/apache/kafka/clients/ConnectionState.java 
 ab7e3220f9b76b92ef981d695299656f041ad5ed 
   clients/src/main/java/org/apache/kafka/clients/KafkaClient.java 
 397695568d3fd8e835d8f923a89b3b00c96d0ead 
   clients/src/main/java/org/apache/kafka/clients/NetworkClient.java 
 6746275d0b2596cd6ff7ce464a3a8225ad75ef00 
   
 clients/src/main/java/org/apache/kafka/clients/RequestCompletionHandler.java 
 PRE-CREATION 
   clients/src/main/java/org/apache/kafka/clients/consumer/CommitType.java 
 PRE-CREATION 
   clients/src/main/java/org/apache/kafka/clients/consumer/Consumer.java 
 c0c636b3e1ba213033db6d23655032c9bbd5e378 
   clients/src/main/java/org/apache/kafka/clients/consumer/ConsumerConfig.java 
 57c1807ccba9f264186f83e91f37c34b959c8060 
   
 clients/src/main/java/org/apache/kafka/clients/consumer/ConsumerRebalanceCallback.java
  e4cf7d1cfa01c2844b53213a7b539cdcbcbeaa3a 
   clients/src/main/java/org/apache/kafka/clients/consumer/ConsumerRecord.java 
 16af70a5de52cca786fdea147a6a639b7dc4a311 
   
 clients/src/main/java/org/apache/kafka/clients/consumer/ConsumerRecords.java 
 bdf4b26942d5a8c8a9503e05908e9a9eff6228a7 
   clients/src/main/java/org/apache/kafka/clients/consumer/KafkaConsumer.java 
 76efc216c9e6c3ab084461d792877092a189ad0f 
   clients/src/main/java/org/apache/kafka/clients/consumer/MockConsumer.java 
 fa88ac1a8b19b4294f211c4467fe68c7707ddbae 
   
 clients/src/main/java/org/apache/kafka/clients/consumer/NoOffsetForPartitionException.java
  PRE-CREATION 
   clients/src/main/java/org/apache/kafka/clients/consumer/OffsetMetadata.java 
 ea423ad15eebd262d20d5ec05d592cc115229177 
   
 clients/src/main/java/org/apache/kafka/clients/consumer/internals/Heartbeat.java
  PRE-CREATION 
   
 clients/src/main/java/org/apache/kafka/clients/consumer/internals/NoOpConsumerRebalanceCallback.java
  PRE-CREATION 
   
 clients/src/main/java/org/apache/kafka/clients/consumer/internals/SubscriptionState.java
  PRE-CREATION 
   clients/src/main/java/org/apache/kafka/clients/producer/KafkaProducer.java 
 fc71710dd5997576d3841a1c3b0f7e19a8c9698e 
   clients/src/main/java/org/apache/kafka/clients/producer/MockProducer.java 
 904976fadf0610982958628eaee810b60a98d725 
   clients/src/main/java/org/apache/kafka/clients/producer/ProducerConfig.java 
 8b3e565edd1ae04d8d34bd9f1a41e9fa8c880a75 
   
 clients/src/main/java/org/apache/kafka/clients/producer/internals/Metadata.java
  dcf46581b912cfb1b5c8d4cbc293d2d1444b7740 
   
 clients/src/main/java/org/apache/kafka/clients/producer/internals/Partitioner.java
  483899d2e69b33655d0e08949f5f64af2519660a 
   
 clients/src/main/java/org/apache/kafka/clients/producer/internals/Sender.java 
 ccc03d8447ebba40131a70e16969686ac4aab58a 
   clients/src/main/java/org/apache/kafka/common/Cluster.java 
 d3299b944062d96852452de455902659ad8af757 
   clients/src/main/java/org/apache/kafka/common/PartitionInfo.java 
 b15aa2c3ef2d7c4b24618ff42fd4da324237a813 
   clients/src/main/java/org/apache/kafka/common/config/ConfigDef.java 
 98cb79b701918eca3f6ca9823b6c7b7c97b3ecec 
   clients/src/main/java/org/apache/kafka/common/errors/ApiException.java 
 7c948b166a8ac07616809f260754116ae7764973 
   clients/src/main/java/org/apache/kafka/common/network/Selectable.java 
 b68bbf00ab8eba6c5867d346c91188142593ca6e

Re: Review Request 27799: New consumer

2015-01-21 Thread Onur Karaman 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/27799/#review69117
---



clients/src/main/java/org/apache/kafka/common/requests/ConsumerMetadataRequest.java
https://reviews.apache.org/r/27799/#comment113735

CURRENT_SCHEMA is sometimes public and sometimes private across the 
different requests / responses in this rb. Are some of these planned to be 
accessed elsewhere?



clients/src/main/java/org/apache/kafka/common/requests/MetadataResponse.java
https://reviews.apache.org/r/27799/#comment113734

Other CURRENT_SCHEMA's throughout the rb were changed to be final.


- Onur Karaman


On Jan. 21, 2015, 4:47 p.m., Jay Kreps wrote:
 
 ---
 This is an automatically generated e-mail. To reply, visit:
 https://reviews.apache.org/r/27799/
 ---
 
 (Updated Jan. 21, 2015, 4:47 p.m.)
 
 
 Review request for kafka.
 
 
 Bugs: KAFKA-1760
 https://issues.apache.org/jira/browse/KAFKA-1760
 
 
 Repository: kafka
 
 
 Description
 ---
 
 New consumer.
 
 Addressed the first round of comments.
 
 
 Diffs
 -
 
   build.gradle c9ac43378c3bf5443f0f47c8ba76067237ecb348 
   clients/src/main/java/org/apache/kafka/clients/ClientRequest.java 
 d32c319d8ee4c46dad309ea54b136ea9798e2fd7 
   clients/src/main/java/org/apache/kafka/clients/ClusterConnectionStates.java 
 8aece7e81a804b177a6f2c12e2dc6c89c1613262 
   clients/src/main/java/org/apache/kafka/clients/CommonClientConfigs.java 
 PRE-CREATION 
   clients/src/main/java/org/apache/kafka/clients/ConnectionState.java 
 ab7e3220f9b76b92ef981d695299656f041ad5ed 
   clients/src/main/java/org/apache/kafka/clients/KafkaClient.java 
 397695568d3fd8e835d8f923a89b3b00c96d0ead 
   clients/src/main/java/org/apache/kafka/clients/NetworkClient.java 
 6746275d0b2596cd6ff7ce464a3a8225ad75ef00 
   
 clients/src/main/java/org/apache/kafka/clients/RequestCompletionHandler.java 
 PRE-CREATION 
   clients/src/main/java/org/apache/kafka/clients/consumer/CommitType.java 
 PRE-CREATION 
   clients/src/main/java/org/apache/kafka/clients/consumer/Consumer.java 
 c0c636b3e1ba213033db6d23655032c9bbd5e378 
   clients/src/main/java/org/apache/kafka/clients/consumer/ConsumerConfig.java 
 57c1807ccba9f264186f83e91f37c34b959c8060 
   
 clients/src/main/java/org/apache/kafka/clients/consumer/ConsumerRebalanceCallback.java
  e4cf7d1cfa01c2844b53213a7b539cdcbcbeaa3a 
   clients/src/main/java/org/apache/kafka/clients/consumer/ConsumerRecord.java 
 16af70a5de52cca786fdea147a6a639b7dc4a311 
   
 clients/src/main/java/org/apache/kafka/clients/consumer/ConsumerRecords.java 
 bdf4b26942d5a8c8a9503e05908e9a9eff6228a7 
   clients/src/main/java/org/apache/kafka/clients/consumer/KafkaConsumer.java 
 76efc216c9e6c3ab084461d792877092a189ad0f 
   clients/src/main/java/org/apache/kafka/clients/consumer/MockConsumer.java 
 fa88ac1a8b19b4294f211c4467fe68c7707ddbae 
   
 clients/src/main/java/org/apache/kafka/clients/consumer/NoOffsetForPartitionException.java
  PRE-CREATION 
   clients/src/main/java/org/apache/kafka/clients/consumer/OffsetMetadata.java 
 ea423ad15eebd262d20d5ec05d592cc115229177 
   
 clients/src/main/java/org/apache/kafka/clients/consumer/internals/Heartbeat.java
  PRE-CREATION 
   
 clients/src/main/java/org/apache/kafka/clients/consumer/internals/NoOpConsumerRebalanceCallback.java
  PRE-CREATION 
   
 clients/src/main/java/org/apache/kafka/clients/consumer/internals/SubscriptionState.java
  PRE-CREATION 
   clients/src/main/java/org/apache/kafka/clients/producer/KafkaProducer.java 
 fc71710dd5997576d3841a1c3b0f7e19a8c9698e 
   clients/src/main/java/org/apache/kafka/clients/producer/MockProducer.java 
 904976fadf0610982958628eaee810b60a98d725 
   clients/src/main/java/org/apache/kafka/clients/producer/ProducerConfig.java 
 8b3e565edd1ae04d8d34bd9f1a41e9fa8c880a75 
   
 clients/src/main/java/org/apache/kafka/clients/producer/internals/Metadata.java
  dcf46581b912cfb1b5c8d4cbc293d2d1444b7740 
   
 clients/src/main/java/org/apache/kafka/clients/producer/internals/Partitioner.java
  483899d2e69b33655d0e08949f5f64af2519660a 
   
 clients/src/main/java/org/apache/kafka/clients/producer/internals/Sender.java 
 ccc03d8447ebba40131a70e16969686ac4aab58a 
   clients/src/main/java/org/apache/kafka/common/Cluster.java 
 d3299b944062d96852452de455902659ad8af757 
   clients/src/main/java/org/apache/kafka/common/PartitionInfo.java 
 b15aa2c3ef2d7c4b24618ff42fd4da324237a813 
   clients/src/main/java/org/apache/kafka/common/config/ConfigDef.java 
 98cb79b701918eca3f6ca9823b6c7b7c97b3ecec 
   clients/src/main/java/org/apache/kafka/common/errors/ApiException.java 
 7c948b166a8ac07616809f260754116ae7764973 
   clients/src/main/java/org/apache/kafka/common/network/Selectable.java 
 b68bbf00ab8eba6c5867d346c91188142593ca6e