Re: 0.8.1 Java Producer API Callbacks
It's beta mostly because it's new. Thanks, Jun On Thu, May 1, 2014 at 10:25 PM, cac...@gmail.com cac...@gmail.com wrote: Thanks, that's quite helpful. According to this post, http://blog.empathybox.com/ , it looks like it will be beta then which seems good enough. Assuming that the beta designation is correct, is that because it won't have as many features/the same flexibility as it's expected to by 0.9? Christian On Thu, May 1, 2014 at 9:19 PM, Neha Narkhede neha.narkh...@gmail.com wrote: The javadoc of the new producer is here for now - http://empathybox.com/kafka-javadoc/index.html?kafka/clients/producer/KafkaProducer.html Thanks, Neha On Thu, May 1, 2014 at 9:16 PM, Jun Rao jun...@gmail.com wrote: The new producer (that supports callbacks) is in trunk. It will be released in 0.8.2. You can look at the java doc of KafkaProducer for the api. Thanks Jun On Thu, May 1, 2014 at 8:43 PM, Christian Csar cac...@gmail.com wrote: On 05/01/2014 07:22 PM, Christian Csar wrote: I'm looking at using the java producer api for 0.8.1 and I'm slightly confused by this passage from section 4.4 of https://kafka.apache.org/documentation.html#theproducer Note that as of Kafka 0.8.1 the async producer does not have a callback, which could be used to register handlers to catch send errors. Adding such callback functionality is proposed for Kafka 0.9, see [Proposed Producer API]( https://cwiki.apache.org/confluence/display/KAFKA/Client+Rewrite#ClientRewrite-ProposedProducerAPI ) . org.apache.kafka.clients.producer.KafkaProducer in 0.8.1 appears to have public FutureRecordMetadata send(ProducerRecord record, Callback callback) which looks like the mentioned callback. How do the callbacks with the async producer? Is it as described in the comment on the send method (see https://github.com/apache/kafka/blob/0.8.1/clients/src/main/java/org/apache/kafka/clients/producer/KafkaProducer.java#L151 for reference)? Looking around it seems plausible the language in the documentation might refer to a separate sort of callback that existed in 0.7 but not 0.8. In our use case we have something useful to do if we can detect messages failing to be sent. Christian It appears that I was looking at the Java client rather than the Scala java api referenced by the documentation https://github.com/apache/kafka/blob/0.8.1/core/src/main/scala/kafka/javaapi/producer/Producer.scala Are both of these currently suited for use from java and still supported? Given the support for callbacks in the event of failure I am inclined to use the Java one despite the currently limited support for specifying partitioners (though it supports specifying the partition) or encoders. Any guidance on this would be appreciated. Christian
Re: 0.8.1 Java Producer API Callbacks
Hi Christian, As Jun mentioned, the new producer is marked beta since it is new. By 0.9 or even sooner, we'd expect it to be deployed at LinkedIn and a few other companies in a stable state. The APIs, guarantees and features will not change by 0.9, just the stability. Thanks, Neha On Fri, May 2, 2014 at 7:50 AM, Jun Rao jun...@gmail.com wrote: It's beta mostly because it's new. Thanks, Jun On Thu, May 1, 2014 at 10:25 PM, cac...@gmail.com cac...@gmail.com wrote: Thanks, that's quite helpful. According to this post, http://blog.empathybox.com/ , it looks like it will be beta then which seems good enough. Assuming that the beta designation is correct, is that because it won't have as many features/the same flexibility as it's expected to by 0.9? Christian On Thu, May 1, 2014 at 9:19 PM, Neha Narkhede neha.narkh...@gmail.com wrote: The javadoc of the new producer is here for now - http://empathybox.com/kafka-javadoc/index.html?kafka/clients/producer/KafkaProducer.html Thanks, Neha On Thu, May 1, 2014 at 9:16 PM, Jun Rao jun...@gmail.com wrote: The new producer (that supports callbacks) is in trunk. It will be released in 0.8.2. You can look at the java doc of KafkaProducer for the api. Thanks Jun On Thu, May 1, 2014 at 8:43 PM, Christian Csar cac...@gmail.com wrote: On 05/01/2014 07:22 PM, Christian Csar wrote: I'm looking at using the java producer api for 0.8.1 and I'm slightly confused by this passage from section 4.4 of https://kafka.apache.org/documentation.html#theproducer Note that as of Kafka 0.8.1 the async producer does not have a callback, which could be used to register handlers to catch send errors. Adding such callback functionality is proposed for Kafka 0.9, see [Proposed Producer API]( https://cwiki.apache.org/confluence/display/KAFKA/Client+Rewrite#ClientRewrite-ProposedProducerAPI ) . org.apache.kafka.clients.producer.KafkaProducer in 0.8.1 appears to have public FutureRecordMetadata send(ProducerRecord record, Callback callback) which looks like the mentioned callback. How do the callbacks with the async producer? Is it as described in the comment on the send method (see https://github.com/apache/kafka/blob/0.8.1/clients/src/main/java/org/apache/kafka/clients/producer/KafkaProducer.java#L151 for reference)? Looking around it seems plausible the language in the documentation might refer to a separate sort of callback that existed in 0.7 but not 0.8. In our use case we have something useful to do if we can detect messages failing to be sent. Christian It appears that I was looking at the Java client rather than the Scala java api referenced by the documentation https://github.com/apache/kafka/blob/0.8.1/core/src/main/scala/kafka/javaapi/producer/Producer.scala Are both of these currently suited for use from java and still supported? Given the support for callbacks in the event of failure I am inclined to use the Java one despite the currently limited support for specifying partitioners (though it supports specifying the partition) or encoders. Any guidance on this would be appreciated. Christian
Re: 0.8.1 Java Producer API Callbacks
This summary is correct. If you are just starting development now it is probably reasonable to start with the new producer. We would certainly appreciate any feedback on it. My recommendation would be to build the producer off trunk as there were a few bug fixes since 0.8.1.x that are worth getting. -Jay On Fri, May 2, 2014 at 9:49 AM, Neha Narkhede neha.narkh...@gmail.comwrote: Hi Christian, As Jun mentioned, the new producer is marked beta since it is new. By 0.9 or even sooner, we'd expect it to be deployed at LinkedIn and a few other companies in a stable state. The APIs, guarantees and features will not change by 0.9, just the stability. Thanks, Neha On Fri, May 2, 2014 at 7:50 AM, Jun Rao jun...@gmail.com wrote: It's beta mostly because it's new. Thanks, Jun On Thu, May 1, 2014 at 10:25 PM, cac...@gmail.com cac...@gmail.com wrote: Thanks, that's quite helpful. According to this post, http://blog.empathybox.com/ , it looks like it will be beta then which seems good enough. Assuming that the beta designation is correct, is that because it won't have as many features/the same flexibility as it's expected to by 0.9? Christian On Thu, May 1, 2014 at 9:19 PM, Neha Narkhede neha.narkh...@gmail.com wrote: The javadoc of the new producer is here for now - http://empathybox.com/kafka-javadoc/index.html?kafka/clients/producer/KafkaProducer.html Thanks, Neha On Thu, May 1, 2014 at 9:16 PM, Jun Rao jun...@gmail.com wrote: The new producer (that supports callbacks) is in trunk. It will be released in 0.8.2. You can look at the java doc of KafkaProducer for the api. Thanks Jun On Thu, May 1, 2014 at 8:43 PM, Christian Csar cac...@gmail.com wrote: On 05/01/2014 07:22 PM, Christian Csar wrote: I'm looking at using the java producer api for 0.8.1 and I'm slightly confused by this passage from section 4.4 of https://kafka.apache.org/documentation.html#theproducer Note that as of Kafka 0.8.1 the async producer does not have a callback, which could be used to register handlers to catch send errors. Adding such callback functionality is proposed for Kafka 0.9, see [Proposed Producer API]( https://cwiki.apache.org/confluence/display/KAFKA/Client+Rewrite#ClientRewrite-ProposedProducerAPI ) . org.apache.kafka.clients.producer.KafkaProducer in 0.8.1 appears to have public FutureRecordMetadata send(ProducerRecord record, Callback callback) which looks like the mentioned callback. How do the callbacks with the async producer? Is it as described in the comment on the send method (see https://github.com/apache/kafka/blob/0.8.1/clients/src/main/java/org/apache/kafka/clients/producer/KafkaProducer.java#L151 for reference)? Looking around it seems plausible the language in the documentation might refer to a separate sort of callback that existed in 0.7 but not 0.8. In our use case we have something useful to do if we can detect messages failing to be sent. Christian It appears that I was looking at the Java client rather than the Scala java api referenced by the documentation https://github.com/apache/kafka/blob/0.8.1/core/src/main/scala/kafka/javaapi/producer/Producer.scala Are both of these currently suited for use from java and still supported? Given the support for callbacks in the event of failure I am inclined to use the Java one despite the currently limited support for specifying partitioners (though it supports specifying the partition) or encoders. Any guidance on this would be appreciated. Christian
Re: 0.8.1 Java Producer API Callbacks
On 05/01/2014 07:22 PM, Christian Csar wrote: I'm looking at using the java producer api for 0.8.1 and I'm slightly confused by this passage from section 4.4 of https://kafka.apache.org/documentation.html#theproducer Note that as of Kafka 0.8.1 the async producer does not have a callback, which could be used to register handlers to catch send errors. Adding such callback functionality is proposed for Kafka 0.9, see [Proposed Producer API](https://cwiki.apache.org/confluence/display/KAFKA/Client+Rewrite#ClientRewrite-ProposedProducerAPI). org.apache.kafka.clients.producer.KafkaProducer in 0.8.1 appears to have public FutureRecordMetadata send(ProducerRecord record, Callback callback) which looks like the mentioned callback. How do the callbacks with the async producer? Is it as described in the comment on the send method (see https://github.com/apache/kafka/blob/0.8.1/clients/src/main/java/org/apache/kafka/clients/producer/KafkaProducer.java#L151 for reference)? Looking around it seems plausible the language in the documentation might refer to a separate sort of callback that existed in 0.7 but not 0.8. In our use case we have something useful to do if we can detect messages failing to be sent. Christian It appears that I was looking at the Java client rather than the Scala java api referenced by the documentation https://github.com/apache/kafka/blob/0.8.1/core/src/main/scala/kafka/javaapi/producer/Producer.scala Are both of these currently suited for use from java and still supported? Given the support for callbacks in the event of failure I am inclined to use the Java one despite the currently limited support for specifying partitioners (though it supports specifying the partition) or encoders. Any guidance on this would be appreciated. Christian signature.asc Description: OpenPGP digital signature
Re: 0.8.1 Java Producer API Callbacks
The new producer (that supports callbacks) is in trunk. It will be released in 0.8.2. You can look at the java doc of KafkaProducer for the api. Thanks Jun On Thu, May 1, 2014 at 8:43 PM, Christian Csar cac...@gmail.com wrote: On 05/01/2014 07:22 PM, Christian Csar wrote: I'm looking at using the java producer api for 0.8.1 and I'm slightly confused by this passage from section 4.4 of https://kafka.apache.org/documentation.html#theproducer Note that as of Kafka 0.8.1 the async producer does not have a callback, which could be used to register handlers to catch send errors. Adding such callback functionality is proposed for Kafka 0.9, see [Proposed Producer API]( https://cwiki.apache.org/confluence/display/KAFKA/Client+Rewrite#ClientRewrite-ProposedProducerAPI) . org.apache.kafka.clients.producer.KafkaProducer in 0.8.1 appears to have public FutureRecordMetadata send(ProducerRecord record, Callback callback) which looks like the mentioned callback. How do the callbacks with the async producer? Is it as described in the comment on the send method (see https://github.com/apache/kafka/blob/0.8.1/clients/src/main/java/org/apache/kafka/clients/producer/KafkaProducer.java#L151 for reference)? Looking around it seems plausible the language in the documentation might refer to a separate sort of callback that existed in 0.7 but not 0.8. In our use case we have something useful to do if we can detect messages failing to be sent. Christian It appears that I was looking at the Java client rather than the Scala java api referenced by the documentation https://github.com/apache/kafka/blob/0.8.1/core/src/main/scala/kafka/javaapi/producer/Producer.scala Are both of these currently suited for use from java and still supported? Given the support for callbacks in the event of failure I am inclined to use the Java one despite the currently limited support for specifying partitioners (though it supports specifying the partition) or encoders. Any guidance on this would be appreciated. Christian
Re: 0.8.1 Java Producer API Callbacks
The javadoc of the new producer is here for now - http://empathybox.com/kafka-javadoc/index.html?kafka/clients/producer/KafkaProducer.html Thanks, Neha On Thu, May 1, 2014 at 9:16 PM, Jun Rao jun...@gmail.com wrote: The new producer (that supports callbacks) is in trunk. It will be released in 0.8.2. You can look at the java doc of KafkaProducer for the api. Thanks Jun On Thu, May 1, 2014 at 8:43 PM, Christian Csar cac...@gmail.com wrote: On 05/01/2014 07:22 PM, Christian Csar wrote: I'm looking at using the java producer api for 0.8.1 and I'm slightly confused by this passage from section 4.4 of https://kafka.apache.org/documentation.html#theproducer Note that as of Kafka 0.8.1 the async producer does not have a callback, which could be used to register handlers to catch send errors. Adding such callback functionality is proposed for Kafka 0.9, see [Proposed Producer API]( https://cwiki.apache.org/confluence/display/KAFKA/Client+Rewrite#ClientRewrite-ProposedProducerAPI ) . org.apache.kafka.clients.producer.KafkaProducer in 0.8.1 appears to have public FutureRecordMetadata send(ProducerRecord record, Callback callback) which looks like the mentioned callback. How do the callbacks with the async producer? Is it as described in the comment on the send method (see https://github.com/apache/kafka/blob/0.8.1/clients/src/main/java/org/apache/kafka/clients/producer/KafkaProducer.java#L151 for reference)? Looking around it seems plausible the language in the documentation might refer to a separate sort of callback that existed in 0.7 but not 0.8. In our use case we have something useful to do if we can detect messages failing to be sent. Christian It appears that I was looking at the Java client rather than the Scala java api referenced by the documentation https://github.com/apache/kafka/blob/0.8.1/core/src/main/scala/kafka/javaapi/producer/Producer.scala Are both of these currently suited for use from java and still supported? Given the support for callbacks in the event of failure I am inclined to use the Java one despite the currently limited support for specifying partitioners (though it supports specifying the partition) or encoders. Any guidance on this would be appreciated. Christian
Re: 0.8.1 Java Producer API Callbacks
Thanks, that's quite helpful. According to this post, http://blog.empathybox.com/ , it looks like it will be beta then which seems good enough. Assuming that the beta designation is correct, is that because it won't have as many features/the same flexibility as it's expected to by 0.9? Christian On Thu, May 1, 2014 at 9:19 PM, Neha Narkhede neha.narkh...@gmail.comwrote: The javadoc of the new producer is here for now - http://empathybox.com/kafka-javadoc/index.html?kafka/clients/producer/KafkaProducer.html Thanks, Neha On Thu, May 1, 2014 at 9:16 PM, Jun Rao jun...@gmail.com wrote: The new producer (that supports callbacks) is in trunk. It will be released in 0.8.2. You can look at the java doc of KafkaProducer for the api. Thanks Jun On Thu, May 1, 2014 at 8:43 PM, Christian Csar cac...@gmail.com wrote: On 05/01/2014 07:22 PM, Christian Csar wrote: I'm looking at using the java producer api for 0.8.1 and I'm slightly confused by this passage from section 4.4 of https://kafka.apache.org/documentation.html#theproducer Note that as of Kafka 0.8.1 the async producer does not have a callback, which could be used to register handlers to catch send errors. Adding such callback functionality is proposed for Kafka 0.9, see [Proposed Producer API]( https://cwiki.apache.org/confluence/display/KAFKA/Client+Rewrite#ClientRewrite-ProposedProducerAPI ) . org.apache.kafka.clients.producer.KafkaProducer in 0.8.1 appears to have public FutureRecordMetadata send(ProducerRecord record, Callback callback) which looks like the mentioned callback. How do the callbacks with the async producer? Is it as described in the comment on the send method (see https://github.com/apache/kafka/blob/0.8.1/clients/src/main/java/org/apache/kafka/clients/producer/KafkaProducer.java#L151 for reference)? Looking around it seems plausible the language in the documentation might refer to a separate sort of callback that existed in 0.7 but not 0.8. In our use case we have something useful to do if we can detect messages failing to be sent. Christian It appears that I was looking at the Java client rather than the Scala java api referenced by the documentation https://github.com/apache/kafka/blob/0.8.1/core/src/main/scala/kafka/javaapi/producer/Producer.scala Are both of these currently suited for use from java and still supported? Given the support for callbacks in the event of failure I am inclined to use the Java one despite the currently limited support for specifying partitioners (though it supports specifying the partition) or encoders. Any guidance on this would be appreciated. Christian
