Hi Paul. This looks great - I am looking forward to replace my hacked wrapper for the Java (and Scala...) API with this code. One question: if I have a zk consumer stream with auto.commit.enable set to false, how do I manually commit the changes?
Thanks, Omri On Friday, June 12, 2015 at 11:35:44 AM UTC-4, Paul Ingles wrote: > > Hi all, > > > I'm delighted to say 0.3.1 is released and available on Clojars. > > > It's been a long while since we've made any big changes to clj-kafka so > I'm very happy to say we've caught up over the past couple of days. Most of > the changes were the result of submissions from the community but I have to > give most of the credit to Ragnar Dahlen :) > > > Notable changes: > > > - Refactored Zookeeper consumer to expose underlying streams (transducer > compatible :) > > - "New Producer" API support > > - Topic Administration > > - Updated to Kafka 0.8.2.1 > > - Removed Kafka deps from clj-kafka > > > https://github.com/pingles/clj-kafka/ > > http://pingles.github.io/clj-kafka/ > > > > Zookeeper Consumer > > ================= > > We've broken the old messages function apart into two pieces: > create-message-stream will connect the consumer to a message stream (topic > + number of threads when consuming more than 1 partition) and stream-seq > (recast the stream as a lazy sequence as with earlier releases). > > > This change addresses a bunch of other requests/changes people had > submitted. Hopefully most stuff that was a little tough to do before is now > easier to integrate- composition with other libs like manifold[1] should be > easier. > > > Transducer Support > > ---------------------------- > > This also means that the lib is easier to use with transducers[2] which > should be useful for people doing stream processing. > > > > New Producer API > > ============== > > We have integrated support for the "New Producer" API[3,4]. This is > asynchronous by default but provides both futures and callbacks to return > the offset/error. > > > Topic Administration > > ================ > > A new namespace exists that wraps some of the Kafka admin utils for > checking if topics exist, creating/removing topics and changing per-topic > configuration. > > > No Kafka deps in clj-kafka > > ==================== > > I think* this was inherited from back when the Kafka releases had no > dependencies specified and so clj-kafka had to specify them instead. This > is no longer necessary so the project config is substantially tidier. This > is quite a substantial difference so please just check that this doesn't > break anything you were depending on. > > > > > 1) https://github.com/ztellman/manifold > > 2) > https://github.com/pingles/clj-kafka/blob/master/README.md#usage-with-transducers > > 3) http://blog.confluent.io/2014/12/02/whats-coming-in-apache-kafka-0-8-2/ > > 4) > http://kafka.apache.org/082/javadoc/org/apache/kafka/clients/producer/KafkaProducer.html > > 5) > https://github.com/pingles/clj-kafka/blob/master/README.md#administration-operations > -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups "Clojure" group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
