Hi all,
I'm delighted to say 0.3.1 is released and available on Clojars. It's been a long while since we've made any big changes to clj-kafka so I'm very happy to say we've caught up over the past couple of days. Most of the changes were the result of submissions from the community but I have to give most of the credit to Ragnar Dahlen :) Notable changes: - Refactored Zookeeper consumer to expose underlying streams (transducer compatible :) - "New Producer" API support - Topic Administration - Updated to Kafka 0.8.2.1 - Removed Kafka deps from clj-kafka https://github.com/pingles/clj-kafka/ http://pingles.github.io/clj-kafka/ Zookeeper Consumer ================= We've broken the old messages function apart into two pieces: create-message-stream will connect the consumer to a message stream (topic + number of threads when consuming more than 1 partition) and stream-seq (recast the stream as a lazy sequence as with earlier releases). This change addresses a bunch of other requests/changes people had submitted. Hopefully most stuff that was a little tough to do before is now easier to integrate- composition with other libs like manifold[1] should be easier. Transducer Support ---------------------------- This also means that the lib is easier to use with transducers[2] which should be useful for people doing stream processing. New Producer API ============== We have integrated support for the "New Producer" API[3,4]. This is asynchronous by default but provides both futures and callbacks to return the offset/error. Topic Administration ================ A new namespace exists that wraps some of the Kafka admin utils for checking if topics exist, creating/removing topics and changing per-topic configuration. No Kafka deps in clj-kafka ==================== I think* this was inherited from back when the Kafka releases had no dependencies specified and so clj-kafka had to specify them instead. This is no longer necessary so the project config is substantially tidier. This is quite a substantial difference so please just check that this doesn't break anything you were depending on. 1) https://github.com/ztellman/manifold 2) https://github.com/pingles/clj-kafka/blob/master/README.md#usage-with-transducers 3) http://blog.confluent.io/2014/12/02/whats-coming-in-apache-kafka-0-8-2/ 4) http://kafka.apache.org/082/javadoc/org/apache/kafka/clients/producer/KafkaProducer.html 5) https://github.com/pingles/clj-kafka/blob/master/README.md#administration-operations -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups "Clojure" group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
