Understood- almost all the functions in the zk namespace were intended for occasional/admin type use so they open/close the ZK connection.
It should be pretty easy to refactor that around so connections are passed in (allowing you to retain a connection across multiple operations). I'll see if I can get time today to do it. If not I'd happily take pull requests :) On Monday, June 15, 2015 at 5:29:30 PM UTC+1, Omri Hurvitz wrote: > > I see it now. I am concerned though that this will create a new zk > connection every time I use it. > > On Friday, June 12, 2015 at 1:46:46 PM UTC-4, Ragnar Dahlén wrote: >> >> Hi Omri, >> >> There's clj-kafka.zk/set-offset! for exactly that purpose: >> https://pingles.github.io/clj-kafka/clj-kafka.zk.html#var-set-offset.21 >> >> Thanks, >> Ragnar >> >> On Friday, 12 June 2015 18:23:23 UTC+1, Omri Hurvitz wrote: >>> >>> Hi Paul. >>> >>> This looks great - I am looking forward to replace my hacked wrapper for >>> the Java (and Scala...) API with this code. >>> One question: if I have a zk consumer stream with auto.commit.enable set >>> to false, how do I manually commit the changes? >>> >>> Thanks, >>> >>> Omri >>> >>> On Friday, June 12, 2015 at 11:35:44 AM UTC-4, Paul Ingles wrote: >>>> >>>> Hi all, >>>> >>>> >>>> I'm delighted to say 0.3.1 is released and available on Clojars. >>>> >>>> >>>> It's been a long while since we've made any big changes to clj-kafka so >>>> I'm very happy to say we've caught up over the past couple of days. Most >>>> of >>>> the changes were the result of submissions from the community but I have >>>> to >>>> give most of the credit to Ragnar Dahlen :) >>>> >>>> >>>> Notable changes: >>>> >>>> >>>> - Refactored Zookeeper consumer to expose underlying streams >>>> (transducer compatible :) >>>> >>>> - "New Producer" API support >>>> >>>> - Topic Administration >>>> >>>> - Updated to Kafka 0.8.2.1 >>>> >>>> - Removed Kafka deps from clj-kafka >>>> >>>> >>>> https://github.com/pingles/clj-kafka/ >>>> >>>> http://pingles.github.io/clj-kafka/ >>>> >>>> >>>> >>>> Zookeeper Consumer >>>> >>>> ================= >>>> >>>> We've broken the old messages function apart into two pieces: >>>> create-message-stream will connect the consumer to a message stream (topic >>>> + number of threads when consuming more than 1 partition) and stream-seq >>>> (recast the stream as a lazy sequence as with earlier releases). >>>> >>>> >>>> This change addresses a bunch of other requests/changes people had >>>> submitted. Hopefully most stuff that was a little tough to do before is >>>> now >>>> easier to integrate- composition with other libs like manifold[1] should >>>> be >>>> easier. >>>> >>>> >>>> Transducer Support >>>> >>>> ---------------------------- >>>> >>>> This also means that the lib is easier to use with transducers[2] which >>>> should be useful for people doing stream processing. >>>> >>>> >>>> >>>> New Producer API >>>> >>>> ============== >>>> >>>> We have integrated support for the "New Producer" API[3,4]. This is >>>> asynchronous by default but provides both futures and callbacks to return >>>> the offset/error. >>>> >>>> >>>> Topic Administration >>>> >>>> ================ >>>> >>>> A new namespace exists that wraps some of the Kafka admin utils for >>>> checking if topics exist, creating/removing topics and changing per-topic >>>> configuration. >>>> >>>> >>>> No Kafka deps in clj-kafka >>>> >>>> ==================== >>>> >>>> I think* this was inherited from back when the Kafka releases had no >>>> dependencies specified and so clj-kafka had to specify them instead. This >>>> is no longer necessary so the project config is substantially tidier. This >>>> is quite a substantial difference so please just check that this doesn't >>>> break anything you were depending on. >>>> >>>> >>>> >>>> >>>> 1) https://github.com/ztellman/manifold >>>> >>>> 2) >>>> https://github.com/pingles/clj-kafka/blob/master/README.md#usage-with-transducers >>>> >>>> 3) >>>> http://blog.confluent.io/2014/12/02/whats-coming-in-apache-kafka-0-8-2/ >>>> >>>> 4) >>>> http://kafka.apache.org/082/javadoc/org/apache/kafka/clients/producer/KafkaProducer.html >>>> >>>> 5) >>>> https://github.com/pingles/clj-kafka/blob/master/README.md#administration-operations >>>> >>> -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups "Clojure" group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
