Re: defrecord, equality, hashing, and performance
On Thursday, June 11, 2015 at 2:19:56 PM UTC-5, Andy Fingerhut wrote: You can override hashCode and/or hasheq methods in deftype. I've been unable to do it. Is there something special needed to allow this to work? user= (defrecord Foo [x] Object (hashCode [this] 42)) CompilerException java.lang.ClassFormatError: Duplicate method namesignature in class file user/Foo, compiling:(/private/var/folders/hv/hv6wzwrEFOujYN382REV6TI/-Tmp-/form-init7644007787939926514.clj:1:1) user= (defrecord Foo [x] Object (hasheq [this] 42)) CompilerException java.lang.ClassFormatError: Duplicate method namesignature in class file user/Foo, compiling:(/private/var/folders/hv/hv6wzwrEFOujYN382REV6TI/-Tmp-/form-init7644007787939926514.clj:1:1) If the reason that defrecord hashing is slow in your application is because _hasheq_ recalculates the hash values from scratch each time, without _caching the value_, consider voting for this ticket so that is improved in the future: http://dev.clojure.org/jira/browse/CLJ-1224 I'm not sure whether this might be a part of what I'm experiencing, but I didn't know about the ticket-voting process. I'll look into that. -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: defrecord, equality, hashing, and performance
Related to this, another thing I would like Clojure to provide is an easy way to opt-in for map-like function application for defrecords. Right now, Clojure doesn't implement function application for records, and it is a glaring incompatibility that causes bugs when you switch back and forth between records and maps to explore performance trade-offs. So it should be easy to opt-in. I assume the reasoning behind it comes from the notion that many people will want to create their own function application behavior for many types of records (which is true). Still, you should at least be able to *choose* to get default function application behavior, because it is tedious and error-prone to implement it on your own. Alternatively, if Clojure did what I suggested in my previous email, and allowed you to override functionality provided by defrecord, then defrecord could safely go ahead and implement function application behavior by default, because people would still be free to override that if they don't want that behavior. The other broader point here is that Clojure fails to provide good default, customizable abstract implementations of all its data structures. There is little to no documentation about what methods need to be overridden to get a minimal implementation of a given data structure. Clojurescript appears to be doing better on this front, since it was designed from the ground up with protocols in mind, but the lack of correspondence between Clojure's classes/abstract classes/interfaces and Clojurescript's implementation makes it difficult to write cross-platform data structures. -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: defrecord, equality, hashing, and performance
So just to explain this a little more, recently I wanted something record-like with custom hashing and equality for my ubergraph library. Unfortunately, you can't do this with defrecord, and starting from deftype and re-implementing all the map-like semantics from scratch is a total pain. So what I did is from potemkin, I referred def-map-type, and from potemkin.collections I referred AbstractMap. The type definition begins like this: (def-map-type Ubergraph [node-map allow-parallel? undirected? attrs cached-hash] AbstractMap and then you have your usual alternation of protocols and implementations. At the end, you must implement the following: (get [this key default-value] ...) (assoc [this key value] ...) (dissoc [this key] ...) (keys [this] ...) (meta [this] ...l) (with-meta [this meta] ...) Then, you have the flexibility to override hasheq and equiv: (hasheq [this] ...) (equiv [this other] ...) I'm not crazy about the fact that I had to implement get, assoc, dissoc, keys, meta, and with-meta in order to override hasheq and equiv. It was routine and tedious, and it is stuff that defrecord's macro does automatically for you. For example, my get implementation looks like this: (get [this key default-value] (case key :node-map node-map :allow-parallel? allow-parallel? :undirected? undirected? :attrs attrs :cached-hash cached-hash default-value)) just taking cases on all the keywords that refer to fields in my deftype. This means that any time you add a field to your record, you have to go back and tweak all those method implementations, which is annoying. Despite these complaints, it was, by far, the simplest way I was able to find to get a record/map-like structure with custom hashing and equality. Even though potemkin's def-map-type makes you implement those methods on your own, it does a lot of the grunt work of fulfilling other map methods that you probably wouldn't remember to implement everything by yourself. Right now Clojure offers this all-or-nothing approach: do it entirely yourself with deftype or use defrecord and lose the ability to customize. Clojure's position statement on this is that data structures generally fall cleanly into one of these two categories. I find that premise to be faulty. The simplest fix would be if Clojure simply allowed one to override the protocols which are implemented for you by defrecord, rather than throwing an error and saying that you can't redefine one of those protocols because they have already been implemented. Keep the errors for the case when you accidentally redefine the same method twice in your own implementation, but don't call it an error if you override the default implementation of something provided by the defrecord macro. -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: defrecord, equality, hashing, and performance
On Thursday, June 11, 2015 at 2:12:12 PM UTC-5, puzzler wrote: Zach Tellman's potemkin library includes several useful ways to tweak deftypes and defrecords. I wish Clojure itself provided more ways to do this, but in the meantime, potemkin is the best way to create something custom, like a deftype that behaves mostly like defrecord with some different hashing and equality behavior. Very interesting. Cool. I'll look at that. -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: defrecord, equality, hashing, and performance
deftype allows you to override hashCode and/or hasheq (I believe defaulting to identity-based implementations from java.lang.Object). defrecord does not. As mentioned in another message, potemkin may provide easier building blocks to build on than deftype. Andy On Thu, Jun 11, 2015 at 11:10 PM, Mars0i marsh...@logical.net wrote: On Thursday, June 11, 2015 at 2:19:56 PM UTC-5, Andy Fingerhut wrote: You can override hashCode and/or hasheq methods in deftype. I've been unable to do it. Is there something special needed to allow this to work? user= (defrecord Foo [x] Object (hashCode [this] 42)) CompilerException java.lang.ClassFormatError: Duplicate method namesignature in class file user/Foo, compiling:(/private/var/folders/hv/hv6wzwrEFOujYN382REV6TI/-Tmp-/form-init7644007787939926514.clj:1:1) user= (defrecord Foo [x] Object (hasheq [this] 42)) CompilerException java.lang.ClassFormatError: Duplicate method namesignature in class file user/Foo, compiling:(/private/var/folders/hv/hv6wzwrEFOujYN382REV6TI/-Tmp-/form-init7644007787939926514.clj:1:1) If the reason that defrecord hashing is slow in your application is because _hasheq_ recalculates the hash values from scratch each time, without _caching the value_, consider voting for this ticket so that is improved in the future: http://dev.clojure.org/jira/browse/CLJ-1224 I'm not sure whether this might be a part of what I'm experiencing, but I didn't know about the ticket-voting process. I'll look into that. -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: defrecord, equality, hashing, and performance
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 Prismatic has some document for deftype/defrecord/map, hope it's useful to you: https://github.com/Prismatic/eng-practices/blob/master/clojure/20130926-data-representation.md On 06/12/2015 02:36 AM, Mars0i wrote: I think that the following is all correct, but I could be wrong about something. The datatypes page at clojure.org http://clojure.org/datatypes says: defrecord provides ... value-based equality and hashCode, while deftype does not. So (defrecord Rec [x]) (= (Rec. 42) (Rec. 42)) ;= true but (deftype Typ [x]) (= (Typ. 42) (Typ. 42)) ;= false. This also means, as I understand it, that data structures that use hashing--maps and sets, for example--hash on value for records, but on identity for types. I believe that the same is true when Java hashtables compare records, or compare types, although I haven't carefully tested this. It makes sense that the primary, most fully functional user-definable datatypes in Clojure should use value-based equality; that's fully in the spirit of a functional language, and is very convenient in many contexts, and hashing should follow the equality semantics of the datatypes. You can always use identical? if you want to test records for identity rather than equality. However, you can't tell maps and sets to use identity for records, afaik. This makes records undesirable if you only care about identity, and need to use a data structure that uses hashing, and care about performance, because value-based hashing is presumably a lot slower than identity-based hashing (which is what my experiments seem to show). On the other hand, records are obviously way, way, more convenient than types in many contexts because the functionality provided by deftype is so bare-bones. It really bothers me that I have to choose between speed and using convenient, idiomatic functionality (defrecord) that is one of several significant reasons that I love Clojure. (This is in an application in which value-based equality would /never/ be what I wanted for my datatypes.) Should there be an alternative? Either a way to define e.g. a special kind of set that uses identity rather than value for records, or a way to make records use identity for equality, or another datatype that's just like defrecord but ..., or a way to get more functionality with deftype, or ...? (I don't think that Clojure should include every option or functionality that someone thinks might be convenient in some context. Another reason that I love Clojure is that it /doesn't/ do that, but instead provides limited, well-thought out options that allow a lot of flexibility.) In an earlier thread https://groups.google.com/forum/#!topic/clojure/EdjnSxRkOPk, Stephen Yi pointed out that although you're allowed to override some Object methods in defrecord, you can't override Object.equals and Object.hashCode. It isn't documented, but the compiler throws an exception if you try it. That makes quite a bit of sense, since changing the meaning of equality for a datatype would be an easy way to introduce confusing bugs. On the other hand, if you do override Object.equals and Object.hashCode for defrecord, you probably know that you're doing something weird, and that you'd better take precautions, if necessary, to avoid later confusion. -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com mailto:clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout. -BEGIN PGP SIGNATURE- Version: GnuPG v2 iQEcBAEBCAAGBQJVen1zAAoJEBnCozJnPTdiq6UH/3pKe17SGwNV0a/PajDOLbfe R86c/U0faXgtr4hL4IrhSEK9HWlGHWy8FHE57TYQstc8OSryJXsbNp3/OjpfIpfB QBpQwgBmwkrA/EkLJEmBsGu50R8nQgwLiG6S0T8gSFPUSM2NkQJyBkczTz0AVX7y bUjliAb3lGaWG8t4uJhPt0feq+acfpjR1bG5V6ckISFtcJW+N8aG0lQKkDmTqcQZ AppdpFrjiDw+SoyPcEB+vpRyQxwnxYk/g9qZHjEoFi7DG0e59gHFG19dLlXeH5Nv tKR6rRGXsJQROxdxGWNS2HYa4ASSJJd80reoehl341A2xZ0z46SASw51T2QqpZs= =3/P/ -END PGP SIGNATURE- -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this
Re: defrecord, equality, hashing, and performance
Re IFn support for defrecord, I don't actually know the reason that's not built-in. I am not aware of an existing ticket for this. Of course, at this point many people patch it in, so we would need to consider potential breakage from that. -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Boot: How to run with the clojure version declared in the :dependencies?
Hi all,
I'm toying around with boot and try to get my leiningen config converted
to it.
My build.boot contains
--8---cut here---start-8---
(set-env!
:resource-paths #{src resources}
:dependencies '[[org.clojure/clojure 1.7.0-RC1]
...])
--8---cut here---end---8---
but boot repl or `M-x cider-jack-in RET` in emacs still give me a REPL
running 1.6.0. I know I can call
$ BOOT_CLOJURE_VERSION=1.7.0-RC1 boot repl
or define the BOOT_CLOJURE_VERSION in a boot.properties file, but do I
really need to duplicate the clojure version?
Bye,
Tassilo
--
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
---
You received this message because you are subscribed to the Google Groups
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Boot: CLASSPATH troubles with javax.tools.JavaCompiler
Hi all, I'm toying around with boot and try to get my leiningen config converted to it. My Clojure project uses a third-party Java library which uses a javax.tools.JavaCompiler to compile classes at runtime. By default, such a JavaCompiler compiles with the CLASSPATH which is the value of the java.class.path property. However, when run via boot I get (System/getProperty java.class.path) ;= /home/horn/bin/boot which is the boot shell script. Cause that's not a correct CLASSPATH, the JavaCompiler won't find any classes and error out. It seems boot puts the real CLASSPATH into the property fake.class.path or boot.class.path (I haven't found out what's exactly the difference). So if I'd call the JavaCompiler myself, I could use its getTask()'s options parameter to specify something like -cp (System/getProperty fake.class.path). But since the compiler is called internally in the lib I depend on, that approach won't work. Well, and IMHO having to consider properties defined by one specific build tool in application code doesn't seem right, too. What works is putting (System/setProperty java.class.path (System/getProperty fake.class.path)) in my build.boot but I guess boot has a reason for messing with java.class.path and doing the above will defeat this purpose. Bye, Tassilo -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: defrecord, equality, hashing, and performance
On Friday, June 12, 2015 at 11:46:17 AM UTC-4, Mars0i wrote: Oh, yes, and that's an interesting idea to wrap a record in an atom or delay. For my present uses that would be more trouble than it's worth, but it's something worth keeping in mind for other situations. Putting a unique primary key like value, such as a UUID, into each new instance will also effectively cause identity semantics. -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
`rational?` `decimal?` Am I misunderstanding these? (also `float?`)
Your definitions are correct, but in a different domain: you are using the mathematical definitions. What these functions give you is information about the representation of the numbers in memory, not information about the numbers themselves. For example, (integer? 1.0) will give you false. A rational number in mathematics is one that can be written as a fraction (of integers), or equivalently one that does not have an infinite and non-periodic expansion. A rational number in the sense of rational? in Clojure is an object in memory which is actually stored as a couple of integers, not merely a valie that could be stored that way. As others have mentioned, the way in which a number is stored will have some impact on how much memory it takes up, how fast the computer can compute operations on it, and how much precision will be lost with these operations. As always, Clojure puts more emphasis on behaviours and interfaces than on implementations, so you should really understand rational? as will I get an exact answer if I use operations that would give an exact answer with a rational number (in the mathematical sense)? For example, (/ 1M 3) will throw an exception rather than returning an inexact, truncated answer, whereas (/ 1.0 3) will happily lie to you. (According to the documentation, BigDecimal always returns a correct value or throws an exception, except if you explicitly tell it to round. It can represent values down to about 1e-2_147_483_647, and up to filling your computer's memory.) For a first step towards understanding floating-point values, I would recommend reading: http://blog.reverberate.org/2014/09/what-every-computer-programmer-should.html On Friday, 12 June 2015, John Gabriele jmg3...@gmail.com javascript:_e(%7B%7D,'cvml','jmg3...@gmail.com'); wrote: My understanding is that a rational number is one that can be written as a fraction. For example, 5.1, which can be written as 51/10. But Clojure seems to disagree: ~~~ (rational? 51/10) ;= true (rational? 5.1) ;= false (?!) ~~~ Is my definition of rational incorrect? Also, my understanding is that a decimal number is one that has a decimal point in it, like, for example, 5.1. However: ~~~ (decimal? 5.1) ;= false (?!) ~~~ And while typing this, I also notice that while `integer?` acts like I'd expect, `float?` does something weird: ~~~ (integer? 5) ;= true Yes (integer? 5N) ;= true Yes (integer? 5.1) ;= false (float? 5.1) ;= true (float? 5.1M) ;= false (?!) ~~~ Maybe I'm confusing floating point number with decimal number here? If so, what's the difference? Thanks! -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: [ANN] clj-kafka 0.3.1
Paul, This is great news. Looking forward to using it. cheers, Bruce On Fri, Jun 12, 2015 at 4:35 PM, Paul Ingles p...@oobaloo.co.uk wrote: Hi all, I'm delighted to say 0.3.1 is released and available on Clojars. It's been a long while since we've made any big changes to clj-kafka so I'm very happy to say we've caught up over the past couple of days. Most of the changes were the result of submissions from the community but I have to give most of the credit to Ragnar Dahlen :) Notable changes: - Refactored Zookeeper consumer to expose underlying streams (transducer compatible :) - New Producer API support - Topic Administration - Updated to Kafka 0.8.2.1 - Removed Kafka deps from clj-kafka https://github.com/pingles/clj-kafka/ http://pingles.github.io/clj-kafka/ Zookeeper Consumer = We've broken the old messages function apart into two pieces: create-message-stream will connect the consumer to a message stream (topic + number of threads when consuming more than 1 partition) and stream-seq (recast the stream as a lazy sequence as with earlier releases). This change addresses a bunch of other requests/changes people had submitted. Hopefully most stuff that was a little tough to do before is now easier to integrate- composition with other libs like manifold[1] should be easier. Transducer Support This also means that the lib is easier to use with transducers[2] which should be useful for people doing stream processing. New Producer API == We have integrated support for the New Producer API[3,4]. This is asynchronous by default but provides both futures and callbacks to return the offset/error. Topic Administration A new namespace exists that wraps some of the Kafka admin utils for checking if topics exist, creating/removing topics and changing per-topic configuration. No Kafka deps in clj-kafka I think* this was inherited from back when the Kafka releases had no dependencies specified and so clj-kafka had to specify them instead. This is no longer necessary so the project config is substantially tidier. This is quite a substantial difference so please just check that this doesn't break anything you were depending on. 1) https://github.com/ztellman/manifold 2) https://github.com/pingles/clj-kafka/blob/master/README.md#usage-with-transducers 3) http://blog.confluent.io/2014/12/02/whats-coming-in-apache-kafka-0-8-2/ 4) http://kafka.apache.org/082/javadoc/org/apache/kafka/clients/producer/KafkaProducer.html 5) https://github.com/pingles/clj-kafka/blob/master/README.md#administration-operations -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Unable to get a response from my server, after re-architecting to get away from global vars
When I first wrote a Clojure web app, I followed a lot of the basic
tutorials out there, and those tend to create a lot of global vars. So, for
instance, I used the defroutes macro from Compojure, which left me with a
global app-routes var.
But lately I wanted to switch to a style that would allow me to call a
start function that would reboot my app, so that I could more easily
re-start from the REPL, without shutting down the JVM and starting over
again.
So I moved all of my global var stuff inside my start function, like this:
(defn start [map-of-config]
(try
(let [port (if (nil? (:port map-of-config))
34000
(Integer/parseInt (:port map-of-config)))
app-routes (routes
(ANY / [] homepage)
(GET /v0.2/token [] token)
(GET /v0.2/errors [] errors)
(ANY /v0.2/:token/:name-of-collection/object-id/:object-id request
query/fetch)
(ANY /v0.2/:token/:name-of-collection/:document-id request
query/fetch)
(ANY /v0.2/:token/:name-of-collection/ request query/fetch)
(route/resources /)
(route/not-found Page not found. Check the http verb that you used
(GET, POST, PUT, DELETE) and make sure you put a collection name in the
URL, and possibly also a document ID. Also, all requests should go to an
URL that starts with /v0.2))
app (- app-routes
(wrap-json-response)
(middleware/wrap-objectid-to-str)
(middleware/wrap-prepare-message)
(middleware/wrap-error-post-with-document-id)
(middleware/wrap-malformed?)
(middleware/wrap-check-content-type)
(middleware/wrap-transaction-id)
(middleware/wrap-token)
(middleware/wrap-token-check)
(middleware/wrap-cors-headers)
(wrap-keyword-params)
(wrap-multipart-params)
(wrap-nested-params)
(wrap-params)
(wrap-json-body {:keywords? true})
(wrap-content-type))
jetty (run-jetty app {:port port :join? false :max-threads 5000})]
;; we want to reboot this app from the REPL, so the start function
needs to overwrite server
(println (str The port number we will listen to: port))
(timbre/log :trace (str The port number we will listen to: port))
(swap! server (fn [old-value] jetty)))
(catch Exception e (println e
This does not throw an exception. And, once it is running, if I do this:
netstat -ntlp | grep LISTEN
then I can see there is suddenly an app listening on port 34000:
tcp0 0 127.0.0.1:27017 0.0.0.0:* LISTEN
-
tcp0 0 127.0.0.1:3306 0.0.0.0:* LISTEN
-
tcp0 0 0.0.0.0:22 0.0.0.0:* LISTEN
-
tcp0 0 127.0.0.1:5432 0.0.0.0:* LISTEN
-
tcp6 0 0 :::34000:::*LISTEN
12835/java
tcp6 0 0 :::80 :::*LISTEN
-
tcp6 0 0 :::22 :::*LISTEN
-
But if I point my browser at this port, I get nothing. I do not get an
exception, I also see nothing printed at the terminal.
I have gone through and added print statements to all of my middleware, but
I can not see any of the print statements in the terminal. I am wondering
if it is possible for a request to come in and get swallowed entirely,
without even triggering the middleware?
--
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
---
You received this message because you are subscribed to the Google Groups
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Re: defrecord, equality, hashing, and performance
On Friday, June 12, 2015 at 1:34:20 AM UTC-5, Andy Fingerhut wrote: deftype allows you to override hashCode and/or hasheq (I believe defaulting to identity-based implementations from java.lang.Object). defrecord does not. Sorry--I misread your earlier statement about this. That's good to know, although for my present application, deftype's hashing behavior is all that I need. I miss defrecord's advantages when I use deftype, but will look at potemkin, as suggested. -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: defrecord, equality, hashing, and performance
Oh, yes, and that's an interesting idea to wrap a record in an atom or delay. For my present uses that would be more trouble than it's worth, but it's something worth keeping in mind for other situations. On Friday, June 12, 2015 at 10:41:57 AM UTC-5, Mars0i wrote: puzzler, thanks for the explanation about how you built on potemkin's map options. Probably not useful for my current application--easier to just work around deftype's limitations on an ad-hoc basis. But that takes some of the fun (convenience) out of Clojure (and unnecessarily, I feel, I guess). -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[ANN] clj-kafka 0.3.1
Hi all, I'm delighted to say 0.3.1 is released and available on Clojars. It's been a long while since we've made any big changes to clj-kafka so I'm very happy to say we've caught up over the past couple of days. Most of the changes were the result of submissions from the community but I have to give most of the credit to Ragnar Dahlen :) Notable changes: - Refactored Zookeeper consumer to expose underlying streams (transducer compatible :) - New Producer API support - Topic Administration - Updated to Kafka 0.8.2.1 - Removed Kafka deps from clj-kafka https://github.com/pingles/clj-kafka/ http://pingles.github.io/clj-kafka/ Zookeeper Consumer = We've broken the old messages function apart into two pieces: create-message-stream will connect the consumer to a message stream (topic + number of threads when consuming more than 1 partition) and stream-seq (recast the stream as a lazy sequence as with earlier releases). This change addresses a bunch of other requests/changes people had submitted. Hopefully most stuff that was a little tough to do before is now easier to integrate- composition with other libs like manifold[1] should be easier. Transducer Support This also means that the lib is easier to use with transducers[2] which should be useful for people doing stream processing. New Producer API == We have integrated support for the New Producer API[3,4]. This is asynchronous by default but provides both futures and callbacks to return the offset/error. Topic Administration A new namespace exists that wraps some of the Kafka admin utils for checking if topics exist, creating/removing topics and changing per-topic configuration. No Kafka deps in clj-kafka I think* this was inherited from back when the Kafka releases had no dependencies specified and so clj-kafka had to specify them instead. This is no longer necessary so the project config is substantially tidier. This is quite a substantial difference so please just check that this doesn't break anything you were depending on. 1) https://github.com/ztellman/manifold 2) https://github.com/pingles/clj-kafka/blob/master/README.md#usage-with-transducers 3) http://blog.confluent.io/2014/12/02/whats-coming-in-apache-kafka-0-8-2/ 4) http://kafka.apache.org/082/javadoc/org/apache/kafka/clients/producer/KafkaProducer.html 5) https://github.com/pingles/clj-kafka/blob/master/README.md#administration-operations -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: defrecord, equality, hashing, and performance
puzzler, thanks for the explanation about how you built on potemkin's map options. Probably not useful for my current application--easier to just work around deftype's limitations on an ad-hoc basis. But that takes some of the fun (convenience) out of Clojure (and unnecessarily, I feel, I guess). -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: [ANN] clj-kafka 0.3.1
Hi Paul. This looks great - I am looking forward to replace my hacked wrapper for the Java (and Scala...) API with this code. One question: if I have a zk consumer stream with auto.commit.enable set to false, how do I manually commit the changes? Thanks, Omri On Friday, June 12, 2015 at 11:35:44 AM UTC-4, Paul Ingles wrote: Hi all, I'm delighted to say 0.3.1 is released and available on Clojars. It's been a long while since we've made any big changes to clj-kafka so I'm very happy to say we've caught up over the past couple of days. Most of the changes were the result of submissions from the community but I have to give most of the credit to Ragnar Dahlen :) Notable changes: - Refactored Zookeeper consumer to expose underlying streams (transducer compatible :) - New Producer API support - Topic Administration - Updated to Kafka 0.8.2.1 - Removed Kafka deps from clj-kafka https://github.com/pingles/clj-kafka/ http://pingles.github.io/clj-kafka/ Zookeeper Consumer = We've broken the old messages function apart into two pieces: create-message-stream will connect the consumer to a message stream (topic + number of threads when consuming more than 1 partition) and stream-seq (recast the stream as a lazy sequence as with earlier releases). This change addresses a bunch of other requests/changes people had submitted. Hopefully most stuff that was a little tough to do before is now easier to integrate- composition with other libs like manifold[1] should be easier. Transducer Support This also means that the lib is easier to use with transducers[2] which should be useful for people doing stream processing. New Producer API == We have integrated support for the New Producer API[3,4]. This is asynchronous by default but provides both futures and callbacks to return the offset/error. Topic Administration A new namespace exists that wraps some of the Kafka admin utils for checking if topics exist, creating/removing topics and changing per-topic configuration. No Kafka deps in clj-kafka I think* this was inherited from back when the Kafka releases had no dependencies specified and so clj-kafka had to specify them instead. This is no longer necessary so the project config is substantially tidier. This is quite a substantial difference so please just check that this doesn't break anything you were depending on. 1) https://github.com/ztellman/manifold 2) https://github.com/pingles/clj-kafka/blob/master/README.md#usage-with-transducers 3) http://blog.confluent.io/2014/12/02/whats-coming-in-apache-kafka-0-8-2/ 4) http://kafka.apache.org/082/javadoc/org/apache/kafka/clients/producer/KafkaProducer.html 5) https://github.com/pingles/clj-kafka/blob/master/README.md#administration-operations -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: [ANN] clj-kafka 0.3.1
Hi Omri, There's clj-kafka.zk/set-offset! for exactly that purpose: https://pingles.github.io/clj-kafka/clj-kafka.zk.html#var-set-offset.21 Thanks, Ragnar On Friday, 12 June 2015 18:23:23 UTC+1, Omri Hurvitz wrote: Hi Paul. This looks great - I am looking forward to replace my hacked wrapper for the Java (and Scala...) API with this code. One question: if I have a zk consumer stream with auto.commit.enable set to false, how do I manually commit the changes? Thanks, Omri On Friday, June 12, 2015 at 11:35:44 AM UTC-4, Paul Ingles wrote: Hi all, I'm delighted to say 0.3.1 is released and available on Clojars. It's been a long while since we've made any big changes to clj-kafka so I'm very happy to say we've caught up over the past couple of days. Most of the changes were the result of submissions from the community but I have to give most of the credit to Ragnar Dahlen :) Notable changes: - Refactored Zookeeper consumer to expose underlying streams (transducer compatible :) - New Producer API support - Topic Administration - Updated to Kafka 0.8.2.1 - Removed Kafka deps from clj-kafka https://github.com/pingles/clj-kafka/ http://pingles.github.io/clj-kafka/ Zookeeper Consumer = We've broken the old messages function apart into two pieces: create-message-stream will connect the consumer to a message stream (topic + number of threads when consuming more than 1 partition) and stream-seq (recast the stream as a lazy sequence as with earlier releases). This change addresses a bunch of other requests/changes people had submitted. Hopefully most stuff that was a little tough to do before is now easier to integrate- composition with other libs like manifold[1] should be easier. Transducer Support This also means that the lib is easier to use with transducers[2] which should be useful for people doing stream processing. New Producer API == We have integrated support for the New Producer API[3,4]. This is asynchronous by default but provides both futures and callbacks to return the offset/error. Topic Administration A new namespace exists that wraps some of the Kafka admin utils for checking if topics exist, creating/removing topics and changing per-topic configuration. No Kafka deps in clj-kafka I think* this was inherited from back when the Kafka releases had no dependencies specified and so clj-kafka had to specify them instead. This is no longer necessary so the project config is substantially tidier. This is quite a substantial difference so please just check that this doesn't break anything you were depending on. 1) https://github.com/ztellman/manifold 2) https://github.com/pingles/clj-kafka/blob/master/README.md#usage-with-transducers 3) http://blog.confluent.io/2014/12/02/whats-coming-in-apache-kafka-0-8-2/ 4) http://kafka.apache.org/082/javadoc/org/apache/kafka/clients/producer/KafkaProducer.html 5) https://github.com/pingles/clj-kafka/blob/master/README.md#administration-operations -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: defrecord, equality, hashing, and performance
I bumped up the priority on CLJ-1224 to make sure it got pulled into 1.8 (based on being reminded about it here). I don't expect it to be included in 1.7. On Thursday, June 11, 2015 at 9:30:21 PM UTC-5, Mike Rodriguez wrote: I agree the hashCode performance for records is a concern due to that lack of caching. I noticed the priority of that Jira 1224 changed to critical about a week ago (June 3). I was curious why that was done and what that means in terms of prioritization. Last minute squeeze into CLJ version 1.7 release?! :) Too hopefully I'm sure. -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: [ANN] clj-kafka 0.3.1
Great stuff! Thanks Paul, Ragnar and others! -- Henrik On Fri, Jun 12, 2015 at 5:35 PM, Paul Ingles p...@oobaloo.co.uk wrote: Hi all, I'm delighted to say 0.3.1 is released and available on Clojars. It's been a long while since we've made any big changes to clj-kafka so I'm very happy to say we've caught up over the past couple of days. Most of the changes were the result of submissions from the community but I have to give most of the credit to Ragnar Dahlen :) Notable changes: - Refactored Zookeeper consumer to expose underlying streams (transducer compatible :) - New Producer API support - Topic Administration - Updated to Kafka 0.8.2.1 - Removed Kafka deps from clj-kafka https://github.com/pingles/clj-kafka/ http://pingles.github.io/clj-kafka/ Zookeeper Consumer = We've broken the old messages function apart into two pieces: create-message-stream will connect the consumer to a message stream (topic + number of threads when consuming more than 1 partition) and stream-seq (recast the stream as a lazy sequence as with earlier releases). This change addresses a bunch of other requests/changes people had submitted. Hopefully most stuff that was a little tough to do before is now easier to integrate- composition with other libs like manifold[1] should be easier. Transducer Support This also means that the lib is easier to use with transducers[2] which should be useful for people doing stream processing. New Producer API == We have integrated support for the New Producer API[3,4]. This is asynchronous by default but provides both futures and callbacks to return the offset/error. Topic Administration A new namespace exists that wraps some of the Kafka admin utils for checking if topics exist, creating/removing topics and changing per-topic configuration. No Kafka deps in clj-kafka I think* this was inherited from back when the Kafka releases had no dependencies specified and so clj-kafka had to specify them instead. This is no longer necessary so the project config is substantially tidier. This is quite a substantial difference so please just check that this doesn't break anything you were depending on. 1) https://github.com/ztellman/manifold 2) https://github.com/pingles/clj-kafka/blob/master/README.md#usage-with-transducers 3) http://blog.confluent.io/2014/12/02/whats-coming-in-apache-kafka-0-8-2/ 4) http://kafka.apache.org/082/javadoc/org/apache/kafka/clients/producer/KafkaProducer.html 5) https://github.com/pingles/clj-kafka/blob/master/README.md#administration-operations -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
