Re: defrecord, equality, hashing, and performance
Thanks for the insight Alex! -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: defrecord, equality, hashing, and performance
On Thursday, June 11, 2015 at 2:19:56 PM UTC-5, Andy Fingerhut wrote: You can override hashCode and/or hasheq methods in deftype. I've been unable to do it. Is there something special needed to allow this to work? user= (defrecord Foo [x] Object (hashCode [this] 42)) CompilerException java.lang.ClassFormatError: Duplicate method namesignature in class file user/Foo, compiling:(/private/var/folders/hv/hv6wzwrEFOujYN382REV6TI/-Tmp-/form-init7644007787939926514.clj:1:1) user= (defrecord Foo [x] Object (hasheq [this] 42)) CompilerException java.lang.ClassFormatError: Duplicate method namesignature in class file user/Foo, compiling:(/private/var/folders/hv/hv6wzwrEFOujYN382REV6TI/-Tmp-/form-init7644007787939926514.clj:1:1) If the reason that defrecord hashing is slow in your application is because _hasheq_ recalculates the hash values from scratch each time, without _caching the value_, consider voting for this ticket so that is improved in the future: http://dev.clojure.org/jira/browse/CLJ-1224 I'm not sure whether this might be a part of what I'm experiencing, but I didn't know about the ticket-voting process. I'll look into that. -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: defrecord, equality, hashing, and performance
Related to this, another thing I would like Clojure to provide is an easy way to opt-in for map-like function application for defrecords. Right now, Clojure doesn't implement function application for records, and it is a glaring incompatibility that causes bugs when you switch back and forth between records and maps to explore performance trade-offs. So it should be easy to opt-in. I assume the reasoning behind it comes from the notion that many people will want to create their own function application behavior for many types of records (which is true). Still, you should at least be able to *choose* to get default function application behavior, because it is tedious and error-prone to implement it on your own. Alternatively, if Clojure did what I suggested in my previous email, and allowed you to override functionality provided by defrecord, then defrecord could safely go ahead and implement function application behavior by default, because people would still be free to override that if they don't want that behavior. The other broader point here is that Clojure fails to provide good default, customizable abstract implementations of all its data structures. There is little to no documentation about what methods need to be overridden to get a minimal implementation of a given data structure. Clojurescript appears to be doing better on this front, since it was designed from the ground up with protocols in mind, but the lack of correspondence between Clojure's classes/abstract classes/interfaces and Clojurescript's implementation makes it difficult to write cross-platform data structures. -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: defrecord, equality, hashing, and performance
So just to explain this a little more, recently I wanted something record-like with custom hashing and equality for my ubergraph library. Unfortunately, you can't do this with defrecord, and starting from deftype and re-implementing all the map-like semantics from scratch is a total pain. So what I did is from potemkin, I referred def-map-type, and from potemkin.collections I referred AbstractMap. The type definition begins like this: (def-map-type Ubergraph [node-map allow-parallel? undirected? attrs cached-hash] AbstractMap and then you have your usual alternation of protocols and implementations. At the end, you must implement the following: (get [this key default-value] ...) (assoc [this key value] ...) (dissoc [this key] ...) (keys [this] ...) (meta [this] ...l) (with-meta [this meta] ...) Then, you have the flexibility to override hasheq and equiv: (hasheq [this] ...) (equiv [this other] ...) I'm not crazy about the fact that I had to implement get, assoc, dissoc, keys, meta, and with-meta in order to override hasheq and equiv. It was routine and tedious, and it is stuff that defrecord's macro does automatically for you. For example, my get implementation looks like this: (get [this key default-value] (case key :node-map node-map :allow-parallel? allow-parallel? :undirected? undirected? :attrs attrs :cached-hash cached-hash default-value)) just taking cases on all the keywords that refer to fields in my deftype. This means that any time you add a field to your record, you have to go back and tweak all those method implementations, which is annoying. Despite these complaints, it was, by far, the simplest way I was able to find to get a record/map-like structure with custom hashing and equality. Even though potemkin's def-map-type makes you implement those methods on your own, it does a lot of the grunt work of fulfilling other map methods that you probably wouldn't remember to implement everything by yourself. Right now Clojure offers this all-or-nothing approach: do it entirely yourself with deftype or use defrecord and lose the ability to customize. Clojure's position statement on this is that data structures generally fall cleanly into one of these two categories. I find that premise to be faulty. The simplest fix would be if Clojure simply allowed one to override the protocols which are implemented for you by defrecord, rather than throwing an error and saying that you can't redefine one of those protocols because they have already been implemented. Keep the errors for the case when you accidentally redefine the same method twice in your own implementation, but don't call it an error if you override the default implementation of something provided by the defrecord macro. -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: defrecord, equality, hashing, and performance
On Thursday, June 11, 2015 at 2:12:12 PM UTC-5, puzzler wrote: Zach Tellman's potemkin library includes several useful ways to tweak deftypes and defrecords. I wish Clojure itself provided more ways to do this, but in the meantime, potemkin is the best way to create something custom, like a deftype that behaves mostly like defrecord with some different hashing and equality behavior. Very interesting. Cool. I'll look at that. -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: defrecord, equality, hashing, and performance
deftype allows you to override hashCode and/or hasheq (I believe defaulting to identity-based implementations from java.lang.Object). defrecord does not. As mentioned in another message, potemkin may provide easier building blocks to build on than deftype. Andy On Thu, Jun 11, 2015 at 11:10 PM, Mars0i marsh...@logical.net wrote: On Thursday, June 11, 2015 at 2:19:56 PM UTC-5, Andy Fingerhut wrote: You can override hashCode and/or hasheq methods in deftype. I've been unable to do it. Is there something special needed to allow this to work? user= (defrecord Foo [x] Object (hashCode [this] 42)) CompilerException java.lang.ClassFormatError: Duplicate method namesignature in class file user/Foo, compiling:(/private/var/folders/hv/hv6wzwrEFOujYN382REV6TI/-Tmp-/form-init7644007787939926514.clj:1:1) user= (defrecord Foo [x] Object (hasheq [this] 42)) CompilerException java.lang.ClassFormatError: Duplicate method namesignature in class file user/Foo, compiling:(/private/var/folders/hv/hv6wzwrEFOujYN382REV6TI/-Tmp-/form-init7644007787939926514.clj:1:1) If the reason that defrecord hashing is slow in your application is because _hasheq_ recalculates the hash values from scratch each time, without _caching the value_, consider voting for this ticket so that is improved in the future: http://dev.clojure.org/jira/browse/CLJ-1224 I'm not sure whether this might be a part of what I'm experiencing, but I didn't know about the ticket-voting process. I'll look into that. -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: defrecord, equality, hashing, and performance
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 Prismatic has some document for deftype/defrecord/map, hope it's useful to you: https://github.com/Prismatic/eng-practices/blob/master/clojure/20130926-data-representation.md On 06/12/2015 02:36 AM, Mars0i wrote: I think that the following is all correct, but I could be wrong about something. The datatypes page at clojure.org http://clojure.org/datatypes says: defrecord provides ... value-based equality and hashCode, while deftype does not. So (defrecord Rec [x]) (= (Rec. 42) (Rec. 42)) ;= true but (deftype Typ [x]) (= (Typ. 42) (Typ. 42)) ;= false. This also means, as I understand it, that data structures that use hashing--maps and sets, for example--hash on value for records, but on identity for types. I believe that the same is true when Java hashtables compare records, or compare types, although I haven't carefully tested this. It makes sense that the primary, most fully functional user-definable datatypes in Clojure should use value-based equality; that's fully in the spirit of a functional language, and is very convenient in many contexts, and hashing should follow the equality semantics of the datatypes. You can always use identical? if you want to test records for identity rather than equality. However, you can't tell maps and sets to use identity for records, afaik. This makes records undesirable if you only care about identity, and need to use a data structure that uses hashing, and care about performance, because value-based hashing is presumably a lot slower than identity-based hashing (which is what my experiments seem to show). On the other hand, records are obviously way, way, more convenient than types in many contexts because the functionality provided by deftype is so bare-bones. It really bothers me that I have to choose between speed and using convenient, idiomatic functionality (defrecord) that is one of several significant reasons that I love Clojure. (This is in an application in which value-based equality would /never/ be what I wanted for my datatypes.) Should there be an alternative? Either a way to define e.g. a special kind of set that uses identity rather than value for records, or a way to make records use identity for equality, or another datatype that's just like defrecord but ..., or a way to get more functionality with deftype, or ...? (I don't think that Clojure should include every option or functionality that someone thinks might be convenient in some context. Another reason that I love Clojure is that it /doesn't/ do that, but instead provides limited, well-thought out options that allow a lot of flexibility.) In an earlier thread https://groups.google.com/forum/#!topic/clojure/EdjnSxRkOPk, Stephen Yi pointed out that although you're allowed to override some Object methods in defrecord, you can't override Object.equals and Object.hashCode. It isn't documented, but the compiler throws an exception if you try it. That makes quite a bit of sense, since changing the meaning of equality for a datatype would be an easy way to introduce confusing bugs. On the other hand, if you do override Object.equals and Object.hashCode for defrecord, you probably know that you're doing something weird, and that you'd better take precautions, if necessary, to avoid later confusion. -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com mailto:clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout. -BEGIN PGP SIGNATURE- Version: GnuPG v2 iQEcBAEBCAAGBQJVen1zAAoJEBnCozJnPTdiq6UH/3pKe17SGwNV0a/PajDOLbfe R86c/U0faXgtr4hL4IrhSEK9HWlGHWy8FHE57TYQstc8OSryJXsbNp3/OjpfIpfB QBpQwgBmwkrA/EkLJEmBsGu50R8nQgwLiG6S0T8gSFPUSM2NkQJyBkczTz0AVX7y bUjliAb3lGaWG8t4uJhPt0feq+acfpjR1bG5V6ckISFtcJW+N8aG0lQKkDmTqcQZ AppdpFrjiDw+SoyPcEB+vpRyQxwnxYk/g9qZHjEoFi7DG0e59gHFG19dLlXeH5Nv tKR6rRGXsJQROxdxGWNS2HYa4ASSJJd80reoehl341A2xZ0z46SASw51T2QqpZs= =3/P/ -END PGP SIGNATURE- -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this
Re: defrecord, equality, hashing, and performance
Re IFn support for defrecord, I don't actually know the reason that's not built-in. I am not aware of an existing ticket for this. Of course, at this point many people patch it in, so we would need to consider potential breakage from that. -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: defrecord, equality, hashing, and performance
On Friday, June 12, 2015 at 11:46:17 AM UTC-4, Mars0i wrote: Oh, yes, and that's an interesting idea to wrap a record in an atom or delay. For my present uses that would be more trouble than it's worth, but it's something worth keeping in mind for other situations. Putting a unique primary key like value, such as a UUID, into each new instance will also effectively cause identity semantics. -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: defrecord, equality, hashing, and performance
On Friday, June 12, 2015 at 1:34:20 AM UTC-5, Andy Fingerhut wrote: deftype allows you to override hashCode and/or hasheq (I believe defaulting to identity-based implementations from java.lang.Object). defrecord does not. Sorry--I misread your earlier statement about this. That's good to know, although for my present application, deftype's hashing behavior is all that I need. I miss defrecord's advantages when I use deftype, but will look at potemkin, as suggested. -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: defrecord, equality, hashing, and performance
Oh, yes, and that's an interesting idea to wrap a record in an atom or delay. For my present uses that would be more trouble than it's worth, but it's something worth keeping in mind for other situations. On Friday, June 12, 2015 at 10:41:57 AM UTC-5, Mars0i wrote: puzzler, thanks for the explanation about how you built on potemkin's map options. Probably not useful for my current application--easier to just work around deftype's limitations on an ad-hoc basis. But that takes some of the fun (convenience) out of Clojure (and unnecessarily, I feel, I guess). -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: defrecord, equality, hashing, and performance
puzzler, thanks for the explanation about how you built on potemkin's map options. Probably not useful for my current application--easier to just work around deftype's limitations on an ad-hoc basis. But that takes some of the fun (convenience) out of Clojure (and unnecessarily, I feel, I guess). -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: defrecord, equality, hashing, and performance
I bumped up the priority on CLJ-1224 to make sure it got pulled into 1.8 (based on being reminded about it here). I don't expect it to be included in 1.7. On Thursday, June 11, 2015 at 9:30:21 PM UTC-5, Mike Rodriguez wrote: I agree the hashCode performance for records is a concern due to that lack of caching. I noticed the priority of that Jira 1224 changed to critical about a week ago (June 3). I was curious why that was done and what that means in terms of prioritization. Last minute squeeze into CLJ version 1.7 release?! :) Too hopefully I'm sure. -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: defrecord, equality, hashing, and performance
Zach Tellman's potemkin library includes several useful ways to tweak deftypes and defrecords. I wish Clojure itself provided more ways to do this, but in the meantime, potemkin is the best way to create something custom, like a deftype that behaves mostly like defrecord with some different hashing and equality behavior. On Thu, Jun 11, 2015 at 11:36 AM, Mars0i marsh...@logical.net wrote: I think that the following is all correct, but I could be wrong about something. The datatypes page at clojure.org http://clojure.org/datatypes says: defrecord provides ... value-based equality and hashCode, while deftype does not. So (defrecord Rec [x]) (= (Rec. 42) (Rec. 42)) ;= true but (deftype Typ [x]) (= (Typ. 42) (Typ. 42)) ;= false. This also means, as I understand it, that data structures that use hashing--maps and sets, for example--hash on value for records, but on identity for types. I believe that the same is true when Java hashtables compare records, or compare types, although I haven't carefully tested this. It makes sense that the primary, most fully functional user-definable datatypes in Clojure should use value-based equality; that's fully in the spirit of a functional language, and is very convenient in many contexts, and hashing should follow the equality semantics of the datatypes. You can always use identical? if you want to test records for identity rather than equality. However, you can't tell maps and sets to use identity for records, afaik. This makes records undesirable if you only care about identity, and need to use a data structure that uses hashing, and care about performance, because value-based hashing is presumably a lot slower than identity-based hashing (which is what my experiments seem to show). On the other hand, records are obviously way, way, more convenient than types in many contexts because the functionality provided by deftype is so bare-bones. It really bothers me that I have to choose between speed and using convenient, idiomatic functionality (defrecord) that is one of several significant reasons that I love Clojure. (This is in an application in which value-based equality would *never* be what I wanted for my datatypes.) Should there be an alternative? Either a way to define e.g. a special kind of set that uses identity rather than value for records, or a way to make records use identity for equality, or another datatype that's just like defrecord but ..., or a way to get more functionality with deftype, or ...? (I don't think that Clojure should include every option or functionality that someone thinks might be convenient in some context. Another reason that I love Clojure is that it *doesn't* do that, but instead provides limited, well-thought out options that allow a lot of flexibility.) In an earlier thread https://groups.google.com/forum/#!topic/clojure/EdjnSxRkOPk, Stephen Yi pointed out that although you're allowed to override some Object methods in defrecord, you can't override Object.equals and Object.hashCode. It isn't documented, but the compiler throws an exception if you try it. That makes quite a bit of sense, since changing the meaning of equality for a datatype would be an easy way to introduce confusing bugs. On the other hand, if you do override Object.equals and Object.hashCode for defrecord, you probably know that you're doing something weird, and that you'd better take precautions, if necessary, to avoid later confusion. -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: defrecord, equality, hashing, and performance
You can override hashCode and/or hasheq methods in deftype. If the reason that defrecord hashing is slow in your application is because hashed recalculates the hash values from scratch each time, without hashing, consider voting for this ticket so that is improved in the future: http://dev.clojure.org/jira/browse/CLJ-1224 Andy On Thu, Jun 11, 2015 at 11:36 AM, Mars0i marsh...@logical.net wrote: I think that the following is all correct, but I could be wrong about something. The datatypes page at clojure.org http://clojure.org/datatypes says: defrecord provides ... value-based equality and hashCode, while deftype does not. So (defrecord Rec [x]) (= (Rec. 42) (Rec. 42)) ;= true but (deftype Typ [x]) (= (Typ. 42) (Typ. 42)) ;= false. This also means, as I understand it, that data structures that use hashing--maps and sets, for example--hash on value for records, but on identity for types. I believe that the same is true when Java hashtables compare records, or compare types, although I haven't carefully tested this. It makes sense that the primary, most fully functional user-definable datatypes in Clojure should use value-based equality; that's fully in the spirit of a functional language, and is very convenient in many contexts, and hashing should follow the equality semantics of the datatypes. You can always use identical? if you want to test records for identity rather than equality. However, you can't tell maps and sets to use identity for records, afaik. This makes records undesirable if you only care about identity, and need to use a data structure that uses hashing, and care about performance, because value-based hashing is presumably a lot slower than identity-based hashing (which is what my experiments seem to show). On the other hand, records are obviously way, way, more convenient than types in many contexts because the functionality provided by deftype is so bare-bones. It really bothers me that I have to choose between speed and using convenient, idiomatic functionality (defrecord) that is one of several significant reasons that I love Clojure. (This is in an application in which value-based equality would *never* be what I wanted for my datatypes.) Should there be an alternative? Either a way to define e.g. a special kind of set that uses identity rather than value for records, or a way to make records use identity for equality, or another datatype that's just like defrecord but ..., or a way to get more functionality with deftype, or ...? (I don't think that Clojure should include every option or functionality that someone thinks might be convenient in some context. Another reason that I love Clojure is that it *doesn't* do that, but instead provides limited, well-thought out options that allow a lot of flexibility.) In an earlier thread https://groups.google.com/forum/#!topic/clojure/EdjnSxRkOPk, Stephen Yi pointed out that although you're allowed to override some Object methods in defrecord, you can't override Object.equals and Object.hashCode. It isn't documented, but the compiler throws an exception if you try it. That makes quite a bit of sense, since changing the meaning of equality for a datatype would be an easy way to introduce confusing bugs. On the other hand, if you do override Object.equals and Object.hashCode for defrecord, you probably know that you're doing something weird, and that you'd better take precautions, if necessary, to avoid later confusion. -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: defrecord, equality, hashing, and performance
Ugh. Too many typos there. Here is what I meant: If the reason that defrecord hashing is slow in your application is because _hasheq_ recalculates the hash values from scratch each time, without _caching the value_, consider voting for this ticket so that is improved in the future: http://dev.clojure.org/jira/browse/CLJ-1224 On Thu, Jun 11, 2015 at 12:19 PM, Andy Fingerhut andy.finger...@gmail.com wrote: You can override hashCode and/or hasheq methods in deftype. If the reason that defrecord hashing is slow in your application is because hashed recalculates the hash values from scratch each time, without hashing, consider voting for this ticket so that is improved in the future: http://dev.clojure.org/jira/browse/CLJ-1224 Andy On Thu, Jun 11, 2015 at 11:36 AM, Mars0i marsh...@logical.net wrote: I think that the following is all correct, but I could be wrong about something. The datatypes page at clojure.org http://clojure.org/datatypes says: defrecord provides ... value-based equality and hashCode, while deftype does not. So (defrecord Rec [x]) (= (Rec. 42) (Rec. 42)) ;= true but (deftype Typ [x]) (= (Typ. 42) (Typ. 42)) ;= false. This also means, as I understand it, that data structures that use hashing--maps and sets, for example--hash on value for records, but on identity for types. I believe that the same is true when Java hashtables compare records, or compare types, although I haven't carefully tested this. It makes sense that the primary, most fully functional user-definable datatypes in Clojure should use value-based equality; that's fully in the spirit of a functional language, and is very convenient in many contexts, and hashing should follow the equality semantics of the datatypes. You can always use identical? if you want to test records for identity rather than equality. However, you can't tell maps and sets to use identity for records, afaik. This makes records undesirable if you only care about identity, and need to use a data structure that uses hashing, and care about performance, because value-based hashing is presumably a lot slower than identity-based hashing (which is what my experiments seem to show). On the other hand, records are obviously way, way, more convenient than types in many contexts because the functionality provided by deftype is so bare-bones. It really bothers me that I have to choose between speed and using convenient, idiomatic functionality (defrecord) that is one of several significant reasons that I love Clojure. (This is in an application in which value-based equality would *never* be what I wanted for my datatypes.) Should there be an alternative? Either a way to define e.g. a special kind of set that uses identity rather than value for records, or a way to make records use identity for equality, or another datatype that's just like defrecord but ..., or a way to get more functionality with deftype, or ...? (I don't think that Clojure should include every option or functionality that someone thinks might be convenient in some context. Another reason that I love Clojure is that it *doesn't* do that, but instead provides limited, well-thought out options that allow a lot of flexibility.) In an earlier thread https://groups.google.com/forum/#!topic/clojure/EdjnSxRkOPk, Stephen Yi pointed out that although you're allowed to override some Object methods in defrecord, you can't override Object.equals and Object.hashCode. It isn't documented, but the compiler throws an exception if you try it. That makes quite a bit of sense, since changing the meaning of equality for a datatype would be an easy way to introduce confusing bugs. On the other hand, if you do override Object.equals and Object.hashCode for defrecord, you probably know that you're doing something weird, and that you'd better take precautions, if necessary, to avoid later confusion. -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this
Re: defrecord, equality, hashing, and performance
Yes, please vote for that issue. I find myself frequently having to work around this limitation of records in Clojure; I mostly avoid using records directly as a consequence of this performance issue. As a side note, one quick-and-dirty way to get identity semantics for your data is to wrap each piece of data in your application in a delay or atom. The wrapper will give all your data a unique identity-based hash value and equality semantics. On Thu, Jun 11, 2015 at 12:29 PM, Andy Fingerhut andy.finger...@gmail.com wrote: If the reason that defrecord hashing is slow in your application is because _hasheq_ recalculates the hash values from scratch each time, without _caching the value_, consider voting for this ticket so that is improved in the future: http://dev.clojure.org/jira/browse/CLJ-1224 -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: defrecord, equality, hashing, and performance
I agree the hashCode performance for records is a concern due to that lack of caching. I noticed the priority of that Jira 1224 changed to critical about a week ago (June 3). I was curious why that was done and what that means in terms of prioritization. Last minute squeeze into CLJ version 1.7 release?! :) Too hopefully I'm sure. -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
