Re: Question regarding java array

2015-06-11 Thread Ritchie Cai
I tried with vectorz here: 
https://github.com/malloc82/imaging/blob/45475b99f564b1ac77e668e04b91cb9c01a096d7/src/imaging/dicom.clj#L130-L161
 
and I'm really impressed with it's performance.
The performance is shown here:

imaging.dicom (def data3 (timer total:  (load-txt-image_matrix 
resources/PCT/CTP404_merged/x_11.txt)))
  == timing:  loading txt data 728.267621 ms
  == timing:   max 25.00652 ms
  == timing:   min 25.575979 ms
  == timing:   update pixel values 111.647122 ms
  == timing:   total:  926.00495 ms

Just wondering though, is there a faster way to load an array than this 
way? 
https://github.com/malloc82/imaging/blob/45475b99f564b1ac77e668e04b91cb9c01a096d7/src/imaging/dicom.clj#L138
the data file I'm trying to read from contains text based pixel values.

Thanks.

On Wednesday, June 10, 2015 at 5:01:25 PM UTC-5, Mikera wrote:

 Consider using core.matrix with vectorz-clj for operations on large 
 numerical arrays / vectors of doubles. It is a *lot* faster than using 
 Clojure vectors for this kind of scenario, plus it has a lot of helpful 
 array operations already defined.

 (use 'clojure.core.matrix)
 (def v (array :vectorz (range 128)))

 (time (emax v))
 = 127.0
 Elapsed time: 1.179533 msecs


 On Wednesday, 10 June 2015 21:07:09 UTC+1, Ritchie Cai wrote:

 I'm working on a java array of double with 128 elements. I need the 
 max and min values of the array. So I initially tried areduce and loop, 
 both gives runs around 20 seconds. But when try (apply max (vec array)) I 
 get result under 90 ms.
 Can anyone explain why there is such a big difference?
 Also if want to iterate large java array like this to do some other 
 operations, e.g. convolution, what's the best way to go? Is there another 
 fast way to iterate through array or do I need to convert array into vector?

 Thanks
 Ritchie



-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


Re: Question regarding java array

2015-06-11 Thread Ritchie Cai
Yup. Reflection is issue, I needed type hint. 
However, on another note, I notice that in your first test case, your 
evaluation takes about 3 ms, but on my machine it takes 76 ms. I'm running 
a Xeon CPU at 3.5 GHZ, clojure-1.7-RC1. What could cause such a huge 
different timing?

Thanks.

On Wednesday, June 10, 2015 at 8:04:00 PM UTC-5, Steven Yi wrote:

 As mentioned by Colin and Andy, I would guess it would be some form of 
 boxing and reflection going on.  I tried the following:

 (defn array-max [^doubles arr]

   (let [len (alength arr)]

 (loop [m Double/NEGATIVE_INFINITY indx 0]

   (if ( indx len)

 (recur (max m (aget arr indx)) (unchecked-inc indx))

 m


 user= (let [vs (amap (double-array 128) idx ret (Math/random))] 

(time (array-max vs))) 

 Elapsed time: 3.719835 msecs

 To note, if you check out the source of areduce:

 user= (source areduce)

 (defmacro areduce

   Reduces an expression across an array a, using an index named idx,

   and return value named ret, initialized to init, setting ret to the 

   evaluation of expr at each step, returning ret.

   {:added 1.0}

   [a idx ret init expr]

   `(let [a# ~a]

  (loop  [~idx 0 ~ret ~init]

(if ( ~idx  (alength a#))

  (recur (unchecked-inc ~idx) ~expr)

  ~ret


 It's just a macro, and so typehinting is going to play a factor.  For 
 example, with areduce and a type hint on the array:


 (defn array-max2 [^doubles arr]

   (areduce arr idx ret Double/NEGATIVE_INFINITY (max ret (aget arr idx

 user= (let [vs (amap (double-array 128) idx ret (Math/random))] (time 
 (array-max vs))) 

  Elapsed time: 3.314599 msecs


 But with no type hint on arr:


 (defn array-max2 [arr]

   (areduce arr idx ret Double/NEGATIVE_INFINITY (max ret (aget arr idx

 user= (let [vs (amap (double-array 128) idx ret (Math/random))] (time 
 (array-max2 vs))) 

 Elapsed time: 35612.919192 msecs


 Without a typehint on the arr argument, I also do get boxed math and 
 reflection warnings:


 Reflection warning, 
 /private/var/folders/0k/xj_drd990xxf4q99n2bdknrcgn/T/form-init1595291808747030463.clj:2:3
  
 - call to static method alength on clojure.lang.RT can't be resolved 
 (argument types: unknown).

 Boxed math warning, 
 /private/var/folders/0k/xj_drd990xxf4q99n2bdknrcgn/T/form-init1595291808747030463.clj:2:3
  
 - call: public static boolean clojure.lang.Numbers.lt
 (long,java.lang.Object).

 Reflection warning, 
 /private/var/folders/0k/xj_drd990xxf4q99n2bdknrcgn/T/form-init1595291808747030463.clj:2:58
  
 - call to static method aget on clojure.lang.RT can't be resolved (argument 
 types: unknown, int).

 Boxed math warning, 
 /private/var/folders/0k/xj_drd990xxf4q99n2bdknrcgn/T/form-init1595291808747030463.clj:2:49
  
 - call: public static java.lang.Object 
 clojure.lang.Numbers.max(double,java.lang.Object).

 form-init1595291808747030463.clj:2 recur arg for primitive local: ret is 
 not matching primitive, had: Object, needed: double

 Auto-boxing loop arg: ret

 Reflection warning, 
 /private/var/folders/0k/xj_drd990xxf4q99n2bdknrcgn/T/form-init1595291808747030463.clj:2:3
  
 - call to static method alength on clojure.lang.RT can't be resolved 
 (argument types: unknown).

 Boxed math warning, 
 /private/var/folders/0k/xj_drd990xxf4q99n2bdknrcgn/T/form-init1595291808747030463.clj:2:3
  
 - call: public static boolean clojure.lang.Numbers.lt
 (long,java.lang.Object).

 Reflection warning, 
 /private/var/folders/0k/xj_drd990xxf4q99n2bdknrcgn/T/form-init1595291808747030463.clj:2:58
  
 - call to static method aget on clojure.lang.RT can't be resolved (argument 
 types: unknown, int).

 Boxed math warning, 
 /private/var/folders/0k/xj_drd990xxf4q99n2bdknrcgn/T/form-init1595291808747030463.clj:2:49
  
 - call: public static java.lang.Object 
 clojure.lang.Numbers.max(java.lang.Object,java.lang.Object).



 On Wednesday, June 10, 2015 at 4:07:09 PM UTC-4, Ritchie Cai wrote:

 I'm working on a java array of double with 128 elements. I need the 
 max and min values of the array. So I initially tried areduce and loop, 
 both gives runs around 20 seconds. But when try (apply max (vec array)) I 
 get result under 90 ms.
 Can anyone explain why there is such a big difference?
 Also if want to iterate large java array like this to do some other 
 operations, e.g. convolution, what's the best way to go? Is there another 
 fast way to iterate through array or do I need to convert array into vector?

 Thanks
 Ritchie



-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You 

Re: Question regarding java array

2015-06-11 Thread Colin Yates
 Lesson learned here for me is that only use java array when absolutely 
 necessary. I always thought since it's primitive array, it should be the 
 fastest. Apparently not!
This bears repeating. I often find it hit-and-miss to know when idiomatic 
Clojure will be faster than turning to Java. Are there any resources (blogs, 
books etc) to help increase intuition as to where to go? Other than time, the 
REPL and criterium of course :)

 On 11 Jun 2015, at 10:26, Ritchie Cai ritchie...@gmail.com wrote:
 
 Yup. As it turns out, the slow down is pretty much due to reflection. After 
 added type hint, it's much better now.
 The code can be viewed here: 
 https://github.com/malloc82/imaging/blob/45475b99f564b1ac77e668e04b91cb9c01a096d7/src/imaging/dicom.clj#L46-L161
 
 They are three different implementations, here is the performance for each 
 one:
 imaging.dicom (def data1 (timer total:  (load-txt-image 
 resources/PCT/CTP404_merged/x_11.txt)))
   == timing:  loading txt data 513.267672 ms
   == timing:   max 61.782047 ms
   == timing:   min 62.4488 ms
   == timing:   update pixel values 281.222481 ms
   == timing:   total:  952.981063 ms
 #'imaging.dicom/data1
 imaging.dicom (def data3 (timer total:  (load-txt-image_matrix 
 resources/PCT/CTP404_merged/x_11.txt)))
   == timing:  loading txt data 728.267621 ms
   == timing:   max 25.00652 ms
   == timing:   min 25.575979 ms
   == timing:   update pixel values 111.647122 ms
   == timing:   total:  926.00495 ms
 #'imaging.dicom/data3
 imaging.dicom (def data2 (timer total:  (load-txt-image_array 
 resources/PCT/CTP404_merged/x_11.txt)))
   == timing:  loading txt data 664.818514 ms
   == timing:min max :  429.855274 ms
   == timing:   update pixel values 361.323422 ms
   == timing:   total:  1491.792197 ms
 #'imaging.dicom/data2
 imaging.dicom 
 
 The core.matrix one is the fastest, persistent vector is the second, but very 
 close. Double array is actually the slowest one.
 Lesson learned here for me is that only use java array when absolutely 
 necessary. I always thought since it's primitive array, it should be the 
 fastest. Apparently not!
 
 Thanks for the hint.
 
 On Wednesday, June 10, 2015 at 3:08:44 PM UTC-5, Andy Fingerhut wrote:
 Add this line at the top of your file where you try areduce and loop, and 
 look for any reflection warnings that occur when loading the file:
 
 (set! *warn-on-reflection* true)
 
 If there are none, probably best to post a link to your code, or paste it in 
 a message here if it is short enough, so others can give more precise 
 suggestions.
 
 Andy
 
 On Wed, Jun 10, 2015 at 1:03 PM, Ritchie Cai ritch...@gmail.com 
 javascript: wrote:
 I'm working on a java array of double with 128 elements. I need the max 
 and min values of the array. So I initially tried areduce and loop, both 
 gives runs around 20 seconds. But when try (apply max (vec array)) I get 
 result under 90 ms.
 Can anyone explain why there is such a big difference?
 Also if want to iterate large java array like this to do some other 
 operations, e.g. convolution, what's the best way to go? Is there another 
 fast way to iterate through array or do I need to convert array into vector?
 
 Thanks
 Ritchie
 
 
 -- 
 You received this message because you are subscribed to the Google
 Groups Clojure group.
 To post to this group, send email to clo...@googlegroups.com javascript:
 Note that posts from new members are moderated - please be patient with your 
 first post.
 To unsubscribe from this group, send email to
 clojure+u...@googlegroups.com javascript:
 For more options, visit this group at
 http://groups.google.com/group/clojure?hl=en 
 http://groups.google.com/group/clojure?hl=en
 --- 
 You received this message because you are subscribed to the Google Groups 
 Clojure group.
 To unsubscribe from this group and stop receiving emails from it, send an 
 email to clojure+u...@googlegroups.com javascript:.
 For more options, visit https://groups.google.com/d/optout 
 https://groups.google.com/d/optout.
 
 
 -- 
 You received this message because you are subscribed to the Google
 Groups Clojure group.
 To post to this group, send email to clojure@googlegroups.com
 Note that posts from new members are moderated - please be patient with your 
 first post.
 To unsubscribe from this group, send email to
 clojure+unsubscr...@googlegroups.com
 For more options, visit this group at
 http://groups.google.com/group/clojure?hl=en 
 http://groups.google.com/group/clojure?hl=en
 --- 
 You received this message because you are subscribed to the Google Groups 
 Clojure group.
 To unsubscribe from this group and stop receiving emails from it, send an 
 email to clojure+unsubscr...@googlegroups.com 
 mailto:clojure+unsubscr...@googlegroups.com.
 For more options, visit https://groups.google.com/d/optout 
 https://groups.google.com/d/optout.

-- 
You 

Re: Question regarding java array

2015-06-11 Thread Ritchie Cai
Yup. As it turns out, the slow down is pretty much due to reflection. After 
added type hint, it's much better now.
The code can be viewed here: 
https://github.com/malloc82/imaging/blob/45475b99f564b1ac77e668e04b91cb9c01a096d7/src/imaging/dicom.clj#L46-L161

They are three different implementations, here is the performance for each 
one:
imaging.dicom (def data1 (timer total:  (load-txt-image 
resources/PCT/CTP404_merged/x_11.txt)))
  == timing:  loading txt data 513.267672 ms
  == timing:   max 61.782047 ms
  == timing:   min 62.4488 ms
  == timing:   update pixel values 281.222481 ms
  == timing:   total:  952.981063 ms
#'imaging.dicom/data1
imaging.dicom (def data3 (timer total:  (load-txt-image_matrix 
resources/PCT/CTP404_merged/x_11.txt)))
  == timing:  loading txt data 728.267621 ms
  == timing:   max 25.00652 ms
  == timing:   min 25.575979 ms
  == timing:   update pixel values 111.647122 ms
  == timing:   total:  926.00495 ms
#'imaging.dicom/data3
imaging.dicom (def data2 (timer total:  (load-txt-image_array 
resources/PCT/CTP404_merged/x_11.txt)))
  == timing:  loading txt data 664.818514 ms
  == timing:min max :  429.855274 ms
  == timing:   update pixel values 361.323422 ms
  == timing:   total:  1491.792197 ms
#'imaging.dicom/data2
imaging.dicom 

The core.matrix one is the fastest, persistent vector is the second, but 
very close. Double array is actually the slowest one.
Lesson learned here for me is that only use java array when absolutely 
necessary. I always thought since it's primitive array, it should be the 
fastest. Apparently not!

Thanks for the hint.

On Wednesday, June 10, 2015 at 3:08:44 PM UTC-5, Andy Fingerhut wrote:

 Add this line at the top of your file where you try areduce and loop, and 
 look for any reflection warnings that occur when loading the file:

 (set! *warn-on-reflection* true)

 If there are none, probably best to post a link to your code, or paste it 
 in a message here if it is short enough, so others can give more precise 
 suggestions.

 Andy

 On Wed, Jun 10, 2015 at 1:03 PM, Ritchie Cai ritch...@gmail.com 
 javascript: wrote:

 I'm working on a java array of double with 128 elements. I need the 
 max and min values of the array. So I initially tried areduce and loop, 
 both gives runs around 20 seconds. But when try (apply max (vec array)) I 
 get result under 90 ms.
 Can anyone explain why there is such a big difference?
 Also if want to iterate large java array like this to do some other 
 operations, e.g. convolution, what's the best way to go? Is there another 
 fast way to iterate through array or do I need to convert array into vector?

 Thanks
 Ritchie

  -- 
 You received this message because you are subscribed to the Google
 Groups Clojure group.
 To post to this group, send email to clo...@googlegroups.com 
 javascript:
 Note that posts from new members are moderated - please be patient with 
 your first post.
 To unsubscribe from this group, send email to
 clojure+u...@googlegroups.com javascript:
 For more options, visit this group at
 http://groups.google.com/group/clojure?hl=en
 --- 
 You received this message because you are subscribed to the Google Groups 
 Clojure group.
 To unsubscribe from this group and stop receiving emails from it, send an 
 email to clojure+u...@googlegroups.com javascript:.
 For more options, visit https://groups.google.com/d/optout.




-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


Re: Question regarding java array

2015-06-11 Thread Ritchie Cai
Unfortunately, I don't get to decide the data format. It's a dump from 
previous stage. Also, it's supposed to be super easy for Physics people to 
look at. If you every work with them, you'll know what I mean XD.

But thanks for the suggestion.

On Thursday, June 11, 2015 at 7:30:07 AM UTC-5, Andy- wrote:

 On Thursday, June 11, 2015 at 5:32:02 AM UTC-4, Ritchie Cai wrote:

 Just wondering though, is there a faster way to load an array than this 
 way? 
 https://github.com/malloc82/imaging/blob/45475b99f564b1ac77e668e04b91cb9c01a096d7/src/imaging/dicom.clj#L138
 the data file I'm trying to read from contains text based pixel values.


  I'm not sure about your use case but you may want to look into HDF5 data 
 format. That's what it's made for: Super fast loading of numerical data 
 from disk. It can even load in parallel (many hundreds MB/s). As soon as my 
 text file takes too long to load I usually just convert it first to HDF5 
 and then go from there. Linux/Mac has HDF tools which can dump and import 
 (h5import) your data from the command line.
 It's also a very common data format.

 HTH



-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


Re: Question regarding java array

2015-06-11 Thread Steven Yi
I'm not sure why you'd see much slower results there.  For reference,
I'm on a Core i7-2720-M (MacbookPro 8,1 13 early 2011), and was using
clojure-1.7-beta3.

Also, I looked at the code you posted and I'm not so sure about your
assumption that Java arrays are slower:

* in load-txt-image_array, you could probably type hint the data var
in the first let binding as ^doubles.  With that, you should be able
to get rid of the type hinting throughout the rest of the function.

* In your areduce code you're using a vector to carry the result,
which requires packing and unpacking, which ends up being somewhat
like auto-boxing. Using a loop-recur would allow you to carry over the
min and max separately between steps, something like:

(let [len (alength data)]
  (loop [i 0 my-min 0.0 my-max 0.0]
(if ( i len)
   (let [v (aget data i)]
 (recur (unchecked-inc i) (Math/min my-min v) (Math/max my-max v)))
   [my-min my-max])))

(could also use min and max instead of Math/min and Math/max)

* In the update pixel values part of the function, you're using a
doseq with a range.  That'd cause a sequence of boxed numbers of be
generated. Even though you have a ^double as a type hint, which will
get you out of the boxed math warning, there's still boxing going on
and you'll still first getting a boxed number and then have a cast to
primitive double.  For example, if you use this function:

user= (defn a [] (doseq [i (range 50)] (println (+ ^double i 1.0

and use no.disassemble, you'll find byte code like this:

278  checkcast java.lang.Number [131]
281  invokestatic
clojure.lang.RT.uncheckedDoubleCast(java.lang.Object) : double [135]
284  dconst_1
285  invokestatic clojure.lang.Numbers.unchecked_add(double,
double) : double [141]

I'd try using a loop-recur here as well instead of the doseq.

As a sidenote, if haven't looked, you might give Prismatic's hiphip[1]
library a try.

[1] - https://github.com/prismatic/hiphip

On Thu, Jun 11, 2015 at 5:43 AM, Ritchie Cai ritchie...@gmail.com wrote:
 Yup. Reflection is issue, I needed type hint.
 However, on another note, I notice that in your first test case, your
 evaluation takes about 3 ms, but on my machine it takes 76 ms. I'm running a
 Xeon CPU at 3.5 GHZ, clojure-1.7-RC1. What could cause such a huge different
 timing?

 Thanks.


 On Wednesday, June 10, 2015 at 8:04:00 PM UTC-5, Steven Yi wrote:

 As mentioned by Colin and Andy, I would guess it would be some form of
 boxing and reflection going on.  I tried the following:

 (defn array-max [^doubles arr]

   (let [len (alength arr)]

 (loop [m Double/NEGATIVE_INFINITY indx 0]

   (if ( indx len)

 (recur (max m (aget arr indx)) (unchecked-inc indx))

 m


 user= (let [vs (amap (double-array 128) idx ret (Math/random))]

(time (array-max vs)))

 Elapsed time: 3.719835 msecs


 To note, if you check out the source of areduce:

 user= (source areduce)

 (defmacro areduce

   Reduces an expression across an array a, using an index named idx,

   and return value named ret, initialized to init, setting ret to the

   evaluation of expr at each step, returning ret.

   {:added 1.0}

   [a idx ret init expr]

   `(let [a# ~a]

  (loop  [~idx 0 ~ret ~init]

(if ( ~idx  (alength a#))

  (recur (unchecked-inc ~idx) ~expr)

  ~ret


 It's just a macro, and so typehinting is going to play a factor.  For
 example, with areduce and a type hint on the array:


 (defn array-max2 [^doubles arr]

   (areduce arr idx ret Double/NEGATIVE_INFINITY (max ret (aget arr idx

 user= (let [vs (amap (double-array 128) idx ret (Math/random))] (time
 (array-max vs)))

 Elapsed time: 3.314599 msecs


 But with no type hint on arr:


 (defn array-max2 [arr]

   (areduce arr idx ret Double/NEGATIVE_INFINITY (max ret (aget arr idx


 user= (let [vs (amap (double-array 128) idx ret (Math/random))] (time
 (array-max2 vs)))

 Elapsed time: 35612.919192 msecs


 Without a typehint on the arr argument, I also do get boxed math and
 reflection warnings:


 Reflection warning,
 /private/var/folders/0k/xj_drd990xxf4q99n2bdknrcgn/T/form-init1595291808747030463.clj:2:3
 - call to static method alength on clojure.lang.RT can't be resolved
 (argument types: unknown).

 Boxed math warning,
 /private/var/folders/0k/xj_drd990xxf4q99n2bdknrcgn/T/form-init1595291808747030463.clj:2:3
 - call: public static boolean
 clojure.lang.Numbers.lt(long,java.lang.Object).

 Reflection warning,
 /private/var/folders/0k/xj_drd990xxf4q99n2bdknrcgn/T/form-init1595291808747030463.clj:2:58
 - call to static method aget on clojure.lang.RT can't be resolved (argument
 types: unknown, int).

 Boxed math warning,
 /private/var/folders/0k/xj_drd990xxf4q99n2bdknrcgn/T/form-init1595291808747030463.clj:2:49
 - call: public static java.lang.Object
 clojure.lang.Numbers.max(double,java.lang.Object).

 form-init1595291808747030463.clj:2 recur arg for 

defrecord, equality, hashing, and performance

2015-06-11 Thread Mars0i
I think that the following is all correct, but I could be wrong about 
something.

The datatypes page at clojure.org http://clojure.org/datatypes says: 
defrecord provides ... value-based equality and hashCode, while deftype 
does not.   So
(defrecord Rec [x])
(= (Rec. 42) (Rec. 42)) ;= true
but
(deftype Typ [x])
(= (Typ. 42) (Typ. 42)) ;= false.
This also means, as I understand it, that data structures that use 
hashing--maps and sets, for example--hash on value for records, but on 
identity for types.  I believe that the same is true when Java hashtables 
compare records, or compare types, although I haven't carefully tested this.

It makes sense that the primary, most fully functional user-definable 
datatypes in Clojure should use value-based equality; that's fully in the 
spirit of a functional language, and is very convenient in many contexts, 
and hashing should follow the equality semantics of the datatypes.  You can 
always use identical? if you want to test records for identity rather than 
equality.

However, you can't tell maps and sets to use identity for records, afaik.  
This makes records undesirable if you only care about identity, and need to 
use a data structure that uses hashing, and care about performance, because 
value-based hashing is presumably a lot slower than identity-based hashing 
(which is what my experiments seem to show).

On the other hand, records are obviously way, way, more convenient than 
types in many contexts because the functionality provided by deftype is so 
bare-bones.  It really bothers me that I have to choose between speed and 
using convenient, idiomatic functionality (defrecord) that is one of 
several significant reasons that I love Clojure.  (This is in an 
application in which value-based equality would *never* be what I wanted 
for my datatypes.)

Should there be an alternative?  Either a way to define e.g. a special kind 
of set that uses identity rather than value for records, or a way to make 
records use identity for equality, or another datatype that's just like 
defrecord but ..., or a way to get more functionality with deftype, or 
...?  (I don't think that Clojure should include every option or 
functionality that someone thinks might be convenient in some context.  
Another reason that I love Clojure is that it *doesn't* do that, but 
instead provides limited, well-thought out options that allow a lot of 
flexibility.)

In an earlier thread 
https://groups.google.com/forum/#!topic/clojure/EdjnSxRkOPk, Stephen Yi 
pointed out that although you're allowed to override some Object methods in 
defrecord, you can't override Object.equals and Object.hashCode.  It isn't 
documented, but the compiler throws an exception if you try it.  That makes 
quite a bit of sense, since changing the meaning of equality for a datatype 
would be an easy way to introduce confusing bugs.  On the other hand, if 
you do override Object.equals and Object.hashCode for defrecord, you 
probably know that you're doing something weird, and that you'd better take 
precautions, if necessary, to avoid later confusion.

-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


Re: defrecord, equality, hashing, and performance

2015-06-11 Thread Mark Engelberg
Zach Tellman's potemkin library includes several useful ways to tweak
deftypes and defrecords.  I wish Clojure itself provided more ways to do
this, but in the meantime, potemkin is the best way to create something
custom, like a deftype that behaves mostly like defrecord with some
different hashing and equality behavior.

On Thu, Jun 11, 2015 at 11:36 AM, Mars0i marsh...@logical.net wrote:

 I think that the following is all correct, but I could be wrong about
 something.

 The datatypes page at clojure.org http://clojure.org/datatypes says:
 defrecord provides ... value-based equality and hashCode, while deftype
 does not.   So
 (defrecord Rec [x])
 (= (Rec. 42) (Rec. 42)) ;= true
 but
 (deftype Typ [x])
 (= (Typ. 42) (Typ. 42)) ;= false.
 This also means, as I understand it, that data structures that use
 hashing--maps and sets, for example--hash on value for records, but on
 identity for types.  I believe that the same is true when Java hashtables
 compare records, or compare types, although I haven't carefully tested this.

 It makes sense that the primary, most fully functional user-definable
 datatypes in Clojure should use value-based equality; that's fully in the
 spirit of a functional language, and is very convenient in many contexts,
 and hashing should follow the equality semantics of the datatypes.  You can
 always use identical? if you want to test records for identity rather
 than equality.

 However, you can't tell maps and sets to use identity for records, afaik.
 This makes records undesirable if you only care about identity, and need to
 use a data structure that uses hashing, and care about performance, because
 value-based hashing is presumably a lot slower than identity-based hashing
 (which is what my experiments seem to show).

 On the other hand, records are obviously way, way, more convenient than
 types in many contexts because the functionality provided by deftype is so
 bare-bones.  It really bothers me that I have to choose between speed and
 using convenient, idiomatic functionality (defrecord) that is one of
 several significant reasons that I love Clojure.  (This is in an
 application in which value-based equality would *never* be what I wanted
 for my datatypes.)

 Should there be an alternative?  Either a way to define e.g. a special
 kind of set that uses identity rather than value for records, or a way to
 make records use identity for equality, or another datatype that's just
 like defrecord but ..., or a way to get more functionality with deftype, or
 ...?  (I don't think that Clojure should include every option or
 functionality that someone thinks might be convenient in some context.
 Another reason that I love Clojure is that it *doesn't* do that, but
 instead provides limited, well-thought out options that allow a lot of
 flexibility.)

 In an earlier thread
 https://groups.google.com/forum/#!topic/clojure/EdjnSxRkOPk, Stephen Yi
 pointed out that although you're allowed to override some Object methods in
 defrecord, you can't override Object.equals and Object.hashCode.  It isn't
 documented, but the compiler throws an exception if you try it.  That makes
 quite a bit of sense, since changing the meaning of equality for a datatype
 would be an easy way to introduce confusing bugs.  On the other hand, if
 you do override Object.equals and Object.hashCode for defrecord, you
 probably know that you're doing something weird, and that you'd better take
 precautions, if necessary, to avoid later confusion.

 --
 You received this message because you are subscribed to the Google
 Groups Clojure group.
 To post to this group, send email to clojure@googlegroups.com
 Note that posts from new members are moderated - please be patient with
 your first post.
 To unsubscribe from this group, send email to
 clojure+unsubscr...@googlegroups.com
 For more options, visit this group at
 http://groups.google.com/group/clojure?hl=en
 ---
 You received this message because you are subscribed to the Google Groups
 Clojure group.
 To unsubscribe from this group and stop receiving emails from it, send an
 email to clojure+unsubscr...@googlegroups.com.
 For more options, visit https://groups.google.com/d/optout.


-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


Re: defrecord, equality, hashing, and performance

2015-06-11 Thread Andy Fingerhut
You can override hashCode and/or hasheq methods in deftype.

If the reason that defrecord hashing is slow in your application is because
hashed recalculates the hash values from scratch each time, without
hashing, consider voting for this ticket so that is improved in the future:
http://dev.clojure.org/jira/browse/CLJ-1224

Andy

On Thu, Jun 11, 2015 at 11:36 AM, Mars0i marsh...@logical.net wrote:

 I think that the following is all correct, but I could be wrong about
 something.

 The datatypes page at clojure.org http://clojure.org/datatypes says:
 defrecord provides ... value-based equality and hashCode, while deftype
 does not.   So
 (defrecord Rec [x])
 (= (Rec. 42) (Rec. 42)) ;= true
 but
 (deftype Typ [x])
 (= (Typ. 42) (Typ. 42)) ;= false.
 This also means, as I understand it, that data structures that use
 hashing--maps and sets, for example--hash on value for records, but on
 identity for types.  I believe that the same is true when Java hashtables
 compare records, or compare types, although I haven't carefully tested this.

 It makes sense that the primary, most fully functional user-definable
 datatypes in Clojure should use value-based equality; that's fully in the
 spirit of a functional language, and is very convenient in many contexts,
 and hashing should follow the equality semantics of the datatypes.  You can
 always use identical? if you want to test records for identity rather
 than equality.

 However, you can't tell maps and sets to use identity for records, afaik.
 This makes records undesirable if you only care about identity, and need to
 use a data structure that uses hashing, and care about performance, because
 value-based hashing is presumably a lot slower than identity-based hashing
 (which is what my experiments seem to show).

 On the other hand, records are obviously way, way, more convenient than
 types in many contexts because the functionality provided by deftype is so
 bare-bones.  It really bothers me that I have to choose between speed and
 using convenient, idiomatic functionality (defrecord) that is one of
 several significant reasons that I love Clojure.  (This is in an
 application in which value-based equality would *never* be what I wanted
 for my datatypes.)

 Should there be an alternative?  Either a way to define e.g. a special
 kind of set that uses identity rather than value for records, or a way to
 make records use identity for equality, or another datatype that's just
 like defrecord but ..., or a way to get more functionality with deftype, or
 ...?  (I don't think that Clojure should include every option or
 functionality that someone thinks might be convenient in some context.
 Another reason that I love Clojure is that it *doesn't* do that, but
 instead provides limited, well-thought out options that allow a lot of
 flexibility.)

 In an earlier thread
 https://groups.google.com/forum/#!topic/clojure/EdjnSxRkOPk, Stephen Yi
 pointed out that although you're allowed to override some Object methods in
 defrecord, you can't override Object.equals and Object.hashCode.  It isn't
 documented, but the compiler throws an exception if you try it.  That makes
 quite a bit of sense, since changing the meaning of equality for a datatype
 would be an easy way to introduce confusing bugs.  On the other hand, if
 you do override Object.equals and Object.hashCode for defrecord, you
 probably know that you're doing something weird, and that you'd better take
 precautions, if necessary, to avoid later confusion.

 --
 You received this message because you are subscribed to the Google
 Groups Clojure group.
 To post to this group, send email to clojure@googlegroups.com
 Note that posts from new members are moderated - please be patient with
 your first post.
 To unsubscribe from this group, send email to
 clojure+unsubscr...@googlegroups.com
 For more options, visit this group at
 http://groups.google.com/group/clojure?hl=en
 ---
 You received this message because you are subscribed to the Google Groups
 Clojure group.
 To unsubscribe from this group and stop receiving emails from it, send an
 email to clojure+unsubscr...@googlegroups.com.
 For more options, visit https://groups.google.com/d/optout.


-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


Re: defrecord, equality, hashing, and performance

2015-06-11 Thread Andy Fingerhut
Ugh.  Too many typos there.  Here is what I meant:

If the reason that defrecord hashing is slow in your application is because
_hasheq_ recalculates the hash values from scratch each time, without
_caching the value_, consider voting for this ticket so that is improved in
the future: http://dev.clojure.org/jira/browse/CLJ-1224

On Thu, Jun 11, 2015 at 12:19 PM, Andy Fingerhut andy.finger...@gmail.com
wrote:

 You can override hashCode and/or hasheq methods in deftype.

 If the reason that defrecord hashing is slow in your application is
 because hashed recalculates the hash values from scratch each time, without
 hashing, consider voting for this ticket so that is improved in the future:
 http://dev.clojure.org/jira/browse/CLJ-1224

 Andy

 On Thu, Jun 11, 2015 at 11:36 AM, Mars0i marsh...@logical.net wrote:

 I think that the following is all correct, but I could be wrong about
 something.

 The datatypes page at clojure.org http://clojure.org/datatypes says:
 defrecord provides ... value-based equality and hashCode, while deftype
 does not.   So
 (defrecord Rec [x])
 (= (Rec. 42) (Rec. 42)) ;= true
 but
 (deftype Typ [x])
 (= (Typ. 42) (Typ. 42)) ;= false.
 This also means, as I understand it, that data structures that use
 hashing--maps and sets, for example--hash on value for records, but on
 identity for types.  I believe that the same is true when Java hashtables
 compare records, or compare types, although I haven't carefully tested this.

 It makes sense that the primary, most fully functional user-definable
 datatypes in Clojure should use value-based equality; that's fully in the
 spirit of a functional language, and is very convenient in many contexts,
 and hashing should follow the equality semantics of the datatypes.  You can
 always use identical? if you want to test records for identity rather
 than equality.

 However, you can't tell maps and sets to use identity for records,
 afaik.  This makes records undesirable if you only care about identity, and
 need to use a data structure that uses hashing, and care about performance,
 because value-based hashing is presumably a lot slower than identity-based
 hashing (which is what my experiments seem to show).

 On the other hand, records are obviously way, way, more convenient than
 types in many contexts because the functionality provided by deftype is so
 bare-bones.  It really bothers me that I have to choose between speed and
 using convenient, idiomatic functionality (defrecord) that is one of
 several significant reasons that I love Clojure.  (This is in an
 application in which value-based equality would *never* be what I wanted
 for my datatypes.)

 Should there be an alternative?  Either a way to define e.g. a special
 kind of set that uses identity rather than value for records, or a way to
 make records use identity for equality, or another datatype that's just
 like defrecord but ..., or a way to get more functionality with deftype, or
 ...?  (I don't think that Clojure should include every option or
 functionality that someone thinks might be convenient in some context.
 Another reason that I love Clojure is that it *doesn't* do that, but
 instead provides limited, well-thought out options that allow a lot of
 flexibility.)

 In an earlier thread
 https://groups.google.com/forum/#!topic/clojure/EdjnSxRkOPk, Stephen
 Yi pointed out that although you're allowed to override some Object methods
 in defrecord, you can't override Object.equals and Object.hashCode.  It
 isn't documented, but the compiler throws an exception if you try it.  That
 makes quite a bit of sense, since changing the meaning of equality for a
 datatype would be an easy way to introduce confusing bugs.  On the other
 hand, if you do override Object.equals and Object.hashCode for defrecord,
 you probably know that you're doing something weird, and that you'd better
 take precautions, if necessary, to avoid later confusion.

 --
 You received this message because you are subscribed to the Google
 Groups Clojure group.
 To post to this group, send email to clojure@googlegroups.com
 Note that posts from new members are moderated - please be patient with
 your first post.
 To unsubscribe from this group, send email to
 clojure+unsubscr...@googlegroups.com
 For more options, visit this group at
 http://groups.google.com/group/clojure?hl=en
 ---
 You received this message because you are subscribed to the Google Groups
 Clojure group.
 To unsubscribe from this group and stop receiving emails from it, send an
 email to clojure+unsubscr...@googlegroups.com.
 For more options, visit https://groups.google.com/d/optout.




-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this 

Re: defrecord, equality, hashing, and performance

2015-06-11 Thread Mark Engelberg
Yes, please vote for that issue.  I find myself frequently having to work
around this limitation of records in Clojure; I mostly avoid using records
directly as a consequence of this performance issue.

As a side note, one quick-and-dirty way to get identity semantics for your
data is to wrap each piece of data in your application in a delay or atom.
The wrapper will give all your data a unique identity-based hash value and
equality semantics.


On Thu, Jun 11, 2015 at 12:29 PM, Andy Fingerhut andy.finger...@gmail.com
wrote:

 If the reason that defrecord hashing is slow in your application is
 because _hasheq_ recalculates the hash values from scratch each time,
 without _caching the value_, consider voting for this ticket so that is
 improved in the future: http://dev.clojure.org/jira/browse/CLJ-1224



-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


Re: Question regarding java array

2015-06-11 Thread Andy-
On Thursday, June 11, 2015 at 5:32:02 AM UTC-4, Ritchie Cai wrote:

 Just wondering though, is there a faster way to load an array than this 
 way? 
 https://github.com/malloc82/imaging/blob/45475b99f564b1ac77e668e04b91cb9c01a096d7/src/imaging/dicom.clj#L138
 the data file I'm trying to read from contains text based pixel values.


 I'm not sure about your use case but you may want to look into HDF5 data 
format. That's what it's made for: Super fast loading of numerical data 
from disk. It can even load in parallel (many hundreds MB/s). As soon as my 
text file takes too long to load I usually just convert it first to HDF5 
and then go from there. Linux/Mac has HDF tools which can dump and import 
(h5import) your data from the command line.
It's also a very common data format.

HTH

-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


Re: Merge optimization question

2015-06-11 Thread Matching Socks
(keys old-map) might not be in the same order as (keys new-map).  The 
every? check is almost right but the test will be false where new-map's 
value is falsey.  You could use (set (keys new-map)).

-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


Merge optimization question

2015-06-11 Thread Leon Grapenthin
If I want merge in an algorithm where it is often the case that the new map 
has the same keys as the old map, I'd assume using the new map as a 
replacement is faster in those cases.

Of course comparing the keys costs and I'd like to know which is the 
fastest way.

Can I do (= (keys old-map) (keys new-map)) or are the orders not guaranteed?

I see an alternative in (every? new-map (keys old-map)) which would also 
account for cases where the keysets are different but old-map would be 
completely replaced by the merge.

Which one would you recommend, performance wise?

On a side note: I noticed Clojures merge merges a map onto nil by replacing 
nil with an empty map. Wouldn't it be faster if it returned the map right 
away? Should I write a ticket?

Kind regards,
 Leon.

-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


Re: `rational?` `decimal?` Am I misunderstanding these? (also `float?`)

2015-06-11 Thread Fluid Dynamics
On Friday, June 12, 2015 at 12:12:01 AM UTC-4, Andy Fingerhut wrote:

 Some related facts, but not sure if they offer a perfect justification for 
 _why_ rational? and decimal? return what they do.

 (source rational?) reveals that it returns true for integers, ratios, and 
 decimal?, where (source decimal?) reveals it is true if it is of type 
 BigDecimal.  Arithmetic operations like + - * / quot rem given integers and 
 ratios should always return exact answers as other integers or ratios 
 (barring division by 0).  I don't think that is true for BigDecimal's, 
 though, at least with division.

 Regarding rational numbers as ones that can be written as a fraction, all 
 float and double values except NaN, and positive and negative infinity can 
 be written exactly as fractions, too.  However, + - * / usually give only 
 approximate answers, not exact.


Also, these floats and doubles always have a denominator that's a power of 
2. Which means thirds, fifths, tenths, and the like generally won't be 
represented exactly (unlike with Ratio), though halves, quarters, and 
eighths can be. BigDecimal has a power-of-10 denominator, so it can 
additionally represent fifths and tenths exactly, but still not thirds 
(rounded, or non-repeating decimal exception if you haven't set a precision 
limit).

Most of the numeric types and their behavior are documented fully in Java's 
documentation -- the language documentation for floats and doubles, and the 
Javadocs for the java.math.BigDecimal class for BigDecimal. The 
clojure.lang.Ratio class is best understood as simply keeping track of a 
numerator/denominator pair of BigIntegers which can stay exact, but in some 
numerical uses will quickly gobble up memory using huger and huger 
denominators (and numerators, most likely) to do so. If the space (and 
slowdown!) becomes a problem you then need to decide on a precision 
restriction and use floats, doubles, or BigDecimals -- or work smarter, 
not harder by using a different algorithm to do whatever you are trying to 
do. (A perturbation theoretic approach might allow you to use a few high 
precision Ratios or BigDecimals and do most of the work with deltas from 
these that are maintained in ordinary doubles, for example. It depends on 
the numerics you're doing whether, and how, you can do this.)

-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


Re: Merge optimization question

2015-06-11 Thread Mohit Thatte
Hi Leon,

If you use merge a lot, check out data.int-map
https://github.com/clojure/data.int-map, it only supports integer keys,
but is optimized for merges.

Merge is essentially (reduce conj) so you could see if using a transient
http://clojure.org/transients version of the original map and merging
into it works for you.

~M


On Fri, Jun 12, 2015 at 4:20 AM, Leon Grapenthin grapenthinl...@gmail.com
wrote:

 If I want merge in an algorithm where it is often the case that the new
 map has the same keys as the old map, I'd assume using the new map as a
 replacement is faster in those cases.

 Of course comparing the keys costs and I'd like to know which is the
 fastest way.

 Can I do (= (keys old-map) (keys new-map)) or are the orders not
 guaranteed?

 I see an alternative in (every? new-map (keys old-map)) which would also
 account for cases where the keysets are different but old-map would be
 completely replaced by the merge.

 Which one would you recommend, performance wise?

 On a side note: I noticed Clojures merge merges a map onto nil by
 replacing nil with an empty map. Wouldn't it be faster if it returned the
 map right away? Should I write a ticket?

 Kind regards,
  Leon.

 --
 You received this message because you are subscribed to the Google
 Groups Clojure group.
 To post to this group, send email to clojure@googlegroups.com
 Note that posts from new members are moderated - please be patient with
 your first post.
 To unsubscribe from this group, send email to
 clojure+unsubscr...@googlegroups.com
 For more options, visit this group at
 http://groups.google.com/group/clojure?hl=en
 ---
 You received this message because you are subscribed to the Google Groups
 Clojure group.
 To unsubscribe from this group and stop receiving emails from it, send an
 email to clojure+unsubscr...@googlegroups.com.
 For more options, visit https://groups.google.com/d/optout.




-- 
-Mohit Thatte

-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


Re: `rational?` `decimal?` Am I misunderstanding these? (also `float?`)

2015-06-11 Thread Andy Fingerhut
Some related facts, but not sure if they offer a perfect justification for
_why_ rational? and decimal? return what they do.

(source rational?) reveals that it returns true for integers, ratios, and
decimal?, where (source decimal?) reveals it is true if it is of type
BigDecimal.  Arithmetic operations like + - * / quot rem given integers and
ratios should always return exact answers as other integers or ratios
(barring division by 0).  I don't think that is true for BigDecimal's,
though, at least with division.

Regarding rational numbers as ones that can be written as a fraction, all
float and double values except NaN, and positive and negative infinity can
be written exactly as fractions, too.  However, + - * / usually give only
approximate answers, not exact.

Andy

On Thu, Jun 11, 2015 at 6:39 PM, John Gabriele jmg3...@gmail.com wrote:



 On Thursday, June 11, 2015 at 9:34:57 PM UTC-4, John Gabriele wrote:


 ~~~
 (integer? 5)   ;= true Yes
 (integer? 5N)  ;= true Yes
 (integer? 5.1) ;= false

 (float? 5.1)  ;= true
 (float? 5.1M) ;= false (?!)
 ~~~


 Whoops. Please ignore the two Yes's.

 Also, I think I just fell further into the rabbit hole:

 ~~~
 (rational? 5.1)  ;= false
 (rational? 5.1M) ;= true (?!)

 (decimal? 5.1)   ;= false
 (decimal? 5.1M)  ;= true (?!)
 ~~~

  --
 You received this message because you are subscribed to the Google
 Groups Clojure group.
 To post to this group, send email to clojure@googlegroups.com
 Note that posts from new members are moderated - please be patient with
 your first post.
 To unsubscribe from this group, send email to
 clojure+unsubscr...@googlegroups.com
 For more options, visit this group at
 http://groups.google.com/group/clojure?hl=en
 ---
 You received this message because you are subscribed to the Google Groups
 Clojure group.
 To unsubscribe from this group and stop receiving emails from it, send an
 email to clojure+unsubscr...@googlegroups.com.
 For more options, visit https://groups.google.com/d/optout.


-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


Re: easy clojure.core.cache interaction

2015-06-11 Thread Ryan Waters
Also worth mentioning is that the implementation is many-thread friendly,
so using a cache:
1) won't be a single thread bottleneck (as in the case of using an
unadorned agent)
2) work is done on the calling threads

Granted, there's a race condition where two threads trying to find the same
thing at the exact same time will result in one thread doing the work and
the other thread waiting (and therefore you don't know which thread would
be doing the work) but this should be exceedingly rare and the work-or-wait
is happening where it should be anyway.

A previous version of the function had the work being done within the agent
which would have killed parallelized I/O for me (in a separate project).


On Thu, Jun 11, 2015 at 9:32 PM, Ryan Waters ryan.or...@gmail.com wrote:

 https://gist.github.com/rwat/4abcebcb4cfae956f382

 I've enjoyed using clojure.core.cache for caching results from time
 expensive operations involving I/O, etc. and feel like this is the sort of
 function that makes c.c.cache as simple to use as possible.  e.g. all it
 becomes is:

 (cache-interact c f k xs)

 where c is the cache (agent), f is the function that generates the value
 you're looking for if it isn't already cached, k is the key to search for
 and xs is optional extra arguments that are apply'ed to f.

 I could have used an agent or atom to store the cache but I don't know how
 much is 'too much' to perform within an atom reset!/swap!.  The operations
 performed within the critical section are all fast but there's a handful of
 them so I decided to err on the side of using an agent.

 Also, I apologize for some of the symbol names.  I don't make it a habit
 to name things in that style.

 I would appreciate any feedback / recommendations for improvement.

 Thank you,
 Ryan


-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


easy clojure.core.cache interaction

2015-06-11 Thread Ryan Waters
https://gist.github.com/rwat/4abcebcb4cfae956f382

I've enjoyed using clojure.core.cache for caching results from time
expensive operations involving I/O, etc. and feel like this is the sort of
function that makes c.c.cache as simple to use as possible.  e.g. all it
becomes is:

(cache-interact c f k xs)

where c is the cache (agent), f is the function that generates the value
you're looking for if it isn't already cached, k is the key to search for
and xs is optional extra arguments that are apply'ed to f.

I could have used an agent or atom to store the cache but I don't know how
much is 'too much' to perform within an atom reset!/swap!.  The operations
performed within the critical section are all fast but there's a handful of
them so I decided to err on the side of using an agent.

Also, I apologize for some of the symbol names.  I don't make it a habit to
name things in that style.

I would appreciate any feedback / recommendations for improvement.

Thank you,
Ryan

-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


`rational?` `decimal?` Am I misunderstanding these? (also `float?`)

2015-06-11 Thread John Gabriele
My understanding is that a rational number is one that can be written as a 
fraction. For example, 5.1, which can be written as 51/10. But Clojure 
seems to disagree:

~~~
(rational? 51/10) ;= true
(rational? 5.1)   ;= false (?!)
~~~

Is my definition of rational incorrect?

Also, my understanding is that a decimal number is one that has a decimal 
point in it, like, for example, 5.1. However:

~~~
(decimal? 5.1) ;= false (?!)
~~~

And while typing this, I also notice that while `integer?` acts like I'd 
expect, `float?` does something weird:

~~~
(integer? 5)   ;= true Yes
(integer? 5N)  ;= true Yes
(integer? 5.1) ;= false

(float? 5.1)  ;= true
(float? 5.1M) ;= false (?!)
~~~

Maybe I'm confusing floating point number with decimal number here? If 
so, what's the difference?

Thanks!

-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


Re: `rational?` `decimal?` Am I misunderstanding these? (also `float?`)

2015-06-11 Thread John Gabriele


On Thursday, June 11, 2015 at 9:34:57 PM UTC-4, John Gabriele wrote:


 ~~~
 (integer? 5)   ;= true Yes
 (integer? 5N)  ;= true Yes
 (integer? 5.1) ;= false

 (float? 5.1)  ;= true
 (float? 5.1M) ;= false (?!)
 ~~~


Whoops. Please ignore the two Yes's.

Also, I think I just fell further into the rabbit hole:

~~~
(rational? 5.1)  ;= false
(rational? 5.1M) ;= true (?!)

(decimal? 5.1)   ;= false
(decimal? 5.1M)  ;= true (?!)
~~~

-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


Re: defrecord, equality, hashing, and performance

2015-06-11 Thread Mike Rodriguez
I agree the hashCode performance for records is a concern due to that lack of 
caching. 

I noticed the priority of that Jira 1224 changed to critical about a week ago 
(June 3). I was curious why that was done and what that means in terms of 
prioritization. 

Last minute squeeze into CLJ version 1.7 release?! :) 

Too hopefully I'm sure. 

-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.