Very interesting thread!
I'm having a similar problem and Michaels approach got me a small speedup
(around 15%). When trying Mike's approach on my real data, if fails with
IllegalArgumentException Too many arguments to struct constructor clojure.
lang.PersistentStructMap.construct (PersistentStr
Hey Michael,
Since your eval solution essentially "cookie-cutters" out maps, each with
the same keys, as fast as it can, I was playing around with what would
happen if you used records, and I cobbled together something that appears
to run twice as fast as the eval approach:
(defn read-to-struc
Zipmap doesn't use transients, so calling it at runtime will be
significantly slower than constructing a literal map.
http://dev.clojure.org/jira/browse/CLJ-1005
On Friday, October 10, 2014 11:42:14 AM UTC-7, Michael Blume wrote:
>
> So I'm reading a bunch of rows from a huge csv file and mars
On Oct 10, 2014, at 1:46 PM, Ben Wolfson wrote:
> it's not quite at compile-time (since it's a dynamic call to eval, after all,
> and "names" and "headers" aren't known at compile time), but it is calling it
> eval-time, for lack of a better phrase.
Yes, I meant when it compiles the code that e
it's not quite at compile-time (since it's a dynamic call to eval, after
all, and "names" and "headers" aren't known at compile time), but it is
calling it eval-time, for lack of a better phrase.
On Fri, Oct 10, 2014 at 1:35 PM, Sean Corfield wrote:
> Ah, interesting... I hadn't considered it wa
Ah, interesting... I hadn't considered it was running the zipmap at
compile-time so it only runs it once as opposed to running it for each row!
Sean
On Oct 10, 2014, at 1:06 PM, Ben Wolfson wrote:
> I believe it's because the `mapper` function is just creating and returning a
> map literal. Th
Did you run it enough times to fully warm up the JVM?
On Fri, Oct 10, 2014 at 4:21 PM, Michael Blume
wrote:
> https://github.com/MichaelBlume/eval-speed
> eval-speed.core=> (time-fn read-to-maps)
> "Elapsed time: 5551.011069 msecs"
> nil
> eval-speed.core=> (time-fn read-to-maps-fn)
> "Elapsed t
https://github.com/MichaelBlume/eval-speed
eval-speed.core=> (time-fn read-to-maps)
"Elapsed time: 5551.011069 msecs"
nil
eval-speed.core=> (time-fn read-to-maps-fn)
"Elapsed time: 5587.256991 msecs"
nil
eval-speed.core=> (time-fn read-to-maps-partial)
"Elapsed time: 5606.649172 msecs"
nil
eval-sp
I believe it's because the `mapper` function is just creating and returning
a map literal. The "mapper" function in the evaled version is something
like this:
user> (def names '[n1 n2 n3 n4])
#'user/names
user> (def headers '[h1 h2 h3 h4])
#'user/headers
user> `(fn [[~@names]] ~(zipmap headers nam
It may be more to do with the difference between `for` and `map`. How do these
versions compare in your benchmark:
(defn read-to-maps-partial [rows]
(let [headers (->>
rows
first
(take-while (complement #{""}))
(map keyword
Hello,
Can you show the code you use for benchmarking?
Le vendredi 10 octobre 2014, Michael Blume a écrit :
> So I'm reading a bunch of rows from a huge csv file and marshalling those
> rows into maps using the first row as keys. I wrote the function two ways:
> https://gist.github.com/MichaelB
So I'm reading a bunch of rows from a huge csv file and marshalling those
rows into maps using the first row as keys. I wrote the function two
ways: https://gist.github.com/MichaelBlume/c67d22df0ff9c225d956 and the
version with eval is twice as fast and I'm kind of curious about why.
Presumably
12 matches
Mail list logo
