Re: Performance optimizations dealing with java collections
That's it! Thanks so much for the help! In what cases is a function
turned into a RestFn? variadic clojure or vararg java? I was confused
about how clojure is turned into java, but things are a little clearer now.
(make-array) uses java.lang.reflect.Array.newInstance, which is bad as
well. I'm going to try and push this interpolation down into the java API.
On Tuesday, June 18, 2013 3:00:23 PM UTC-7, John Hume wrote:
Offhand it looks like the only RestFn you call from filter-link is
clojure.core/format. Have you tried replacing that with something like this?
(String/format (.get link 1) (doto (make-array String 1) (aset 0 (.get
link 2)))
I'm not suggesting that's idiomatic, but if it addresses the issue then
you can focus on the difference and look for a happy medium.
On Tue, Jun 18, 2013 at 4:10 PM, Tim Jones timoth...@hp.com javascript:
wrote:
(defn- filter-link
Format lang and cc into the link returned from the product.
[^List link]
[(.get link 0) (.get link 1) (.get link 2) (.get link 3)
(- (.get link 4)
(URLDecoder/decode UTF-8)
(format (.get link 1) (.get link 2)))])
(defn link-info
Retrieve all link info from a product as a lazy-seq of vectors of
String. The function
handles parameter substitution within the URL
[^Product p]
(map filter-link
(.getAllLinkInfo p @lib {h_lang %1$s h_cc %2$s})))
--
--
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
---
You received this message because you are subscribed to the Google Groups
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
Re: Performance optimizations dealing with java collections
Let's try this again. Maybe the question is what is the most performant way to consume a native java collection (List, array, or other) in clojure? What I've tried so far hits clojure-imposed performance issues. How do I get to near-java performance? I'm trying to create a polemic for clojure and the java version of this program is 30% faster isn't where I want to start. Am I missing something? On Monday, June 17, 2013 3:34:07 PM UTC-7, Tim Jones wrote: I'm working on a small clojure program which pulls data from a custom memory-backed data store via a java api. When looking at performance, there is a hotspot at the point of interaction with this API. One of the fields of each record being exported contains a list of lists of strings (ListListString) with about 60-150 items in the outer list and 5 items in the inner list. The problem is that at the point of assimilating this java data type into clojure, there are a couple calls taking ~30-40% of the execution time. Here are the senarios; I'm just starting to get familiar with clojure internals, so forgive obvious blunders in this. Processing it as: (map parse-field (.getAllFields datastore)) with parse-field defined like: (defn parse-field [^List field] ...) or (if I change the API call) (defn parse-field [^[Ljava.lang.String; field] ...) it calls clojure.lang.RestFn.invoke() which wants to build an ArraySeq, which calls java.lang.Class.getComponentType, which is a killer (20 of 50 seconds on 500 records). With parse-field using destructuring, on either the List or String array, clojure.lang.RT.nthFrom() is invoked, which calls java.lang.Class.isArray ( https://github.com/clojure/clojure/blob/master/src/jvm/clojure/lang/RT.java#L773), which is a killer as well. So I inlined parse-field and used loop/recur (instead of map) and .get on the list to get the next item, but this still results in the RestFn.invoke()-ArraysSeq-Class.getComponentType path. This is related to http://dev.clojure.org/jira/browse/CLJ-1200, although called via a different path. Does anyone have any ideas on how I could avoid these calls? Thanks, Tim -- -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
Re: Performance optimizations dealing with java collections
On Tuesday, June 18, 2013 9:58:20 AM UTC-7, Michael Klishin wrote:
2013/6/18 Tim Jones timoth...@hp.com javascript:
How do I get to near-java performance?
Start by providing a snippet of your code and profiling.
Great. Here's the context: iterate through a list of Product, and for
each Product, get a ListListString which is URL info for that product.
Convert each ListString to a vector and do a little processing on it.
Here is the code:
(defn- filter-link
Format lang and cc into the link returned from the product.
[^List link]
[(.get link 0) (.get link 1) (.get link 2) (.get link 3)
(- (.get link 4)
(URLDecoder/decode UTF-8)
(format (.get link 1) (.get link 2)))])
(defn link-info
Retrieve all link info from a product as a lazy-seq of vectors of
String. The function
handles parameter substitution within the URL
[^Product p]
(map filter-link
(.getAllLinkInfo p @lib {h_lang %1$s h_cc %2$s})))
Here's a VisualVM screenshot. 500 products takes ~45s, with 18s spent in
one call:
https://lh4.googleusercontent.com/-gLONfrNtJPs/UcDMkxHchuI/ABU/GXqaevewpqs/s1600/clj-java-perf.png
--
--
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
---
You received this message because you are subscribed to the Google Groups
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
Performance optimizations dealing with java collections
I'm working on a small clojure program which pulls data from a custom memory-backed data store via a java api. When looking at performance, there is a hotspot at the point of interaction with this API. One of the fields of each record being exported contains a list of lists of strings (ListListString) with about 60-150 items in the outer list and 5 items in the inner list. The problem is that at the point of assimilating this java data type into clojure, there are a couple calls taking ~30-40% of the execution time. Here are the senarios; I'm just starting to get familiar with clojure internals, so forgive obvious blunders in this. Processing it as: (map parse-field (.getAllFields datastore)) with parse-field defined like: (defn parse-field [^List field] ...) or (if I change the API call) (defn parse-field [^[Ljava.lang.String; field] ...) it calls clojure.lang.RestFn.invoke() which wants to build an ArraySeq, which calls java.lang.Class.getComponentType, which is a killer (20 of 50 seconds on 500 records). With parse-field using destructuring, on either the List or String array, clojure.lang.RT.nthFrom() is invoked, which calls java.lang.Class.isArray (https://github.com/clojure/clojure/blob/master/src/jvm/clojure/lang/RT.java#L773), which is a killer as well. So I inlined parse-field and used loop/recur (instead of map) and .get on the list to get the next item, but this still results in the RestFn.invoke()-ArraysSeq-Class.getComponentType path. This is related to http://dev.clojure.org/jira/browse/CLJ-1200, although called via a different path. Does anyone have any ideas on how I could avoid these calls? Thanks, Tim -- -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
Re: A tutorial for how to setup your clojure development environment for: Emacs, Leiningen and Linux.
On Thu, Jun 14, 2012 at 10:31:44AM -0700, Phil Hagelberg wrote: On Wed, Jun 13, 2012 at 8:01 PM, David Della Costa ddellaco...@gmail.com wrote: Similarly, it's easy to get lost (as a beginner) between namespace issues with packages and how to set things up properly with Leiningen. It'd be good to have some documentation on that. Maybe if the Leiningen tutorial linked to http://blog.8thlight.com/colin-jones/2010/12/05/clojure-libs-and-namespaces-require-use-import-and-ns.html? I'm new to SLIME and spent about a day before I figured out C-x M-p to switch namespaces within a project. Tim -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en
