Eh, I said cons, but it's 6:30 in the morning. I meant the for's lazy seq
requires at least one allocation per item. Your time spent in
PersistentHashSet.cons is probably due to the use of set.
Timothy
On Wed, Oct 23, 2013 at 6:21 AM, Timothy Baldridge <tbaldri...@gmail.com>wrote:
> This code will have a ton of allocations:
>
> (defn solve [rows cols pieces]
> (if (empty? pieces)
> #{#{}}
>
> (set
> (let [candidate (first pieces)]
> (for [solution (solve rows cols (rest pieces))
> x (range 0 cols)
> y (range 0 rows)
> :when (allowed? candidate [x y] solution)]
> (conj solution [candidate [x y]]))))))
>
>
>
> Every item in the cons will require an allocation. After that, the entire
> thing has to be turned into a set. This entire thing is also recursive, so
> that compounds the problem. Perhaps re-write this as a reducer?
>
> Timothy
>
>
>
> On Wed, Oct 23, 2013 at 6:18 AM, Timothy Baldridge
> <tbaldri...@gmail.com>wrote:
>
>> Great! you have a profiler, use that. Find the hotspots, use YourKit to
>> find where the .cons is being called from, find things to optimize, and go
>> from there. This is exactly the same process I would use any optimizations
>> I attempted.
>>
>> Timothy
>>
>>
>> On Wed, Oct 23, 2013 at 6:09 AM, Paul Butcher <p...@paulbutcher.com>wrote:
>>
>>> On 23 Oct 2013, at 12:44, Timothy Baldridge <tbaldri...@gmail.com>
>>> wrote:
>>>
>>> That being said, the #1 rule of benchmarking with lein is don't
>>> benchmark with lein. The JVM lein spins up has a different set of goals
>>> (namely startup time). The best way to benchmark is to run the test several
>>> thousand times, from a jar created with lein uberjar.
>>>
>>>
>>> I'm well aware of the issues associated with micro-benchmarking on the
>>> JVM. Bear in mind that what I'm doing isn't micro-benchmarking. I have a
>>> Scala program that runs in 2.5 minutes, which when re-implemented in
>>> Clojure doesn't finish after being run overnight. I.e. the Clojure is (at
>>> least) 2 orders of magnitude slower than the Scala.
>>>
>>> I'd love to be able to run it just once - running it thousands of times
>>> with its current performance would require *years*.
>>>
>>> I've now got it recompiled as an uberjar and I'm running it with 12G RAM
>>> (-Xms12G -Xmx12G). So far, it's been running for 12 minutes (i.e. 6x longer
>>> than the Scala version takes to complete) and YourKit shows me that it's
>>> used less than 300MB RAM (i.e. the vast majority of the 12G is unused).
>>>
>>> YourKit also shows me that it's still the case that it's spending 99% of
>>> its time in PersistentHashSet.cons.
>>>
>>> I completely understand that playing around with pre-compilation and JVM
>>> settings will make a difference to the observed performance. But not two
>>> orders of magnitude difference, surely?
>>>
>>> It strikes me that there's clearly something that I'm missing in the
>>> Clojure implementation that's making a dramatic difference to performance.
>>> I'd love some help determining what that something is.
>>>
>>> --
>>> paul.butcher->msgCount++
>>>
>>> Snetterton, Castle Combe, Cadwell Park...
>>> Who says I have a one track mind?
>>>
>>> http://www.paulbutcher.com/
>>> LinkedIn: http://www.linkedin.com/in/paulbutcher
>>> MSN: p...@paulbutcher.com
>>> AIM: paulrabutcher
>>> Skype: paulrabutcher
>>>
>>> --
>>> --
>>> You received this message because you are subscribed to the Google
>>> Groups "Clojure" group.
>>> To post to this group, send email to clojure@googlegroups.com
>>> Note that posts from new members are moderated - please be patient with
>>> your first post.
>>> To unsubscribe from this group, send email to
>>> clojure+unsubscr...@googlegroups.com
>>> For more options, visit this group at
>>> http://groups.google.com/group/clojure?hl=en
>>> ---
>>> You received this message because you are subscribed to the Google
>>> Groups "Clojure" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to clojure+unsubscr...@googlegroups.com.
>>> For more options, visit https://groups.google.com/groups/opt_out.
>>>
>>
>>
>>
>> --
>> “One of the main causes of the fall of the Roman Empire was that–lacking
>> zero–they had no way to indicate successful termination of their C
>> programs.”
>> (Robert Firth)
>>
>
>
>
> --
> “One of the main causes of the fall of the Roman Empire was that–lacking
> zero–they had no way to indicate successful termination of their C
> programs.”
> (Robert Firth)
>
--
“One of the main causes of the fall of the Roman Empire was that–lacking
zero–they had no way to indicate successful termination of their C
programs.”
(Robert Firth)
--
--
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
---
You received this message because you are subscribed to the Google Groups
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
