Why does the following Clojure code take 10x the time the C# version does? How to improve the Clojure version?

I wrote the following code to solve this challenge - https://www.reddit.com/r/dailyprogrammer/comments/35s2ds/20150513_challenge_214_intermediate_pile_of_paper/. Code - https://github.com/amithgeorge/reddit-dailyprogrammer-clojure/blob/56ce1dbb6a08e96150dc85934caecfeb68108a53/src/rdp/214_interme

Why does the following Clojure code take 10x the time the C# version does? How to improve the Clojure version?

Actually, I'm more than a little curious about performance optimizations to my solution as well[0]. Running Yourkit shows that most of the execution time is spent in reduce, so I've tried switching to group-by instead. Also tried replacing with iterate. Neither of these improved overall executio

Re: Why does the following Clojure code take 10x the time the C# version does? How to improve the Clojure version?

I forgot to link the input files https://raw.githubusercontent.com/fsufitch/dailyprogrammer/0e4bb5ba1e3bc6e749b9e9bb49387513d5a623b7/ideas/pile_of_paper/100rects100x100.in https://raw.githubusercontent.com/fsufitch/dailyprogrammer/0e4bb5ba1e3bc6e749b9e9bb49387513d5a623b7/ideas/pile_of_paper/100re

Re: Why does the following Clojure code take 10x the time the C# version does? How to improve the Clojure version?

My general rule of thumb is that "idiomatic" Clojure is about 10x slower than Java. So this result doesn't really surprise me. If you want to get maximum performance, you will have to do some more advanced / non-idiomatic things like: - Add type hints for classes and primitive values - Use "deft

Re: Why does the following Clojure code take 10x the time the C# version does? How to improve the Clojure version?

Ditto F# vs. C#. One has to wonder when / where / if functional-pure-immutable approaches will ever under the covers get "fast enough"? -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note

Re: Why does the following Clojure code take 10x the time the C# version does? How to improve the Clojure version?

I think it false view that immutable/functional constructs are slow by default. Instead, a lot of it comes down to lack of support in the popular VMs for these constructs. For example the following code: (defn add-fn [& args] (reduce -add 0 args)) (loop [x 0] (if (eq x 1) x (recur

Re: Why does the following Clojure code take 10x the time the C# version does? How to improve the Clojure version?

I could be a bit off here, but I'd say the speed differences between Clojure and Java are actually due to immutability AND weak-typing, while in C# vs F# the difference should be less pronounced and governed primarily by the use of immutability (i.e., using mutable data-structures yield results in

Re: Why does the following Clojure code take 10x the time the C# version does? How to improve the Clojure version?

The major problem here is that you are using boxed math for everything instead of primitives. 0) Switch to Clojure 1.7.0-beta3 - it's faster and some things below are dependent on it for best performance. And use Java 1.8. 1) Parse the lines you're reading directly into longs (Clojure focuses on

Re: Why does the following Clojure code take 10x the time the C# version does? How to improve the Clojure version?

Gah. Meant in project.clj: :jvm-opts ^:replace ["-server"] ;; maybe also set max heap with "-Xmx1g" in there On Thursday, May 14, 2015 at 2:46:59 PM UTC-5, Alex Miller wrote: > > Oh also, the default Leiningen settings are optimized for reducing > startup, not for speed. Use: > > ^:jvm-opt

Re: Why does the following Clojure code take 10x the time the C# version does? How to improve the Clojure version?

Oh also, the default Leiningen settings are optimized for reducing startup, not for speed. Use: ^:jvm-opts ^:replace ["-server"] ;; maybe also set max heap with "-Xmx1g" in there Or just don't use lein when perf testing. On Thursday, May 14, 2015 at 2:44:38 PM UTC-5, Alex Miller wrote: >

Re: Why does the following Clojure code take 10x the time the C# version does? How to improve the Clojure version?

Also using `lein run` without supplying correct JVM settings (like -server) is going to lead to pretty misleading results. On Thu, May 14, 2015 at 3:44 PM, Alex Miller wrote: > The major problem here is that you are using boxed math for everything > instead of primitives. > > 0) Switch to Clojur

Re: Why does the following Clojure code take 10x the time the C# version does? How to improve the Clojure version?

I'd like to get more guidance on how to specify :jvm-opts for maximum performance. I've received some help on this topic from people on this list in the past (thanks!), but I'm pretty sure that I'm still not doing things right. I'm almost always interested in maximum performance for long-running

Re: Why does the following Clojure code take 10x the time the C# version does? How to improve the Clojure version?

Probably not helpful, but I tend to rely on the jvm optimisations and just -server. I figured this is an area where a little knowledge is a dangerous thing. At the very least I would have a realistic benchmark suite to prove to myself that these gains were worth it. In my experience the performanc

Re: Why does the following Clojure code take 10x the time the C# version does? How to improve the Clojure version?

I know that remembering to put "-server" used to be a huge issue, but I thought that on all recent versions of Java on all modern 64-bit machines, -server is now the default. I thought it was a non-issue at this point. Is that incorrect? On Thu, May 14, 2015 at 1:54 PM, Colin Yates wrote: > Pro

Re: Why does the following Clojure code take 10x the time the C# version does? How to improve the Clojure version?

Now I feel even more of an ignoramus :) On 14 May 2015 21:57, "Mark Engelberg" wrote: > I know that remembering to put "-server" used to be a huge issue, but I > thought that on all recent versions of Java on all modern 64-bit machines, > -server is now the default. I thought it was a non-issue

Re: Why does the following Clojure code take 10x the time the C# version does? How to improve the Clojure version?

FWIW if I run with no :jvm-opts at all then I often crash with an out-of-memory error, so I do know that whatever happens by default doesn't do what I'm doing with respect to memory, at least. I don't know what happens with respect to the other issues (tiered compilation and whatever else) by d

Re: Why does the following Clojure code take 10x the time the C# version does? How to improve the Clojure version?

I always run my REPL inside of Counterclockwise, but as far as I know it uses leiningen to launch the REPL. In any case, when I attach a profiler such as the built-in jvisualvm to the REPL process, it clearly says that the REPL is running in 64-bit server mode. On Thu, May 14, 2015 at 2:29 PM, Le

Re: Why does the following Clojure code take 10x the time the C# version does? How to improve the Clojure version?

By default, I believe that lein will set: -XX:+TieredCompilation -XX:TieredStopAtLevel=1 for the JVM opts. If you supply :jvm-opts, they will be *added* to these. Using ^:replace will instead *replace* the defaults with what you supply. There are a bunch of rules in the JVM for whether -server

Re: Why does the following Clojure code take 10x the time the C# version does? How to improve the Clojure version?

In general, running any kind of JVM via Leiningen, using the 'ps' command with lots of '' on the end of the command line options should give full command line options used to start the JVM, so you can see what is happening. Understanding exactly which options are good or bad for performance, I

Re: Why does the following Clojure code take 10x the time the C# version does? How to improve the Clojure version?

I can heartily recommend Java Performance: The Definitive Guide to anyone interested in digging further into all the knobs you can set on the command line: http://www.amazon.com/Java-Performance-The-Definitive-Guide/dp/1449358454 -- You received this message because you are subscribed to the Go

Re: Why does the following Clojure code take 10x the time the C# version does? How to improve the Clojure version?

Thanks Colin and also Alex and Andy. I'm trying to determine a reasonable way to do this without reading a book about it. It sounds like I should use ^:replace, -server, and also -XX:-TieredCompilation (is that right, the way I've changed + to -?), and also -XX:+AggressiveOpts. Does it make se

Re: Why does the following Clojure code take 10x the time the C# version does? How to improve the Clojure version?

On Thursday, May 14, 2015 at 8:45:09 PM UTC-4, Lee wrote: > > Thanks Colin and also Alex and Andy. > > I'm trying to determine a reasonable way to do this without reading a book > about it. > > It sounds like I should use ^:replace, -server, and also > -XX:-TieredCompilation (is that right, the

Re: Why does the following Clojure code take 10x the time the C# version does? How to improve the Clojure version?

I agree that the problem is immutable/functional constructs per se., but I don't think the problem is the lack of VM support either. It is possible to get *very* fast code on the JVM. In Clojure the issue is more that the dynamic nature of many Clojure constructs and lack of static type inferen

Re: Why does the following Clojure code take 10x the time the C# version does? How to improve the Clojure version?

> On May 14, 2015, at 9:15 PM, Fluid Dynamics wrote: > > Umm, the :replace metadata needs to be on the vector. Attaching it to the let > form won't do much of anything useful. So: > > :jvm-opts ~(let [mem-to-use >(long (* (.getTotalPhysicalMemorySize >

Re: Why does the following Clojure code take 10x the time the C# version does? How to improve the Clojure version?

All the advice here is valid; type hinting + eliminating boxed math will probably give you the biggest gains. By adding one type hint in the proper place sometimes I've been able to make my code go 5-10x faster. I've used tools like YourKit to great advantage to be able to pinpoint exactly whic

Re: Why does the following Clojure code take 10x the time the C# version does? How to improve the Clojure version?

> I.e. your time is better spent optimizing a fn that's called 1k times per > second and it's a little slow (for example, missing a type hint and has to > do reflection or using boxed math) vs. a fn that's very slow but is only > called once a minute. not all apps, and not all developers, end up w

Re: Why does the following Clojure code take 10x the time the C# version does? How to improve the Clojure version?

Ok, IF there's such a spot : ) . On Thursday, May 14, 2015 at 9:57:53 PM UTC-7, raould wrote: > > > I.e. your time is better spent optimizing a fn that's called 1k times > per > > second and it's a little slow (for example, missing a type hint and has > to > > do reflection or using boxed math

Re: Why does the following Clojure code take 10x the time the C# version does? How to improve the Clojure version?

Thanks for the detailed suggestions. Implementing them did bring the execution time down to around 250secs. Though that value is still much longer than 45secs. Could you please verify if I have implemented them correctly? Code - https://github.com/amithgeorge/reddit-dailyprogrammer-clojure/blo

Re: Why does the following Clojure code take 10x the time the C# version does? How to improve the Clojure version?

@puzzler -server is the default mostly, but Leiningen overrides some of the important options that -server enables, as detailed by Alex. @Lee the first 15 minutes of this talk by Tom Crayford has some useful info about performance and JVM options in it: https://skillsmatter.com/skillscasts/6148

Re: Why does the following Clojure code take 10x the time the C# version does? How to improve the Clojure version?

Thanks Jony -- very helpful! -Lee > On May 15, 2015, at 7:52 AM, Jony Hudson wrote: > > @puzzler -server is the default mostly, but Leiningen overrides some of the > important options that -server enables, as detailed by Alex. > > @Lee the first 15 minutes of this talk by Tom Crayford has so

Re: Why does the following Clojure code take 10x the time the C# version does? How to improve the Clojure version?

Hi Amith, I checked out your project from git and just doing 'lein run' I got a reported: "Elapsed time: 185.651689 msecs" However, if I modify the -main function in 214_intermediate.clj to wrap the time testing with (doseq [_ (range 20)]), to run the test multiple times, the behavior is much

Re: Why does the following Clojure code take 10x the time the C# version does? How to improve the Clojure version?

Hi Steven, My bad. You need to invoke the code using the command lein run -m rdp.214-intermediate-arr 1 true The `1` tells it to select a certain input file, (in this case the biggest) and the `true` tells it to use the function that internally uses a java array (as opposed to the function th

Re: Why does the following Clojure code take 10x the time the C# version does? How to improve the Clojure version?

Ah, I see. Well, I think then you can ignore the stuff about warming up, as this certainly takes a while to run here: "Elapsed time: 314763.93 msecs" I tried profiling with Yourkit and saw a couple of things to change: ;; I think lte with more than two args ends up being slower than unrolli

Re: Why does the following Clojure code take 10x the time the C# version does? How to improve the Clojure version?

Summary: With a new algorithm + parallelism, I reduced it from 528s to 11s. This sounded fun, so I took a crack at it, starting with your solution. Description and patch files here: https://gist.github.com/leifp/a864bca941ecdacb5840 Starting with your solution, I: 1. Divided the canvas in

Re: Why does the following Clojure code take 10x the time the C# version does? How to improve the Clojure version?

Hi, Thanks for taking the time to profile the code. I implemented the two suggestions (using the two argument arity of lte and using aset inste

Re: Why does the following Clojure code take 10x the time the C# version does? How to improve the Clojure version?

Hi, Thank you for taking the time. That is a rather smart algo. The code and the changes were easy to understand. I didn't implement the parallelized version as my aim was to understand why the clojure version was so slow compared to the equivalent C# version. (Btw, you have access to a 12 co

Re: Why does the following Clojure code take 10x the time the C# version does? How to improve the Clojure version?

Hi Amith, Just got back from a trip and had a chance to look at this again. I'm not sure why you didn't see a change there; I'm running the same version of Clojure and Java here. I did another change to move from using a reduce to nested loop-recurs. That with a some additional type hints for ^

Re: Why does the following Clojure code take 10x the time the C# version does? How to improve the Clojure version?

Hi Amith, One last optimization yields a 4x increase over the last version, and now 12x over the original: "Elapsed time: 22848.384642 msecs" Code here: https://gist.github.com/kunstmusik/db6e14118e818abf3bb8 Changes from last version: - parse-inputs - Put parsed Paper objects into an array i

Re: Why does the following Clojure code take 10x the time the C# version does? How to improve the Clojure version?

Wow. This is rather hard to believe. But the execution time is now 10s. :D All of your suggestions were on the mark. I implemented them in stages/commits. The command to execute the file remains the same - `lein run -m rdp.214-intermediate-arr 1 true` 1. Replaced the coordinates vector with sep

Re: Why does the following Clojure code take 10x the time the C# version does? How to improve the Clojure version?

Oh, a few more things I observed; - I figured out how to use *unchecked-math* and *warn-on-reflection* :) Earlier, I was setting them within a binding, but that didn't seem to do anything. Seems the correct way is to set them at the very top of the file, the first 2 lines after the namespace

Re: Why does the following Clojure code take 10x the time the C# version does? How to improve the Clojure version?

"Elapsed time: 1155.723592 msecs" I implemented the changes Amith and Steven suggested on my multithreaded version. I think we can safely say that, with a modest effort, Clojure can compare favorably to C#. :) https://gist.github.com/leifp/a864bca941ecdacb5840 Cheers, Leif On Tuesday, May 19,

Re: Why does the following Clojure code take 10x the time the C# version does? How to improve the Clojure version?

Hi Amith, Very glad you're seeing the performance increase there too! I think you're right about both the extraneous type hints and the unchecked-math. I tend to turn on unchecked-math in a leiningen profile that I use while developing, but leave it set to default for normal builds. Because of th