"Wm. Josiah Erikson" <wmjos...@gmail.com> writes: > Am I reading this right that this is actually a Java problem, and not > clojure-specific? Wouldn't the rest of the Java community have noticed > this? Or maybe massive parallelism in this particular way isn't > something commonly done with Java in the industry? > > Thanks for the patches though - it's nice to see some improvement... > I'll be fascinated to see how this turns out in the end. Have we found > a large Java bug?
Apologies for my very-slow reply here. I keep thinking that I’ll have more time to look into this issue, and keep having other things requiring my attention. And on top of that, I’ve temporarily lost the many-way AMD system I was using as a test-bed. I very much want to see if I can get my hands on an Intel system to compare to. My AMD system is in theory 32-way – two physical CPUs, each with 16 cores. However, Linux reports (via /proc/cpuinfo) the cores in groups of 8 (“cpu cores : 8” etc). And something very strange happens when extending parallelism beyond 8-way... I ran several experiments using a version of your whole-application benchmark I modified to control the level of parallelism. At parallelism 9+, the real time it takes to complete the benchmark hardly budges, but the user/CPU time increases linearly with the level of parallelism! As far as I can tell, multi-processor AMD *is* a NUMA architecture, which might potentially explain things. But enabling the JVM NUMA options doesn’t seem to affect the benchmark. I think next steps are two-fold: (1) examine parallelism vs real & CPU time on an Intel system, and (2) attempt to reproduce the observed behavior in pure Java. I’m keeping my fingers crossed that I’ll have some time to look at this more soon, but I’m honestly not very hopeful. In the mean time, I hope you’ve managed to exploit multi-process parallelism to run more efficiently? -Marshall -- -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups "Clojure" group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
