Thanks Seth, It seems from your email that you got the emails I forwarded you as well.
If indeed, we are no longer benchmarking before release to guard against regressions, it seems like we should resume this policy. —Uri Uri Wilensky Lorraine H. Morton chaired Professor Professor of Learning Sciences, Computer Science and Complex Systems Director, Center for Connected Learning and Computer-Based Modeling (CCL) Northwestern University > On Jul 7, 2020, at 1:06 PM, Seth Tisue <s...@tisue.net> wrote: > > (in response to a private query) > > The documentation page of record on NetLogo's benchmark suite is: > https://github.com/NetLogo/NetLogo/wiki/Benchmarking > I'd suggest that this be updated with anything useful from the current > conversation, so that any info or conclusions aren't lost for next time. > > For as long as I remained lead developer of NetLogo, running the benchmarks > prior to release, to guard against performance regressions, was part of our > process. This continued at least through the 5.0.x series and probably also > 5.1.0 (which was led by Frank Duncan, but I was still actively involved as > well). What happened after that, I don't know. > > So 5.0 and/or 5.1 are reasonable choices to use as a baseline. > > Trying to go back farther than that might be difficult, but if someone is > determined to try, the 4.x code is at https://github.com/NetLogo/old-NetLogo > (private link only CCL'ers be able to access, since NetLogo was still > closed-source at that time). I believe that repo also includes the 4.x > versions of the benchmark models themselves. > > Old measurements taken on old hardware/OS/JVM combinations can't be compared > directly to current measurements, of course. If someone wants to run the > NetLogo 5.0 or 5.1 benchmarks on modern hardware+OS+JVM but you run into > practical difficulties, I could probably be of assistance. > > These days everyone benchmarks JVM things using JMH. JMH didn't exist yet > when I created NetLogo's benchmarking infrastructure. Probably that old > stuff still works okay for guarding against performance regressions, but if > there is to be any renewed work on actually improving performance of > JVM-based NetLogo, I would suggest moving to JMH for that. > > Seth > > -- > You received this message because you are subscribed to the Google Groups > "netlogo-devel" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to netlogo-devel+unsubscr...@googlegroups.com > <mailto:netlogo-devel+unsubscr...@googlegroups.com>. > To view this discussion on the web visit > https://groups.google.com/d/msgid/netlogo-devel/bdd3af1a-2247-43ac-91c8-d82bf5d6f84an%40googlegroups.com > > <https://urldefense.com/v3/__https://groups.google.com/d/msgid/netlogo-devel/bdd3af1a-2247-43ac-91c8-d82bf5d6f84an*40googlegroups.com?utm_medium=email&utm_source=footer__;JQ!!Dq0X2DkFhyF93HkjWTBQKhk!HJOq6d4wheLGMAdwQbFLenPBgjHcM_2IIscxzejc9Z0o9W3iEPbnIieNUaukSGArvw$>. -- You received this message because you are subscribed to the Google Groups "netlogo-devel" group. To unsubscribe from this group and stop receiving emails from it, send an email to netlogo-devel+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/netlogo-devel/6D60B9FD-92AD-4684-BAC5-8CB62623A69C%40northwestern.edu.
