I only do those benchmarks on a desktop linux box with the cpu governer set to a fixed frequency. Imo that's the only way to get reproducible numbers. Especially when working on a notebook it also depends how much heat the notebook can dissipate. I've seen my MBP not using his full turbo when he is really hot after benchmarks (even singlethreaded).
LieGrue, strub ----- Original Message ----- > From: Kristian Rosenvold <kristian.rosenv...@gmail.com> > To: Maven Developers List <dev@maven.apache.org> > Cc: > Sent: Wednesday, December 12, 2012 9:35 AM > Subject: Re: Logback in Maven Core > > Yeah, measuring performance on modern cpu's is totally borked. To get > any real measurements one probably needs to to average of 100 non-stop > builds or similar, to counter for all the dark magics intel do with > temperature-based overclocking. > > I think I've seen somewhere that it's possible to disable all the > cpu-voodo in the bios. But what a pain to reboot to change those > settings ! Or dig out the old Pentium4 from the closet. > > Kristian > > 2012/12/12 Ralph Goers <ralph.go...@dslextreme.com>: >> I checked out Maven and used its build as a comparison. First, I ran the > log4j 2 build and it was taking around 59 seconds. I then changed the > log4j2.xml to remove the colors. I then got an average time for Log4j 2 of > 54.76s and for Logback I get an average of 55.225s. I consider these > differences to be meaningless. >> >> For reference, the log4j2.xml I used is attached. >> >> Ralph >> >> >> >> >> >> >> >> >> >> >> On Dec 11, 2012, at 11:19 PM, Stephen Connolly wrote: >> >>> Well I am not going to tar and feather log4j2 based on one set of runs > on >>> my machine. I would like somebody else to repeat and confirm first as > there >>> could have been some background OS update or other process stealing CPU >>> while doing the 3 log4j2 runs. >>> >>> Also I do not know if I am comparing the same things. Afaik the log > back >>> branch has the latest fixes in it, while the log4j2 branch is the > colorized >>> one from a few weeks back and likely has not got the fixes required for > the >>> issues you identified with the last 3.1.0 RC >>> >>> We need to compare like with like to make an informed decision... I am > just >>> putting some numbers down as a starting point >>> >>> -Stephen >>> >>> On Wednesday, 12 December 2012, Kristian Rosenvold wrote: >>> >>>> Finally some interesting numbers, and if (heaven forbid) this > decision >>>> should be based on >>>> technical grounds, this is one of the first significant pieces to > come >>>> up in this discussion. >>>> >>>> Since I am quite unfamiliar with logging (I use loose coupling and >>>> tests instead ;), I took the opportunity to read >>>> http://logging.apache.org/log4j/2.x/performance.html Somehow the >>>> real-life results don't seem to match up with the advertising > blurp on >>>> the log4j site. While it hardly surprises me, I was wondering if >>>> anyone actually knows why? >>>> >>>> Kristian >>>> >>>> >>>> >>>> 2012/12/12 Stephen Connolly > <stephen.alan.conno...@gmail.com<javascript:;> >>>>> : >>>>> The consistent times (i.e. repeated runs after discarding the > first) are: >>>>> >>>>> 3.0.4: 1min18sec >>>>> logback: 1min13sec >>>>> log4j2: 1min34sec >>>>> >>>>> The second test was building GIT hash >>>>> 85dd6e37456d30d2661e10b044efa9036c528023 of jszip-maven-plugin > (@ >>>> jszip.org) >>>>> with the following command line: >>>>> >>>>> mvn -o -X clean verify -DskipTests -Dinvoker.skip >>>>> >>>>> [Testing heavy logging] >>>>> >>>>> 3.0.4: 12.1sec >>>>> logback: 12.2sec >>>>> log4j2: 12.5sec >>>> >>>> > --------------------------------------------------------------------- >>>> To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org > <javascript:;> >>>> For additional commands, e-mail: dev-h...@maven.apache.org > <javascript:;> >>>> >>>> >> >> >> >> >> --------------------------------------------------------------------- >> To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org >> For additional commands, e-mail: dev-h...@maven.apache.org > > --------------------------------------------------------------------- > To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org > For additional commands, e-mail: dev-h...@maven.apache.org > --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org For additional commands, e-mail: dev-h...@maven.apache.org
