Hi, Claes, Thank you for fixing that. --lx
On 10/28/20, 4:07 PM, "Claes Redestad" <[email protected]> wrote: CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you can confirm the sender and know the content is safe. FWIW I just integrated the change to remove --enable-preview from the micros build, so the UnsupportedClassVersionError should be gone if you sync up. No --enable-preview needed. /Claes On 2020-10-28 11:09, Liu, Xin wrote: > Hi, Claes, > > Thank you for the great guideline of micro benchmarking. now I have a clearer picture for it. > I'm happy to learn that 5-10 minutes for each Benchmark! Let me polish mine. > > Thanks, > --lx > > On 10/27/20, 3:59 AM, "Claes Redestad" <[email protected]> wrote: > > CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you can confirm the sender and know the content is safe. > > > > Hi lx, > > the need to add --enable-preview is known, and it's a bug introduced a > couple of months ago thanks to a suggestion I did (without realizing > that compiling one micro with --enable-preview would poison all of > them..). This issue is tracked by a few bugs, see > https://bugs.openjdk.java.net/browse/JDK-8250669 and > https://bugs.openjdk.java.net/browse/JDK-8253828 > > The fix will likely be to remove the --enable-preview from the micros > build after Records goes out of preview and then think more carefully on > how to add support for benchmarking preview features. > > As to your other question: When adding a microbenchmark it's appreciated > if the default configuration makes it as quick to run as possible > without sacrificing quality. Often the default number of forks, > iteration runtime et.c. can be tuned down by using custom settings in > @Fork et.c. Custom parameters used can be limited to a select handful. > > Still it's normal for a "good" microbenchmark to require > at least five, usually ten minutes per @Benchmark to produce reasonably > trustworthy results. > > I can't speak for what others do, but we (Oracle) select what to run > regularly from the available microbenchmarks at our discretion, though. > That means adding a microbenchmark that takes a prohibitively long time > won't be disruptive to our nightly testing, but for obvious (mostly > economic) reasons we'd be unlikely to add an extremely long-running > micro to regular coverage. > > /Claes > > On 2020-10-27 10:06, Liu, Xin wrote: > > Hi, > > > > I follow the instruction on https://urldefense.com/v3/__https://htmlpreview.github.io/?https:**Agithub.com*openjdk*jdk*blob*master*doc*testing.html*microbenchmarks__;Ly8vLy8vLy8j!!GqivPVa7Brio!ND3mbOTCkCdkICq5NKxlPCWh7dWYhtMdQdg0GabOEKe0FlDTJDkj-lfNyWujH7AmDts$ to run a new microbenchmark. > > I got error message as follows. > > java.lang.UnsupportedClassVersionError: Preview features are not enabled for org/openjdk/bench/java/lang/reflect/proxy/jmh_generated/ProxyBench_newProxyInstance1i_jmhTest (class file version 60.65535). Try running with '--enable-preview' > > at java.base/java.lang.ClassLoader.defineClass1(Native Method) > > at java.base/java.lang.ClassLoader.defineClass(ClassLoader.java:1010) > > at java.base/java.security.SecureClassLoader.defineClass(SecureClassLoader.java:150) > > at java.base/jdk.internal.loader.BuiltinClassLoader.defineClass(BuiltinClassLoader.java:855) > > at java.base/jdk.internal.loader.BuiltinClassLoader.findClassOnClassPathOrNull(BuiltinClassLoader.java:753) > > at java.base/jdk.internal.loader.BuiltinClassLoader.loadClassOrNull(BuiltinClassLoader.java:676) > > at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:634) > > at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:182) > > at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:519) > > at java.base/java.lang.Class.forName0(Native Method) > > at java.base/java.lang.Class.forName(Class.java:377) > > at org.openjdk.jmh.util.ClassUtils.loadClass(ClassUtils.java:72) > > at org.openjdk.jmh.runner.BenchmarkHandler.<init>(BenchmarkHandler.java:68) > > at org.openjdk.jmh.runner.BaseRunner.runBenchmark(BaseRunner.java:232) > > at org.openjdk.jmh.runner.BaseRunner.doSingle(BaseRunner.java:138) > > at org.openjdk.jmh.runner.BaseRunner.runBenchmarksForked(BaseRunner.java:75) > > at org.openjdk.jmh.runner.ForkedRunner.run(ForkedRunner.java:72) > > at org.openjdk.jmh.runner.ForkedMain.main(ForkedMain.java:84) > > > > > > If I add an extra flag “—enable-preview” , the micro bench will work. > > make test TEST="micro:java.lang.reflect" MICRO="VM_OPTIONS=--enable-preview" > > > > I am using jmh from make/devkit/createJMHBundle.sh, which gives me jmh-core-1.26.jar > > Is that jmh-core too old or we should adjust RunTest.gmk a little bit to have that flag? > > > > Does Openjdk CI run benchmarks to detect performance regression? If so, could you point me to a guideline of microbenmark? > > I lack of the basic sense of a “good” microbenchmark. My concern is the new micro I added is too long to break the established test run. > > > > Thanks, > > --lx > > > > >
