Hi, I had tested SPECjbb2005 on Harmony and submitted the results for reviewing to SPEC. SPEC committee has some questions on my submission concerning Harmony support and testing of command-line options.
1) Support of Harmony: The issue is section (1) of the SPECjbb2005 Run Rules [1], citing: "The implementation is generally available, documented and supported by the providing vendor." Though the support requirement for SPECjbb2005 is loosely defined, all current publications are on Java implementations that provide various levels of support. SPEC argues that this level of support cannot be compared to forum support provided by Harmony developers. SPEC argues that current production JVMs on the market today are fully supported by the companies that develop them and they need to clarify these issues before any result using Harmony can be accepted. What can you comment on this? 2) Testing: SPEC also concerned about testing required for performance optimizations to be considered as generally available, broad and suggested for production use. They want to know what checks are in place to prevent unsafe changes that benefit SPECjbb2005 only? I will be glad to see the input on these questions from Harmony developers. Thanks, Sergey Nesterov, CS Department of MSU [1] http://www.spec.org/jbb2005/docs/RunRules.html
