On 4/6/2013 7:26 PM, Greg Trasuk wrote: ...
Once we have a stable set of regression tests, then OK, we could think about improving performance or using Maven repositories as the codebase server.
...
I think there is something else you need before it would be a good idea to release any changes for the sake of performance - some examples of workloads whose performance you want to improve, and that are in fact improved by the changes.
I've worked on many performance campaigns, at several different companies including Cray Research and Sun Microsystems. Each campaign has been based on one or more benchmarks. The benchmarks could be industry-standard benchmarks, or could be sanitized versions of user workloads.
I've had many opportunities to compare measurements to expert predictions, including my own predictions, about what needs to be changed to improve performance. In no case have the expert predictions matched measurement.
Based on that experience I had serious concerns about working on River performance with no benchmarks. Are there any River users who care enough about performance to help with creating benchmarks, and running them in highly parallel environments? If not, maybe performance changes should not be a priority.
Without benchmarks, changes intended to improve performance may carry functional risk for no gain, or even for a performance regression.
Patricia
