On Tue, Apr 15, 2014 at 5:47 PM, Josh Berkus <j...@agliodbs.com> wrote:
> Hackers, > > I think 9.3 has given us evidence that our users aren't giving new > versions of PostgreSQL substantial beta testing, or if they are, they > aren't sharing the results with us. > > How can we make beta testing better and more effective? How can we get > more users to actually throw serious workloads at new versions and share > the results? > > I've tried a couple of things over the last two years and they haven't > worked all that well. Since we're about to go into another beta testing > period, we need something new. Ideas? > I think it boils down to making it really easy to create a workload generator. Most companies have simple single-threaded regression tests for functionality but very few companies have good parallel workload generators which reflect activities in their production environment. A documented beta test process/toolset which does the following would help: 1) Enables full query logging 2) Creates a replica of a production DB, record $TIME when it stops. 3) Allow user to make changes (upgrade to 9.4, change hardware, change kernel settings, ...) 4) Plays queries from the CSV logs starting from $TIME mimicking actual timing and transaction boundaries If Pg can make it easy to duplicate activities currently going on in production inside another environment, I would be pleased to fire a couple billion queries through it over the next few weeks. #4 should include reporting useful to the project, such as a sampling of queries which performed significantly worse and a few relative performance stats for overall execution time.
