On Sat, May 15, 2021 at 02:22:24PM -0400, Andrew Dunstan wrote: > PostgresNode is currently able to create nodes suitable for upgrade down > to release 10. If we add '-w' to the 'pg_ctl start' flags that can > extend down to release 9.5. (Just tested) I think we should do that > forthwith. '-w' is now the default, but having it there explicitly does > no harm.
Agreed. When testing manually, I have personally never worked on any patches that required binaries older than 9.4, so I would be fine if the TAP tests are able to work easily down to 9.5, even if pg_upgrade is supported down to 8.4. > If people are interested in what's incompatible on older versions, they > can look at > <https://gitlab.com/adunstan/postgresnodeng/-/blob/master/PostgresNode.pm> > starting at about line 2764. We should really have adjust_conf() at some point in the in-core module. > I don't think this will work, though, unless there is enough data to > exercise pg_upgrade fairly thoroughly. The approach taken by both > test.sh and (somewhat more comprehensively) by the buildfarm cross > version upgrade module is to test a cluster where the regression tests > have been run. Yeah, that's what my patch is doing with pg_regress, FWIW. This requires regress.so from the old version. > That might be more difficult when testing against older > versions, so I have published a snapshot of the dumps of each of the > versions we tests against in the buildfarm animal crake. These could be > loaded into PostgresNode instances and then an upgrade attempted. See > <https://gitlab.com/adunstan/pg-old-bin/-/tree/master/data>. The data > goes back to 9.2. These compressed dumps are a couple of megabytes each, > not huge. I agree that this can be simpler in some cases. In your experience, how much of an issue is it when it becomes necessary to keep around binaries that rely on libraries older than what a system can support? It is easy to counter issues in this area with OpenSSL and non-necessary things, but we had in the past also cases where we had code that conflicted with the kernel, aka 3e68686. At the end of this exercise, what I think we should achieve is to: 1) Reduce the diff between the buildfarm code and the in-core code. 2) Get rid of test.sh. 3) Be able to run easily upgrade tests across major versions for developers. As of now, 3) requires some extra facilities depending on if this is done by the buildfarm or the in-core tests: 1) Path to the source code of the old version. This is required once it becomes necessary to find out src/test/regress/ for the schedule, the tests to run and its regress.so. There is no need to do that if you have a dump of the old instance. 2) Path to a custom dump to replace the run with pg_regress from 1). 3) Location of the old binaries, for pg_upgrade. When it comes to PostgresNode, we require an install path, so we cannot use directly the location of the binaries. Looking at the code of the buildfarm, its code does something smarter than what my patch or HEAD's test.sh does now, as these require the path for the old source. The buildfarm code first scans for the probin's used in the regression database and then updates any references. What test.sh and my patch do is using the path to the old source code and run a single UPDATE. The approach taken by the buildfarm code is more portable, as a path to the old source code becomes necessary only if running pg_regress manually. So, what about doing the following thing? 1) Update the TAP test so as probin entries are updated in the same way as the buildfarm. 2) Allow one to specify a path to a custom dump, or a path to the old source code for pg_regress. If we do that, then it should be possible to reduce the code footprint in the buildfarm code, while still allowing people to test major upgrades in the same old-fashioned way, right? That's assuming that PostgresNode is made compatible down to 9.2, of course, as a first step, as that's the range of the dumps you are keeping around for the buildfarm. Thoughts? -- Michael
signature.asc
Description: PGP signature
