Re: [HACKERS] pg_upgrade libraries check

2012-05-29 Thread Tom Lane
Jeff Davis writes: > Also, I think it needs to force the extension version to match the old > cluster. Otherwise, we could be dealing with on-disk format changes, or > other complexities. Yeah, I was thinking we might want --binary-upgrade to specify a particular extension version in CREATE EXTEN

Re: [HACKERS] pg_upgrade libraries check

2012-05-29 Thread Jeff Davis
On Mon, 2012-05-28 at 16:06 -0400, Robert Haas wrote: > On Sun, May 27, 2012 at 11:31 AM, Tom Lane wrote: > > I don't recall exactly what problems drove us to make pg_upgrade do > > what it does with extensions, but we need a different fix for them. > > Well, you need pg_upgrade to preserve the O

Re: [HACKERS] pg_upgrade libraries check

2012-05-29 Thread Bruce Momjian
On Tue, May 29, 2012 at 05:35:18PM -0400, Tom Lane wrote: > Bruce Momjian writes: > > The bottom line is that already needed function shared objects checking, > > so we just wrapped languages into that as well. > > Well, that'd be fine if it actually *worked*, but as per this > discussion, it doe

Re: [HACKERS] pg_upgrade libraries check

2012-05-29 Thread Tom Lane
Bruce Momjian writes: > The bottom line is that already needed function shared objects checking, > so we just wrapped languages into that as well. Well, that'd be fine if it actually *worked*, but as per this discussion, it doesn't work; you have to kluge around the fact that the shared library n

Re: [HACKERS] pg_upgrade libraries check

2012-05-29 Thread Bruce Momjian
On Tue, May 29, 2012 at 04:10:41PM -0400, Tom Lane wrote: > Hmmm ... that is a good point; right now, there are probably still an > awful lot of installations that have PL languages installed "bare" > rather than wrapped in an extension, including all the ones where the > plpython library has the o

Re: [HACKERS] pg_upgrade libraries check

2012-05-29 Thread Tom Lane
Bruce Momjian writes: > On Tue, May 29, 2012 at 02:38:03PM -0400, Tom Lane wrote: >> Well, it'd eliminate the kluges for plpython, as well as possibly fixing >> Andrew's JSON backport issue, and going forward there are going to be >> more things like that. So I think it's something to pursue, but

Re: [HACKERS] pg_upgrade libraries check

2012-05-29 Thread Bruce Momjian
On Tue, May 29, 2012 at 02:38:03PM -0400, Tom Lane wrote: > Bruce Momjian writes: > > On Tue, May 29, 2012 at 01:46:28PM -0400, Tom Lane wrote: > >> Bruce Momjian writes: > >>> In fact, can we identify right now if a function is used only by an > >>> extension? > > >> ITYM is the function define

Re: [HACKERS] pg_upgrade libraries check

2012-05-29 Thread Tom Lane
Bruce Momjian writes: > On Tue, May 29, 2012 at 01:46:28PM -0400, Tom Lane wrote: >> Bruce Momjian writes: >>> In fact, can we identify right now if a function is used only by an >>> extension? >> ITYM is the function defined by an extension, and the answer to that is >> "look in pg_depend". >

Re: [HACKERS] pg_upgrade libraries check

2012-05-29 Thread Bruce Momjian
On Tue, May 29, 2012 at 01:46:28PM -0400, Tom Lane wrote: > Bruce Momjian writes: > > I assumed I could just have pg_upgrade create and drop the extension in > > the new database to make sure it works. In the JSON backpatch case, the > > extension file would just do nothing, as has already been s

Re: [HACKERS] pg_upgrade libraries check

2012-05-29 Thread Tom Lane
Bruce Momjian writes: > I assumed I could just have pg_upgrade create and drop the extension in > the new database to make sure it works. In the JSON backpatch case, the > extension file would just do nothing, as has already been suggested. It seems like checking for the control file being prese

Re: [HACKERS] pg_upgrade libraries check

2012-05-29 Thread Bruce Momjian
On Sun, May 27, 2012 at 02:32:52PM -0400, Tom Lane wrote: > Andrew Dunstan writes: > > AIUI, for Tom's scheme to work pg_upgrade would need not to check > > libraries that belonged to extension functions. Currently it simply > > assumes that what is present in the old cluster is exactly what wil

Re: [HACKERS] pg_upgrade libraries check

2012-05-29 Thread Bruce Momjian
On Sun, May 27, 2012 at 02:39:28PM -0400, Tom Lane wrote: > Bruce Momjian writes: > > On Sun, May 27, 2012 at 11:31:12AM -0400, Tom Lane wrote: > >> I don't recall exactly what problems drove us to make pg_upgrade do > >> what it does with extensions, but we need a different fix for them. > > > U

Re: [HACKERS] pg_upgrade libraries check

2012-05-29 Thread Dimitri Fontaine
Tom Lane writes: > is not going to happen. Putting shared libraries into a > postgres-writable directory will be seen (correctly) as a security hole > of the first magnitude, not to mention that in many systems it'd require > root privilege anyway to adjust the dynamic linker's search path. It s

Re: [HACKERS] pg_upgrade libraries check

2012-05-28 Thread Robert Haas
On Sun, May 27, 2012 at 11:31 AM, Tom Lane wrote: > I don't recall exactly what problems drove us to make pg_upgrade do > what it does with extensions, but we need a different fix for them. Well, you need pg_upgrade to preserve the OIDs of objects that are part of extensions just as you do for an

Re: [HACKERS] pg_upgrade libraries check

2012-05-28 Thread Tom Lane
Dimitri Fontaine writes: > I have some plans that we will be discussing later in the new dev cycle > and that would impact such a method if we're to follow them. To better > solve both the per-system (not even cluster) and per-database extension > versions and the inline/os-packaged extension disc

Re: [HACKERS] pg_upgrade libraries check

2012-05-28 Thread Dimitri Fontaine
Tom Lane writes: > Well, the scheme I had in mind would require pg_upgrade to verify that > the new cluster contains an extension control file for each extension in > the old cluster (which is something it really oughta do anyway, if it > doesn't already). After that, though, it ought not be look

Re: [HACKERS] pg_upgrade libraries check

2012-05-27 Thread Andrew Dunstan
On 05/27/2012 02:32 PM, Tom Lane wrote: Andrew Dunstan writes: AIUI, for Tom's scheme to work pg_upgrade would need not to check libraries that belonged to extension functions. Currently it simply assumes that what is present in the old cluster is exactly what will be needed in the new cluste

Re: [HACKERS] pg_upgrade libraries check

2012-05-27 Thread Tom Lane
Bruce Momjian writes: > On Sun, May 27, 2012 at 11:31:12AM -0400, Tom Lane wrote: >> I don't recall exactly what problems drove us to make pg_upgrade do >> what it does with extensions, but we need a different fix for them. > Uh, pg_upgrade doesn't do anything special with extensions, so it must

Re: [HACKERS] pg_upgrade libraries check

2012-05-27 Thread Tom Lane
Andrew Dunstan writes: > AIUI, for Tom's scheme to work pg_upgrade would need not to check > libraries that belonged to extension functions. Currently it simply > assumes that what is present in the old cluster is exactly what will be > needed in the new cluster. Tom's proposed scheme would com

Re: [HACKERS] pg_upgrade libraries check

2012-05-27 Thread Andrew Dunstan
On 05/27/2012 12:50 PM, Bruce Momjian wrote: On Sun, May 27, 2012 at 12:31:09PM -0400, Andrew Dunstan wrote: On 05/27/2012 11:31 AM, Tom Lane wrote: Having said that, I've got to also say that I think we've fundamentally blown it with the current approach to upgrading extensions. Because w

Re: [HACKERS] pg_upgrade libraries check

2012-05-27 Thread Bruce Momjian
On Sun, May 27, 2012 at 12:31:09PM -0400, Andrew Dunstan wrote: > > > On 05/27/2012 11:31 AM, Tom Lane wrote: > > > > > >Having said that, I've got to also say that I think we've fundamentally > >blown it with the current approach to upgrading extensions. Because we > >dump all the extension mem

Re: [HACKERS] pg_upgrade libraries check

2012-05-27 Thread Andrew Dunstan
On 05/27/2012 11:31 AM, Tom Lane wrote: Having said that, I've got to also say that I think we've fundamentally blown it with the current approach to upgrading extensions. Because we dump all the extension member objects, the extension contents have got to be restorable into a new database v

Re: [HACKERS] pg_upgrade libraries check

2012-05-27 Thread Bruce Momjian
On Sun, May 27, 2012 at 12:08:14PM -0400, Bruce Momjian wrote: > > > We are not writing a one-off pg_upgrade for JSON-backpatchers here. > > > > I tend to agree with that position, and particularly think that we > > should not allow the not-community-approved design of the existing > > JSON backpo

Re: [HACKERS] pg_upgrade libraries check

2012-05-27 Thread Bruce Momjian
On Sun, May 27, 2012 at 11:31:12AM -0400, Tom Lane wrote: > Bruce Momjian writes: > > On Sun, May 27, 2012 at 08:48:54AM -0400, Andrew Dunstan wrote: > >> "things like CREATE LANGUAGE plperl" is a rather vague phrase. The > >> PL case could be easily handled by adding this to the query: > >> OR EX

Re: [HACKERS] pg_upgrade libraries check

2012-05-27 Thread Tom Lane
Bruce Momjian writes: > On Sun, May 27, 2012 at 08:48:54AM -0400, Andrew Dunstan wrote: >> "things like CREATE LANGUAGE plperl" is a rather vague phrase. The >> PL case could be easily handled by adding this to the query: >> OR EXISTS (SELECT 1 FROM pg_catalog.pg_language WHERE lanplcallfoid >> =

Re: [HACKERS] pg_upgrade libraries check

2012-05-27 Thread Bruce Momjian
On Sun, May 27, 2012 at 08:48:54AM -0400, Andrew Dunstan wrote: > > >I just realized the problem as part of debugging the report of a problem > >with plpython_call_handler(): > > > > http://archives.postgresql.org/pgsql-hackers/2012-03/msg01101.php > > http://archives.postgresql.org/pgsql-

Re: [HACKERS] pg_upgrade libraries check

2012-05-27 Thread Andrew Dunstan
On 05/27/2012 06:40 AM, Bruce Momjian wrote: On Fri, May 25, 2012 at 11:08:10PM -0400, Bruce Momjian wrote: On Fri, May 25, 2012 at 10:20:29AM -0400, Andrew Dunstan wrote: pg_upgrade is a little over-keen about checking for shared libraries that back functions. In particular, it checks for li

Re: [HACKERS] pg_upgrade libraries check

2012-05-27 Thread Bruce Momjian
On Fri, May 25, 2012 at 11:08:10PM -0400, Bruce Momjian wrote: > On Fri, May 25, 2012 at 10:20:29AM -0400, Andrew Dunstan wrote: > > pg_upgrade is a little over-keen about checking for shared libraries > > that back functions. In particular, it checks for libraries that > > support functions create

Re: [HACKERS] pg_upgrade libraries check

2012-05-25 Thread Bruce Momjian
On Fri, May 25, 2012 at 10:20:29AM -0400, Andrew Dunstan wrote: > pg_upgrade is a little over-keen about checking for shared libraries > that back functions. In particular, it checks for libraries that > support functions created in pg_catalog, even if pg_dump doesn't > export the function. > > The

[HACKERS] pg_upgrade libraries check

2012-05-25 Thread Andrew Dunstan
pg_upgrade is a little over-keen about checking for shared libraries that back functions. In particular, it checks for libraries that support functions created in pg_catalog, even if pg_dump doesn't export the function. The attached patch mimics the filter that pg_dump uses for functions so t