Re: [HACKERS] Upgrading Extension, version numbers
Robert Haas writes: > My understanding of the consensus is that it wasn't felt necessary for > the purpose for which it was proposed. I think it could be > re-proposed with a different argument and very possibly accepted. Sure. I'd still prefer us to adopt the solution I've been promoting, obviously, which I think has more merits. Namely no directory scanning, easy to support extension names such as postgis-1.5, and easy to support for a single upgrade file supporting upgrades from more than a single version, and bypassing entirely the need to know what version numbering scheme is in use: you just don't need to know how to compute previous or next version number. Now it's all about tradeoffs, and I'm just trying to explain what the one I'm doing here seems to me to have lot of sense. Regards, -- Dimitri Fontaine http://2ndQuadrant.fr PostgreSQL : Expertise, Formation et Support -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Upgrading Extension, version numbers
On Jan 5, 2011, at 10:05 AM, Robert Haas wrote: >> There's no consensus to publish a bakend \i like function. So there's >> no support for this upgrade script organizing you're promoting. Unless >> the consensus changes again (but a commit has been done). > > My understanding of the consensus is that it wasn't felt necessary for > the purpose for which it was proposed. I think it could be > re-proposed with a different argument and very possibly accepted. +1 Yes, exactly. Best, David -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Upgrading Extension, version numbers
On Tue, Jan 4, 2011 at 2:48 PM, Dimitri Fontaine wrote: > "David E. Wheeler" writes: >> As Tom pointed out, you can do the same with naming conventions by having >> scripts \i each other as appropriate. > > This is a deprecated idea, though. We're talking about the > pg_execute_from_file() patch that has been applied, but without the > pg_execute_sql_file() function. So that part is internal to the backend > extension code and not available from SQL anymore. > > There's no consensus to publish a bakend \i like function. So there's > no support for this upgrade script organizing you're promoting. Unless > the consensus changes again (but a commit has been done). My understanding of the consensus is that it wasn't felt necessary for the purpose for which it was proposed. I think it could be re-proposed with a different argument and very possibly accepted. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Upgrading Extension, version numbers
On Jan 4, 2011, at 12:05 PM, Dimitri Fontaine wrote: > "David E. Wheeler" writes: >> * Prefer convention over configuration > > The previous idea about the convention is not flying well with the very > recent proposal of ALTER EXTENSION ... UPGRADE TO VERSION ..., because > it would certainly require that the extension's name include its major > version number, like debian is doing for a number of packages. No, just the file. > Also, how are PostGIS 1.4 and 1.5 (and 2.0) packaged nowadays? Tarballs. >> * Not make me do more work that the computer can do > > No computer will guess reliably which upgrade file to apply given the > currently installed version and the newer one, as soon as the same file > can get used for more than a single combination of those two strings. Why not? Version numbers would have to be part of the file names. The only wrinkle is being able to properly order version numbers, and we could address that by requiring a specific version format. Tom suggested integers; I suggested semantic versions. > I much prefer to avoid shipping that many files, and thinks that even in > the worst case where you have to add a setup line per supported upgrade > setup, the control file support for that is better. Well, for a version that requires no upgrade script, there just wouldn't be one. It's a matter of taste. > Now I perfectly understand that there's more to this world than my eyes > can see, that's why we're talking about alternatives. You are? Best, David -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Upgrading Extension, version numbers
"David E. Wheeler" writes: > * Prefer convention over configuration The previous idea about the convention is not flying well with the very recent proposal of ALTER EXTENSION ... UPGRADE TO VERSION ..., because it would certainly require that the extension's name include its major version number, like debian is doing for a number of packages. Also, how are PostGIS 1.4 and 1.5 (and 2.0) packaged nowadays? > * Not make me do more work that the computer can do No computer will guess reliably which upgrade file to apply given the currently installed version and the newer one, as soon as the same file can get used for more than a single combination of those two strings. I much prefer to avoid shipping that many files, and thinks that even in the worst case where you have to add a setup line per supported upgrade setup, the control file support for that is better. Now I perfectly understand that there's more to this world than my eyes can see, that's why we're talking about alternatives. Regards, -- Dimitri Fontaine http://2ndQuadrant.fr PostgreSQL : Expertise, Formation et Support -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Upgrading Extension, version numbers
On Jan 4, 2011, at 11:48 AM, Dimitri Fontaine wrote: >> As Tom pointed out, you can do the same with naming conventions by having >> scripts \i each other as appropriate. > > This is a deprecated idea, though. We're talking about the > pg_execute_from_file() patch that has been applied, but without the > pg_execute_sql_file() function. So that part is internal to the backend > extension code and not available from SQL anymore. > > There's no consensus to publish a bakend \i like function. So there's > no support for this upgrade script organizing you're promoting. Unless > the consensus changes again (but a commit has been done). To be clear, consensus was not reached, by my reading. It may be that it makes sense to restore pg_execute_sql_file(), perhaps to run only in the context of ALTER EXTENSION. Just to be clear where I'm coming from, as an extension developer, I would like PostgreSQL extensions to: * Prefer convention over configuration * Not make me do more work that the computer can do Best, David -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Upgrading Extension, version numbers
"David E. Wheeler" writes: > As Tom pointed out, you can do the same with naming conventions by having > scripts \i each other as appropriate. This is a deprecated idea, though. We're talking about the pg_execute_from_file() patch that has been applied, but without the pg_execute_sql_file() function. So that part is internal to the backend extension code and not available from SQL anymore. There's no consensus to publish a bakend \i like function. So there's no support for this upgrade script organizing you're promoting. Unless the consensus changes again (but a commit has been done). Regards, -- Dimitri Fontaine http://2ndQuadrant.fr PostgreSQL : Expertise, Formation et Support -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Upgrading Extension, version numbers
On Jan 4, 2011, at 12:46 AM, Dimitri Fontaine wrote: > "David E. Wheeler" writes: >> Just so long as you're aware that you might get more challenges on this >> going forward. > > Sure, thanks for the reminder. That said I also remember the reaction > when I used to scan the SHARE/contrib directory to find the extension > control file having the right name property, and I don't see scanning > the same directory in order to find out which upgrade file to consider > depending on several parts of its name as so different. Silly programmer! You don't have to do that yourself! You can teach the computer to do it for you. It's very good at that sort of thing! > Current code allows you to use the same upgrade script for more than one > source version, and does so in a way that it's easy to determine which > upgrade file to seek for. As Tom pointed out, you can do the same with naming conventions by having scripts \i each other as appropriate. >> I guess. I'll have to think about how to support it in PGXN, though. And the >> upgrade keys if they stay in. > > Disclaimer: the following is based on my understanding of how you want > to bundle things, from several discussions we had together at pubs or > on IRC, please don't read further if you're changed your mind about > generating the control file from your PGXN YAML specification. s/YAML/JSON/, and okay. :-) > Well, I think you're having a dependency inversion problem here. PGXN > depends on extensions, not the other way round. What? That makes no sense, so I must be misunderstanding what you're trying to say. > Also, I really expect > the extension facility to be mainly used for internal proprietary code, > mainly procedure collections, and only occasionaly for publishing Open > Source components. This is because you're not a Perl programmer. See CPAN. > So you should be considering the control file as an input to your > processes, a source file, not something that your service will hide for > extension authors: there's no benefit that I can see in doing so. I know, but then you're not a CPAN guy. You're a Debian package guy. It's hardly surprising that we'll have inverted views of this sort of thing. Frankly, I think that you might find StackBuilder a better fit with your world view. http://pgfoundry.org/projects/stackbuilder/ Best, David -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Upgrading Extension, version numbers
"David E. Wheeler" writes: > Just so long as you're aware that you might get more challenges on this going > forward. Sure, thanks for the reminder. That said I also remember the reaction when I used to scan the SHARE/contrib directory to find the extension control file having the right name property, and I don't see scanning the same directory in order to find out which upgrade file to consider depending on several parts of its name as so different. Current code allows you to use the same upgrade script for more than one source version, and does so in a way that it's easy to determine which upgrade file to seek for. >> I don't see the benefit of having the 'relocatable' property optional in >> the control file, but I see a huge drawback. Requiring it will force >> extension authors to at least have a glance at the docs to understand >> how to set it. It's important not to overlook it. > > I guess. I'll have to think about how to support it in PGXN, though. And the > upgrade keys if they stay in. Disclaimer: the following is based on my understanding of how you want to bundle things, from several discussions we had together at pubs or on IRC, please don't read further if you're changed your mind about generating the control file from your PGXN YAML specification. Well, I think you're having a dependency inversion problem here. PGXN depends on extensions, not the other way round. Also, I really expect the extension facility to be mainly used for internal proprietary code, mainly procedure collections, and only occasionaly for publishing Open Source components. So you should be considering the control file as an input to your processes, a source file, not something that your service will hide for extension authors: there's no benefit that I can see in doing so. Regards, -- Dimitri Fontaine http://2ndQuadrant.fr PostgreSQL : Expertise, Formation et Support -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Upgrading Extension, version numbers
On Jan 3, 2011, at 12:23 PM, Dimitri Fontaine wrote: > "David E. Wheeler" writes: >> The fact that the last two messages in the thread say something else >> does not mean that they represent the consensus. > > Yeah, but as I'm the one writing the code, I gave myself more than one > vote. And did consider the alternatives but didn't like them so much. Just so long as you're aware that you might get more challenges on this going forward. > Most are, but it's not for granted. See adminpack. Or earthdistance > that I had to make not-relocatable for lack of dependency support, as it > depends on cube and ALTER EXTENSION earthdistance SET SCHEMA foo would > have relocated cube too. We said dependency can wait until v2. > > I don't see the benefit of having the 'relocatable' property optional in > the control file, but I see a huge drawback. Requiring it will force > extension authors to at least have a glance at the docs to understand > how to set it. It's important not to overlook it. I guess. I'll have to think about how to support it in PGXN, though. And the upgrade keys if they stay in. David -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Upgrading Extension, version numbers
"David E. Wheeler" writes: > The fact that the last two messages in the thread say something else > does not mean that they represent the consensus. Yeah, but as I'm the one writing the code, I gave myself more than one vote. And did consider the alternatives but didn't like them so much. > Right, I forgot about the relocatable parameter. I kind of expect that most > extensions *would* be relocatable, though. Maybe it should be expected to be > true if it's not present? Or perhaps require non-relocatable extensions to > have a "fixed_schema" control key or something? Either will work, just trying > to find the likely convention to avoid configuration in most cases. Maybe I'm > wrong, though, and most extensions wouldn't be relocatable? Most are, but it's not for granted. See adminpack. Or earthdistance that I had to make not-relocatable for lack of dependency support, as it depends on cube and ALTER EXTENSION earthdistance SET SCHEMA foo would have relocated cube too. We said dependency can wait until v2. I don't see the benefit of having the 'relocatable' property optional in the control file, but I see a huge drawback. Requiring it will force extension authors to at least have a glance at the docs to understand how to set it. It's important not to overlook it. Regards, -- Dimitri Fontaine http://2ndQuadrant.fr PostgreSQL : Expertise, Formation et Support -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Upgrading Extension, version numbers
On Jan 3, 2011, at 11:46 AM, Dimitri Fontaine wrote: > Not what I have understood. > > http://archives.postgresql.org/pgsql-hackers/2010-12/msg01014.php > http://archives.postgresql.org/pgsql-hackers/2010-12/msg01045.php > > AS there was no answer, the meaning for me is that it was ok to > proceed. On this list people agreeing often remain silent. There were several of us who were not silent. http://archives.postgresql.org/pgsql-hackers/2010-12/msg00804.php http://archives.postgresql.org/pgsql-hackers/2010-12/msg00796.php The fact that the last two messages in the thread say something else does not mean that they represent the consensus. >>> Note that we always need to support the placeholder here, because of >>> course following dependencies at this point isn't possible. >> >> I thought placeholders were going away, too. Did I lose track? > > Oh, dear, yes :) See the documentation for the relocatable parameter. > We know handle two kinds of extensions, some of them you can't offer > better than placeholders to allow users to define the schema where they > will land. Also, at upgrade time, I don't see any other way to solve > the problem. Do you? > > http://pgsql.tapoueh.org/extensions/doc/html/extend-extension.html Right, I forgot about the relocatable parameter. I kind of expect that most extensions *would* be relocatable, though. Maybe it should be expected to be true if it's not present? Or perhaps require non-relocatable extensions to have a "fixed_schema" control key or something? Either will work, just trying to find the likely convention to avoid configuration in most cases. Maybe I'm wrong, though, and most extensions wouldn't be relocatable? > Yeah. Before extension existed, it has always been working like that, > our users already depend on such a behavior, nothing new here. I just > don't see how extension could solve that is all I'm saying. Fair enough. >> The new .so should not be installed until the upgrade is been run. > > Nice statement. How do you make that happen? Nope. David -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Upgrading Extension, version numbers
"David E. Wheeler" writes: > I thought we were going to try to avoid having entries for upgrades in > the control file. Not what I have understood. http://archives.postgresql.org/pgsql-hackers/2010-12/msg01014.php http://archives.postgresql.org/pgsql-hackers/2010-12/msg01045.php AS there was no answer, the meaning for me is that it was ok to proceed. On this list people agreeing often remain silent. >> Note that we always need to support the placeholder here, because of >> course following dependencies at this point isn't possible. > > I thought placeholders were going away, too. Did I lose track? Oh, dear, yes :) See the documentation for the relocatable parameter. We know handle two kinds of extensions, some of them you can't offer better than placeholders to allow users to define the schema where they will land. Also, at upgrade time, I don't see any other way to solve the problem. Do you? http://pgsql.tapoueh.org/extensions/doc/html/extend-extension.html >> At the time you tell PostgreSQL about the new extension, the shared >> object file has been in place for some time already, and the upgrade SQL >> script has not been ran yet. > > That sounds dangerous. Been doing that countless times. Yet to see a case where the new .so is not compatible at all with the previous .sql and the author don't give you any warning about the situation. In theory that's possible, in practice we value upgrades high enough around here. Other than that, I don't have another idea to make it work reliably. I'm still reading, though. Meanwhile I've done what seems like a good compromise and to follow practical use cases. >> What I hope extension authors will do is document whether any upgrade >> requires a restart or will otherwise be responsible for instability in >> the server for backend started with the newer .so before the upgrade >> script has been run. So that users/DBA will know whether the upgrade >> calls for a maintenance window. > > But if a new connection comes in, the .so will be loaded into the new child, > no? Very dangerous. Yeah. Before extension existed, it has always been working like that, our users already depend on such a behavior, nothing new here. I just don't see how extension could solve that is all I'm saying. > The new .so should not be installed until the upgrade is been run. Nice statement. How do you make that happen? Regards, -- Dimitri Fontaine http://2ndQuadrant.fr PostgreSQL : Expertise, Formation et Support -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
