Re: [HACKERS] Modifying and solidifying contrib
Are we doing this? --- Joshua D. Drake wrote: Hello, With all the recent discussion on contrib modules etc.. I would like to offer the following suggestion. I am willing to do a good portion of the work myself and I can get it done before feature freeze. I will need help with the global make file stuff however so that is one dependency. Add directory /modules Modules are compiled and installed by default but not enabled. Modules in 8.3 currently are: pgrowlocks pg_freespacemap (to be renameed pgfreespacemap to be consistent) pgstattuple pgcrypto xml2 pgbuffercache initagg Requirements for /modules Must go through normal vetting process on -hackers Must include patches to core documentation in Docbook Must include test cases? I don't recall if we have regress for all contrib stuff. Keep directory contrib Contrib is not compiled or installed by default Contrib in 8.3 would be: start-scripts pgbench (which I think really should be a foundry project) vacuumlo (is this even required anymore?) adminpack btree_gist etc... Requirements for /contrib Must go through normal vetting process on -hackers Must includes README Must include test cases? Same questions for modules Thoughts, flames? Sincerely, Joshua D. Drake -- === The PostgreSQL Company: Command Prompt, Inc. === Sales/Support: +1.503.667.4564 || 24x7/Emergency: +1.800.492.2240 Providing the most comprehensive PostgreSQL solutions since 1997 http://www.commandprompt.com/ Donate to the PostgreSQL Project: http://www.postgresql.org/about/donate PostgreSQL Replication: http://www.commandprompt.com/products/ ---(end of broadcast)--- TIP 7: You can help support the PostgreSQL project by donating at http://www.postgresql.org/about/donate -- Bruce Momjian [EMAIL PROTECTED] http://momjian.us EnterpriseDB http://www.enterprisedb.com + If your life is a hard drive, Christ can be your backup. + ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] Modifying and solidifying contrib
On 2/6/07, Andrew Dunstan [EMAIL PROTECTED] wrote: If the extension installs everything in dedicated namespace(s), I think we would want to have an option to add those namespaces easily to search paths. Right now all we can do is to set a search path. It would be nice, for example, to have support for appending or prepending something to the search path. I suspect most apps/extensions don't currently use namespaces much, or we might well have seen more demand in this area. I still do not understand why is it so needed. Your argument is some apps aren't able to call functions as schemaname.functionname(arg1, arg2, ..), right? First of all, I do not think that the number of such apps is huge. Second, this is really the problem of those apps themselves. I still think that separate namespaces for extensions is a good idea while adjusting search_path is not. I've explained my POV in details several messages ago in this thread... Separation of extensions with fully specified names schemaname.functionname(...) is good improvement (for simplification and clarity) and while adjusting search_path should be DBA/DBD's decision. -- Best regards, Nikolay ---(end of broadcast)--- TIP 1: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] Modifying and solidifying contrib
On 2/7/07, Nikolay Samokhvalov [EMAIL PROTECTED] wrote: I still think that separate namespaces for extensions is a good idea while adjusting search_path is not. I've explained my POV in details several messages ago in this thread... Separation of extensions with fully specified names schemaname.functionname(...) is good improvement (for simplification and clarity) and while adjusting search_path should be DBA/DBD's decision. Oh, I've just recalled the problem that could arise in this scenario... We cannot use schema name as prefix for operator calling (tsearch2.ts_debug(...) works, while ... tsearch2.@@ ... doesn't). This is one specific issue, maybe it's worth to resolve it? Or it's impossible for some reasons... -- Best regards, Nikolay ---(end of broadcast)--- TIP 9: In versions below 8.0, the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [HACKERS] Modifying and solidifying contrib
Nikolay Samokhvalov wrote: I still do not understand why is it so needed. Your argument is some apps aren't able to call functions as schemaname.functionname(arg1, arg2, ..), right? wrong. I still think that separate namespaces for extensions is a good idea while adjusting search_path is not. I've explained my POV in details several messages ago in this thread... The difference between us is that I am less inclined to be prescriptive about such matters than you are. I think that as namespace use expands we should also probably provide better support for adding things to the search path (or indeed taking things away). If you don't want to use it then don't, but I don't see why you are so insistent on denying such facilities to others. cheers andrew ---(end of broadcast)--- TIP 6: explain analyze is your friend
Re: [HACKERS] Modifying and solidifying contrib
On 2/7/07, Andrew Dunstan [EMAIL PROTECTED] wrote: Nikolay Samokhvalov wrote: I still do not understand why is it so needed. Your argument is some apps aren't able to call functions as schemaname.functionname(arg1, arg2, ..), right? wrong. I still think that separate namespaces for extensions is a good idea while adjusting search_path is not. I've explained my POV in details several messages ago in this thread... The difference between us is that I am less inclined to be prescriptive about such matters than you are. I think that as namespace use expands we should also probably provide better support for adding things to the search path (or indeed taking things away). If you don't want to use it then don't, but I don't see why you are so insistent on denying such facilities to others. ok, looks like I've misunderstood your mesages. Sorry for that. Surely additional capabilities for manipulation with search_path cannot hinder anybody. -- Best regards, Nikolay ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
Re: [HACKERS] Modifying and solidifying contrib
Nikolay Samokhvalov [EMAIL PROTECTED] writes: We cannot use schema name as prefix for operator calling You can, but it's kind of ugly: regression=# SELECT 2 OPERATOR(pg_catalog.+) 2; ?column? -- 4 (1 row) regards, tom lane ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
Re: [HACKERS] Modifying and solidifying contrib
Jim Nasby wrote: In addition to Martijn's tsearch case, there's also PostGIS. And I believe this is a pretty big pain for them. Hear hear! It would be nice to dump from an old PostgreSQL/PostGIS combination and restore to a new version combination, without taking all the function definitions along for a ride in the dump process. What we really want is just the data. -- Paul Ramsey Refractions Research http://www.refractions.net [EMAIL PROTECTED] Phone: 250-383-3022 Cell: 250-885-0632 ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings
Re: [HACKERS] Modifying and solidifying contrib
On Mon, Jan 29, 2007 at 04:22:43PM -0500, Tom Lane wrote: (A) I'm not sure we would have heard about it, and (B) any one user is probably only using a subset of what has been proposed to be loaded by default, so the odds of collisions would go way up. As a data point, some time ago (7.2 days) I used to do this as a matter of completeness, and never had a collision. A -- Andrew Sullivan | [EMAIL PROTECTED] If they don't do anything, we don't need their acronym. --Josh Hamilton, on the US FEMA ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] Modifying and solidifying contrib
Andrew Sullivan wrote: On Mon, Jan 29, 2007 at 04:22:43PM -0500, Tom Lane wrote: (A) I'm not sure we would have heard about it, and (B) any one user is probably only using a subset of what has been proposed to be loaded by default, so the odds of collisions would go way up. As a data point, some time ago (7.2 days) I used to do this as a matter of completeness, and never had a collision. (bangs head against brick wall) of course there isn't a collision. The point I at least have been trying to make is that extensions generally (e.g. from pgfoundry) should protect themselves from possible collisions with core and other unknown extensions that might be loaded, by using unique namespace(s), and further, that the standard extensions (i.e. what we now load from contrib) should act as good exemplars by doing likewise, with some support given for legacy uses that expect them to use the public schema. cheers andrew ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
Re: [HACKERS] Modifying and solidifying contrib
On Tue, Feb 06, 2007 at 11:43:24AM -0500, Andrew Dunstan wrote: Andrew Sullivan wrote: As a data point, some time ago (7.2 days) I used to do this as a matter of completeness, and never had a collision. The point I at least have been trying to make is that extensions generally (e.g. from pgfoundry) should protect themselves from possible collisions with core and other unknown extensions that might be loaded, by using unique namespace(s), and further, that the standard extensions This wasn't a disagreement with your general point. I was just trying to say that the problem did not ine fact exist at some point, so the empirical rathole perhaps doesn't need to be explored. The style question is the only one that is relevant, I think. (I happen to agree with you on that, and it seems to me that a more complete proposal for namespace guidelines might be nice.) A -- Andrew Sullivan | [EMAIL PROTECTED] I remember when computers were frustrating because they *did* exactly what you told them to. That actually seems sort of quaint now. --J.D. Baldwin ---(end of broadcast)--- TIP 6: explain analyze is your friend
Re: [HACKERS] Modifying and solidifying contrib
Nikolay Samokhvalov wrote: On 2/5/07, Andrew Dunstan [EMAIL PROTECTED] wrote: [...] I would suggest we start with what is (I think) simplest and clearest: . catalog support via a simple extension-schema(s) map . initdb installs standard extensions if it finds them, unless told not to . support for adjusting search path. Why adjusting search_path is needed at all? If the extension installs everything in dedicated namespace(s), I think we would want to have an option to add those namespaces easily to search paths. Right now all we can do is to set a search path. It would be nice, for example, to have support for appending or prepending something to the search path. I suspect most apps/extensions don't currently use namespaces much, or we might well have seen more demand in this area. cheers andrew ---(end of broadcast)--- TIP 1: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] Modifying and solidifying contrib
On Feb 5, 2007, at 11:19 AM, Andrew Dunstan wrote: Jim Nasby wrote: There was also mention of having a means to tell pg_dump not to dump extensions... What's the use case for that? What will we do if there are db objects that depend on some extensions? Given that there will be some uninstall support, this one seems less necessary. In addition to Martijn's tsearch case, there's also PostGIS. And I believe this is a pretty big pain for them. I really think we should approach this by not trying to do everything at once. That's fine; I just wanted to point out the use case while lists were being made. -- Jim Nasby[EMAIL PROTECTED] EnterpriseDB http://enterprisedb.com 512.569.9461 (cell) ---(end of broadcast)--- TIP 7: You can help support the PostgreSQL project by donating at http://www.postgresql.org/about/donate
Re: [HACKERS] Modifying and solidifying contrib
Jim Nasby wrote: There was also mention of having a means to tell pg_dump not to dump extensions... What's the use case for that? What will we do if there are db objects that depend on some extensions? Given that there will be some uninstall support, this one seems less necessary. I really think we should approach this by not trying to do everything at once. cheers andrew ---(end of broadcast)--- TIP 1: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] Modifying and solidifying contrib
On Mon, Feb 05, 2007 at 12:19:51PM -0500, Andrew Dunstan wrote: Jim Nasby wrote: There was also mention of having a means to tell pg_dump not to dump extensions... What's the use case for that? What will we do if there are db objects that depend on some extensions? Given that there will be some uninstall support, this one seems less necessary. Well, the use case is someone using tsearch2 on version A and wants to a do a dump to restore into later version B. It would be helpful if pg_dump compacted the whole tsearch2 infrastrcutre into a single INSTALL tsearch2 command. Obviously, the tsearch2 uninstall script for version B isn't going to work properly for a database restore from version A. And this way a dump/restore will pickup any new features added in the later version. I really think we should approach this by not trying to do everything at once. That's true, but it's something to keep in mind. Have a nice day, -- Martijn van Oosterhout kleptog@svana.org http://svana.org/kleptog/ From each according to his ability. To each according to his ability to litigate. signature.asc Description: Digital signature
Re: [HACKERS] Modifying and solidifying contrib
Martijn van Oosterhout wrote: On Mon, Feb 05, 2007 at 12:19:51PM -0500, Andrew Dunstan wrote: Jim Nasby wrote: There was also mention of having a means to tell pg_dump not to dump extensions... What's the use case for that? What will we do if there are db objects that depend on some extensions? Given that there will be some uninstall support, this one seems less necessary. Well, the use case is someone using tsearch2 on version A and wants to a do a dump to restore into later version B. It would be helpful if pg_dump compacted the whole tsearch2 infrastrcutre into a single INSTALL tsearch2 command. Obviously, the tsearch2 uninstall script for version B isn't going to work properly for a database restore from version A. And this way a dump/restore will pickup any new features added in the later version. And if there's an API change everything will blow up. I would suggest we start with what is (I think) simplest and clearest: . catalog support via a simple extension-schema(s) map . initdb installs standard extensions if it finds them, unless told not to . support for adjusting search path. If that gets done nicely for 8.3 we'll be doing well. cheers andrew ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] Modifying and solidifying contrib
On 2/5/07, Andrew Dunstan [EMAIL PROTECTED] wrote: [...] I would suggest we start with what is (I think) simplest and clearest: . catalog support via a simple extension-schema(s) map . initdb installs standard extensions if it finds them, unless told not to . support for adjusting search path. Why adjusting search_path is needed at all? -- Best regards, Nikolay ---(end of broadcast)--- TIP 7: You can help support the PostgreSQL project by donating at http://www.postgresql.org/about/donate
Re: [HACKERS] Modifying and solidifying contrib
There was also mention of having a means to tell pg_dump not to dump extensions... On Jan 30, 2007, at 2:49 PM, Andrew Dunstan wrote: Bruce Momjian wrote: Joshua D. Drake wrote: This seems like a good first step in growing a packaging infrastructure. I'd rather grow it organically than try to design it all up front. I am in Denver and have spotty inet access so forgive me. So where does this above leave us? What are we doing? I was kind of unclear on that too. It seems we are trying to address several issues: visibility of contrib, installation of contrib, etc. We discussed whether we put the functions in public, a schema for all contrib, or a schema for each contrib module, and then there was the discussion of how to configure someone using ten /contrib modules, or at least wanting them all to be accessible. And then there was the idea of allowing schema permissions to control access, so perhaps we could install more of /contrib by default, and allow the administrator to just enable/disable them via permissions. Personally, I think that might be the best approach because it allows us to eliminate the install process, but doesn't make the database less secure --- the administrator enables/disables them at runtime, or at least could. The issues I see are: 1. the 'thing name - the only name I have not seen some objection to is extension. 2. namespace - I think the consensus is tending towards one or more per extension. 3. install/uninstall support: Tom's proposal for an extension- schema map in the catalog will deal with that nicely, I think. 4. visibility/searchpath issues. I don't think long search paths are a huge issue, but I think we can make life a bit easier by tweaking searchpath support a bit (David's clever SQL notwithstanding). 5. legacy support - we need an option to load existing extensions to the public schema as now, or support for aliases/synonyms (the latter might be good to have regardless). 6. they all need proper docs. READMEs and the like are nowhere near good enough. Richard mentioned special testing requirements, but I don't see why we can't continue to use our standard regression mechanism. Mention has also been made of autoloading extensions with initdb. A case could perhaps be made for doing it in createdb - maybe not every db needs ltree, say. OTOH, if it's sitting quietly in its own schema than it's probably not doing any harm either, so maybe initdb should just load all the extensions it finds, and as you say make one less hoop to make people jump through. If we do that I think at least we'd need an option to inhibit autoloading. cheers andrew ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org -- Jim Nasby[EMAIL PROTECTED] EnterpriseDB http://enterprisedb.com 512.569.9461 (cell) ---(end of broadcast)--- TIP 7: You can help support the PostgreSQL project by donating at http://www.postgresql.org/about/donate
Re: [HACKERS] Modifying and solidifying contrib
David Fetter wrote: On Tue, Jan 30, 2007 at 03:49:14PM -0500, Andrew Dunstan wrote: 4. visibility/searchpath issues. I don't think long search paths are a huge issue, but I think we can make life a bit easier by tweaking searchpath support a bit (David's clever SQL notwithstanding). The only clever bit I added was the CASE statement. Credit for the rest belongs to Andrew at Supernews. It's not a bad thing for people to keep around, either way. :) I dislike on principle things that mangle the catalogs directly. As soon as I see one I think why aren't we providing an SQL command for that? By and large, I think users should be able to work as though the catalog were not visible, or at least not directly writable. 5. legacy support - we need an option to load existing extensions to the public schema as now, or support for aliases/synonyms (the latter might be good to have regardless). Hrm. This gets tricky. When things are mandated to be in their own namespace, they need not check what everybody else's things are doing each time, whereas when they go into the public schema... :P Why is it tricky? This is for legacy only, i.e. for object we know of today. Any future objects in existing extensions, or objects in new extensions, should not have this support - they should use their own namespaces, pure and simple. Richard mentioned special testing requirements, but I don't see why we can't continue to use our standard regression mechanism. A subdirectory in src/tests/regression for each one? No. One of the reasons for us to maintain some standard extensions is to act as exemplars. You should be able to build, install and test an extension without having a complete source tree present. So each extension should keep its own sql and expected directory as now, I think. I don't think it would be too much trouble to do extensions the way we now do tables and schemas in pg_dump, i.e. with multiple possible regular expression entries like --include-extension= and --exclude-extension= where the includes get evaluated before the excludes. OK, as long as --exclude-extension has an all option, or we have a --no-extensions option also. cheers andrew ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
Re: [HACKERS] Modifying and solidifying contrib
On Wed, Jan 31, 2007 at 09:31:00AM -0500, Andrew Dunstan wrote: David Fetter wrote: On Tue, Jan 30, 2007 at 03:49:14PM -0500, Andrew Dunstan wrote: 4. visibility/searchpath issues. I don't think long search paths are a huge issue, but I think we can make life a bit easier by tweaking searchpath support a bit (David's clever SQL notwithstanding). The only clever bit I added was the CASE statement. Credit for the rest belongs to Andrew at Supernews. It's not a bad thing for people to keep around, either way. :) I dislike on principle things that mangle the catalogs directly. As soon as I see one I think why aren't we providing an SQL command for that? By and large, I think users should be able to work as though the catalog were not visible, or at least not directly writable. So are you proposing user-visible functions in pg_catalog like the following? append_to_search_path(role NAME, database NAME, paths NAME[]) prepend_to_search_path(role NAME, database NAME, paths NAME[]) remove_from_search_path(role NAME, database NAME, paths NAME[]) The above is how I'm picturing how this would fit in with the TODO of allowing things to be set on a per-role-and-database basis. There could be two-argument short-cuts of each of those which would do the above for the current user. 5. legacy support - we need an option to load existing extensions to the public schema as now, or support for aliases/synonyms (the latter might be good to have regardless). Hrm. This gets tricky. When things are mandated to be in their own namespace, they need not check what everybody else's things are doing each time, whereas when they go into the public schema... :P Why is it tricky? This is for legacy only, i.e. for object we know of today. Any future objects in existing extensions, or objects in new extensions, should not have this support - they should use their own namespaces, pure and simple. OK Richard mentioned special testing requirements, but I don't see why we can't continue to use our standard regression mechanism. A subdirectory in src/tests/regression for each one? No. One of the reasons for us to maintain some standard extensions is to act as exemplars. You should be able to build, install and test an extension without having a complete source tree present. So each extension should keep its own sql and expected directory as now, I think. Right :) I don't think it would be too much trouble to do extensions the way we now do tables and schemas in pg_dump, i.e. with multiple possible regular expression entries like --include-extension= and --exclude-extension= where the includes get evaluated before the excludes. OK, as long as --exclude-extension has an all option, or we have a --no-extensions option also. While we're at it, both cases should be straight-forward to do. Cheers, D -- David Fetter [EMAIL PROTECTED] http://fetter.org/ phone: +1 415 235 3778AIM: dfetter666 Skype: davidfetter Remember to vote! ---(end of broadcast)--- TIP 2: Don't 'kill -9' the postmaster
Re: [HACKERS] Modifying and solidifying contrib
Bruce Momjian wrote: Joshua D. Drake wrote: This seems like a good first step in growing a packaging infrastructure. I'd rather grow it organically than try to design it all up front. I am in Denver and have spotty inet access so forgive me. So where does this above leave us? What are we doing? I was kind of unclear on that too. It seems we are trying to address several issues: visibility of contrib, installation of contrib, etc. We discussed whether we put the functions in public, a schema for all contrib, or a schema for each contrib module, and then there was the discussion of how to configure someone using ten /contrib modules, or at least wanting them all to be accessible. And then there was the idea of allowing schema permissions to control access, so perhaps we could install more of /contrib by default, and allow the administrator to just enable/disable them via permissions. Personally, I think that might be the best approach because it allows us to eliminate the install process, but doesn't make the database less secure --- the administrator enables/disables them at runtime, or at least could. The issues I see are: 1. the 'thing name - the only name I have not seen some objection to is extension. 2. namespace - I think the consensus is tending towards one or more per extension. 3. install/uninstall support: Tom's proposal for an extension-schema map in the catalog will deal with that nicely, I think. 4. visibility/searchpath issues. I don't think long search paths are a huge issue, but I think we can make life a bit easier by tweaking searchpath support a bit (David's clever SQL notwithstanding). 5. legacy support - we need an option to load existing extensions to the public schema as now, or support for aliases/synonyms (the latter might be good to have regardless). 6. they all need proper docs. READMEs and the like are nowhere near good enough. Richard mentioned special testing requirements, but I don't see why we can't continue to use our standard regression mechanism. Mention has also been made of autoloading extensions with initdb. A case could perhaps be made for doing it in createdb - maybe not every db needs ltree, say. OTOH, if it's sitting quietly in its own schema than it's probably not doing any harm either, so maybe initdb should just load all the extensions it finds, and as you say make one less hoop to make people jump through. If we do that I think at least we'd need an option to inhibit autoloading. cheers andrew ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] Modifying and solidifying contrib
On 1/30/07, Andrew Dunstan [EMAIL PROTECTED] wrote: [...] 4. visibility/searchpath issues. I don't think long search paths are a huge issue, but I think we can make life a bit easier by tweaking searchpath support a bit (David's clever SQL notwithstanding). As for search_path -- is it really needed to change it? I think it'd be better to leave default search_path even if we have many extensions each sitting in its own schema. If DBA/DBD wants, he can change it himself. The reasons to follow this way are: 1. two or more extensions might have functions with the same name (actually, that's what schemes/namespaces serve for) = we do not know which function should have higher priority (what order for schemas to choose?); 2. originally, when I've proposed to use separate schema name for each contrib module I've forgotten to mention another cause to do it -- this helps in development because everyone always knows what function is used (the code becomes a little bit larger, but understanding and code readability are improved) = so, it's better to not tweak search_path, it's better to encourage DBD to use full function names (if he wants to avoid using schema names, he can set search_path himself, resolving possible names priority issues mentioned above). Finally, AFAIK other DBMSs use the similar approaches (provide additional extensions/packages/extensions/... using separate namespaces and do not try to avoid writing namespace in function calls). -- Best regards, Nikolay
Re: [HACKERS] Modifying and solidifying contrib
On Tue, Jan 30, 2007 at 03:49:14PM -0500, Andrew Dunstan wrote: Bruce Momjian wrote: Joshua D. Drake wrote: This seems like a good first step in growing a packaging infrastructure. I'd rather grow it organically than try to design it all up front. I am in Denver and have spotty inet access so forgive me. So where does this above leave us? What are we doing? I was kind of unclear on that too. It seems we are trying to address several issues: visibility of contrib, installation of contrib, etc. We discussed whether we put the functions in public, a schema for all contrib, or a schema for each contrib module, and then there was the discussion of how to configure someone using ten /contrib modules, or at least wanting them all to be accessible. And then there was the idea of allowing schema permissions to control access, so perhaps we could install more of /contrib by default, and allow the administrator to just enable/disable them via permissions. Personally, I think that might be the best approach because it allows us to eliminate the install process, but doesn't make the database less secure --- the administrator enables/disables them at runtime, or at least could. The issues I see are: 1. the 'thing name - the only name I have not seen some objection to is extension. +1 for name extension. 2. namespace - I think the consensus is tending towards one or more per extension. +1 here too. I understand that this may result in schemas that have few functions in them. That's OK :) 3. install/uninstall support: Tom's proposal for an extension-schema map in the catalog will deal with that nicely, I think. +1 :) 4. visibility/searchpath issues. I don't think long search paths are a huge issue, but I think we can make life a bit easier by tweaking searchpath support a bit (David's clever SQL notwithstanding). The only clever bit I added was the CASE statement. Credit for the rest belongs to Andrew at Supernews. It's not a bad thing for people to keep around, either way. :) 5. legacy support - we need an option to load existing extensions to the public schema as now, or support for aliases/synonyms (the latter might be good to have regardless). Hrm. This gets tricky. When things are mandated to be in their own namespace, they need not check what everybody else's things are doing each time, whereas when they go into the public schema... :P 6. they all need proper docs. READMEs and the like are nowhere near good enough. Agreed. I'm thinking a new major section in the SGML docs is in order with a subsection for each contrib/ piece underneath. Richard mentioned special testing requirements, but I don't see why we can't continue to use our standard regression mechanism. A subdirectory in src/tests/regression for each one? Mention has also been made of autoloading extensions with initdb. A case could perhaps be made for doing it in createdb - maybe not every db needs ltree, say. OTOH, if it's sitting quietly in its own schema than it's probably not doing any harm either, so maybe initdb should just load all the extensions it finds, and as you say make one less hoop to make people jump through. If we do that I think at least we'd need an option to inhibit autoloading. I don't think it would be too much trouble to do extensions the way we now do tables and schemas in pg_dump, i.e. with multiple possible regular expression entries like --include-extension= and --exclude-extension= where the includes get evaluated before the excludes. Just my $.02 :) Cheers, D -- David Fetter [EMAIL PROTECTED] http://fetter.org/ phone: +1 415 235 3778AIM: dfetter666 Skype: davidfetter Remember to vote! ---(end of broadcast)--- TIP 9: In versions below 8.0, the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [HACKERS] Modifying and solidifying contrib
On Jan 31, 2007, at 12:42 , David Fetter wrote: On Tue, Jan 30, 2007 at 03:49:14PM -0500, Andrew Dunstan wrote: 6. they all need proper docs. READMEs and the like are nowhere near good enough. Agreed. I'm thinking a new major section in the SGML docs is in order with a subsection for each contrib/ piece underneath. I agree re: new section. Are you thinking that all contrib docs would be built automatically, even if the individual extensions (neé contrib modules?) aren't installed? I think that would definitely raise awareness of the extensions that are available. I'd also like to see being able to add docs for non-core extensions (e.g., ip4r) to the main documentation. Not sure what that would involve: rebuilding the tocs and index, besides the new pages themselves? Or perhaps just a rebuild of the complete docs? I haven't had docs building on a local system for a couple of years, so I'm not it a position currently to play around with this, but it's something I'd love to learn how to do. Michael Glaesemann grzm seespotcode net ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings
Re: [HACKERS] Modifying and solidifying contrib
David Fetter wrote: On Sun, Jan 28, 2007 at 02:14:36PM -0800, Joshua D. Drake wrote: I don't think all or nothing is a good way to do this. 500 functions in a schema called extensions isn't much more helpful than 500 in public. There's a reason namespaces were invented long ago, and this is classic use case for same. :) I disagree, see my post previously about initializing the extensions schema to not be accessible initially. It would be there, it would be loaded, but it would take a superuser to grant ability to access functions. This allows a clean distinction between the modules while allowing their access on a case by case basis. It's 982 functions as of this writing in CVS TIP's contrib. Do you not get how wacky it is to have that many functions, none of which have any collision-prevention built into their install scripts, in a flat namespace? We currently have 1695 standard functions. I don't see a problem with putting the extensions all in one schema, but I also don't see the point. -- Bruce Momjian [EMAIL PROTECTED] EnterpriseDBhttp://www.enterprisedb.com + If your life is a hard drive, Christ can be your backup. + ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings
Re: [HACKERS] Modifying and solidifying contrib
Bruce Momjian wrote: David Fetter wrote: It's 982 functions as of this writing in CVS TIP's contrib. Do you not get how wacky it is to have that many functions, none of which have any collision-prevention built into their install scripts, in a flat namespace? We currently have 1695 standard functions. I don't see a problem with putting the extensions all in one schema, but I also don't see the point. I certainly don't see the point. But I do see a considerable point in having extensions use their own schemas. The fact that we have 1695 standard functions means we bear the responsibility of ensuring we don't have name clashes among them. We should encourage extension authors by example to use the namespace facility to relieve themselves of having to ensure they don't clash not only with the standard functions but with other extensions. IOW we should act with respect to the namespace for extensions we distribute just like we would reasonably expect authors of third party extensions to behave. For backwards compatibility, we might be well advised also to distribute load scripts that put extension objects in the public schema as is done now, but this should be a deprecated practice, IMNSHO. cheers andrew ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
Re: [HACKERS] Modifying and solidifying contrib
Keep in mind all contrib loads into public, and I don't remember any namespace conflict issues in the past. --- Andrew Dunstan wrote: Bruce Momjian wrote: David Fetter wrote: It's 982 functions as of this writing in CVS TIP's contrib. Do you not get how wacky it is to have that many functions, none of which have any collision-prevention built into their install scripts, in a flat namespace? We currently have 1695 standard functions. I don't see a problem with putting the extensions all in one schema, but I also don't see the point. I certainly don't see the point. But I do see a considerable point in having extensions use their own schemas. The fact that we have 1695 standard functions means we bear the responsibility of ensuring we don't have name clashes among them. We should encourage extension authors by example to use the namespace facility to relieve themselves of having to ensure they don't clash not only with the standard functions but with other extensions. IOW we should act with respect to the namespace for extensions we distribute just like we would reasonably expect authors of third party extensions to behave. For backwards compatibility, we might be well advised also to distribute load scripts that put extension objects in the public schema as is done now, but this should be a deprecated practice, IMNSHO. cheers andrew ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq -- Bruce Momjian [EMAIL PROTECTED] EnterpriseDBhttp://www.enterprisedb.com + If your life is a hard drive, Christ can be your backup. + ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
Re: [HACKERS] Modifying and solidifying contrib
Bruce Momjian [EMAIL PROTECTED] writes: Keep in mind all contrib loads into public, and I don't remember any namespace conflict issues in the past. (A) I'm not sure we would have heard about it, and (B) any one user is probably only using a subset of what has been proposed to be loaded by default, so the odds of collisions would go way up. regards, tom lane ---(end of broadcast)--- TIP 9: In versions below 8.0, the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [HACKERS] Modifying and solidifying contrib
Bruce Momjian wrote: Keep in mind all contrib loads into public, and I don't remember any namespace conflict issues in the past. That is beside the point. Of course there haven't been conflicts - precisely because a single group controls the whole lot. What I said was that we should behave as sane third party extension authors would, namely to use their own namespace to protect themselves from conflicts with other unknown extensions. It's called setting a good example or eating your own dog food. cheers andrew ---(end of broadcast)--- TIP 2: Don't 'kill -9' the postmaster
Re: [HACKERS] Modifying and solidifying contrib
Andrew Dunstan wrote: Bruce Momjian wrote: Keep in mind all contrib loads into public, and I don't remember any namespace conflict issues in the past. That is beside the point. Of course there haven't been conflicts - precisely because a single group controls the whole lot. What I said was that we should behave as sane third party extension authors would, namely to use their own namespace to protect themselves from conflicts with other unknown extensions. It's called setting a good example or eating your own dog food. The problem I see controlling per-user search_path if +10 extensions are instlalled, and you want them always to be available by default. Also, are we going to use per-schema permissions so that the schemas are not visible by default? That _might_ allow us to install more by default. -- Bruce Momjian [EMAIL PROTECTED] EnterpriseDBhttp://www.enterprisedb.com + If your life is a hard drive, Christ can be your backup. + ---(end of broadcast)--- TIP 2: Don't 'kill -9' the postmaster
Re: [HACKERS] Modifying and solidifying contrib
Bruce Momjian wrote: Andrew Dunstan wrote: Bruce Momjian wrote: Keep in mind all contrib loads into public, and I don't remember any namespace conflict issues in the past. That is beside the point. Of course there haven't been conflicts - precisely because a single group controls the whole lot. What I said was that we should behave as sane third party extension authors would, namely to use their own namespace to protect themselves from conflicts with other unknown extensions. It's called setting a good example or eating your own dog food. The problem I see controlling per-user search_path if +10 extensions are instlalled, and you want them always to be available by default. This suggests maybe we need to look at beefing up a few things. For example, an alias facility that provided a name in schema X for things in schema Y would help lots here. (You want everything visible? Just alias it in public.) ISTR such things in DB2, although it's now many years since I laid hands on it, so my memory could well be very faulty. Also, ability to append to the search path rather than just set it could help, as might ability to add names of non-existent schemas (which would be ignored at run time when found not to exist). cheers andrew ---(end of broadcast)--- TIP 6: explain analyze is your friend
Re: [HACKERS] Modifying and solidifying contrib
Andrew Dunstan wrote: Bruce Momjian wrote: Andrew Dunstan wrote: Bruce Momjian wrote: Keep in mind all contrib loads into public, and I don't remember any namespace conflict issues in the past. That is beside the point. Of course there haven't been conflicts - precisely because a single group controls the whole lot. What I said was that we should behave as sane third party extension authors would, namely to use their own namespace to protect themselves from conflicts with other unknown extensions. It's called setting a good example or eating your own dog food. The problem I see controlling per-user search_path if +10 extensions are instlalled, and you want them always to be available by default. This suggests maybe we need to look at beefing up a few things. For example, an alias facility that provided a name in schema X for things in schema Y would help lots here. (You want everything visible? Just alias it in public.) ISTR such things in DB2, although it's now many years since I laid hands on it, so my memory could well be very faulty. I think that is SYNONYM. Also, ability to append to the search path rather than just set it could help, as might ability to add names of non-existent schemas (which would be ignored at run time when found not to exist). Agreed. -- Bruce Momjian [EMAIL PROTECTED] EnterpriseDBhttp://www.enterprisedb.com + If your life is a hard drive, Christ can be your backup. + ---(end of broadcast)--- TIP 1: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] Modifying and solidifying contrib
On Mon, Jan 29, 2007 at 04:44:44PM -0500, Andrew Dunstan wrote: Bruce Momjian wrote: Andrew Dunstan wrote: Bruce Momjian wrote: Keep in mind all contrib loads into public, and I don't remember any namespace conflict issues in the past. That is beside the point. Of course there haven't been conflicts - precisely because a single group controls the whole lot. What I said was that we should behave as sane third party extension authors would, namely to use their own namespace to protect themselves from conflicts with other unknown extensions. It's called setting a good example or eating your own dog food. The problem I see controlling per-user search_path if +10 extensions are instlalled, and you want them always to be available by default. This suggests maybe we need to look at beefing up a few things. For example, an alias facility that provided a name in schema X for things in schema Y would help lots here. (You want everything visible? Just alias it in public.) ISTR such things in DB2, although it's now many years since I laid hands on it, so my memory could well be very faulty. Also, ability to append to the search path rather than just set it could help, as might ability to add names of non-existent schemas (which would be ignored at run time when found not to exist). You can already do this via the following (baroque, but idempotent) method: UPDATE pg_catalog.pg_settings SET setting = CASE WHEN 'foo' = ANY(string_to_array(setting, ',')) THEN setting ELSE setting || ',foo' END WHERE name = 'search_path' ; Cheers, D -- David Fetter [EMAIL PROTECTED] http://fetter.org/ phone: +1 415 235 3778AIM: dfetter666 Skype: davidfetter Remember to vote! ---(end of broadcast)--- TIP 9: In versions below 8.0, the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [HACKERS] Modifying and solidifying contrib
Joshua D. Drake wrote: Tom Lane wrote: Joshua D. Drake [EMAIL PROTECTED] writes: So what are we thinking here? Along with my suggestion of extensions / contrib that we modify initdb to load an extensions schema with all extensions into template1? No, I don't think so. If you do that it's effectively moving all that stuff into core, especially if you haven't provided a way to turn it off. O.k. any thoughts there? What if we didn't make the extensions schema PUBLIC? Meaning that explicits rights would have to be given for the extensions to be used by anyone but a super user? Obviously the initdb switch could also be selective: I was thinking last week about what I wanted from a packaging system... 1. Add a new column for all system objects, separate from schema: package. So you would have core, tsearch, ltree, etc. At the very least this lets you manually uninstall a package by searching for and dropping by package-name. Most of what is currently considered PostgreSQL would be core package I suppose, but you could split out various of the types (numeric, text, internet etc.) I suppose). Each project in contrib/ would be a package, as would each procedural language. 2. All packages have the following attributes: a. Name, description, version number, supplier name/website b. Installation script, removal script (as functions - obviously they'll need some bootstrapping)/ We might want hooks for various upgrade functions too. c. Test function(s) with predictable names (perhaps just test001() test002()). These might be dropp-able for a production deployment. d. A list of dependencies on other packages/versions. This implies some pg_package_is_installed() function perhaps. e. A default search_path to override any user search_path settings (so your package always calls the desired version of foo()). f. Permissions at a package-level, so that a package can be loaded but not installed, or installed but not usable by non-superusers. 3. All packages that ship with the standard PG distribution are: a. built by default b. installed to a suitable extensions/packages directory. c. have install/remove functions d. have some semi-standard naming (pgx_...) e. have test functions f. are documented in the main manuals From my limited knowledge of the code, none of this should require major surgery except perhaps permissions at the package level and the bootstrapping for installation. Bootstrapping could consist of nothing more than a set of SQL scripts which set up some temporary tables and create and call the _install() function. Any of this in the direction that other people were thinking of? -- Richard Huxton Archonet Ltd ---(end of broadcast)--- TIP 2: Don't 'kill -9' the postmaster
Re: [HACKERS] Modifying and solidifying contrib
Richard Huxton dev@archonet.com writes: 1. Add a new column for all system objects, separate from schema: package. Wouldn't it be a whole lot easier just to drive it off schema, rather than inventing duplicative parallel infrastructure? That is, say that a package has one or more schemas and what it owns is whatever is in those schemas. This lets you, for example, use schema permissions to manage access to the package. regards, tom lane ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
Re: [HACKERS] Modifying and solidifying contrib
Tom Lane wrote: Richard Huxton dev@archonet.com writes: 1. Add a new column for all system objects, separate from schema: package. Wouldn't it be a whole lot easier just to drive it off schema, rather than inventing duplicative parallel infrastructure? That is, say that a package has one or more schemas and what it owns is whatever is in those schemas. This lets you, for example, use schema permissions to manage access to the package. Four differences: 1. You couldn't have a tsearch package with functions in public. At least not without some IMPORT TSEARCH.foo() INTO public 2. You can't easily disable access to a whole package if it's spread over multiple schemas. 3. You can't determine what package various objects belong to - is this stopwords table from tsearch2 or ArchonetSearch17? 4. You can't have one package depending upon another (webstats v2.1 depends on internet_addr v2.3). With the current search_path functionality I think it's important the package names are separate. I think I'm right in saying there are two packaging schemes out there - you either have a single hierarchy (Perl/Java) or something parallel (at 90 degrees to?) to an existing setup (RPM/deb). I think our search_path means we're dealing with something more like a Linux packaging setup. -- Richard Huxton Archonet Ltd ---(end of broadcast)--- TIP 6: explain analyze is your friend
Re: [HACKERS] Modifying and solidifying contrib
Richard Huxton dev@archonet.com writes: Tom Lane wrote: Wouldn't it be a whole lot easier just to drive it off schema, rather than inventing duplicative parallel infrastructure? Four differences: 1. You couldn't have a tsearch package with functions in public. At least not without some IMPORT TSEARCH.foo() INTO public So? That's what a search path is for. 2. You can't easily disable access to a whole package if it's spread over multiple schemas. The main reason I can see for separating a package into more than one schema is exactly that some parts would be public and others private. So schema-level access controls are what you want, *not* package-level. 3. You can't determine what package various objects belong to - is this stopwords table from tsearch2 or ArchonetSearch17? Sure you can; you look to see what schema it's in. 4. You can't have one package depending upon another (webstats v2.1 depends on internet_addr v2.3). What's that have to do with it? Perhaps I should be clearer: I agree with having an explicit representation of a package in some new system catalog for that purpose. That does not translate to needing to add package hooks to every other catalog. Indirect links through schemas seem more than sufficient. regards, tom lane ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] Modifying and solidifying contrib
Tom Lane wrote: I agree with having an explicit representation of a package in some new system catalog for that purpose. That does not translate to needing to add package hooks to every other catalog. Indirect links through schemas seem more than sufficient. This seems like a good first step in growing a packaging infrastructure. I'd rather grow it organically than try to design it all up front. cheers andrew ---(end of broadcast)--- TIP 2: Don't 'kill -9' the postmaster
Re: [HACKERS] Modifying and solidifying contrib
This seems like a good first step in growing a packaging infrastructure. I'd rather grow it organically than try to design it all up front. I am in Denver and have spotty inet access so forgive me. So where does this above leave us? What are we doing? Joshua D. Drake cheers andrew ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings
Re: [HACKERS] Modifying and solidifying contrib
Joshua D. Drake wrote: This seems like a good first step in growing a packaging infrastructure. I'd rather grow it organically than try to design it all up front. I am in Denver and have spotty inet access so forgive me. So where does this above leave us? What are we doing? I was kind of unclear on that too. It seems we are trying to address several issues: visibility of contrib, installation of contrib, etc. We discussed whether we put the functions in public, a schema for all contrib, or a schema for each contrib module, and then there was the discussion of how to configure someone using ten /contrib modules, or at least wanting them all to be accessible. And then there was the idea of allowing schema permissions to control access, so perhaps we could install more of /contrib by default, and allow the administrator to just enable/disable them via permissions. Personally, I think that might be the best approach because it allows us to eliminate the install process, but doesn't make the database less secure --- the administrator enables/disables them at runtime, or at least could. -- Bruce Momjian [EMAIL PROTECTED] EnterpriseDBhttp://www.enterprisedb.com + If your life is a hard drive, Christ can be your backup. + ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings
Re: [HACKERS] Modifying and solidifying contrib
Kevin Barnard wrote: The basic idea of my proposal would be to have a function that checks for the existence of C libraries. The next piece I would need would be a mechanism for psql to halt the restore SQL script if these checks failed. Finally I would need to add a switch to tell pg_dump to do the checks first thereby causing the restore to fail, hopefully with a nice message that clues the admin that something is wrong. You already get this today. If you create a C function and the library doesn't exist, you get an error, and if you use the right flags for psql or pg_restore, the restoration aborts. -- Peter Eisentraut http://developer.postgresql.org/~petere/ ---(end of broadcast)--- TIP 2: Don't 'kill -9' the postmaster
Re: [HACKERS] Modifying and solidifying contrib
David Fetter wrote: On Sat, Jan 27, 2007 at 09:49:25PM -0800, Joshua D. Drake wrote: Tom Lane wrote: Joshua D. Drake [EMAIL PROTECTED] writes: So what are we thinking here? Along with my suggestion of extensions / contrib that we modify initdb to load an extensions schema with all extensions into template1? No, I don't think so. If you do that it's effectively moving all that stuff into core, especially if you haven't provided a way to turn it off. O.k. any thoughts there? What if we didn't make the extensions schema PUBLIC? Meaning that explicits rights would have to be given for the extensions to be used by anyone but a super user? Whether they're auto-installable or not, I'd vote for putting each one in its own schema by default. That way, people can get an excellent idea just by looking at what schemas exist what extensions are installed in a given DB, and it's fairly straight-forward to remove the thing simply by dropping the schema cascade. Well to me that gets a little messy. I mean: pg_catalog,public,user schemas,xml2,ltree (just to get a could functions?) etc... Obviously the initdb switch could also be selective: initdb --enable-extensions If it were an initdb switch, I'd want to have something more like --enable-extension=earthdistance And have to parse for each extension? Sincerely, Joshua D. Drake Cheers, D -- === The PostgreSQL Company: Command Prompt, Inc. === Sales/Support: +1.503.667.4564 || 24x7/Emergency: +1.800.492.2240 Providing the most comprehensive PostgreSQL solutions since 1997 http://www.commandprompt.com/ Donate to the PostgreSQL Project: http://www.postgresql.org/about/donate PostgreSQL Replication: http://www.commandprompt.com/products/ ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
Re: [HACKERS] Modifying and solidifying contrib
On Sun, Jan 28, 2007 at 10:10:14AM -0800, Joshua D. Drake wrote: David Fetter wrote: On Sat, Jan 27, 2007 at 09:49:25PM -0800, Joshua D. Drake wrote: Tom Lane wrote: Joshua D. Drake [EMAIL PROTECTED] writes: So what are we thinking here? Along with my suggestion of extensions / contrib that we modify initdb to load an extensions schema with all extensions into template1? No, I don't think so. If you do that it's effectively moving all that stuff into core, especially if you haven't provided a way to turn it off. O.k. any thoughts there? What if we didn't make the extensions schema PUBLIC? Meaning that explicits rights would have to be given for the extensions to be used by anyone but a super user? Whether they're auto-installable or not, I'd vote for putting each one in its own schema by default. That way, people can get an excellent idea just by looking at what schemas exist what extensions are installed in a given DB, and it's fairly straight-forward to remove the thing simply by dropping the schema cascade. Well to me that gets a little messy. I mean: pg_catalog,public,user schemas,xml2,ltree (just to get a could functions?) etc... Not as messy as trying to drop or re-create a package when there are already 500 functions in the public schema. Obviously the initdb switch could also be selective: initdb --enable-extensions If it were an initdb switch, I'd want to have something more like --enable-extension=earthdistance And have to parse for each extension? I don't see this as a big problem. Cheers, D -- David Fetter [EMAIL PROTECTED] http://fetter.org/ phone: +1 415 235 3778AIM: dfetter666 Skype: davidfetter Remember to vote! ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
Re: [HACKERS] Modifying and solidifying contrib
Michael Glaesemann wrote: On Jan 28, 2007, at 11:25 , Joshua D. Drake wrote: David Fetter wrote: Not so great. SQL:2003 has a special meaning for the word module. Yeah I saw mention of that in another thread, but I really didn't like the word plugins. Do you have another thought? Extensions? Extensions would tie in nicely with its common use in the docs, especially wrt pgxs: I don't mind this term, BUT, what we need to get across is not just that these are extensions, but that they are *standard* extensions, supplied with PostgreSQL core code and supported by the PostgreSQL core team. This would be analogous with, say, the standard perl modules (like Exporter or IO::Handle) that come with the standard perl source distribution. If we can get that idea across then we might lower the resistance of people like hosting providers to loading them. cheers andrew ---(end of broadcast)--- TIP 7: You can help support the PostgreSQL project by donating at http://www.postgresql.org/about/donate
Re: [HACKERS] Modifying and solidifying contrib
On Sun, Jan 28, 2007 at 04:52:27PM -0500, Andrew Dunstan wrote: Michael Glaesemann wrote: On Jan 28, 2007, at 11:25 , Joshua D. Drake wrote: David Fetter wrote: Not so great. SQL:2003 has a special meaning for the word module. Yeah I saw mention of that in another thread, but I really didn't like the word plugins. Do you have another thought? Extensions? Extensions would tie in nicely with its common use in the docs, especially wrt pgxs: I don't mind this term, BUT, what we need to get across is not just that these are extensions, but that they are *standard* extensions, supplied with PostgreSQL core code and supported by the PostgreSQL core team. This would be analogous with, say, the standard perl modules (like Exporter or IO::Handle) that come with the standard perl source distribution. If we can get that idea across then we might lower the resistance of people like hosting providers to loading them. Integrating their docs into the standard PostgreSQL SGML (or XML, should we go there) docs would go a long, long way toward helping with this. Cheers, D -- David Fetter [EMAIL PROTECTED] http://fetter.org/ phone: +1 415 235 3778AIM: dfetter666 Skype: davidfetter Remember to vote! ---(end of broadcast)--- TIP 2: Don't 'kill -9' the postmaster
Re: [HACKERS] Modifying and solidifying contrib
Well to me that gets a little messy. I mean: pg_catalog,public,user schemas,xml2,ltree (just to get a could functions?) etc... Not as messy as trying to drop or re-create a package when there are already 500 functions in the public schema. I am not sure I understand the correlation. I am not suggesting we install anything into public. They would all go into a single additional schema called extensions or some such. Obviously the initdb switch could also be selective: initdb --enable-extensions If it were an initdb switch, I'd want to have something more like --enable-extension=earthdistance And have to parse for each extension? I don't see this as a big problem. Well I am not really interesting in this. Someone else is welcome to try that. Sincerely, Joshua D. Drake -- === The PostgreSQL Company: Command Prompt, Inc. === Sales/Support: +1.503.667.4564 || 24x7/Emergency: +1.800.492.2240 Providing the most comprehensive PostgreSQL solutions since 1997 http://www.commandprompt.com/ Donate to the PostgreSQL Project: http://www.postgresql.org/about/donate PostgreSQL Replication: http://www.commandprompt.com/products/ ---(end of broadcast)--- TIP 1: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] Modifying and solidifying contrib
I don't mind this term, BUT, what we need to get across is not just that these are extensions, but that they are *standard* extensions, supplied with PostgreSQL core code and supported by the PostgreSQL core team. This would be analogous with, say, the standard perl modules (like Exporter or IO::Handle) that come with the standard perl source distribution. If we can get that idea across then we might lower the resistance of people like hosting providers to loading them. Furthering the perception is reality rule perhaps the schema should be called std_ext? Integrating their docs into the standard PostgreSQL SGML (or XML, should we go there) docs would go a long, long way toward helping with this. Agreed. Sincerely, Joshua D. Drake -- === The PostgreSQL Company: Command Prompt, Inc. === Sales/Support: +1.503.667.4564 || 24x7/Emergency: +1.800.492.2240 Providing the most comprehensive PostgreSQL solutions since 1997 http://www.commandprompt.com/ Donate to the PostgreSQL Project: http://www.postgresql.org/about/donate PostgreSQL Replication: http://www.commandprompt.com/products/ ---(end of broadcast)--- TIP 7: You can help support the PostgreSQL project by donating at http://www.postgresql.org/about/donate
Re: [HACKERS] Modifying and solidifying contrib
On Sun, Jan 28, 2007 at 01:58:38PM -0800, Joshua D. Drake wrote: Well to me that gets a little messy. I mean: pg_catalog,public,user schemas,xml2,ltree (just to get a could functions?) etc... Not as messy as trying to drop or re-create a package when there are already 500 functions in the public schema. I am not sure I understand the correlation. I am not suggesting we install anything into public. They would all go into a single additional schema called extensions or some such. I don't think all or nothing is a good way to do this. 500 functions in a schema called extensions isn't much more helpful than 500 in public. There's a reason namespaces were invented long ago, and this is classic use case for same. :) Obviously the initdb switch could also be selective: initdb --enable-extensions If it were an initdb switch, I'd want to have something more like --enable-extension=earthdistance And have to parse for each extension? I don't see this as a big problem. Well I am not really interesting in this. Someone else is welcome to try that. It's really not hard, even for a C n00b like me. :) The only trick here, and again it's not a huge one, is to modify the postgresql.conf to have the correct default search_path. Cheers, D -- David Fetter [EMAIL PROTECTED] http://fetter.org/ phone: +1 415 235 3778AIM: dfetter666 Skype: davidfetter Remember to vote! ---(end of broadcast)--- TIP 2: Don't 'kill -9' the postmaster
Re: [HACKERS] Modifying and solidifying contrib
I don't think all or nothing is a good way to do this. 500 functions in a schema called extensions isn't much more helpful than 500 in public. There's a reason namespaces were invented long ago, and this is classic use case for same. :) I disagree, see my post previously about initializing the extensions schema to not be accessible initially. It would be there, it would be loaded, but it would take a superuser to grant ability to access functions. This allows a clean distinction between the modules while allowing their access on a case by case basis. --enable-extension=earthdistance And have to parse for each extension? I don't see this as a big problem. Well I am not really interesting in this. Someone else is welcome to try that. It's really not hard, even for a C n00b like me. :) I didn't say it was hard. I said I wasn't interested :) Sincerely, Joshua D. Drake -- === The PostgreSQL Company: Command Prompt, Inc. === Sales/Support: +1.503.667.4564 || 24x7/Emergency: +1.800.492.2240 Providing the most comprehensive PostgreSQL solutions since 1997 http://www.commandprompt.com/ Donate to the PostgreSQL Project: http://www.postgresql.org/about/donate PostgreSQL Replication: http://www.commandprompt.com/products/ ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] Modifying and solidifying contrib
On Sun, Jan 28, 2007 at 02:14:36PM -0800, Joshua D. Drake wrote: I don't think all or nothing is a good way to do this. 500 functions in a schema called extensions isn't much more helpful than 500 in public. There's a reason namespaces were invented long ago, and this is classic use case for same. :) I disagree, see my post previously about initializing the extensions schema to not be accessible initially. It would be there, it would be loaded, but it would take a superuser to grant ability to access functions. This allows a clean distinction between the modules while allowing their access on a case by case basis. It's 982 functions as of this writing in CVS TIP's contrib. Do you not get how wacky it is to have that many functions, none of which have any collision-prevention built into their install scripts, in a flat namespace? Then again, you started the PL/PHP project, so maybe I shouldn't ask ;) --enable-extension=earthdistance And have to parse for each extension? I don't see this as a big problem. Well I am not really interesting in this. Someone else is welcome to try that. It's really not hard, even for a C n00b like me. :) I didn't say it was hard. I said I wasn't interested :) I think it's necessary to get each in its own schema whether we have an initdb flag or not. Cheers, D -- David Fetter [EMAIL PROTECTED] http://fetter.org/ phone: +1 415 235 3778AIM: dfetter666 Skype: davidfetter Remember to vote! ---(end of broadcast)--- TIP 9: In versions below 8.0, the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [HACKERS] Modifying and solidifying contrib
David Fetter wrote: I think it's necessary to get each in its own schema whether we have an initdb flag or not. In any case, the initdb flag idea is about as much a nonstarter as the configure flag idea, for the same (packaging) reasons. -- Peter Eisentraut http://developer.postgresql.org/~petere/ ---(end of broadcast)--- TIP 2: Don't 'kill -9' the postmaster
Re: [HACKERS] Modifying and solidifying contrib
--enable-extension=earthdistance And have to parse for each extension? I don't see this as a big problem. Well I am not really interesting in this. Someone else is welcome to try that. It's really not hard, even for a C n00b like me. :) I didn't say it was hard. I said I wasn't interested :) I think it's necessary to get each in its own schema whether we have an initdb flag or not. I can see schemas for certain ones. Tsearch (just as an example as it appears to be going into core) but ltree? Sincerely, Joshua D. Drake Cheers, D -- === The PostgreSQL Company: Command Prompt, Inc. === Sales/Support: +1.503.667.4564 || 24x7/Emergency: +1.800.492.2240 Providing the most comprehensive PostgreSQL solutions since 1997 http://www.commandprompt.com/ Donate to the PostgreSQL Project: http://www.postgresql.org/about/donate PostgreSQL Replication: http://www.commandprompt.com/products/ ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] Modifying and solidifying contrib
Peter Eisentraut wrote: David Fetter wrote: I think it's necessary to get each in its own schema whether we have an initdb flag or not. In any case, the initdb flag idea is about as much a nonstarter as the configure flag idea, for the same (packaging) reasons. I don't see your point but it makes my life simpler. So do you opinions on the schema idea? Sincerely, Joshua D. Drake -- === The PostgreSQL Company: Command Prompt, Inc. === Sales/Support: +1.503.667.4564 || 24x7/Emergency: +1.800.492.2240 Providing the most comprehensive PostgreSQL solutions since 1997 http://www.commandprompt.com/ Donate to the PostgreSQL Project: http://www.postgresql.org/about/donate PostgreSQL Replication: http://www.commandprompt.com/products/ ---(end of broadcast)--- TIP 7: You can help support the PostgreSQL project by donating at http://www.postgresql.org/about/donate
[HACKERS] Modifying and solidifying contrib
Hello, With all the recent discussion on contrib modules etc.. I would like to offer the following suggestion. I am willing to do a good portion of the work myself and I can get it done before feature freeze. I will need help with the global make file stuff however so that is one dependency. Add directory /modules Modules are compiled and installed by default but not enabled. Modules in 8.3 currently are: pgrowlocks pg_freespacemap (to be renameed pgfreespacemap to be consistent) pgstattuple pgcrypto xml2 pgbuffercache initagg Requirements for /modules Must go through normal vetting process on -hackers Must include patches to core documentation in Docbook Must include test cases? I don't recall if we have regress for all contrib stuff. Keep directory contrib Contrib is not compiled or installed by default Contrib in 8.3 would be: start-scripts pgbench (which I think really should be a foundry project) vacuumlo (is this even required anymore?) adminpack btree_gist etc... Requirements for /contrib Must go through normal vetting process on -hackers Must includes README Must include test cases? Same questions for modules Thoughts, flames? Sincerely, Joshua D. Drake -- === The PostgreSQL Company: Command Prompt, Inc. === Sales/Support: +1.503.667.4564 || 24x7/Emergency: +1.800.492.2240 Providing the most comprehensive PostgreSQL solutions since 1997 http://www.commandprompt.com/ Donate to the PostgreSQL Project: http://www.postgresql.org/about/donate PostgreSQL Replication: http://www.commandprompt.com/products/ ---(end of broadcast)--- TIP 7: You can help support the PostgreSQL project by donating at http://www.postgresql.org/about/donate
Re: [HACKERS] Modifying and solidifying contrib
Joshua D. Drake [EMAIL PROTECTED] writes: With all the recent discussion on contrib modules etc.. I would like to offer the following suggestion. AFAICT you're proposing an entirely cosmetic reclassification of /contrib. Aside from the difficulty of getting agreement on which ones should be in and which out, what does that really buy us? The thing that would be really useful to work on is developing a concrete representation of a module that pg_dump would understand, so that you could e.g. tell it to omit btree_gist from a dump. It might be that just segregating a contrib module into its own schema would be sufficient, or maybe that wouldn't work well because of making people need to deal with long search paths. regards, tom lane ---(end of broadcast)--- TIP 7: You can help support the PostgreSQL project by donating at http://www.postgresql.org/about/donate
Re: [HACKERS] Modifying and solidifying contrib
Tom Lane wrote: Joshua D. Drake [EMAIL PROTECTED] writes: With all the recent discussion on contrib modules etc.. I would like to offer the following suggestion. AFAICT you're proposing an entirely cosmetic reclassification of /contrib. For the most part yes. Perception is reality and all. The exception being that modules are installed by default. Aside from the difficulty of getting agreement on which ones should be in and which out, what does that really buy us? True and I am sure that people with more time to waste than I would like to spend days creating a thread that is 500 responses long on why their particular module should be a module or a contrib. The thing that would be really useful to work on is developing a concrete representation of a module that pg_dump would understand, so that you could e.g. tell it to omit btree_gist from a dump. I am offering what I can. It might be that just segregating a contrib module into its own schema would be sufficient, or maybe that wouldn't work well because of making people need to deal with long search paths. I do like the contrib schema idea would could easily be melded into this proposal. I don't like the idea that all of contrib would automatically be included which is one of the reasons I wanted to split this up. Sincerely, Joshua D. Drake regards, tom lane -- === The PostgreSQL Company: Command Prompt, Inc. === Sales/Support: +1.503.667.4564 || 24x7/Emergency: +1.800.492.2240 Providing the most comprehensive PostgreSQL solutions since 1997 http://www.commandprompt.com/ Donate to the PostgreSQL Project: http://www.postgresql.org/about/donate PostgreSQL Replication: http://www.commandprompt.com/products/ ---(end of broadcast)--- TIP 9: In versions below 8.0, the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [HACKERS] Modifying and solidifying contrib
On Sat, Jan 27, 2007 at 08:59:47AM -0800, Joshua D. Drake wrote: Hello, With all the recent discussion on contrib modules etc.. I would like to offer the following suggestion. I am willing to do a good portion of the work myself and I can get it done before feature freeze. I will need help with the global make file stuff however so that is one dependency. Add directory /modules Not so great. SQL:2003 has a special meaning for the word module. Cheers, D -- David Fetter [EMAIL PROTECTED] http://fetter.org/ phone: +1 415 235 3778AIM: dfetter666 Skype: davidfetter Remember to vote! ---(end of broadcast)--- TIP 6: explain analyze is your friend
Re: [HACKERS] Modifying and solidifying contrib
David Fetter wrote: On Sat, Jan 27, 2007 at 08:59:47AM -0800, Joshua D. Drake wrote: Hello, With all the recent discussion on contrib modules etc.. I would like to offer the following suggestion. I am willing to do a good portion of the work myself and I can get it done before feature freeze. I will need help with the global make file stuff however so that is one dependency. Add directory /modules Not so great. SQL:2003 has a special meaning for the word module. Yeah I saw mention of that in another thread, but I really didn't like the word plugins. Do you have another thought? Extensions? Sincerely, Joshua D. Drake Cheers, D -- === The PostgreSQL Company: Command Prompt, Inc. === Sales/Support: +1.503.667.4564 || 24x7/Emergency: +1.800.492.2240 Providing the most comprehensive PostgreSQL solutions since 1997 http://www.commandprompt.com/ Donate to the PostgreSQL Project: http://www.postgresql.org/about/donate PostgreSQL Replication: http://www.commandprompt.com/products/ ---(end of broadcast)--- TIP 1: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] Modifying and solidifying contrib
On Jan 28, 2007, at 11:25 , Joshua D. Drake wrote: David Fetter wrote: Not so great. SQL:2003 has a special meaning for the word module. Yeah I saw mention of that in another thread, but I really didn't like the word plugins. Do you have another thought? Extensions? Extensions would tie in nicely with its common use in the docs, especially wrt pgxs: PostgreSQL can be extended by the user in many ways ... PostgreSQL also accepts escape string constants, which are an extension to the SQL standard To use the infrastructure for your extension ... Here is an example that builds an extension module ... They test standard SQL operations as well as the extended capabilities of PostgreSQL. The pgxs docs do use module as well, but as previously mentioned module already has a particular meaning in the spec. Michael Glaesemann grzm seespotcode net ---(end of broadcast)--- TIP 1: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] Modifying and solidifying contrib
PostgreSQL can be extended by the user in many ways ... PostgreSQL also accepts escape string constants, which are an extension to the SQL standard To use the infrastructure for your extension ... Here is an example that builds an extension module ... They test standard SQL operations as well as the extended capabilities of PostgreSQL. The pgxs docs do use module as well, but as previously mentioned module already has a particular meaning in the spec. So what are we thinking here? Along with my suggestion of extensions / contrib that we modify initdb to load an extensions schema with all extensions into template1? Sincerely, Joshua D. Drake Michael Glaesemann grzm seespotcode net ---(end of broadcast)--- TIP 1: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly -- === The PostgreSQL Company: Command Prompt, Inc. === Sales/Support: +1.503.667.4564 || 24x7/Emergency: +1.800.492.2240 Providing the most comprehensive PostgreSQL solutions since 1997 http://www.commandprompt.com/ Donate to the PostgreSQL Project: http://www.postgresql.org/about/donate PostgreSQL Replication: http://www.commandprompt.com/products/ ---(end of broadcast)--- TIP 2: Don't 'kill -9' the postmaster
Re: [HACKERS] Modifying and solidifying contrib
Joshua D. Drake [EMAIL PROTECTED] writes: So what are we thinking here? Along with my suggestion of extensions / contrib that we modify initdb to load an extensions schema with all extensions into template1? No, I don't think so. If you do that it's effectively moving all that stuff into core, especially if you haven't provided a way to turn it off. I don't plan to hold still for any back-door avenues to suddenly turning large parts of contrib into core code. regards, tom lane ---(end of broadcast)--- TIP 7: You can help support the PostgreSQL project by donating at http://www.postgresql.org/about/donate
Re: [HACKERS] Modifying and solidifying contrib
Tom Lane wrote: Joshua D. Drake [EMAIL PROTECTED] writes: So what are we thinking here? Along with my suggestion of extensions / contrib that we modify initdb to load an extensions schema with all extensions into template1? No, I don't think so. If you do that it's effectively moving all that stuff into core, especially if you haven't provided a way to turn it off. O.k. any thoughts there? What if we didn't make the extensions schema PUBLIC? Meaning that explicits rights would have to be given for the extensions to be used by anyone but a super user? Obviously the initdb switch could also be selective: initdb --enable-extensions ? Sincerely, Joshua D. Drake -- === The PostgreSQL Company: Command Prompt, Inc. === Sales/Support: +1.503.667.4564 || 24x7/Emergency: +1.800.492.2240 Providing the most comprehensive PostgreSQL solutions since 1997 http://www.commandprompt.com/ Donate to the PostgreSQL Project: http://www.postgresql.org/about/donate PostgreSQL Replication: http://www.commandprompt.com/products/ ---(end of broadcast)--- TIP 2: Don't 'kill -9' the postmaster
Re: [HACKERS] Modifying and solidifying contrib
On Sat, Jan 27, 2007 at 09:49:25PM -0800, Joshua D. Drake wrote: Tom Lane wrote: Joshua D. Drake [EMAIL PROTECTED] writes: So what are we thinking here? Along with my suggestion of extensions / contrib that we modify initdb to load an extensions schema with all extensions into template1? No, I don't think so. If you do that it's effectively moving all that stuff into core, especially if you haven't provided a way to turn it off. O.k. any thoughts there? What if we didn't make the extensions schema PUBLIC? Meaning that explicits rights would have to be given for the extensions to be used by anyone but a super user? Whether they're auto-installable or not, I'd vote for putting each one in its own schema by default. That way, people can get an excellent idea just by looking at what schemas exist what extensions are installed in a given DB, and it's fairly straight-forward to remove the thing simply by dropping the schema cascade. Obviously the initdb switch could also be selective: initdb --enable-extensions If it were an initdb switch, I'd want to have something more like --enable-extension=earthdistance Cheers, D -- David Fetter [EMAIL PROTECTED] http://fetter.org/ phone: +1 415 235 3778AIM: dfetter666 Skype: davidfetter Remember to vote! ---(end of broadcast)--- TIP 2: Don't 'kill -9' the postmaster
Re: [HACKERS] Modifying and solidifying contrib
On 1/27/07, Joshua D. Drake [EMAIL PROTECTED] wrote: Tom Lane wrote: Joshua D. Drake [EMAIL PROTECTED] writes: So what are we thinking here? Along with my suggestion of extensions / contrib that we modify initdb to load an extensions schema with all extensions into template1? No, I don't think so. If you do that it's effectively moving all that stuff into core, especially if you haven't provided a way to turn it off. O.k. any thoughts there? What if we didn't make the extensions schema PUBLIC? Meaning that explicits rights would have to be given for the extensions to be used by anyone but a super user? Obviously the initdb switch could also be selective: initdb --enable-extensions ? Sincerely, Joshua D. Drake I've been listening in on this discussion and it is closely tied to an extension that I would like to try. The only thing holding me back is that I am still far to green on the source code to actually propose the work I would like to do. It makes sense the you would want to include the contrib into the install, but I agree with Tom this isn't something you would want in template1. plpgsql is not even in template1 IIRC. It makes more sense to compile the libraries and have an activation module that can be represented in a pg_dump. Maybe the best way to do this would be to place the SQL that modifies system catalog into a compiled library and call this as a function. The function call could then be placed into the pg_dump if needed. Here is where this ties into what I am working on proposing. I have several custom C functions in a library that I need in a restore/build. It seems like at least a couple times every year a new cluster is built and the Admin forgets to compile the custom functions. He builds the DB and then something doesn't work. The basic idea of my proposal would be to have a function that checks for the existence of C libraries. The next piece I would need would be a mechanism for psql to halt the restore SQL script if these checks failed. Finally I would need to add a switch to tell pg_dump to do the checks first thereby causing the restore to fail, hopefully with a nice message that clues the admin that something is wrong. Yes all of this work so I don't get called after hours yet here I am working on the fix after hours. :-) It seems to me like this might be useful in incorporating an extension/module/whatever into Postgres. -- Kevin Barnard Great Beauty, great strength, and great Riches, are really and truly of no great Use; a right Heart exceeds all. -- Benjamin Franklin
