Re: RFC: Additional Directory for Extensions

2024-08-29 Thread David E. Wheeler
On Aug 27, 2024, at 22:24, Craig Ringer wrote: > `pg_config` only cares about compile-time settings, so I would not > expect its output to change. Right, of course that’s its original purpose, but extensions depend on it to determine where to install extensions. Not just PGXS, but also pgrx and

Re: RFC: Additional Directory for Extensions

2024-08-27 Thread Craig Ringer
On Wed, 28 Aug 2024 at 03:26, David E. Wheeler wrote: > Right. ISTM it could complicate PGXS quite a bit. If we set, say, > > SET extension_search_path = $extsdir, /mnt/extensions/pg16, > /mnt/extensions/pg16/gosuperfast/extensions; > > What should be the output of `pg_config --sharedir`? `pg_co

Re: RFC: Additional Directory for Extensions

2024-08-27 Thread David E. Wheeler
On Aug 26, 2024, at 17:35, Craig Ringer wrote: > This looks like a good suggestion to me, it would make the packaging, > distribution and integration of 3rd party extensions significantly > easier without any obvious large or long term cost. Yes! > Also PGXS, the windows extension build support

Re: RFC: Additional Directory for Extensions

2024-08-27 Thread David E. Wheeler
On Aug 27, 2024, at 04:56, Gabriele Bartolini wrote: > The extension image could follow a naming convention like this (order can be > adjusted): `-- version>-(-)`. For example, `pgvector-16-0.7.4-bookworm-1` would > represent the first image built in a repository for pgvector 0.7.4 for > Post

Re: RFC: Additional Directory for Extensions

2024-08-27 Thread Gabriele Bartolini
Hi David, Thanks for your email. On Mon, 26 Aug 2024 at 16:07, David E. Wheeler wrote: > I would assume BINDIR, DOCDIR, HTMLDIR, PKGLIBDIR, MANDIR, SHAREDIR, and > perhaps LOCALEDIR. > > But even if it’s just one or two, the only proper way an extension > directory would work, IME, is to define

Re: RFC: Additional Directory for Extensions

2024-08-26 Thread Craig Ringer
On Tue, 27 Aug 2024 at 02:07, David E. Wheeler wrote: > On Aug 21, 2024, at 19:07, Craig Ringer wrote: > But even if it’s just one or two, the only proper way an extension directory > would work, IME, is to define a directory-based structure for extensions, > where every file for an extension

Re: RFC: Additional Directory for Extensions

2024-08-26 Thread David E. Wheeler
Hi Hackers, Apologies for the delay in reply; I’ve been at the XOXO Festival and almost completely unplugged for the first time in ages. Happy to see this thread coming alive, though. Thank you Gabriele, Craig, and Jelte! On Aug 21, 2024, at 19:07, Craig Ringer wrote: > So IMO this should be

Re: RFC: Additional Directory for Extensions

2024-08-22 Thread Craig Ringer
On Fri, 23 Aug 2024 at 10:14, Craig Ringer wrote: > On Thu, 22 Aug 2024 at 21:00, Gabriele Bartolini > wrote: > > On Thu, 22 Aug 2024 at 09:32, Jelte Fennema-Nio wrote: > >> SET extension_search_path = /mnt/extensions/pg16/* > > > > That'd be great. +1. > > Agreed, that'd be handy, but not worth

Re: RFC: Additional Directory for Extensions

2024-08-22 Thread Craig Ringer
On Thu, 22 Aug 2024 at 21:00, Gabriele Bartolini wrote: > On Thu, 22 Aug 2024 at 09:32, Jelte Fennema-Nio wrote: >> SET extension_search_path = /mnt/extensions/pg16/* > > That'd be great. +1. Agreed, that'd be handy, but not worth blocking the underlying capability for. Except possibly to the d

Re: RFC: Additional Directory for Extensions

2024-08-22 Thread Gabriele Bartolini
Hi Jelte, On Thu, 22 Aug 2024 at 09:32, Jelte Fennema-Nio wrote: > It looks like you want one directory per extension, so that list would > get pretty long if you have multiple extensions. Maybe (as a follow up > change), we should start to support a * as a wildcard in both of these > GUCs. So y

Re: RFC: Additional Directory for Extensions

2024-08-22 Thread Gabriele Bartolini
Hi Craig, On Thu, 22 Aug 2024 at 01:07, Craig Ringer wrote: > It's also very relevant for local development and testing. > Yep, which is the original goal of Christoph IIRC. > It may be possible to weaken this restriction somewhat thanks to the > upcoming > https://kubernetes.io/blog/2024/08/

Re: RFC: Additional Directory for Extensions

2024-08-22 Thread Jelte Fennema-Nio
On Thu, 22 Aug 2024 at 01:08, Craig Ringer wrote: > SET extension_search_path = $extsdir, > /mnt/extensions/pg16/postgis-vX.Y/extensions, > /mnt/extensions/pg16/gosuperfast/extensions; It looks like you want one directory per extension, so that list would get pretty long if you have multiple

Re: RFC: Additional Directory for Extensions

2024-08-21 Thread Craig Ringer
On Thu, 22 Aug 2024 at 08:00, Gabriele Bartolini wrote: > > Hi everyone, > > Apologies for only starting to look into this now. Thanks, David, for pushing > this forward. 100%. I've wanted this for some time but never had time to cook up a patch. > I want to emphasize the importance of this p

Re: RFC: Additional Directory for Extensions

2024-08-21 Thread Nathan Bossart
On Tue, Jun 25, 2024 at 12:12:33PM +0200, Alvaro Herrera wrote: > archive_command and so on: we could disable these too. Nathan did some > work to implement those using dynamic libraries, so it shouldn't be too > much of a loss; anything that is done with a shell script can also be > done with a s

Re: RFC: Additional Directory for Extensions

2024-08-21 Thread Gabriele Bartolini
Hi everyone, Apologies for only starting to look into this now. Thanks, David, for pushing this forward. I want to emphasize the importance of this patch for the broader adoption of extensions in immutable container environments, such as those used by the CloudNativePG operator in Kubernetes. To

Re: RFC: Additional Directory for Extensions

2024-07-08 Thread David E. Wheeler
On Jun 25, 2024, at 18:31, David E. Wheeler wrote: > For those who prefer a GitHub patch review experience, see this PR: > > https://github.com/theory/postgres/pull/3/files Rebased and restored PGC_SUSET in the attached v5 patch, plus noted the required privileges in the docs. Best, David

Re: RFC: Additional Directory for Extensions

2024-06-26 Thread Jelte Fennema-Nio
On Tue, 25 Jun 2024 at 19:33, David E. Wheeler wrote: > > On Jun 24, 2024, at 5:32 PM, Jelte Fennema-Nio wrote: > > > Still, for the sake of completeness it might make sense to support > > this whole list in extension_destdir. (assuming it's easy to do) > > It should be with the current patch, wh

Re: RFC: Additional Directory for Extensions

2024-06-26 Thread Jelte Fennema-Nio
On Wed, 26 Jun 2024 at 00:32, David E. Wheeler wrote: > In other news, here’s an updated patch that expands the documentation to > record that the destination directory is a prefix, and full paths should be > used under it. Also take the opportunity to document the PGXS DESTDIR > variable as th

Re: RFC: Additional Directory for Extensions

2024-06-25 Thread David E . Wheeler
On Jun 25, 2024, at 10:43 AM, Robert Haas wrote: > If we want to work on making the sorts of changes that > you're proposing, let's do it on a separate thread. It's not going to > be meaningfully harder to move in that direction after some patch like > this than it is today. I appreciate this se

Re: RFC: Additional Directory for Extensions

2024-06-25 Thread David E. Wheeler
On Jun 24, 2024, at 5:32 PM, Jelte Fennema-Nio wrote: > Still, for the sake of completeness it might make sense to support > this whole list in extension_destdir. (assuming it's easy to do) It should be with the current patch, which just uses a prefix to paths in `pg_config`. So if SHAREDIR is

Re: RFC: Additional Directory for Extensions

2024-06-25 Thread Robert Haas
On Tue, Jun 25, 2024 at 6:12 AM Alvaro Herrera wrote: > Now, I'm not saying that this is an easy journey. But if we don't > start, we're not going to get there. I actually kind of agree with you. I think I've said something similar in a previous email to the list somewhere. But I don't agree tha

Re: RFC: Additional Directory for Extensions

2024-06-25 Thread Jelte Fennema-Nio
On Tue, 25 Jun 2024 at 12:12, Alvaro Herrera wrote: > They can mutilate the system catalogs: yes, they can TRUNCATE pg_type. > So what? They've just destroyed their own ability to do anything else. > The real issue here is that they can edit pg_proc to cause SQL function > calls to call arbitrary

Re: RFC: Additional Directory for Extensions

2024-06-25 Thread Alvaro Herrera
On 2024-Jun-24, Robert Haas wrote: > Is "tighten up what the superuser can do" on our list of objectives? > Personally, I think we should be focusing mostly, and maybe entirely, > on letting non-superusers do more things, with appropriate security > controls. The superuser can ultimately do anythi

Re: RFC: Additional Directory for Extensions

2024-06-25 Thread Christoph Berg
Re: Nathan Bossart > At first glance, the general idea seems reasonable to me. I'm wondering > whether there is a requirement for this directory to be prepended or if it > could be appended to the end. That way, the existing ones would take > priority, which might be desirable from a security sta

Re: RFC: Additional Directory for Extensions

2024-06-24 Thread Jelte Fennema-Nio
On Mon, 24 Jun 2024 at 22:42, David E. Wheeler wrote: > >> BINDIR > >> DOCDIR > >> HTMLDIR > >> PKGINCLUDEDIR > >> LOCALEDIR > >> MANDIR > >> > >> I can imagine an extension wanting or needing to use any and all of these. > > > > Are these really all relevant to backend code? > > Oh I think so. Es

Re: RFC: Additional Directory for Extensions

2024-06-24 Thread David E. Wheeler
On Jun 24, 2024, at 17:17, Jelte Fennema-Nio wrote: > If you want to only change $libdir during CREATE EXTENSION (or ALTER > EXTENSION UPDATE), then why not just change it there. And really you'd > only want to change it when creating an extension from which the > control file is coming from exte

Re: RFC: Additional Directory for Extensions

2024-06-24 Thread Jelte Fennema-Nio
On Mon, 24 Jun 2024 at 18:11, Nathan Bossart wrote: > At first glance, the general idea seems reasonable to me. I'm wondering > whether there is a requirement for this directory to be prepended or if it > could be appended to the end. That way, the existing ones would take > priority, which migh

Re: RFC: Additional Directory for Extensions

2024-06-24 Thread Jelte Fennema-Nio
On Thu, 11 Apr 2024 at 19:52, David E. Wheeler wrote: > I realize this probably isn’t going to happen for 17, given the freeze, but I > would very much welcome feedback and pointers to address concerns about > providing a second directory for extensions and DSOs. Quite a few people have > talke

Re: RFC: Additional Directory for Extensions

2024-06-24 Thread David E. Wheeler
On Jun 24, 2024, at 4:28 PM, Robert Haas wrote: > As long as the GUC is superuser-only, I'm not sure what else there is > to do here. The only question is whether there's some reason to > disallow this even from the superuser, but I'm not quite seeing such a > reason. I can switch it back from r

Re: RFC: Additional Directory for Extensions

2024-06-24 Thread Robert Haas
On Mon, Jun 24, 2024 at 3:37 PM David E. Wheeler wrote: > I guess the question then is what security controls are appropriate for this > feature, which after all tells the postmaster what directories to read files > from. It feels a little outside the scope of a regular user to even be aware >

Re: RFC: Additional Directory for Extensions

2024-06-24 Thread David E. Wheeler
On Jun 24, 2024, at 1:53 PM, Robert Haas wrote: > Is "tighten up what the superuser can do" on our list of objectives? > Personally, I think we should be focusing mostly, and maybe entirely, > on letting non-superusers do more things, with appropriate security > controls. The superuser can ultima

Re: RFC: Additional Directory for Extensions

2024-06-24 Thread Robert Haas
On Wed, Apr 3, 2024 at 3:13 AM Alvaro Herrera wrote: > I support the idea of there being a second location from where to load > shared libraries ... but I don't like the idea of making it > runtime-configurable. If we want to continue to tighten up what > superuser can do, then one of the things

Re: RFC: Additional Directory for Extensions

2024-06-24 Thread Nathan Bossart
On Thu, Apr 11, 2024 at 01:52:26PM -0400, David E. Wheeler wrote: > I realize this probably isn´t going to happen for 17, given the freeze, > but I would very much welcome feedback and pointers to address concerns > about providing a second directory for extensions and DSOs. Quite a few > people ha

Re: RFC: Additional Directory for Extensions

2024-04-11 Thread David E. Wheeler
On Apr 4, 2024, at 1:20 PM, David E. Wheeler wrote: > I’ve added some docs based on your GUC description; updated patch attached. Here’s a rebase. I realize this probably isn’t going to happen for 17, given the freeze, but I would very much welcome feedback and pointers to address concerns abo

Re: RFC: Additional Directory for Extensions

2024-04-04 Thread David E. Wheeler
On Apr 3, 2024, at 12:46 PM, Christoph Berg wrote: > Thanks for bringing this up, I should have submitted this years ago. > (The patch is originally from September 2020.) That’s okay, it’s still 2020 in some ways. 😂 > I designed the patch to require a superuser to set it, so it doesn't > matter

Re: RFC: Additional Directory for Extensions

2024-04-03 Thread Christoph Berg
> Fwiw, I wrote this patch to solve the problem of testing extensions at > build-time where the build process does not have write access to > PGSHAREDIR. It solves that problem quite well, almost all PG > extensions have build-time test coverage now (where there was > basically 0 before). Also, it

Re: RFC: Additional Directory for Extensions

2024-04-03 Thread Christoph Berg
Re: David E. Wheeler > > Yes, I like the suggestion to make it require a restart, which lets the > > sysadmin control it and not limited to whatever the person who compiled it > > thought would make sense. > > Here’s a revision of the Debian patch that requires a server start. Thanks for bringi

Re: RFC: Additional Directory for Extensions

2024-04-03 Thread David E. Wheeler
On Apr 3, 2024, at 8:54 AM, David E. Wheeler wrote: > Yes, I like the suggestion to make it require a restart, which lets the > sysadmin control it and not limited to whatever the person who compiled it > thought would make sense. Here’s a revision of the Debian patch that requires a server st

Re: RFC: Additional Directory for Extensions

2024-04-03 Thread David E. Wheeler
On Apr 3, 2024, at 8:54 AM, David E. Wheeler wrote: > Yes, I like the suggestion to make it require a restart, which lets the > sysadmin control it and not limited to whatever the person who compiled it > thought would make sense. Or SIGHUP? D

Re: RFC: Additional Directory for Extensions

2024-04-03 Thread David E. Wheeler
On Apr 3, 2024, at 3:57 AM, walt...@technowledgy.de wrote: > I can also imagine that it would be very helpful in a container setup to be > able to set an environment variable with this path instead of having to > recompile all of postgres to change it. Yes, I like the suggestion to make it requ

Re: RFC: Additional Directory for Extensions

2024-04-03 Thread Daniel Gustafsson
> On 3 Apr 2024, at 09:13, Alvaro Herrera wrote: > > On 2024-Apr-02, David E. Wheeler wrote: > >> That quotation comes from this Debian patch[2] maintained by Christoph >> Berg. I’d like to formally propose integrating this patch into the >> core. And not only because it’s overhead for package m

Re: RFC: Additional Directory for Extensions

2024-04-03 Thread walther
Alvaro Herrera: I support the idea of there being a second location from where to load shared libraries ... but I don't like the idea of making it runtime-configurable. If we want to continue to tighten up what superuser can do, then one of the things that has to go away is the ability to load s

Re: RFC: Additional Directory for Extensions

2024-04-03 Thread Alvaro Herrera
On 2024-Apr-02, David E. Wheeler wrote: > That quotation comes from this Debian patch[2] maintained by Christoph > Berg. I’d like to formally propose integrating this patch into the > core. And not only because it’s overhead for package maintainers like > Christoph, but because a number of use cas