Re: Add new protocol message to change GUCs for usage with future protocol-only GUCs

2024-05-25 Thread Dave Cramer
On Sat, 25 May 2024 at 06:40, Jelte Fennema-Nio wrote: > On Thu, 23 May 2024 at 20:12, Tom Lane wrote: > > > > Jelte Fennema-Nio writes: > > > On Fri, 17 May 2024 at 21:24, Robert Haas > wrote: > > >> Perhaps these are all minority positions, but I can't tell you what > > >> everyone thinks,

Re: Add new protocol message to change GUCs for usage with future protocol-only GUCs

2024-05-25 Thread Jelte Fennema-Nio
(pressed send to early) On Sat, 25 May 2024 at 12:39, Jelte Fennema-Nio wrote: > But again if I'm alone in this, then I don't ... mind budging on this to move this decision along. Using _pq_.xxx parameters for all protocol changes would totally be acceptable to me.

Re: Add new protocol message to change GUCs for usage with future protocol-only GUCs

2024-05-25 Thread Jelte Fennema-Nio
On Thu, 23 May 2024 at 20:12, Tom Lane wrote: > > Jelte Fennema-Nio writes: > > On Fri, 17 May 2024 at 21:24, Robert Haas wrote: > >> Perhaps these are all minority positions, but I can't tell you what > >> everyone thinks, only what I think. > > > I'd love to hear some opinions from others on

Re: Add new protocol message to change GUCs for usage with future protocol-only GUCs

2024-05-25 Thread Jelte Fennema-Nio
On Thu, 23 May 2024 at 20:40, Tom Lane wrote: > > Jacob Champion writes: > > Would it be good to expand on that idea of criticality? IIRC one of > > Jelte's complaints earlier was that middleware has to know all the > > extension types anyway, to be able to figure out whether it has to do > >

Re: Add new protocol message to change GUCs for usage with future protocol-only GUCs

2024-05-24 Thread Jelte Fennema-Nio
On Fri, 24 May 2024 at 15:28, Robert Haas wrote: > > On Thu, May 23, 2024 at 4:00 PM Tom Lane wrote: > > I don't recall exactly what I thought earlier, but now I think we'd > > be better off with separate infrastructure. guc.c is unduly complex > > already. Perhaps there are bits of it that

Re: Add new protocol message to change GUCs for usage with future protocol-only GUCs

2024-05-24 Thread Robert Haas
On Thu, May 23, 2024 at 4:00 PM Tom Lane wrote: > I don't recall exactly what I thought earlier, but now I think we'd > be better off with separate infrastructure. guc.c is unduly complex > already. Perhaps there are bits of it that could be factored out > and shared, but I bet not a lot. OK.

Re: Add new protocol message to change GUCs for usage with future protocol-only GUCs

2024-05-23 Thread Tom Lane
Robert Haas writes: > I think part of the reason we ended up with the protocol parameters = > GUCs thing is because you seemed to be concurring with that approach > upthread. I think it was Jelte's idea originally, but I interpreted > some of your earlier remarks to be supporting it. I'm not sure

Re: Add new protocol message to change GUCs for usage with future protocol-only GUCs

2024-05-23 Thread Robert Haas
On Thu, May 23, 2024 at 2:12 PM Tom Lane wrote: > I got around to looking through this thread in preparation for next > week's patch review session. I have a couple of opinions to offer: I agree with these opinions. Independently of that, I'm glad you shared them. I think part of the reason we

Re: Add new protocol message to change GUCs for usage with future protocol-only GUCs

2024-05-23 Thread Tom Lane
Jacob Champion writes: > Would it be good to expand on that idea of criticality? IIRC one of > Jelte's complaints earlier was that middleware has to know all the > extension types anyway, to be able to figure out whether it has to do > something about them or not. HTTP has the concept of

Re: Add new protocol message to change GUCs for usage with future protocol-only GUCs

2024-05-23 Thread Jacob Champion
On Thu, May 23, 2024 at 11:12 AM Tom Lane wrote: > If a reader doesn't recognize a particular > chunk code, it can still tell whether the chunk is "critical" or not, > and thereby decide if it must give up or can proceed while ignoring > that chunk.) Would it be good to expand on that idea of

Re: Add new protocol message to change GUCs for usage with future protocol-only GUCs

2024-05-23 Thread Tom Lane
Jelte Fennema-Nio writes: > On Fri, 17 May 2024 at 21:24, Robert Haas wrote: >> Perhaps these are all minority positions, but I can't tell you what >> everyone thinks, only what I think. > I'd love to hear some opinions from others on these design choices. So > far it seems like we're the only

Re: Add new protocol message to change GUCs for usage with future protocol-only GUCs

2024-05-17 Thread Jelte Fennema-Nio
On Fri, 17 May 2024 at 21:24, Robert Haas wrote: > I think the > fear that we're going to run into cases where such complex handshaking > is necessary is a major reason why I'm afraid of Jelte's ideas about > ParameterSet: it seems much more opinionated to me than he seems to > think it is. I

Re: Add new protocol message to change GUCs for usage with future protocol-only GUCs

2024-05-17 Thread Robert Haas
On Fri, May 17, 2024 at 1:26 PM Jacob Burroughs wrote: > I was leaving the details around triggering that for this conversation > and in that patch just designing the messages in a way that would > allow adding the reconfiguration/restarting to be easily added in a > backwards-compatible way in a

Re: Add new protocol message to change GUCs for usage with future protocol-only GUCs

2024-05-17 Thread Jacob Burroughs
On Fri, May 17, 2024 at 3:15 AM Robert Haas wrote: > > OK, so you made it so that compressed data is fully self-identifying. > Hence, there's no need to worry if something gets changed: the > receiver, if properly implemented, can't help but notice. The only > downside that I can see to this

Re: Add new protocol message to change GUCs for usage with future protocol-only GUCs

2024-05-17 Thread Robert Haas
On Thu, May 16, 2024 at 1:39 PM Jacob Burroughs wrote: > As currently implemented [1], the client sends the server the list of > all compression algorithms it is willing to accept, and the server can > use one of them. If the server knows what `_pq_.compression` means > but doesn't actually

Re: Add new protocol message to change GUCs for usage with future protocol-only GUCs

2024-05-16 Thread Jelte Fennema-Nio
On Thu, 16 May 2024 at 18:57, Robert Haas wrote: > Ugh, it's so hard to communicate clearly about this stuff. I didn't > really have any thought that we'd ever try to handle something as > complicated as compression using ParameterSet. Okay, then it's definitely very hard to communicate clearly

Re: Add new protocol message to change GUCs for usage with future protocol-only GUCs

2024-05-16 Thread Jacob Burroughs
On Thu, May 16, 2024 at 6:57 AM Robert Haas wrote: > > Ugh, it's so hard to communicate clearly about this stuff. I didn't > really have any thought that we'd ever try to handle something as > complicated as compression using ParameterSet. I tend to agree that > for compression I'd like to see

Re: Add new protocol message to change GUCs for usage with future protocol-only GUCs

2024-05-16 Thread Robert Haas
On Thu, May 16, 2024 at 12:09 PM Jelte Fennema-Nio wrote: > I don't really understand the benefit of your proposal over option 2 > that I proposed. Afaict you're proposing that for e.g. compression we > first set _pq_.supports_compression=1 in the StartupMessage and use > that to do feature

Re: Add new protocol message to change GUCs for usage with future protocol-only GUCs

2024-05-16 Thread Jelte Fennema-Nio
On Thu, 16 May 2024 at 17:29, Robert Haas wrote: > You're probably not going to like this answer, but I feel like this is > another sign that you're trying to use the protocol extensibility > facilities in the wrong way. In my first reply to the thread, I > proposed having the client send

Re: Add new protocol message to change GUCs for usage with future protocol-only GUCs

2024-05-16 Thread Robert Haas
On Thu, May 16, 2024 at 5:22 AM Jelte Fennema-Nio wrote: > If we're not setting the protocol parameter in the StartupMessage, > there's currently no way for us to know if the protocol parameter is > supported by the server. If protocol parameters were unchangable then > that would be fine, but

Re: Add new protocol message to change GUCs for usage with future protocol-only GUCs

2024-05-16 Thread Jelte Fennema-Nio
On Fri, 5 Apr 2024 at 18:30, Dave Cramer wrote: > On Fri, 5 Apr 2024 at 12:09, Jelte Fennema-Nio wrote: >> I'll take a look at redesigning the protocol parameter stuff. To work >> with dedicated functions instead. > > +1 It's been a while, but I now actually took the time to look into this. And

Re: Add new protocol message to change GUCs for usage with future protocol-only GUCs

2024-05-06 Thread Jelte Fennema-Nio
On Tue, 23 Apr 2024 at 19:39, Jacob Champion wrote: > > On Mon, Apr 22, 2024 at 2:20 PM Jelte Fennema-Nio wrote: > > 1. I strongly believe minor protocol version bumps after the initial > > 3.1 one can be made painless for clients/poolers (so the ones to > > 3.2/3.3/etc). Similar to how TLS 1.3

Re: Add new protocol message to change GUCs for usage with future protocol-only GUCs

2024-05-06 Thread Jelte Fennema-Nio
On Tue, 23 Apr 2024 at 17:03, Robert Haas wrote: > If a client doesn't know about encrypted columns and sets that > bit at random, that will break things, and formally I think that's a > risk, because I don't believe we document anywhere that you shouldn't > set unused bits in the format mask.

Re: Add new protocol message to change GUCs for usage with future protocol-only GUCs

2024-04-23 Thread Jacob Champion
On Mon, Apr 22, 2024 at 2:20 PM Jelte Fennema-Nio wrote: > 1. I strongly believe minor protocol version bumps after the initial > 3.1 one can be made painless for clients/poolers (so the ones to > 3.2/3.3/etc). Similar to how TLS 1.3 can be safely introduced, and not > having to worry about

Re: Add new protocol message to change GUCs for usage with future protocol-only GUCs

2024-04-23 Thread Robert Haas
On Mon, Apr 22, 2024 at 5:19 PM Jelte Fennema-Nio wrote: > On Mon, 22 Apr 2024 at 16:26, Robert Haas wrote: > > That's a fair point, but I'm still not seeing much practical > > advantage. It's unlikely that a client is going to set a random bit in > > a format parameter for no reason. > > I

Re: Add new protocol message to change GUCs for usage with future protocol-only GUCs

2024-04-22 Thread Jelte Fennema-Nio
On Mon, 22 Apr 2024 at 16:26, Robert Haas wrote: > That's a fair point, but I'm still not seeing much practical > advantage. It's unlikely that a client is going to set a random bit in > a format parameter for no reason. I think you're missing an important point of mine here. The client wouldn't

Re: Add new protocol message to change GUCs for usage with future protocol-only GUCs

2024-04-22 Thread Robert Haas
On Thu, Apr 18, 2024 at 5:36 PM Jelte Fennema-Nio wrote: > To clarify: My proposed approach is to use a protocol extension > parameter for to enable the new messages that the server can send > (like Peter is doing now). And **in addition to that** gate the new > Bind format type behind a feature

Re: Add new protocol message to change GUCs for usage with future protocol-only GUCs

2024-04-18 Thread Jelte Fennema-Nio
On Thu, 18 Apr 2024 at 23:36, Jelte Fennema-Nio wrote: > To clarify: My proposed approach is to use a protocol extension > parameter for to enable the new messages that the server can send > (like Peter is doing now). And **in addition to that** gate the new > Bind format type behind a feature

Re: Add new protocol message to change GUCs for usage with future protocol-only GUCs

2024-04-18 Thread Jelte Fennema-Nio
On Thu, 18 Apr 2024 at 22:17, Robert Haas wrote: > If we go with Peter's approach, every driver that supports his feature > will work perfectly, and every driver that doesn't will work exactly > as it does today. The risk of breaking anything is as near to zero as > human developers can

Re: Add new protocol message to change GUCs for usage with future protocol-only GUCs

2024-04-18 Thread Robert Haas
On Thu, Apr 18, 2024 at 3:34 PM Jelte Fennema-Nio wrote: > I really don't understand what exactly you're worried about. What do > you expect will break when bumping the protocol version? As Dave said, > clients should never bail out due to protocol version differences. Sure, and I should never

Re: Add new protocol message to change GUCs for usage with future protocol-only GUCs

2024-04-18 Thread Jelte Fennema-Nio
On Mon, 15 Apr 2024 at 21:47, Dave Cramer wrote: >> On Mon, 15 Apr 2024 at 19:52, Robert Haas wrote: >> > surely it can't be right to use protocol >> > version 3.0 to refer to a bunch of different things. But at the same >> > time, surely we don't want clients to start panicking and bailing out

Re: Add new protocol message to change GUCs for usage with future protocol-only GUCs

2024-04-15 Thread Dave Cramer
On Mon, 15 Apr 2024 at 15:38, Jelte Fennema-Nio wrote: > On Mon, 15 Apr 2024 at 19:43, Robert Haas wrote: > > > > On Sat, Apr 6, 2024 at 6:14 PM Jelte Fennema-Nio wrote: > > > I think for clients/drivers, the work would generally be pretty > > > minimal. For almost all proposed changes,

Re: Add new protocol message to change GUCs for usage with future protocol-only GUCs

2024-04-15 Thread Jelte Fennema-Nio
On Mon, 15 Apr 2024 at 19:43, Robert Haas wrote: > > On Sat, Apr 6, 2024 at 6:14 PM Jelte Fennema-Nio wrote: > > I think for clients/drivers, the work would generally be pretty > > minimal. For almost all proposed changes, clients can "support" the > > protocol version update by simply not using

Re: Add new protocol message to change GUCs for usage with future protocol-only GUCs

2024-04-15 Thread Robert Haas
[ Hit send too early, sorry. ] On Mon, Apr 15, 2024 at 1:43 PM Robert Haas wrote: > But the wire protocol changes very slowly, and I think that is a > difference that actually matters quite a bit here. Broadly speaking, I > can use a psq ...a psql that I just built today to talk to a server

Re: Add new protocol message to change GUCs for usage with future protocol-only GUCs

2024-04-15 Thread Robert Haas
On Sat, Apr 6, 2024 at 6:14 PM Jelte Fennema-Nio wrote: > I think for clients/drivers, the work would generally be pretty > minimal. For almost all proposed changes, clients can "support" the > protocol version update by simply not using the new features, ... I mean, I agree if a particular

Re: Add new protocol message to change GUCs for usage with future protocol-only GUCs

2024-04-10 Thread Peter Eisentraut
On 05.04.24 14:55, Robert Haas wrote: I also wonder how the protocol negotiation for column encryption is actually going to work. What are the actual wire protocol changes that are needed? What does the server need to know from the client, or the client from the server, about what is supported?

Re: Add new protocol message to change GUCs for usage with future protocol-only GUCs

2024-04-06 Thread Jelte Fennema-Nio
On Fri, 5 Apr 2024 at 18:48, Robert Haas wrote: > Maybe we'd be better off adding a libpq connection > option that forces the use of a specific minor protocol version, but > then we'll need backward-compatibility code in libpq basically > forever. But maybe we need that anyway to avoid older and

Re: Add new protocol message to change GUCs for usage with future protocol-only GUCs

2024-04-05 Thread Dave Cramer
> > > Plus, you've got all of the consequences for non-core drivers, which > have to both add support for the new wire protocol - if they don't > want to seem outdated and eventually obsolete - and also test that > they're still compatible with all supported server versions. > Connection poolers

Re: Add new protocol message to change GUCs for usage with future protocol-only GUCs

2024-04-05 Thread Jelte Fennema-Nio
On Fri, 5 Apr 2024 at 18:43, Tom Lane wrote: > I don't buy that argument, actually. libpq, and pretty much every > other client AFAIK, has provisions to let higher code levels insert > random options into the startup packet. So to make this work libpq > would have to filter or at least inspect

Re: Add new protocol message to change GUCs for usage with future protocol-only GUCs

2024-04-05 Thread Jelte Fennema-Nio
On Fri, 5 Apr 2024 at 18:30, Dave Cramer wrote: >> > I really intended the _pq_ prefix as a way of taking something out of >> > the GUC namespace, not as a part of the GUC namespace that users would >> > see. And I'm reluctant to go back on that. If we want to make >> > pg_protocol.${NAME} mean a

Re: Add new protocol message to change GUCs for usage with future protocol-only GUCs

2024-04-05 Thread Robert Haas
On Fri, Apr 5, 2024 at 12:09 PM Jelte Fennema-Nio wrote: > But I do think that we shouldn't be at all averse to the act of > bumping the protocol version itself. If we have a single small > protocol change in one release, then imho it's no problem to bump the > protocol version. Bumping the

Re: Add new protocol message to change GUCs for usage with future protocol-only GUCs

2024-04-05 Thread Tom Lane
Dave Cramer writes: > On Fri, 5 Apr 2024 at 12:09, Jelte Fennema-Nio wrote: >> Setting PGC_PROTOCOL gucs would be allowed in the startup packet, >> which is fine afaict because that's also something that's part of the >> protocol level and is thus fully controlled by client libraries and >>

Re: Add new protocol message to change GUCs for usage with future protocol-only GUCs

2024-04-05 Thread Dave Cramer
On Fri, 5 Apr 2024 at 12:09, Jelte Fennema-Nio wrote: > On Fri, 5 Apr 2024 at 16:02, Robert Haas wrote: > > Often? > > > > I kind of hope that the protocol starts to evolve a bit more than it > > has, but I don't want a continuous stream of changes. That will be > > very hard to test and verify

Re: Add new protocol message to change GUCs for usage with future protocol-only GUCs

2024-04-05 Thread Jelte Fennema-Nio
On Fri, 5 Apr 2024 at 16:02, Robert Haas wrote: > Often? > > I kind of hope that the protocol starts to evolve a bit more than it > has, but I don't want a continuous stream of changes. That will be > very hard to test and verify correctness, and a hassle for drivers to > keep up with, and a mess

Re: Add new protocol message to change GUCs for usage with future protocol-only GUCs

2024-04-05 Thread Jelte Fennema-Nio
On Fri, 5 Apr 2024 at 16:04, Robert Haas wrote: > > On Thu, Apr 4, 2024 at 1:10 PM Jelte Fennema-Nio wrote: > > Attached is a rebased patchset > > We should keep talking about this, but I think we're far too close to > the wire at this point to think about committing anything for v17 at > this

Re: Add new protocol message to change GUCs for usage with future protocol-only GUCs

2024-04-05 Thread Robert Haas
On Thu, Apr 4, 2024 at 1:10 PM Jelte Fennema-Nio wrote: > Attached is a rebased patchset We should keep talking about this, but I think we're far too close to the wire at this point to think about committing anything for v17 at this point. These are big changes, they haven't been thoroughly

Re: Add new protocol message to change GUCs for usage with future protocol-only GUCs

2024-04-05 Thread Robert Haas
On Thu, Apr 4, 2024 at 12:45 PM Jelte Fennema-Nio wrote: > Yeah, we definitely think differently here then. To me bumping the > minor protocol version shouldn't be a thing that we would need to > carefully consider. It should be easy to do, and probably done often. Often? I kind of hope that

Re: Add new protocol message to change GUCs for usage with future protocol-only GUCs

2024-04-05 Thread Robert Haas
On Thu, Apr 4, 2024 at 8:50 AM Peter Eisentraut wrote: > It appears there are several different perspectives about this. My > intuition was that a protocol version change indicates something that we > eventually want all client libraries to support. Whereas protocol > extensions are truly

Re: Add new protocol message to change GUCs for usage with future protocol-only GUCs

2024-04-05 Thread Dave Cramer
On Thu, 4 Apr 2024 at 12:45, Jelte Fennema-Nio wrote: > On Thu, 4 Apr 2024 at 14:50, Peter Eisentraut > wrote: > > It appears there are several different perspectives about this. My > > intuition was that a protocol version change indicates something that we > > eventually want all client

Re: Add new protocol message to change GUCs for usage with future protocol-only GUCs

2024-04-04 Thread Jelte Fennema-Nio
On Thu, 4 Apr 2024 at 14:50, Peter Eisentraut wrote: > It appears there are several different perspectives about this. My > intuition was that a protocol version change indicates something that we > eventually want all client libraries to support. Whereas protocol > extensions are truly opt-in.

Re: Add new protocol message to change GUCs for usage with future protocol-only GUCs

2024-04-04 Thread Jelte Fennema-Nio
Ugh, I seem to have somehow missed this response completely. On Thu, 14 Mar 2024 at 21:33, Robert Haas wrote: > While I think that will be a common > pattern, I do not think it will be a universal one. I do agree, > however, that every possible variation of the protocol is either >

Re: Add new protocol message to change GUCs for usage with future protocol-only GUCs

2024-04-04 Thread Peter Eisentraut
On 14.03.24 21:33, Robert Haas wrote: You seem to be supposing here that all protocol changes will consist of adding new message types. While I think that will be a common pattern, I do not think it will be a universal one. As an additional data point, the column encryption patch that is

Re: Add new protocol message to change GUCs for usage with future protocol-only GUCs

2024-03-14 Thread Robert Haas
On Thu, Mar 14, 2024 at 1:54 PM Jelte Fennema-Nio wrote: > In my view there can be, **by definition,** only two general types of > protocol changes: > 1. A "simple protocol change": This is one that requires agreement by > both the client and server, that they understand the new message types >

Re: Add new protocol message to change GUCs for usage with future protocol-only GUCs

2024-03-14 Thread Jelte Fennema-Nio
On Thu, 14 Mar 2024 at 16:45, Robert Haas wrote: > I feel bad arguing about the patches that you think are a slam-dunk, > but I find myself disagreeing with the design choices. No worries, thanks a lot for responding. I'm happy to discuss this design further. I didn't necessarily mean these

Re: Add new protocol message to change GUCs for usage with future protocol-only GUCs

2024-03-14 Thread Robert Haas
On Fri, Mar 8, 2024 at 6:47 AM Jelte Fennema-Nio wrote: > 1. 0001-0005 are needed for any protocol change, and hopefully > shouldn't require much discussion I feel bad arguing about the patches that you think are a slam-dunk, but I find myself disagreeing with the design choices. Regarding

Re: Add new protocol message to change GUCs for usage with future protocol-only GUCs

2024-03-13 Thread Jelte Fennema-Nio
Fixed some conflicts again, as well as adding a connection option to choose the requested protocol version (as discussed in[1]). This new connection option is not useful when connecting to any of the supported postgres versions. But it can be useful when connecting to PG versions before 9.3. Or

Re: Add new protocol message to change GUCs for usage with future protocol-only GUCs

2024-01-21 Thread Peter Smith
2024-01 Commitfest. Hi, This patch has a CF status of "Needs Review", but it seems like there were some CFbot test failures last time it was run [1]. Please have a look and post an updated version if necessary. == [1]

Re: Add new protocol message to change GUCs for usage with future protocol-only GUCs

2024-01-17 Thread Jelte Fennema-Nio
On Wed, 17 Jan 2024 at 14:39, Robert Haas wrote: > I think it's hard to say for sure what API is going to work well here, > because we just don't have much experience with this. Agreed, but I strongly believe PQunsupportedProtocolExtensions() is useful regardless of the API choice. > I also

Re: Add new protocol message to change GUCs for usage with future protocol-only GUCs

2024-01-17 Thread Robert Haas
On Tue, Jan 16, 2024 at 9:21 PM Jelte Fennema-Nio wrote: > It's understandable you're worried about breaking clients, but afaict > **you are actually misunderstanding the situation**. I totally agree > that we cannot bump the protocol version without also merging 0002 > (that's why the version

Re: Add new protocol message to change GUCs for usage with future protocol-only GUCs

2024-01-16 Thread Jelte Fennema-Nio
On Tue, 16 Jan 2024 at 21:36, Robert Haas wrote: > Sorry for not getting back to this right away; there are quite a few > threads competing for my attention. No worries, I know it's a busy time. > But I think we would want to have those changes in all supported > branches before we could think

Re: Add new protocol message to change GUCs for usage with future protocol-only GUCs

2024-01-16 Thread Robert Haas
On Tue, Jan 16, 2024 at 8:43 AM Jelte Fennema-Nio wrote: > I haven't removed 0008 yet, since I'd like some feedback first if that > makes sense. But I did add two new patches in the middle of the > patchset (which shift the later patch numbers by 2): > > 0007: Adds a new \parameterset

Re: Add new protocol message to change GUCs for usage with future protocol-only GUCs

2024-01-11 Thread Jelte Fennema-Nio
On Wed, 10 Jan 2024 at 23:53, Jelte Fennema-Nio wrote: > Honestly, I care more about patch 0010 than patch 0008. Patch 0008 > simply seemed like the easiest way to demonstrate the ParameterSet > message. So to be clear, if you consider 0008 the most controversial/risky part of this patchset

Re: Add new protocol message to change GUCs for usage with future protocol-only GUCs

2024-01-10 Thread Jelte Fennema-Nio
On Wed, 10 Jan 2024 at 22:12, Robert Haas wrote: > Could you explain why you think that the protocol version bump is necessary? Patch 0006 adds a new protocol message to the protocol. Somehow the client needs to be told that the server understands that message. Using the protocol version seems

Re: Add new protocol message to change GUCs for usage with future protocol-only GUCs

2024-01-10 Thread Robert Haas
On Mon, Jan 8, 2024 at 5:52 PM Jelte Fennema-Nio wrote: > It's still bumping the protocol version. I think this is a necessity > with the current changeset, since ParameterSet now applies to plain > GUCs too and. As I clarified in a previous email, this does **not** > break old clients, since the

Re: Add new protocol message to change GUCs for usage with future protocol-only GUCs

2024-01-10 Thread Jelte Fennema-Nio
v6 attached with the following changes: 1. Fixed rebase conflicts with master 2. removed PGC_S_PROTOCOL (but kept PGC_PROTOCOL and PGC_SU_PROTOCOL). This extra source level was not needed. And after some more testing I realized this extra source level even caused problems, since protocol

Re: Add new protocol message to change GUCs for usage with future protocol-only GUCs

2024-01-08 Thread Jelte Fennema-Nio
Okay, attempt number 5 attached. The primary changes with the previous version are: 1. Split up commits a bit differently. I think each commit now stands on its own and is an incremental improvement that could be committed without any of the later ones being necessary. Descriptions of why each

Re: Add new protocol message to change GUCs for usage with future protocol-only GUCs

2024-01-05 Thread Robert Haas
On Fri, Jan 5, 2024 at 12:35 PM Tom Lane wrote: > I think it'd be quite simple. As I said, it's just a small variation > on how some GUCs already work. The only thing that's really > transactional is SQL-driven updates, which'd be disallowed for this > class of variables. Well, I know better

Re: Add new protocol message to change GUCs for usage with future protocol-only GUCs

2024-01-05 Thread Tom Lane
Robert Haas writes: > On Fri, Jan 5, 2024 at 11:53 AM Tom Lane wrote: >> So my thought was that this should be implemented as an (unchangeable) >> flag bit for a GUC variable, GUC_PROTOCOL_ONLY or something like that, >> and then we would refuse SQL-based set attempts on that. The behavior >>

Re: Add new protocol message to change GUCs for usage with future protocol-only GUCs

2024-01-05 Thread Jelte Fennema-Nio
On Fri, 5 Jan 2024 at 17:28, Robert Haas wrote: > First, I don't see a reason to bump the protocol version. The whole > reason for adding support for protocol options (_pq_.whatever) is so > that we wouldn't have to change the protocol version to add new > message types. At some point we may want

Re: Add new protocol message to change GUCs for usage with future protocol-only GUCs

2024-01-05 Thread Robert Haas
On Fri, Jan 5, 2024 at 12:20 PM Jelte Fennema-Nio wrote: > They are not fundamentally transactional afaict based on the changes > that were needed so far. It makes sense too, because e.g. SIGHUP > should change the GUC value if the config changed no matter if the > current transaction aborts or

Re: Add new protocol message to change GUCs for usage with future protocol-only GUCs

2024-01-05 Thread Jelte Fennema-Nio
On Fri, 5 Jan 2024 at 18:08, Robert Haas wrote: > On Fri, Jan 5, 2024 at 11:53 AM Tom Lane wrote: > > There is a lot of infrastructure we'll have to re-invent if > > we make this completely independent of GUCs, notably: > > * a way to establish the initial/default value > > * a way to display

Re: Add new protocol message to change GUCs for usage with future protocol-only GUCs

2024-01-05 Thread Robert Haas
On Fri, Jan 5, 2024 at 11:53 AM Tom Lane wrote: > There is a lot of infrastructure we'll have to re-invent if > we make this completely independent of GUCs, notably: > * a way to establish the initial/default value > * a way to display the active value > > So my thought was that this should be

Re: Add new protocol message to change GUCs for usage with future protocol-only GUCs

2024-01-05 Thread Tom Lane
Robert Haas writes: > Second, I don't really like the idea of selectively turning GUCs into > protocol-managed parameters. I think there are a relatively small > number of things that we'd ever want to manage on a protocol level, > but this design would force us to make it work for any GUC

Re: Add new protocol message to change GUCs for usage with future protocol-only GUCs

2024-01-05 Thread Robert Haas
On Fri, Jan 5, 2024 at 10:14 AM Jelte Fennema-Nio wrote: > New patchset attached, where I split up the patches in smaller logical units. > Note that the first 4 patches in this series are not making any > protocol changes. All they do is set up infrastructure in the code > that allows us to make

Re: Add new protocol message to change GUCs for usage with future protocol-only GUCs

2024-01-05 Thread Jelte Fennema-Nio
New patchset attached, where I split up the patches in smaller logical units. Note that the first 4 patches in this series are not making any protocol changes. All they do is set up infrastructure in the code that allows us to make protocol changes in the future. I hope that those 4 should all be

Re: Add new protocol message to change GUCs for usage with future protocol-only GUCs

2024-01-03 Thread Jelte Fennema-Nio
On Wed, 3 Jan 2024 at 00:43, Jacob Burroughs wrote: > What if we... Great suggestions! Attached is a v3 version of the patchset that implements all of this, including documentation. v3-0001-libpq-Handle-NegotiateProtocolVersion-message-mor.patch Description: Binary data

Re: Add new protocol message to change GUCs for usage with future protocol-only GUCs

2024-01-02 Thread Jacob Burroughs
What if we created a new guc flag `GUC_PROTOCOL_EXTENSION` (or a better name), used that for any of the GUCs options that should *only* be set via a `ParameterSet` protocol message, and then prevent changing those through SET/RESET/RESET ALL (but I don't see a reason to prevent reading them

Re: Add new protocol message to change GUCs for usage with future protocol-only GUCs

2024-01-02 Thread Jelte Fennema-Nio
On Tue, 2 Jan 2024 at 18:51, Robert Haas wrote: > > On the whole, this feels like you are trying to force some things > > into the GUC model that should not be there. I do perceive that > > there are things that could be protocol-level variables, but > > trying to say they are a kind of GUC

Re: Add new protocol message to change GUCs for usage with future protocol-only GUCs

2024-01-02 Thread Robert Haas
On Fri, Dec 29, 2023 at 1:38 PM Tom Lane wrote: > Jelte Fennema-Nio writes: > > 1. Protocol messages are much easier to inspect for connection poolers > > than queries > > Unless you somehow forbid clients from setting GUCs in the old way, > exactly how will that help a pooler? I agree that for

Re: Add new protocol message to change GUCs for usage with future protocol-only GUCs

2024-01-02 Thread Jelte Fennema-Nio
On Sat, 30 Dec 2023 at 00:07, Jelte Fennema-Nio wrote: > > On Fri, 29 Dec 2023 at 19:32, Jeff Davis wrote: > > That is my biggest concern right now: what will new clients connecting > > to old servers do? > > This is not that big of a deal. Since it's only an addition of a new > message type,

Re: Add new protocol message to change GUCs for usage with future protocol-only GUCs

2023-12-30 Thread Jelte Fennema-Nio
On Sat, 30 Dec 2023 at 16:05, Jacob Burroughs wrote: > What if we allowed the client to send `ParameterStatus` messages to > the server to reconfigure protocol extensions that were requested at > startup? Sending ParameterStatus from the frontend is not possible, because Sync already claimed the

Re: Add new protocol message to change GUCs for usage with future protocol-only GUCs

2023-12-30 Thread Jacob Burroughs
> How about instead of talking about protocol-only GUCs (which are > indeed currently a theoretical concept), we start considering this > patch as a way to modify parameters for protocol extensions. Protocol > extension parameters (starting with _pq_.) can currently only be > configured using the

Re: Add new protocol message to change GUCs for usage with future protocol-only GUCs

2023-12-29 Thread Jelte Fennema-Nio
On Fri, 29 Dec 2023 at 19:38, Tom Lane wrote: > > 2. It paves the way for GUCs that can only be set using a protocol > > message (and not using SET). > > This is assuming facts not in evidence. How about instead of talking about protocol-only GUCs (which are indeed currently a theoretical

Re: Add new protocol message to change GUCs for usage with future protocol-only GUCs

2023-12-29 Thread Jelte Fennema-Nio
On Fri, 29 Dec 2023 at 19:38, Tom Lane wrote: >> Jelte Fennema-Nio writes: > > 1. Protocol messages are much easier to inspect for connection poolers > > than queries > > Unless you somehow forbid clients from setting GUCs in the old way, > exactly how will that help a pooler? A co-operating

Re: Add new protocol message to change GUCs for usage with future protocol-only GUCs

2023-12-29 Thread Jelte Fennema-Nio
On Fri, 29 Dec 2023 at 19:32, Jeff Davis wrote: > On Fri, 2023-12-29 at 18:29 +0100, Jelte Fennema-Nio wrote: > > There's definitely still some more work that needs to be done > > (docs for new libpq APIs, protocol version bump, working protocol > > version negotiation). > > That is my biggest

Re: Add new protocol message to change GUCs for usage with future protocol-only GUCs

2023-12-29 Thread Tom Lane
Jeff Davis writes: > On Fri, 2023-12-29 at 13:38 -0500, Tom Lane wrote: >> This is assuming facts not in evidence.  Why would we want such a >> thing? > The problem came up during the binary_formats GUC discussion: it > doesn't really make sense to change that with a SQL query, and doing so >

Re: Add new protocol message to change GUCs for usage with future protocol-only GUCs

2023-12-29 Thread Jeff Davis
On Fri, 2023-12-29 at 13:38 -0500, Tom Lane wrote: > > 2. It paves the way for GUCs that can only be set using a protocol > > message (and not using SET). > > This is assuming facts not in evidence.  Why would we want such a > thing? The problem came up during the binary_formats GUC discussion:

Re: Add new protocol message to change GUCs for usage with future protocol-only GUCs

2023-12-29 Thread Tom Lane
Jelte Fennema-Nio writes: > Currently the only way to set GUCs from a client is by using SET > commands or set them in the StartupMessage. I think it would be very > useful to be able to change settings using a message at the protocol > level. For the following reasons: > 1. Protocol messages

Re: Add new protocol message to change GUCs for usage with future protocol-only GUCs

2023-12-29 Thread Jeff Davis
On Fri, 2023-12-29 at 18:29 +0100, Jelte Fennema-Nio wrote: > 2. It paves the way for GUCs that can only be set using a protocol > message (and not using SET). That sounds useful for GUCs that can interfere with the client, such as client_encoding or the proposed GUC in you referred to at [1]. >

Re: Add new protocol message to change GUCs for usage with future protocol-only GUCs

2023-12-29 Thread Pavel Stehule
Hi pá 29. 12. 2023 v 18:29 odesílatel Jelte Fennema-Nio napsal: > Currently the only way to set GUCs from a client is by using SET > commands or set them in the StartupMessage. I think it would be very > useful to be able to change settings using a message at the protocol > level. For the

Add new protocol message to change GUCs for usage with future protocol-only GUCs

2023-12-29 Thread Jelte Fennema-Nio
Currently the only way to set GUCs from a client is by using SET commands or set them in the StartupMessage. I think it would be very useful to be able to change settings using a message at the protocol level. For the following reasons: 1. Protocol messages are much easier to inspect for