Re: [EXTERNAL] Re: Support load balancing in libpq

2023-03-15 Thread Jelte Fennema
Rebased On Tue, 14 Mar 2023 at 19:05, Gregory Stark (as CFM) wrote: > > The pgindent run in b6dfee28f is causing this patch to need a rebase > for the cfbot to apply it. v12-0002-Refactor-libpq-to-store-addrinfo-in-a-libpq-owne.patch Description: Binary data

Re: [EXTERNAL] Re: Support load balancing in libpq

2023-03-14 Thread Gregory Stark (as CFM)
The pgindent run in b6dfee28f is causing this patch to need a rebase for the cfbot to apply it.

Re: [EXTERNAL] Re: Support load balancing in libpq

2023-03-06 Thread Jelte Fennema
Small update. Improved some wording in the docs. On Fri, 3 Mar 2023 at 15:37, Jelte Fennema wrote: > > > I want to note that the Fisher-Yates algorithm is implemented in a > > difficult to understand manner. > > +if (j < i) /* avoid fetching undefined data if j=i */ > > This stuff does not make

Re: [EXTERNAL] Re: Support load balancing in libpq

2023-03-03 Thread Jelte Fennema
> I want to note that the Fisher-Yates algorithm is implemented in a > difficult to understand manner. > +if (j < i) /* avoid fetching undefined data if j=i */ > This stuff does not make sense in case of shuffling arrays inplace. It > is important only for making a new copy of an array and only in

Re: [EXTERNAL] Re: Support load balancing in libpq

2023-03-02 Thread Andrey Borodin
On Wed, Mar 1, 2023 at 12:03 PM Jelte Fennema wrote: > > done and updated cf entry > Hi Jelte! I've looked into the patch. Although so many improvements can be suggested, It definitely makes sense as-is too. These improvements might be, for example, sorting hosts according to ping latency or

Re: [EXTERNAL] Re: Support load balancing in libpq

2023-03-01 Thread Jelte Fennema
done and updated cf entry On Wed, 1 Mar 2023 at 20:13, Greg S wrote: > > This patch seems to need a rebase. > > I'll update the status to Waiting on Author for now. After rebasing > please update it to either Needs Review or Ready for Committer > depending on how simple the rebase was and

Re: [EXTERNAL] Re: Support load balancing in libpq

2023-03-01 Thread Greg S
This patch seems to need a rebase. I'll update the status to Waiting on Author for now. After rebasing please update it to either Needs Review or Ready for Committer depending on how simple the rebase was and whether there are open questions to finish it.

Re: [EXTERNAL] Re: Support load balancing in libpq

2023-01-26 Thread Jelte Fennema
After discussing this patch privately with Andres here's a new version of this patch. The major differences are: 1. Use the pointer value of the connection as a randomness source 2. Use more precise time as randomness source 3. Move addrinfo changes into a separate commit. This is both to make the

Re: [EXTERNAL] Re: Support load balancing in libpq

2023-01-18 Thread Jelte Fennema
As far as I can tell this is ready for committer feedback now btw. I'd really like to get this into PG16. > It hadn't been my intention to block the patch on it, sorry. Just > registering a preference. No problem. I hadn't looked into the shared PRNG solution closely enough to determine if I

Re: [EXTERNAL] Re: Support load balancing in libpq

2023-01-17 Thread Jacob Champion
On Fri, Jan 13, 2023 at 10:44 AM Jacob Champion wrote: > And my thought was that the one-time > initialization could be moved to a place that doesn't need to know the > connection options at all, to make it easier to reason about the > architecture. Say, next to the WSAStartup machinery. (And

Re: [EXTERNAL] Re: Support load balancing in libpq

2023-01-13 Thread Jacob Champion
On Fri, Jan 13, 2023 at 9:10 AM Jelte Fennema wrote: > > > Just a quick single-issue review, but I agree with Maxim that having > > one PRNG, seeded once, would be simpler > > I don't agree that it's simpler. Because now there's a mutex you have > to manage, and honestly cross-platform threading

Re: [EXTERNAL] Re: Support load balancing in libpq

2023-01-13 Thread Jelte Fennema
> Just a quick single-issue review, but I agree with Maxim that having > one PRNG, seeded once, would be simpler I don't agree that it's simpler. Because now there's a mutex you have to manage, and honestly cross-platform threading in C is not simple. However, I attached two additional patches

Re: [EXTERNAL] Re: Support load balancing in libpq

2023-01-12 Thread Jacob Champion
On Wed, Sep 14, 2022 at 7:54 AM Maxim Orlov wrote: > For the patch itself, I think it is better to use a more precise time > function in libpq_prng_init or call it only once. > Thought it is a special corner case, imagine all the connection attempts at > first second will be seeded with the

Re: [EXTERNAL] Re: Support load balancing in libpq

2023-01-09 Thread Jelte Fennema
Attached an updated patch which should address your feedback and I updated the commit message. > I wonder whether making the parameter a boolean will paint us into a > corner I made it a string option, just like target_session_attrs. I'm pretty sure a round-robin load balancing policy could be

Re: [EXTERNAL] Re: Support load balancing in libpq

2022-09-17 Thread Michael Banck
Hi, On Mon, Sep 12, 2022 at 02:16:56PM +, Jelte Fennema wrote: > Attached is an updated patch with the following changes: > 1. rebased (including solved merge conflict) > 2. fixed failing tests in CI > 3. changed the commit message a little bit > 4. addressed the two remarks from Micheal > 5.

Re: [EXTERNAL] Re: Support load balancing in libpq

2022-09-17 Thread Michael Banck
Hi, On Wed, Sep 14, 2022 at 05:53:48PM +0300, Maxim Orlov wrote: > > Also, IMO, the solution must have a fallback mechanism if the > > standby/chosen host isn't reachable. > > Yeah, I think it should. I'm not insisting on a particular name of options > here, but in my view, the overall idea may

Re: [EXTERNAL] Re: Support load balancing in libpq

2022-09-14 Thread Maxim Orlov
+1 for overall idea of load balancing via random host selection. For the patch itself, I think it is better to use a more precise time function in libpq_prng_init or call it only once. Thought it is a special corner case, imagine all the connection attempts at first second will be seeded with the

Re: [EXTERNAL] Re: Support load balancing in libpq

2022-09-12 Thread Jelte Fennema
Attached is an updated patch with the following changes: 1. rebased (including solved merge conflict) 2. fixed failing tests in CI 3. changed the commit message a little bit 4. addressed the two remarks from Micheal 5. changed the prng_state from a global to a connection level value for

Re: [EXTERNAL] Re: Support load balancing in libpq

2022-07-05 Thread Jelte Fennema
> I'm quoting a previous attempt by Satyanarayana Narlapuram on this > topic [1], it also has a patch set. Thanks for sharing that. It's indeed a different approach to solve the same problem. I think my approach is much simpler, since it only requires minimal changes to the libpq client and none