Re: libpq: PQgetCopyData() and allocation overhead

2023-03-03 Thread Jeroen Vermeulen
mb for now, hoping a sensible rule will become obvious later. Saw no obvious performance impact, so that's good. Jeroen On Fri, 3 Mar 2023 at 19:53, Tom Lane wrote: > Jeroen Vermeulen writes: > > The printf() is just the simplest example that sprang to mind though. > > There ma

Re: libpq: PQgetCopyData() and allocation overhead

2023-03-03 Thread Jeroen Vermeulen
On Fri, 3 Mar 2023 at 18:14, Tom Lane wrote: > Jeroen Vermeulen writes: > > On Fri, 3 Mar 2023 at 17:33, Tom Lane wrote: > >> Let's not do that. Declare it const char *, or maybe better const void > *. > > > Personally I would much prefer "char"

Re: libpq: PQgetCopyData() and allocation overhead

2023-03-03 Thread Jeroen Vermeulen
On Fri, 3 Mar 2023 at 17:33, Tom Lane wrote: > > If you mean exposing PQExpBuffer to users of libpq-fe.h, I'd be very > seriously against that. I realize that libpq exposes it at an ABI > level, but that doesn't mean we want non-Postgres code to use it. > I also don't see what it'd add for this

Re: libpq: PQgetCopyData() and allocation overhead

2023-03-03 Thread Jeroen Vermeulen
; On Thu, 2 Mar 2023 at 20:45, Jeroen Vermeulen wrote: > > I'm attaching a diff now. This is not a patch, it's just a discussion > piece. > > Did you try with PQExpBuffer? I still think that probably fits better > in the API design of libpq. Obviously if it's s

Re: libpq: PQgetCopyData() and allocation overhead

2023-03-02 Thread Jeroen Vermeulen
use a lot of CPU time, and this alternative reduces that by a lot. Jeroen On Thu, 2 Mar 2023 at 13:38, Daniel Gustafsson wrote: > > On 1 Mar 2023, at 15:23, Jeroen Vermeulen wrote: > > > PR for easy discussion: https://github.com/jtv/postgres/pull/1 > > The process for disc

Re: libpq: PQgetCopyData() and allocation overhead

2023-03-01 Thread Jeroen Vermeulen
https://github.com/jtv/postgres/pull/1 Jeroen On Mon, 27 Feb 2023 at 17:08, Jeroen Vermeulen wrote: > Done. Thanks for looking! > > Jelte Fennema pointed out that I should probably be using PQExpBuffer for > this. I'll look into that later; this is a proof of concept, not a

Re: libpq: PQgetCopyData() and allocation overhead

2023-02-27 Thread Jeroen Vermeulen
forpostg...@gmail.com> wrote: > On Fri, Feb 10, 2023 at 5:49 PM Jeroen Vermeulen wrote: > > > > OK, I've updated the PR with a benchmark (in the main directory). > > > > On this benchmark I'm seeing about a 24% reduction in "user" CPU time, > and a 8% re

Re: libpq: PQgetCopyData() and allocation overhead

2023-02-10 Thread Jeroen Vermeulen
OK, I've updated the PR with a benchmark (in the main directory). On this benchmark I'm seeing about a 24% reduction in "user" CPU time, and a 8% reduction in "system" CPU time. (Almost no reduction in wall-clock time.) Jeroen On Fri, 10 Feb 2023 at 11:32, Jero

Re: libpq: PQgetCopyData() and allocation overhead

2023-02-10 Thread Jeroen Vermeulen
arath Rupireddy < bharath.rupireddyforpostg...@gmail.com> wrote: > On Fri, Feb 10, 2023 at 3:43 PM Jeroen Vermeulen wrote: > > > > Would there be interest in a variant of PQgetCopyData() that re-uses the > same buffer for a new row, rather than allocating a new buffer on each > iteration?

libpq: PQgetCopyData() and allocation overhead

2023-02-10 Thread Jeroen Vermeulen
Would there be interest in a variant of PQgetCopyData() that re-uses the same buffer for a new row, rather than allocating a new buffer on each iteration? I tried it on a toy benchmark, and it reduced client-side CPU time by about 12%. (Less of a difference in wall-clock time of course; the clien