Re: libpq compression (part 2)

2023-08-11 Thread Andrey M. Borodin
> On 10 Aug 2023, at 19:47, Jonah H. Harris wrote: > > Pinging to see if anyone has continued to work on this behind-the-scenes or > whether this is the latest patch set there is. It's still on my TODO list, but I haven't done much review cycles yet. And the patch series already needs

Re: libpq compression (part 2)

2023-08-10 Thread Jonah H. Harris
Pinging to see if anyone has continued to work on this behind-the-scenes or whether this is the latest patch set there is. -- Jonah H. Harris

Re: libpq compression (part 2)

2022-11-18 Thread Peter Eisentraut
On 18.11.22 02:07, Andrey Borodin wrote: 2. This literal {no_compression_name} should be replaced by explicit form {no_compression_name, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL} That doesn't seem better.

Re: libpq compression (part 2)

2022-11-17 Thread Andrey Borodin
On Thu, Nov 17, 2022 at 7:09 AM Peter Eisentraut wrote: > > Note that the above code was just changed in dce92e59b1. Thanks! > I don't know > how that affects this patch set. With dce92e59b1 it would be much easier to find a bug in the compression patch. Some more notes about the patch. (sorry

Re: libpq compression (part 2)

2022-11-17 Thread Peter Eisentraut
On 17.11.22 01:27, Andrey Borodin wrote: Also I've found one more TODO item for the patch. Currently in fe-connect.c patch is doing buffer reset: conn->inStart = conn->inCursor = conn->inEnd = 0; This effectively consumes butes up tu current cursor. However, packet inspection is continued. The

Re: libpq compression (part 2)

2022-11-16 Thread Andrey Borodin
On Tue, Nov 15, 2022 at 7:17 PM Justin Pryzby wrote: > Also I've found one more TODO item for the patch. Currently in fe-connect.c patch is doing buffer reset: conn->inStart = conn->inCursor = conn->inEnd = 0; This effectively consumes butes up tu current cursor. However, packet inspection is

Re: libpq compression (part 2)

2022-11-15 Thread Justin Pryzby
On Mon, Nov 14, 2022 at 07:44:24PM -0800, Andrey Borodin wrote: > patchset needs a heavy rebase. If no one shows up to fix it I'll do Despite what its git timestamp says, this is based on the most recent patch from January, which I've had floating around since then. It needed to be rebased over

Re: libpq compression (part 2)

2022-11-14 Thread Andrey Borodin
On Sat, Nov 12, 2022 at 8:04 PM Andrey Borodin wrote: > > While testing patch some more I observe unpleasant segfaults: > > #27 0x0b08fda2 in lz4_decompress (d_stream=0x18cf82a0, > src=0x7feae4fa505d, src_size=92, > (gdb) select-frame 27 > (gdb) info locals > ds = 0x18cf82a0 > decPtr =

Re: libpq compression (part 2)

2022-11-12 Thread Andrey Borodin
On Sat, Nov 12, 2022 at 1:47 PM Andrey Borodin wrote: > > I've tried the patch, it works as advertised. While testing patch some more I observe unpleasant segfaults: #26 0x7fecafa1e058 in __memcpy_ssse3_back () from target:/lib64/libc.so.6 #27 0x0b08fda2 in lz4_decompress

Re: libpq compression (part 2)

2022-11-12 Thread Andrey Borodin
On Tue, Aug 2, 2022 at 1:53 PM Jacob Champion wrote: > > and changing the status to "Needs Review" I've tried the patch, it works as advertised. The code generally is OK, maybe some functions require comments (because at least their neighbours have some). Some linkers complain about

Re: libpq compression (part 2)

2022-08-02 Thread Jacob Champion
This entry has been waiting on author input for a while (our current threshold is roughly two weeks), so I've marked it Returned with Feedback. Once you think the patchset is ready for review again, you (or any interested party) can resurrect the patch entry by visiting

Re: libpq compression (part 2)

2022-03-03 Thread Daniil Zakhlystov
Ok, thanks > On 3 Mar 2022, at 02:33, Justin Pryzby wrote: > > If there's no objection, I'd like to move this to the next CF for > consideration > in PG16. > > On Mon, Jan 17, 2022 at 10:39:19PM -0600, Justin Pryzby wrote: >> On Tue, Jan 18, 2022 at 02:06:32AM +0500, Daniil Zakhlystov wrote:

Re: libpq compression (part 2)

2022-03-02 Thread Justin Pryzby
If there's no objection, I'd like to move this to the next CF for consideration in PG16. On Mon, Jan 17, 2022 at 10:39:19PM -0600, Justin Pryzby wrote: > On Tue, Jan 18, 2022 at 02:06:32AM +0500, Daniil Zakhlystov wrote: > > > => Since March, errmsg doesn't need extra parenthesis around it

Re: libpq compression (part 2)

2022-01-17 Thread Justin Pryzby
On Tue, Jan 18, 2022 at 02:06:32AM +0500, Daniil Zakhlystov wrote: > > => Since March, errmsg doesn't need extra parenthesis around it (e3a87b4). > I’ve resolved the stuck tests and added zlib support for CI Windows builds to > patch 0003-*. Thanks > for the suggestion, all tests seem to be OK

Re: libpq compression (part 2)

2022-01-13 Thread Justin Pryzby
On Fri, Jan 14, 2022 at 02:12:17AM +0500, Daniil Zakhlystov wrote: > Hi, Justin! > > First of all, thanks for the detailed review. I’ve applied your patches to > the current version. Note that my message had other comments that weren't addressed in this patch. Your 0003 patch has a couple

Re: libpq compression (part 2)

2022-01-12 Thread Justin Pryzby
zlib still causes check-world to get stuck. I first mentioned this last March: 20210319062800.gi11...@telsasoft.com Actually all the compression methods seems to get stuck with time make check -C src/bin/pg_rewind time make check -C src/test/isolation For CI purposes, there should be an 0003

Re: libpq compression (part 2)

2022-01-12 Thread Andrey Borodin
> 8 янв. 2022 г., в 01:56, Stephen Frost написал(а): >> >> It's discussed in last year's thread. The thinking is that there tends to be >> *fewer* exploitable opportunities between application->DB than between >> browser->app. > > Yes, this was discussed previously and addressed. What else

Re: libpq compression (part 2)

2022-01-07 Thread Stephen Frost
Greetings, * Bruce Momjian (br...@momjian.us) wrote: > On Fri, Jan 7, 2022 at 03:56:39PM -0500, Stephen Frost wrote: > > As for the details of how we allow control over it, I suppose there's a > > number of options. Having it in the HBA doesn't seem terrible, though I > > suspect most will just

Re: libpq compression (part 2)

2022-01-07 Thread Bruce Momjian
On Fri, Jan 7, 2022 at 03:56:39PM -0500, Stephen Frost wrote: > As for the details of how we allow control over it, I suppose there's a > number of options. Having it in the HBA doesn't seem terrible, though I > suspect most will just want to enable it across the board and having to > have

Re: libpq compression (part 2)

2022-01-07 Thread Stephen Frost
Greetings, * Justin Pryzby (pry...@telsasoft.com) wrote: > On Fri, Jan 07, 2022 at 01:46:00PM -0500, Bruce Momjian wrote: > > On Sat, Jan 1, 2022 at 11:25:05AM -0600, Justin Pryzby wrote: > > > Thanks for working on this. The patch looks to be in good shape - I hope > > > more > > > people

Re: libpq compression (part 2)

2022-01-07 Thread Justin Pryzby
On Fri, Jan 07, 2022 at 01:46:00PM -0500, Bruce Momjian wrote: > On Sat, Jan 1, 2022 at 11:25:05AM -0600, Justin Pryzby wrote: > > Thanks for working on this. The patch looks to be in good shape - I hope > > more > > people will help to review and test it. I took the liberty of creating a > >

Re: libpq compression (part 2)

2022-01-07 Thread Bruce Momjian
On Sat, Jan 1, 2022 at 11:25:05AM -0600, Justin Pryzby wrote: > Thanks for working on this. The patch looks to be in good shape - I hope more > people will help to review and test it. I took the liberty of creating a new > CF entry. http://cfbot.cputube.org/daniil-zakhlystov.html > >

Re: libpq compression (part 2)

2022-01-03 Thread Andrey Borodin
> Maybe you should reset the streams between each compression message (even if > it's using the same compression algorithm). This might allow better > compression. AFAIK on the contrary - longer data sequence usually compresses better. The codec can use knowledge about prior data to better