Re: remaining sql/json patches

2024-04-05 Thread Amit Langote
Hi Alexander, On Fri, Apr 5, 2024 at 3:00 PM Alexander Lakhin wrote: > > Hello Amit, > > 04.04.2024 15:02, Amit Langote wrote: > > Pushed after fixing these and a few other issues. I didn't include > > the testing function you proposed in your other email. It sounds > > useful for testing local

Re: Popcount optimization using AVX512

2024-04-05 Thread Ants Aasma
On Fri, 5 Apr 2024 at 07:15, Nathan Bossart wrote: > Here is an updated patch set. IMHO this is in decent shape and is > approaching committable. I checked the code generation on various gcc and clang versions. It looks mostly fine starting from versions where avx512 is supported, gcc-7.1 and cl

Re: Introduce XID age and inactive timeout based replication slot invalidation

2024-04-05 Thread Bertrand Drouvot
Hi, On Fri, Apr 05, 2024 at 11:21:43AM +0530, Bharath Rupireddy wrote: > On Wed, Apr 3, 2024 at 9:57 PM Bertrand Drouvot > wrote: > Please find the attached v36 patch. Thanks! A few comments: 1 === + +The timeout is measured from the time since the slot has become +inac

Re: Improve eviction algorithm in ReorderBuffer

2024-04-05 Thread Masahiko Sawada
On Fri, Apr 5, 2024 at 2:55 AM Jeff Davis wrote: > > On Thu, 2024-04-04 at 17:28 +0900, Masahiko Sawada wrote: > > > Perhaps it's not worth the effort though, if performance is already > > > good enough? > > > > Yeah, it would be better to measure the overhead first. I'll do that. > > I have some

Re: remaining sql/json patches

2024-04-05 Thread Alexander Lakhin
05.04.2024 10:09, Amit Langote wrote: Seems like it might be a pre-existing issue, because I can also reproduce the crash with: SELECT * FROM COALESCE(row(1)) AS (a int, b int); server closed the connection unexpectedly This probably means the server terminated abnormally before or while process

Re: remaining sql/json patches

2024-04-05 Thread Amit Langote
On Fri, Apr 5, 2024 at 5:00 PM Alexander Lakhin wrote: > 05.04.2024 10:09, Amit Langote wrote: > > Seems like it might be a pre-existing issue, because I can also > > reproduce the crash with: > > That's strange, because I get the error (on master, 6f132ed69). > With backtrace_functions = 'tuplede

Bad estimation for NOT IN clause with big null fraction

2024-04-05 Thread Donghang Lin
Hi hackers Discussion[1] and the relevant commit[2] improved the selectivity calculation for IN/NOT IN. This is the current logic for NOT IN selectivity calculation and it loops over the array elements. else { s1 = s1 * s2; if (isInequality) s1disjoint += s2 - 1.0; } By calcula

Re: LogwrtResult contended spinlock

2024-04-05 Thread Alvaro Herrera
On 2024-Apr-05, Bharath Rupireddy wrote: > 1. > /* > * Update local copy of shared XLogCtl->log{Write,Flush}Result > + * > + * It's critical that Flush always trails Write, so the order of the reads is > + * important, as is the barrier. > */ > #define RefreshXLogWriteResult(_target) \ >

Re: Looking for an index that supports top-n searches by enforcing a max-n automatically

2024-04-05 Thread Morris de Oryx
Just about as soon as I sent the above, I realized that it's unlikely to make sense in the real world in a row-store. If the goal is to keep the top-25 results and trim the rest, what happens when values are added/modified/deleted? You now *have to go look at all of the data you aren't caching in t

Recovery of .partial WAL segments

2024-04-05 Thread Stefan Fercot
Dear hackers, Generating a ".partial" WAL segment is pretty common nowadays (using pg_receivewal or during standby promotion). However, we currently don't do anything with it unless the user manually removes that ".partial" extension. The 028_pitr_timelines tests are highlighting that fact: wit

Re: Synchronizing slots from primary to standby

2024-04-05 Thread shveta malik
On Fri, Apr 5, 2024 at 10:09 AM Bertrand Drouvot wrote: > > What about something like? > > ereport(LOG, > errmsg("synchronized confirmed_flush_lsn for slot \"%s\" differs from > remote slot", > remote_slot->name), > errdetail("Remote slot has LSN %X/%X but local slot has L

Re: CSN snapshots in hot standby

2024-04-05 Thread Andrey M. Borodin
> On 5 Apr 2024, at 02:08, Kirill Reshke wrote: > > maybe we need some hooks here? Or maybe, we can take CSN here from extension > somehow. I really like the idea of CSN-provider-as-extension. But it's very important to move on with CSN, at least on standby, to make CSN actually happen some

Re: Another WaitEventSet resource leakage in back branches

2024-04-05 Thread Etsuro Fujita
On Fri, Mar 22, 2024 at 9:15 PM Etsuro Fujita wrote: > While working on [1], I noticed $SUBJECT: WaitLatchOrSocket in back > branches is ignoring the possibility of failing partway through, too. > I added a PG_FAINALLY block to that function, like commit 555276f85. > Patch attached. I noticed tha

Re: Synchronizing slots from primary to standby

2024-04-05 Thread Bertrand Drouvot
Hi, On Fri, Apr 05, 2024 at 04:09:01PM +0530, shveta malik wrote: > On Fri, Apr 5, 2024 at 10:09 AM Bertrand Drouvot > wrote: > > > > What about something like? > > > > ereport(LOG, > > errmsg("synchronized confirmed_flush_lsn for slot \"%s\" differs > > from remote slot", > > re

Re: Slow catchup of 2PC (twophase) transactions on replica in LR

2024-04-05 Thread Ajin Cherian
On Thu, Apr 4, 2024 at 4:38 PM Amit Kapila wrote: > > I think this would probably be better than the current situation but > can we think of a solution to allow toggling the value of two_phase > even when prepared transactions are present? Can you please summarize > the reason for the problems in

Re: Synchronizing slots from primary to standby

2024-04-05 Thread shveta malik
On Fri, Apr 5, 2024 at 4:31 PM Bertrand Drouvot wrote: > > BTW, I just realized that the LSN I used in my example in the > LSN_FORMAT_ARGS() > are not the right ones. Noted. Thanks. Please find v3 with the comments addressed. thanks Shveta v3-0001-Correct-sanity-check-to-compare-confirmed_ls

Re: Synchronizing slots from primary to standby

2024-04-05 Thread Amit Kapila
On Thu, Apr 4, 2024 at 2:59 PM shveta malik wrote: > > There is an intermittent BF failure observed at [1] after this commit > (2ec005b). > Thanks for analyzing and providing the patch. I'll look into it. There is another BF failure [1] which I have analyzed. The main reason for failure is the f

Re: LogwrtResult contended spinlock

2024-04-05 Thread Alvaro Herrera
Couldn't push: I tested with --disable-atomics --disable-spinlocks and the tests fail because the semaphore for the atomic variables is not always initialized. This is weird -- it's like a client process is running at a time when StartupXLOG has not initialized the variable ... so the initializati

Re: Add new protocol message to change GUCs for usage with future protocol-only GUCs

2024-04-05 Thread Dave Cramer
On Thu, 4 Apr 2024 at 12:45, Jelte Fennema-Nio wrote: > On Thu, 4 Apr 2024 at 14:50, Peter Eisentraut > wrote: > > It appears there are several different perspectives about this. My > > intuition was that a protocol version change indicates something that we > > eventually want all client libra

Re: meson vs windows perl

2024-04-05 Thread Andrew Dunstan
On 2024-04-02 Tu 09:34, Andrew Dunstan wrote: meson.build has this code     ldopts = run_command(perl, '-MExtUtils::Embed', '-e', 'ldopts', check: true).stdout().strip()     undesired = run_command(perl_conf_cmd, 'ccdlflags', check: true).stdout().split()     undesired += run_c

Re: remaining sql/json patches

2024-04-05 Thread Amit Langote
On Thu, Apr 4, 2024 at 9:02 PM Amit Langote wrote: > I'll post the rebased 0002 tomorrow after addressing your comments. Here's one. Main changes: * Fixed a bug in get_table_json_columns() which caused nested columns to be deparsed incorrectly, something Jian reported upthread. * Simplified the

Re: Synchronizing slots from primary to standby

2024-04-05 Thread Amit Kapila
On Fri, Apr 5, 2024 at 5:17 PM Amit Kapila wrote: > > On Thu, Apr 4, 2024 at 2:59 PM shveta malik wrote: > > > > There is an intermittent BF failure observed at [1] after this commit > > (2ec005b). > > > > Thanks for analyzing and providing the patch. I'll look into it. There > is another BF fai

Re: Add new protocol message to change GUCs for usage with future protocol-only GUCs

2024-04-05 Thread Robert Haas
On Thu, Apr 4, 2024 at 8:50 AM Peter Eisentraut wrote: > It appears there are several different perspectives about this. My > intuition was that a protocol version change indicates something that we > eventually want all client libraries to support. Whereas protocol > extensions are truly opt-in

Re: Allow non-superuser to cancel superuser tasks.

2024-04-05 Thread Nathan Bossart
On Fri, Apr 05, 2024 at 02:39:05PM +0900, Michael Paquier wrote: > + /* > + * If the backend is autovacuum worker, allow user with the privileges > of > + * pg_signal_autovacuum role to signal the backend. > + */ > + if (pgstat_get_backend_type(GetNumberFromPGProc(proc)) ==

Re: IPC::Run::time[r|out] vs our TAP tests

2024-04-05 Thread Dagfinn Ilmari Mannsåker
Tom Lane writes: > Erik Wienhold writes: >> Libedit 20191025-3.1 is the first version where ":{?VERB" works as >> expected. The previous release 20190324-3.1 still produces the escaped >> output that Michael found. That narrows down the changes to everything >> between [1] (changed on 2019-03-

Re: Fixing backslash dot for COPY FROM...CSV

2024-04-05 Thread Daniel Verite
Tom Lane wrote: > I've looked over this patch and I generally agree that this is a > reasonable solution. Thanks for reviewing this! > I'm also wondering why the patch adds a test for > "PQprotocolVersion(conn) >= 3" in handleCopyIn. I've removed this in the attached update. > I concu

Re: Popcount optimization using AVX512

2024-04-05 Thread Nathan Bossart
On Fri, Apr 05, 2024 at 10:33:27AM +0300, Ants Aasma wrote: > The main issue I saw was that clang was able to peel off the first > iteration of the loop and then eliminate the mask assignment and > replace masked load with a memory operand for vpopcnt. I was not able > to convince gcc to do that re

Re: broken JIT support on Fedora 40

2024-04-05 Thread Thomas Munro
On Sun, Mar 31, 2024 at 12:49 PM Thomas Munro wrote: > https://github.com/llvm/llvm-project/pull/87093 Oh, with those clues, I think I might see... It is a bit strange that we copy attributes from AttributeTemplate(), a function that returns Datum, to our void deform function. It works (I mean

Re: broken JIT support on Fedora 40

2024-04-05 Thread Dmitry Dolgov
> On Sat, Apr 06, 2024 at 02:00:38AM +1300, Thomas Munro wrote: > On Sun, Mar 31, 2024 at 12:49 PM Thomas Munro wrote: > > https://github.com/llvm/llvm-project/pull/87093 > > Oh, with those clues, I think I might see... It is a bit strange that > we copy attributes from AttributeTemplate(), a fun

Re: Security lessons from liblzma

2024-04-05 Thread Robert Haas
On Thu, Apr 4, 2024 at 4:48 PM Daniel Gustafsson wrote: > AFAIK we haven't historically enforced that installations have the openssl > binary in PATH, but it would be a pretty low bar to add. The bigger issue is > likely to find someone to port this to Windows, it probably won't be too hard > but

Re: Add bump memory context type and use it for tuplesorts

2024-04-05 Thread Matthias van de Meent
On Thu, 4 Apr 2024 at 22:43, Tom Lane wrote: > > Matthias van de Meent writes: > > On Mon, 25 Mar 2024 at 22:44, Tom Lane wrote: > >> Basically, I'm not happy with consuming the last reasonably-available > >> pattern for a memory context type that has little claim to being the > >> Last Context

Re: Speed up clean meson builds by ~25%

2024-04-05 Thread Jelte Fennema-Nio
On Fri, 5 Apr 2024 at 00:45, Jelte Fennema-Nio wrote: > It improved clean build times on my machine (10 cores/20 threads) from ~40 > seconds to ~30 seconds. After discussing this off-list with Bilal, I realized that this gain is only happening for clang builds on my system. Because those take a l

Re: broken JIT support on Fedora 40

2024-04-05 Thread Dmitry Dolgov
> On Fri, Apr 05, 2024 at 03:21:06PM +0200, Dmitry Dolgov wrote: > > On Sat, Apr 06, 2024 at 02:00:38AM +1300, Thomas Munro wrote: > > On Sun, Mar 31, 2024 at 12:49 PM Thomas Munro > > wrote: > > > https://github.com/llvm/llvm-project/pull/87093 > > > > Oh, with those clues, I think I might see..

Re: Add new protocol message to change GUCs for usage with future protocol-only GUCs

2024-04-05 Thread Robert Haas
On Thu, Apr 4, 2024 at 12:45 PM Jelte Fennema-Nio wrote: > Yeah, we definitely think differently here then. To me bumping the > minor protocol version shouldn't be a thing that we would need to > carefully consider. It should be easy to do, and probably done often. Often? I kind of hope that the

Re: Add new protocol message to change GUCs for usage with future protocol-only GUCs

2024-04-05 Thread Robert Haas
On Thu, Apr 4, 2024 at 1:10 PM Jelte Fennema-Nio wrote: > Attached is a rebased patchset We should keep talking about this, but I think we're far too close to the wire at this point to think about committing anything for v17 at this point. These are big changes, they haven't been thoroughly revie

Re: meson vs windows perl

2024-04-05 Thread Andrew Dunstan
On 2024-04-05 Fr 08:25, Andrew Dunstan wrote: Here is an attempt to fix all that. It's ugly, but I think it's more principled. First, instead of getting the ldopts and then trying to filter out the ldflags and ccdlflags, it tells perl not to include those in the first place, by overridi

Re: IPC::Run::time[r|out] vs our TAP tests

2024-04-05 Thread Andrew Dunstan
On 2024-04-04 Th 17:24, Tom Lane wrote: TIL that IPC::Run::timer is not the same as IPC::Run::timeout. With a timer object you have to check $timer->is_expired to see if the timeout has elapsed, but with a timeout object you don't because it will throw a Perl exception upon timing out, probably

Re: WIP Incremental JSON Parser

2024-04-05 Thread Andrew Dunstan
On 2024-04-04 Th 15:54, Nathan Bossart wrote: On Thu, Apr 04, 2024 at 03:31:12PM -0400, Andrew Dunstan wrote: Does the attached patch fix it for you? It clears up 2 of the 3 warnings for me: ../postgresql/src/common/jsonapi.c: In function ‘IsValidJsonNumber’: ../postgresql/src/common/jsonapi.

Re: Add bump memory context type and use it for tuplesorts

2024-04-05 Thread Tom Lane
Matthias van de Meent writes: > On Thu, 4 Apr 2024 at 22:43, Tom Lane wrote: >> The only objection I can think of is that perhaps this would slow >> things down a tad by requiring more complicated shifting/masking. >> I wonder if we could redo the performance checks that were done >> on the way t

Re: Synchronizing slots from primary to standby

2024-04-05 Thread Bertrand Drouvot
Hi, On Fri, Apr 05, 2024 at 06:23:10PM +0530, Amit Kapila wrote: > On Fri, Apr 5, 2024 at 5:17 PM Amit Kapila wrote: > Thinking more on this, it doesn't seem related to > c9920a9068eac2e6c8fb34988d18c0b42b9bf811 as that commit doesn't change > any locking or something like that which impacts writ

Re: Psql meta-command conninfo+

2024-04-05 Thread Peter Eisentraut
On 04.04.24 18:15, Maiquel Grassi wrote: Well, I can revert \conninfo to its original state and keep \conninfo+ as it is, perhaps removing the application name, as I believe everything else is important for a DBA/SysAdmin. Do you think we can proceed with the patch this way? I am open to ideas th

Re: documentation structure

2024-04-05 Thread Robert Haas
On Fri, Mar 29, 2024 at 9:40 AM Robert Haas wrote: > 2. Demote "Monitoring Disk Usage" from a chapter on its own to a > section of the "Monitoring Database Activity" chapter. I haven't seen > any objections to this, and I'd like to move ahead with it. > > 3. Merge the separate chapters on various

Re: Synchronizing slots from primary to standby

2024-04-05 Thread Bertrand Drouvot
Hi, On Fri, Apr 05, 2024 at 02:35:42PM +, Bertrand Drouvot wrote: > I think that maybe as a first step we should move the "elog(DEBUG2," message > as > proposed above to help debugging (that could help to confirm the above > theory). If you agree and think that makes sense, pleae find attac

Re: Speed up clean meson builds by ~25%

2024-04-05 Thread Andres Freund
Hi, On 2024-04-05 15:36:34 +0200, Jelte Fennema-Nio wrote: > On Fri, 5 Apr 2024 at 00:45, Jelte Fennema-Nio wrote: > > It improved clean build times on my machine (10 cores/20 threads) from ~40 > > seconds to ~30 seconds. > > After discussing this off-list with Bilal, I realized that this gain >

Re: Add new protocol message to change GUCs for usage with future protocol-only GUCs

2024-04-05 Thread Jelte Fennema-Nio
On Fri, 5 Apr 2024 at 16:04, Robert Haas wrote: > > On Thu, Apr 4, 2024 at 1:10 PM Jelte Fennema-Nio wrote: > > Attached is a rebased patchset > > We should keep talking about this, but I think we're far too close to > the wire at this point to think about committing anything for v17 at > this po

Re: Streaming read-ready sequential scan code

2024-04-05 Thread Melanie Plageman
On Thu, Apr 4, 2024 at 12:39 PM Andres Freund wrote: > > On 2024-04-04 22:37:39 +1300, Thomas Munro wrote: > > On Thu, Apr 4, 2024 at 10:31 PM Thomas Munro wrote: > > > Alright what about this? > > I think it's probably worth adding a bit more of the commit message to the > function comment. Yes,

Re: Popcount optimization using AVX512

2024-04-05 Thread Nathan Bossart
On Fri, Apr 05, 2024 at 07:58:44AM -0500, Nathan Bossart wrote: > On Fri, Apr 05, 2024 at 10:33:27AM +0300, Ants Aasma wrote: >> The main issue I saw was that clang was able to peel off the first >> iteration of the loop and then eliminate the mask assignment and >> replace masked load with a memor

Re: WIP Incremental JSON Parser

2024-04-05 Thread Nathan Bossart
On Fri, Apr 05, 2024 at 10:15:45AM -0400, Andrew Dunstan wrote: > On 2024-04-04 Th 15:54, Nathan Bossart wrote: >> On Thu, Apr 04, 2024 at 03:31:12PM -0400, Andrew Dunstan wrote: >> > Does the attached patch fix it for you? >> It clears up 2 of the 3 warnings for me: >> >> ../postgresql/src/common

Re: Table AM Interface Enhancements

2024-04-05 Thread Pavel Borisov
Hi, hackers! On Tue, 2 Apr 2024 at 19:17, Jeff Davis wrote: > On Tue, 2024-04-02 at 11:49 +0300, Alexander Korotkov wrote: > > I don't like the idea that every custom table AM reltoptions should > > begin with StdRdOptions. I would rather introduce the new data > > structure with table options,

Re: documentation structure

2024-04-05 Thread Robert Haas
On Mon, Mar 25, 2024 at 11:40 AM Peter Eisentraut wrote: > I think a possible problem we need to consider with these proposals to > combine chapters is that they could make the chapters themselves too > deep and harder to navigate. I looked into various options for further combining chapters and/

Re: broken JIT support on Fedora 40

2024-04-05 Thread Dmitry Dolgov
> On Fri, Apr 05, 2024 at 03:50:50PM +0200, Dmitry Dolgov wrote: > > On Fri, Apr 05, 2024 at 03:21:06PM +0200, Dmitry Dolgov wrote: > > > On Sat, Apr 06, 2024 at 02:00:38AM +1300, Thomas Munro wrote: > > > On Sun, Mar 31, 2024 at 12:49 PM Thomas Munro > > > wrote: > > > > https://github.com/llvm/

Re: Fixing backslash dot for COPY FROM...CSV

2024-04-05 Thread Tom Lane
"Daniel Verite" writes: > Tom Lane wrote: >> I've looked over this patch and I generally agree that this is a >> reasonable solution. > Thanks for reviewing this! While testing this, I tried running the tests with an updated server and non-updated psql, and not only did the new test case f

Re: Psql meta-command conninfo+

2024-04-05 Thread Imseih (AWS), Sami
> The original \conninfo was designed to report values from the libpq API > about what libpq connected to. And the convention in psql is that "+" > provide more or less the same information but a bit more. So I think it > is wrong to make "\conninfo+" something fundamentally different than > "\conn

Re: Add new protocol message to change GUCs for usage with future protocol-only GUCs

2024-04-05 Thread Jelte Fennema-Nio
On Fri, 5 Apr 2024 at 16:02, Robert Haas wrote: > Often? > > I kind of hope that the protocol starts to evolve a bit more than it > has, but I don't want a continuous stream of changes. That will be > very hard to test and verify correctness, and a hassle for drivers to > keep up with, and a mess

Re: AIX support

2024-04-05 Thread Sriram RK
> What you do need to do to reproduce the described problems is > check out the Postgres git tree and rewind to just before > commit 0b16bb877, where we deleted AIX support. Any attempt > to restore AIX support would have to start with reverting that > commit (and perhaps the followup f0827b443).

Re: documentation structure

2024-04-05 Thread David G. Johnston
On Fri, Apr 5, 2024 at 9:01 AM Robert Haas wrote: > > > The rendering can be adjusted to some degree, but then we also need to > > make sure any new chunking makes sense in other chapters. (And it might > > also change a bunch of externally known HTML links.) > > I looked into this and I'm uncle

Re: documentation structure

2024-04-05 Thread Robert Haas
On Fri, Apr 5, 2024 at 12:15 PM David G. Johnston wrote: > Here is a link to my attempt at this a couple of years ago. It basically > "abuses" refentry. > > https://www.postgresql.org/message-id/CAKFQuwaVm%3D6d_sw9Wrp4cdSm5_k%3D8ZVx0--v2v4BH4KnJtqXqg%40mail.gmail.com > > I never did dive into th

Re: Speed up clean meson builds by ~25%

2024-04-05 Thread Jelte Fennema-Nio
On Fri, 5 Apr 2024 at 17:24, Andres Freund wrote: > I recommend opening a bug report for clang, best with an already preprocessed > input file. > We're going to need to do something about this from our side as well, I > suspect. The times aren't great with gcc either, even if not as bad as with >

Re: documentation structure

2024-04-05 Thread David G. Johnston
On Fri, Apr 5, 2024 at 9:18 AM Robert Haas wrote: > On Fri, Apr 5, 2024 at 12:15 PM David G. Johnston > wrote: > > Here is a link to my attempt at this a couple of years ago. It > basically "abuses" refentry. > > > > > https://www.postgresql.org/message-id/CAKFQuwaVm%3D6d_sw9Wrp4cdSm5_k%3D8ZVx0

Re: Fixing backslash dot for COPY FROM...CSV

2024-04-05 Thread Tom Lane
I wrote: > So this means that the patch introduces a rather serious cross-version > compatibility problem. I doubt we can consider inlined CSV data to be > a niche case that few people use; but it will fail every time if your > psql is older than your server. On third thought, that may not be as

Re: Add new protocol message to change GUCs for usage with future protocol-only GUCs

2024-04-05 Thread Dave Cramer
On Fri, 5 Apr 2024 at 12:09, Jelte Fennema-Nio wrote: > On Fri, 5 Apr 2024 at 16:02, Robert Haas wrote: > > Often? > > > > I kind of hope that the protocol starts to evolve a bit more than it > > has, but I don't want a continuous stream of changes. That will be > > very hard to test and verify

Re: Cutting support for OpenSSL 1.0.1 and 1.0.2 in 17~?

2024-04-05 Thread Jacob Champion
On Thu, Apr 4, 2024 at 6:37 PM Michael Paquier wrote: > From where did you pull the LibreSSL sources? Directly from the > OpenBSD tree? I've been building LibreSSL Portable: https://github.com/libressl/portable > Ah, right. OpenSSL_add_all_algorithms() is documented as having no > effect in 1.

Re: Add new protocol message to change GUCs for usage with future protocol-only GUCs

2024-04-05 Thread Tom Lane
Dave Cramer writes: > On Fri, 5 Apr 2024 at 12:09, Jelte Fennema-Nio wrote: >> Setting PGC_PROTOCOL gucs would be allowed in the startup packet, >> which is fine afaict because that's also something that's part of the >> protocol level and is thus fully controlled by client libraries and >> poole

Re: Add new protocol message to change GUCs for usage with future protocol-only GUCs

2024-04-05 Thread Robert Haas
On Fri, Apr 5, 2024 at 12:09 PM Jelte Fennema-Nio wrote: > But I do think that we shouldn't be at all averse to the act of > bumping the protocol version itself. If we have a single small > protocol change in one release, then imho it's no problem to bump the > protocol version. Bumping the versio

Re: Add new protocol message to change GUCs for usage with future protocol-only GUCs

2024-04-05 Thread Jelte Fennema-Nio
On Fri, 5 Apr 2024 at 18:30, Dave Cramer wrote: >> > I really intended the _pq_ prefix as a way of taking something out of >> > the GUC namespace, not as a part of the GUC namespace that users would >> > see. And I'm reluctant to go back on that. If we want to make >> > pg_protocol.${NAME} mean a

Re: Fixing backslash dot for COPY FROM...CSV

2024-04-05 Thread Daniel Verite
Tom Lane wrote: > Not sure what to do here. One idea is to install just the psql-side > fix, which should break nothing now that version-2 protocol is dead, > and then wait a few years before introducing the server-side change. > That seems kind of sad though. Wouldn't backpatching solve

Re: Add new protocol message to change GUCs for usage with future protocol-only GUCs

2024-04-05 Thread Jelte Fennema-Nio
On Fri, 5 Apr 2024 at 18:43, Tom Lane wrote: > I don't buy that argument, actually. libpq, and pretty much every > other client AFAIK, has provisions to let higher code levels insert > random options into the startup packet. So to make this work libpq > would have to filter or at least inspect s

Re: Cutting support for OpenSSL 1.0.1 and 1.0.2 in 17~?

2024-04-05 Thread Daniel Gustafsson
> On 5 Apr 2024, at 03:37, Michael Paquier wrote: > On Thu, Apr 04, 2024 at 11:03:35AM -0700, Jacob Champion wrote: >> v3 does that by putting back checks for symbols that aren't part of >> LibreSSL (tested back to 2.7, which is where the 1.1.x APIs started to >> arrive). > > From where did you

Re: Cutting support for OpenSSL 1.0.1 and 1.0.2 in 17~?

2024-04-05 Thread Daniel Gustafsson
> On 5 Apr 2024, at 18:41, Jacob Champion > wrote: > On Thu, Apr 4, 2024 at 6:37 PM Michael Paquier wrote: >> I would be OK to draw a line to what we test in the buildfarm if it >> comes to that, down to OpenBSD 6.9. > > That would correspond to LibreSSL 3.3 if I'm not mistaken. Any > particul

Re: AIX support

2024-04-05 Thread Noah Misch
On Fri, Apr 05, 2024 at 04:12:06PM +, Sriram RK wrote: > > > What you do need to do to reproduce the described problems is > > check out the Postgres git tree and rewind to just before > > commit 0b16bb877, where we deleted AIX support. Any attempt > > to restore AIX support would have to sta

Re: Can't compile PG 17 (master) from git under Msys2 autoconf

2024-04-05 Thread 'Alvaro Herrera'
On 2024-Apr-04, Regina Obe wrote: > I think I got something not too far off from what's there now that works > under my msys2 setup again. This is partly using your idea of using > $(top_builddir) to qualify the path but in the LN_S section that is causing > me grief. > This seems to work okay

Re: incremental backup breakage in BlockRefTableEntryGetBlocks

2024-04-05 Thread Robert Haas
On Fri, Apr 5, 2024 at 2:59 AM Jakub Wartak wrote: > And of course i'm attaching reproducer with some braindump notes in > case in future one hits similiar issue and wonders where to even start > looking (it's very primitive though but might help). Thanks. I've committed the patch now. -- Rober

Re: LogwrtResult contended spinlock

2024-04-05 Thread Alvaro Herrera
Pushed 0001. Here's the patch that adds the Copy position one more time, with the monotonic_advance function returning the value. -- Álvaro Herrera PostgreSQL Developer — https://www.EnterpriseDB.com/ >From 3f5c860576245b92701e7bfc517947c418c68510 Mon Sep 17 00:00:00 2001 From: Alvaro

Re: Streaming read-ready sequential scan code

2024-04-05 Thread Melanie Plageman
On Fri, Apr 5, 2024 at 12:15 AM Thomas Munro wrote: > > Yeah, I plead benchmarking myopia, sorry. The fastpath as committed > is only reached when distance goes 2->1, as pg_prewarm does. Oops. > With the attached minor rearrangement, it works fine. I also poked > some more at that memory prefet

RE: Can't compile PG 17 (master) from git under Msys2 autoconf

2024-04-05 Thread Regina Obe
> > I think I got something not too far off from what's there now that works > under my msys2 setup again. This is partly using your idea of using > $(top_builddir) to qualify the path but in the LN_S section that is causing me > grief. > > This seems to work okay building in tree and out of tree.

Re: Add new protocol message to change GUCs for usage with future protocol-only GUCs

2024-04-05 Thread Dave Cramer
> > > Plus, you've got all of the consequences for non-core drivers, which > have to both add support for the new wire protocol - if they don't > want to seem outdated and eventually obsolete - and also test that > they're still compatible with all supported server versions. > Connection poolers ha

Re: Can't compile PG 17 (master) from git under Msys2 autoconf

2024-04-05 Thread 'Alvaro Herrera'
On 2024-Apr-05, Regina Obe wrote: > I think it ends up doing a copy thus the copy error in my log failures which > don't exist anywhere in the Makefil > > cp -pR ../../backend/storage/lmgr/lwlocknames.h > > Sorry for not checking on a linux system. I was thinking I should have done > that fir

Re: [HACKERS] make async slave to wait for lsn to be replayed

2024-04-05 Thread Alvaro Herrera
Hello, BTW I noticed that https://coverage.postgresql.org/src/backend/commands/waitlsn.c.gcov.html says that lsn_cmp is not covered by the tests. This probably indicates that the tests are a little too light, but I'm not sure how much extra effort we want to spend. I'm still concerned that Wait

Re: LogwrtResult contended spinlock

2024-04-05 Thread Jeff Davis
On Fri, 2024-04-05 at 13:54 +0200, Alvaro Herrera wrote: > Couldn't push: I tested with --disable-atomics --disable-spinlocks > and > the tests fail because the semaphore for the atomic variables is not > always initialized.  This is weird -- it's like a client process is > running at a time when S

Re: Security lessons from liblzma

2024-04-05 Thread Jacob Champion
On Fri, Apr 5, 2024 at 6:24 AM Robert Haas wrote: > I wonder how hard it would be to just code up our own binary to do > this. If it'd be a pain to do that, or to maintain it across SSL > versions, then it's a bad plan and we shouldn't do it. But if it's not > that much code, maybe it'd be worth c

Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

2024-04-05 Thread stephane tachoires
The following review has been posted through the commitfest application: make installcheck-world: tested, passed Implements feature: tested, passed Spec compliant: tested, passed Documentation:tested, passed All three patches applied nivcely. Code fits standart, commen

Re: Fixing backslash dot for COPY FROM...CSV

2024-04-05 Thread Tom Lane
"Daniel Verite" writes: > Tom Lane wrote: >> Not sure what to do here. One idea is to install just the psql-side >> fix, which should break nothing now that version-2 protocol is dead, >> and then wait a few years before introducing the server-side change. >> That seems kind of sad though.

Obsolete comment in CopyReadLineText()

2024-04-05 Thread Tom Lane
CopyReadLineText quoth: * The objective of this loop is to transfer the entire next input line * into line_buf. Hence, we only care for detecting newlines (\r and/or * \n) and the end-of-copy marker (\.). * * In CSV mode, \r and \n inside a quoted field are just part of t

Re: WIP Incremental JSON Parser

2024-04-05 Thread Andrew Dunstan
On 2024-04-05 Fr 11:43, Nathan Bossart wrote: On Fri, Apr 05, 2024 at 10:15:45AM -0400, Andrew Dunstan wrote: On 2024-04-04 Th 15:54, Nathan Bossart wrote: On Thu, Apr 04, 2024 at 03:31:12PM -0400, Andrew Dunstan wrote: Does the attached patch fix it for you? It clears up 2 of the 3 warning

Re: meson vs windows perl

2024-04-05 Thread Andrew Dunstan
On 2024-04-05 Fr 10:12, Andrew Dunstan wrote: On 2024-04-05 Fr 08:25, Andrew Dunstan wrote: Here is an attempt to fix all that. It's ugly, but I think it's more principled. First, instead of getting the ldopts and then trying to filter out the ldflags and ccdlflags, it tells perl not to

Re: Fixing backslash dot for COPY FROM...CSV

2024-04-05 Thread Tom Lane
After some more poking at this topic, I realize that there is already very strange and undocumented behavior for backslash-dot even in non-CSV mode. Create a file like this: $ cat eofdata foobar foobaz\. more \. yet more and try importing it with COPY: regression=# create table eofdata(f1 text)

Re: Improve eviction algorithm in ReorderBuffer

2024-04-05 Thread Jeff Davis
On Fri, 2024-04-05 at 16:58 +0900, Masahiko Sawada wrote: > IIUC for example in ReorderBuffer, the heap key is transaction size > and the hash key is xid. Yes. > > I see your point. It assumes that the bh_node_type is a pointer or at > least unique. So it cannot work with Datum being a value. R

fasthash32() returning uint64?

2024-04-05 Thread Jeff Davis
In hashfn_unstable.h, fasthash32() is declared as: /* like fasthash64, but returns a 32-bit hashcode */ static inline uint64 fasthash32(const char *k, size_t len, uint64 seed) Is the return type of uint64 a typo? Regards, Jeff Davis

Re: Cutting support for OpenSSL 1.0.1 and 1.0.2 in 17~?

2024-04-05 Thread Jacob Champion
On Fri, Apr 5, 2024 at 9:59 AM Daniel Gustafsson wrote: > Attached is a WIP patch to get more eyes on it, the Meson test for 1.1.1 fails > on Windows in CI which I will investigate next. The changes for SSL_OP_NO_CLIENT_RENEGOTIATION and SSL_R_VERSION_TOO_LOW look good to me. > -Remove s

Re: IPC::Run::time[r|out] vs our TAP tests

2024-04-05 Thread Tom Lane
=?utf-8?Q?Dagfinn_Ilmari_Manns=C3=A5ker?= writes: > Tom Lane writes: >> So it seems like the bug does not exist in any currently-supported >> NetBSD release. Debian has been known to ship obsolete libedit >> versions, though. > Both the current (bokworm/12) and previous (bullseye/11) versions o

Re: Cutting support for OpenSSL 1.0.1 and 1.0.2 in 17~?

2024-04-05 Thread Peter Eisentraut
On 05.04.24 18:59, Daniel Gustafsson wrote: Attached is a WIP patch to get more eyes on it, the Meson test for 1.1.1 fails on Windows in CI which I will investigate next. I'm not a fan of the new PGAC_CHECK_OPENSSL. It creates a second place where the OpenSSL version number has to be updated.

Re: Cutting support for OpenSSL 1.0.1 and 1.0.2 in 17~?

2024-04-05 Thread Daniel Gustafsson
> On 5 Apr 2024, at 23:26, Peter Eisentraut wrote: > > On 05.04.24 18:59, Daniel Gustafsson wrote: >> Attached is a WIP patch to get more eyes on it, the Meson test for 1.1.1 >> fails >> on Windows in CI which I will investigate next. > > I'm not a fan of the new PGAC_CHECK_OPENSSL. It creates

Re: Cutting support for OpenSSL 1.0.1 and 1.0.2 in 17~?

2024-04-05 Thread Jacob Champion
On Fri, Apr 5, 2024 at 2:48 PM Daniel Gustafsson wrote: > But does that actually work? If I change the API_COMPAT to the 1.1.1 version > number and run configure against 1.0.2 it passes just fine. Am I missing some > clever trick here? Similarly, I changed my API_COMPAT to a nonsense 0x9010

Re: Add last_commit_lsn to pg_stat_database

2024-04-05 Thread Cary Huang
The following review has been posted through the commitfest application: make installcheck-world: tested, passed Implements feature: tested, passed Spec compliant: not tested Documentation:not tested Hello I think it is convenient to know the last commit LSN of a data

Re: Cutting support for OpenSSL 1.0.1 and 1.0.2 in 17~?

2024-04-05 Thread Daniel Gustafsson
> On 5 Apr 2024, at 22:55, Jacob Champion > wrote: > > On Fri, Apr 5, 2024 at 9:59 AM Daniel Gustafsson wrote: >> Attached is a WIP patch to get more eyes on it, the Meson test for 1.1.1 >> fails >> on Windows in CI which I will investigate next. The attached version fixes the Windows 1.1.1 c

RE: Can't compile PG 17 (master) from git under Msys2 autoconf

2024-04-05 Thread Regina Obe
> > I think it ends up doing a copy thus the copy error in my log failures > > which don't exist anywhere in the Makefil > > > > cp -pR ../../backend/storage/lmgr/lwlocknames.h > > > > Sorry for not checking on a linux system. I was thinking I should have done > that first. > > Ah yeah, that's pe

Re: Popcount optimization using AVX512

2024-04-05 Thread David Rowley
On Sat, 6 Apr 2024 at 04:38, Nathan Bossart wrote: > This seems to provide a small performance boost, so I've incorporated it > into v27. Won't Valgrind complain about this? +pg_popcount_avx512(const char *buf, int bytes) + buf = (const char *) TYPEALIGN_DOWN(sizeof(__m512i), buf); + val = _mm

DROP OWNED BY fails to clean out pg_init_privs grants

2024-04-05 Thread Tom Lane
I wondered why buildfarm member copperhead has started to fail xversion-upgrade-HEAD-HEAD tests. I soon reproduced the problem here: pg_restore: creating ACL "regress_pg_dump_schema.TYPE "test_type"" pg_restore: while PROCESSING TOC: pg_restore: from TOC entry 4355; 0 0 ACL TYPE "test_type" build

Re: Streaming read-ready sequential scan code

2024-04-05 Thread Thomas Munro
On Sat, Apr 6, 2024 at 6:55 AM Melanie Plageman wrote: > On Fri, Apr 5, 2024 at 12:15 AM Thomas Munro wrote: > > The interesting column is hot. The 200ms->211ms regression is due to > > the extra bookkeeping in the slow path. The rejiggered fastpath code > > fixes it for me, or maybe sometimes

Re: Is this a problem in GenericXLogFinish()?

2024-04-05 Thread Michael Paquier
On Fri, Apr 05, 2024 at 01:26:29PM +0900, Michael Paquier wrote: > On Wed, Feb 07, 2024 at 06:42:21PM +0900, Michael Paquier wrote: >> On Wed, Feb 07, 2024 at 02:08:42PM +0530, Amit Kapila wrote: >> > Thanks for the report and looking into it. Pushed! >> >> Thanks Amit for the commit and Kuroda-sa

  1   2   >