Re: Log connection establishment timings

2025-03-06 Thread Fujii Masao
On 2025/03/07 8:16, Melanie Plageman wrote: On Thu, Mar 6, 2025 at 2:10 PM Andres Freund wrote: I think it'd be better to use absolute times and store them as such in ConnectionTimes or whatever. That way we have information about when a connection was established for some future SQL functi

Re: Parallel heap vacuum

2025-03-06 Thread Peter Smith
Hi Sawada-San. Here are some review comments for patch v10-0002. == src/backend/access/heap/heapam_handler.c 1. .scan_bitmap_next_block = heapam_scan_bitmap_next_block, .scan_bitmap_next_tuple = heapam_scan_bitmap_next_tuple, .scan_sample_next_block = heapam_scan_sample_next_block, - .s

Re: Doc fix of aggressive vacuum threshold for multixact members storage

2025-03-06 Thread John Naylor
On Wed, Mar 5, 2025 at 12:06 PM Alex Friedman wrote: > > Good points, thank you. I'm good with going ahead as you've suggested. Pushed, thanks for the patch! -- John Naylor Amazon Web Services

Re: Logical replication timeout

2025-03-06 Thread RECHTÉ Marc
Hayato Kuroda kindly rebased the patch. v2-0001-WIP-track-wal-segments.patch Description: application/mbox

Trigger more frequent autovacuums

2025-03-06 Thread wenhui qiu
HI Nathan Bossart Melanie Plageman Firstly, congratulations on the submission of this path: https://commitfest.postgresql.org/patch/5320/ vacthresh = (float4) vac_base_thresh + vac_scale_factor * reltuples; anlthresh = (float4) anl_base_thresh + anl_scale_factor * reltuples; vacinsthresh = (float

Re: Removing unneeded self joins

2025-03-06 Thread Andrei Lepikhov
On 26/2/2025 13:14, Alexander Korotkov wrote: On Mon, Feb 24, 2025 at 2:22 PM Andrei Lepikhov wrote: On 24/2/2025 11:57, Alexander Korotkov wrote: Could you, please, elaborate more on what you mean by "new technique of query tree reduction"? I mean any transformations and optimisations that r

Re: [PoC] Federated Authn/z with OAUTHBEARER

2025-03-06 Thread Jacob Champion
On Thu, Mar 6, 2025 at 12:57 PM Jacob Champion wrote: > Problem 1 is a simple patch. I am working on a fix for Problem 2, but > I got stuck trying to get a "perfect" solution working yesterday... > Since this is a partial(?) blocker for getting NetBSD going, I'm going > to pivot to an ugly-but-sim

Re: Parallel heap vacuum

2025-03-06 Thread Peter Smith
Hi Sawada-San. FYI. I am observing the following test behaviour: I apply patch v10-0001, do a clean rebuild and run 'make check', and all tests are OK. Then, after I apply just patch v10-0002 on top of 0001, do a clean rebuild and run 'make check' there are many test fails. == Kind Regards,

Re: [PoC] Federated Authn/z with OAUTHBEARER

2025-03-06 Thread Thomas Munro
On Fri, Mar 7, 2025 at 9:57 AM Jacob Champion wrote: > 2) macOS's EVFILT_TIMER implementation seems to be different from the > other BSDs. On Mac, when you re-add a timer to a kqueue, any existing > timer-fired events for it are not cleared out and the kqueue might > remain readable. This breaks a

Re: Add column name to error description

2025-03-06 Thread Tom Lane
Erik Wienhold writes: > On 2025-03-07 04:05 +0100, Tom Lane wrote: >> I think the idea of the original coding was to keep those values in >> registers in the inner loop rather than re-fetching them each time. > Could be. But the main reason was to hold the output column type as the > inner loop

Re: Change COPY ... ON_ERROR ignore to ON_ERROR ignore_row

2025-03-06 Thread jian he
hi. rebase only. From ce0ce6438094cad553e509db65b7fd27de2b9af6 Mon Sep 17 00:00:00 2001 From: jian he Date: Fri, 7 Mar 2025 11:43:51 +0800 Subject: [PATCH v12 1/1] COPY (on_error set_to_null) Extent "on_error action", introduce new option: on_error set_to_null. Current grammar makes us unable to

Re: [PATCH] SVE popcount support

2025-03-06 Thread chiranmoy.bhattacha...@fujitsu.com
> Interesting. I do see different assembly with the 2 and 4 register > versions, but I didn't get to testing it on a machine with SVE support > today. > Besides some additional benchmarking, I might make some small adjustments > to the patch. But overall, it seems to be in decent shape. Sounds

Re: Statistics Import and Export

2025-03-06 Thread Tom Lane
Andres Freund writes: >And in contrast to analyzing the database in parallel, the pg_dump/restore >work to restore stats afaict happens single-threaded for each database. In principle we should be able to do stats dump/restore parallelized just as we do for data. There are some stumbling

Re: Commitfest app release on Feb 17 with many improvements

2025-03-06 Thread Tom Lane
Jelte Fennema-Nio writes: > Okay, I went for the approach of just trying everything until one > works. Starting with "git am", then patch(1), and as a final attempt > "git apply". +1, thanks! regards, tom lane

Re: Add column name to error description

2025-03-06 Thread Tom Lane
Erik Wienhold writes: > But I don't see the point in keeping variables atttypid and atttypmod > around when those values are now available via outatt. Removing these > two variables makes the code easier to read IMO. Done so in the > attached v4. I think the idea of the original coding was to k

Re: Statistics Import and Export

2025-03-06 Thread Corey Huinker
> > > Patch attached. This patch does NOT change the default; stats are still > opt-out. But it makes it easier for users to start specifying what they > want or not explicitly, or to rely on the defaults if they prefer. > > Note that the patch would mean we go from 2 options in v17: > --{schema|

Re: Statistics Import and Export

2025-03-06 Thread Andres Freund
Hi, On 2025-03-06 10:07:43 -0800, Jeff Davis wrote: > On Thu, 2025-03-06 at 12:16 -0500, Andres Freund wrote: > > I don't follow. We already have the tablenames, schemanames and oids > > of the > > to-be-dumped tables/indexes collected in pg_dump, all that's needed > > is to send > > a list of tho

Re: Query ID Calculation Fix for DISTINCT / ORDER BY and LIMIT / OFFSET

2025-03-06 Thread Sami Imseih
Hi, It seems like there are multiple threads on this topic. This is the original [0], but I suggest continuing the discussion in this thread since it includes the examples and patches. Regarding the issue itself, query jumbling behavior is often subjective, making it difficult to classify as a bu

Re: Statistics Import and Export

2025-03-06 Thread Jeff Davis
On Thu, 2025-03-06 at 11:15 -0500, Robert Haas wrote: > To be honest, I am a bit surprised that we decided to enable this by > default. It's not obvious to me that statistics should be regarded as > part of the database in the same way that table definitions or table > data are. That said, I'm not

Re: Add column name to error description

2025-03-06 Thread Erik Wienhold
On 2025-03-06 21:56 +0100, Tom Lane wrote: > jian he writes: > > if we print out NameStr(att->attname) then error becomes: > > +DETAIL: Returned type unknown does not match expected type character > > varying in column "f2" (position 2). > > > In this case, printing out {column \"%s\"} is not h

Re: Parallel heap vacuum

2025-03-06 Thread Peter Smith
Some minor review comments for patch v10-0001. == src/include/access/tableam.h 1. struct IndexInfo; +struct ParallelVacuumState; +struct ParallelContext; +struct ParallelWorkerContext; struct SampleScanState; Use alphabetical order for consistency with existing code. ~~~ 2. + /* + * Esti

Re: Add Pipelining support in psql

2025-03-06 Thread Jelte Fennema-Nio
On Tue, 25 Feb 2025 at 02:11, Michael Paquier wrote: > Initial digestion has gone well. One thing I've noticed is that \startpipeline throws warnings when copy pasting multiple lines. It seems to still execute everything as expected though. As an example you can copy paste this tiny script: \sta

Re: what's going on with lapwing?

2025-03-06 Thread Julien Rouhaud
On Thu, Mar 06, 2025 at 06:25:29PM -0500, Robert Haas wrote: > On Thu, Mar 6, 2025 at 5:12 PM Julien Rouhaud wrote:\ > > I indeed don't want to keep lapwing up unless there is any value. Note > > that it > > started to fail on 2 branches after the last buildfarm client update for > > reasons I d

Re: log_min_messages per backend type

2025-03-06 Thread Euler Taveira
On Wed, Mar 5, 2025, at 1:40 PM, Andrew Dunstan wrote: > Just bikeshedding a bit ... > > I'm not mad keen on this design. I think the value should be either a single > setting like "WARNING" or a set of type:setting pairs. I agree that "all" is > a bad name, but I think "default" would make sens

Re: explain analyze rows=%.0f

2025-03-06 Thread Robert Haas
On Thu, Mar 6, 2025 at 4:18 PM Alena Rybakina wrote: > To be honest, I initially took it as the total number of tuples and > couldn't figure out for myself how to interpret the result - 0 tuples or > 1 tuple in the end. Maybe it wasn't quite correct to perceive it that > way, but Matthias's explan

Re: Statistics Import and Export

2025-03-06 Thread Andres Freund
Hi, On 2025-03-06 14:51:26 -0500, Tom Lane wrote: > Andres Freund writes: > > On 2025-03-06 13:47:34 -0500, Tom Lane wrote: > >> ... I wonder if we could just rip out pg_upgrade's support > >> for DB-level parallelism, which is not terribly pretty anyway, and > >> simply pass the -j switch straig

Re: Log connection establishment timings

2025-03-06 Thread Melanie Plageman
On Thu, Mar 6, 2025 at 2:10 PM Andres Freund wrote: > > I think it'd be better to use absolute times and store them as such in > ConnectionTimes or whatever. That way we have information about when a > connection was established for some future SQL functions and for debugging > problems. Attached

Re: Commitfest app release on Feb 17 with many improvements

2025-03-06 Thread Jelte Fennema-Nio
On Thu, 6 Mar 2025 at 18:39, Jelte Fennema-Nio wrote: > > On Thu, 6 Mar 2025 at 18:10, Tom Lane wrote: > > Please see if you can make it use patch(1). IME git is too > > stiff-necked about slightly stale patches no matter which > > subcommand you use. > > It was using patch(1) in the past for th

Re: [PATCH] pg_stat_activity: make slow/hanging authentication more visible

2025-03-06 Thread Jacob Champion
On Thu, Mar 6, 2025 at 3:15 PM Michael Paquier wrote: > I have applied the simplest patch for now, to silence the failures in > the CI, and included your suggestion to add a check on the > backend_type for the extra safety it offers. Thanks! Initial CI run looks green, so that's a good start. I'

Re: Statistics Import and Export

2025-03-06 Thread Robert Haas
On Thu, Mar 6, 2025 at 3:50 PM Nathan Bossart wrote: > That being said, I do think in-database parallelism would be useful in some > cases. I frequently hear about problems with huge numbers of large objects > on a cluster with one big database. But that's probably less likely than > the many da

Re: what's going on with lapwing?

2025-03-06 Thread Robert Haas
On Thu, Mar 6, 2025 at 5:12 PM Julien Rouhaud wrote:\ > I indeed don't want to keep lapwing up unless there is any value. Note that > it > started to fail on 2 branches after the last buildfarm client update for > reasons I don't understand. Since everyone is complaining about lapwing > already

Re: [PATCH] pg_stat_activity: make slow/hanging authentication more visible

2025-03-06 Thread Michael Paquier
On Thu, Mar 06, 2025 at 02:25:07PM -0800, Jacob Champion wrote: > On Wed, Mar 5, 2025 at 8:08 PM Michael Paquier wrote: >> + WHERE state = 'starting' and wait_event = 'init-pre-auth';}); > > Did you have thoughts on expanding the check to backend_type [1]? > >> + # Give up. The output of the l

strange valgrind reports about wrapper_handler on 64-bit arm

2025-03-06 Thread Tomas Vondra
Hi, while running check-world on 64-bit arm (rpi5 with Debian 12.9), I got a couple reports like this: ==64550== Use of uninitialised value of size 8 ==64550==at 0xA62FE0: wrapper_handler (pqsignal.c:107) ==64550==by 0x580BB9E7: ??? (in /usr/libexec/valgrind/memcheck-arm64-linux) ==64550=

Re: Separate GUC for replication origins

2025-03-06 Thread Amit Kapila
On Wed, Mar 5, 2025 at 4:38 PM Peter Eisentraut wrote: > > On 11.02.25 21:25, Euler Taveira wrote: > > Here is another patch that only changes the GUC name to > > max_replication_origin_sessions. > > I think the naming and description of this is still confusing. > ... ... > > I agree that the orig

Re: ZStandard (with dictionaries) compression support for TOAST compression

2025-03-06 Thread Nikhil Kumar Veldanda
Hi Tom, On Thu, Mar 6, 2025 at 11:33 AM Tom Lane wrote: > > Robert Haas writes: > > On Thu, Mar 6, 2025 at 12:43 AM Nikhil Kumar Veldanda > > wrote: > >> Notably, this is the first compression algorithm for Postgres that can > >> make use of a dictionary to provide higher levels of compression

Re: what's going on with lapwing?

2025-03-06 Thread Julien Rouhaud
On Thu, Mar 06, 2025 at 02:28:20PM -0500, Andres Freund wrote: > Hi, > > On 2025-03-06 14:13:40 -0500, Tom Lane wrote: > > Robert Haas writes: > > > On Thu, Mar 6, 2025 at 1:07 PM Tom Lane wrote: > > >> Maybe invent a build-farm.conf option like "newest_branch_to_build"? > > > > > Yes, that would

Re: Statistics Import and Export

2025-03-06 Thread Nathan Bossart
On Thu, Mar 06, 2025 at 01:04:55PM -0500, Andres Freund wrote: > To be clear, I think this is a very important improvement that most people > should use. +1 > I just don't think it's quite there yet. I agree that we should continue working on the performance/memory stuff. > 1) It's a difference

Re: PoC. The saving of the compiled jit-code in the plan cache

2025-03-06 Thread Matheus Alcantara
Hi, I've spent some time learning more about jit in the last few weeks and I think that this patch could be very useful, thanks for working on this! I'm new on this subject but I would like to share some thoughts about it. > 1. Changes in jit-code generation. > > a) the load of the absolute addre

Re: what's going on with lapwing?

2025-03-06 Thread Robert Haas
On Thu, Mar 6, 2025 at 4:54 PM Melanie Plageman wrote: > One thing I've been wishing for recently is access to the discussion > and lore around individual buildfarm animals in a consolidated place. > As a new committer, I haven't been part of all of these discussions > over the last N years and so

Re: dblink: Add SCRAM pass-through authentication

2025-03-06 Thread Jacob Champion
On Thu, Mar 6, 2025 at 12:33 PM Peter Eisentraut wrote: > AFAICT, in pgfdw_security_check(), if SCRAM has been used for the > outgoing server connection, then PQconnectionUsedPassword() is true, and > then this check should fail if no "password" parameter was given. That > check should be expande

Trivial comment fix for tsquerysend()

2025-03-06 Thread Emre Hasegeli
Patch is attached. 0001-Fix-comment-of-tsquerysend.patch Description: Binary data

Re: Add contrib/pg_logicalsnapinspect

2025-03-06 Thread Masahiko Sawada
On Wed, Mar 5, 2025 at 4:05 AM Bertrand Drouvot wrote: > > Hi, > > On Wed, Mar 05, 2025 at 02:42:15PM +0530, Amit Kapila wrote: > > On Wed, Mar 5, 2025 at 12:47 PM Bertrand Drouvot > > wrote: > > > > > > Agree, PFA a patch doing so. > > > > > > > It would be better if you could add a few comments

Re: ZStandard (with dictionaries) compression support for TOAST compression

2025-03-06 Thread Nikhil Kumar Veldanda
Hi On Thu, Mar 6, 2025 at 5:35 AM Aleksander Alekseev wrote: > > Hi Nikhil, > > Many thanks for working on this. I proposed a similar patch some time > ago [1] but the overall feedback was somewhat mixed so I choose to > focus on something else. Thanks for peeking this up. > > > test=# select bui

Re: what's going on with lapwing?

2025-03-06 Thread Tom Lane
Robert Haas writes: > On Thu, Mar 6, 2025 at 2:13 PM Tom Lane wrote: >> It's hard to "mandate" anything in a distributed project like this. >> I don't really see a need to either, at least for cases where an >> old animal isn't causing us extra work. > I don't know, to me it feels like we have t

Re: explain analyze rows=%.0f

2025-03-06 Thread Alena Rybakina
On 06.03.2025 17:13, Robert Haas wrote: On Thu, Mar 6, 2025 at 8:30 AM Matthias van de Meent wrote: On Thu, 6 Mar 2025 at 14:18, Alena Rybakina wrote: Hi! I got a query plan with a strange number of rows. Could you please help me understand it? To be honest I can't understand why 0.50 number

Re: explain analyze rows=%.0f

2025-03-06 Thread Alena Rybakina
On 06.03.2025 16:30, Matthias van de Meent wrote: On Thu, 6 Mar 2025 at 14:18, Alena Rybakina wrote: Hi! I got a query plan with a strange number of rows. Could you please help me understand it? To be honest I can't understand why 0.50 number of rows here? Because the scan matched only ~(500

Re: what's going on with lapwing?

2025-03-06 Thread Tom Lane
Andrew Dunstan writes: > The only reason it's not there is that nobody's ever asked for it ;-) > You can specify an actual list right now instead of a keyword for > branches_to_build. e.g. "branches_to_build => [qw(REL_13_STABLE > REL_14_STABLE REL_15_STABLE)]". The only difficulty about this i

Re: what's going on with lapwing?

2025-03-06 Thread Robert Haas
On Thu, Mar 6, 2025 at 2:13 PM Tom Lane wrote: > It's hard to "mandate" anything in a distributed project like this. > I don't really see a need to either, at least for cases where an > old animal isn't causing us extra work. I don't know, to me it feels like we have the argument about whether St

Re: Next commitfest app release is planned for March 18th

2025-03-06 Thread Peter Eisentraut
On 04.03.25 21:37, Jelte Fennema-Nio wrote: 1. This new homepage includes open patches from*all* commitfests. And there's currently no page with that information. Ok, that's interesting, but I'm even less sure why that should be the default view. The whole point of chunking things into commit

Re: Statistics Import and Export

2025-03-06 Thread Nathan Bossart
On Thu, Mar 06, 2025 at 03:20:16PM -0500, Andres Freund wrote: > There are many systems with hundreds of databases, removing all parallelism > for those from pg_upgrade would likely hurt way more than what we can gain > here. I just did a quick test on a freshly analyzed database with 1,000 sequen

Re: [PoC] Federated Authn/z with OAUTHBEARER

2025-03-06 Thread Jacob Champion
On Tue, Mar 4, 2025 at 2:44 PM Jacob Champion wrote: > Maybe. My first attempt gets all the BSDs green except macOS -- which > now fails in a completely different test, haha... -_- Small update: there is not one bug, but three that interact. ಠ_ಠ 1) The test server advertises an issuer of `https:

Re: Statistics Import and Export

2025-03-06 Thread Corey Huinker
> > To be honest, I am a bit surprised that we decided to enable this by > default. It's not obvious to me that statistics should be regarded as > part of the database in the same way that table definitions or table > data are. That said, I'm not overwhelmingly opposed to that choice. > However, ev

Re: what's going on with lapwing?

2025-03-06 Thread Andrew Dunstan
On 2025-03-06 Th 2:28 PM, Andres Freund wrote: Hi, On 2025-03-06 14:13:40 -0500, Tom Lane wrote: Robert Haas writes: On Thu, Mar 6, 2025 at 1:07 PM Tom Lane wrote: Maybe invent a build-farm.conf option like "newest_branch_to_build"? Yes, that would be nice. I also think we should mandate

Re: ZStandard (with dictionaries) compression support for TOAST compression

2025-03-06 Thread Nikhil Kumar Veldanda
Hi Robert, > I think that solving the problems around using a dictionary is going > to be really hard. Can we see some evidence that the results will be > worth it? With the latest patch I've shared, Using a Kaggle dataset of Nintendo-related tweets[1], we leveraged PostgreSQL's acquire_sample_r

Re: Add column name to error description

2025-03-06 Thread Tom Lane
[ sorry about having let this thread fall off my radar ] jian he writes: > if we print out NameStr(att->attname) then error becomes: > +DETAIL: Returned type unknown does not match expected type character > varying in column "f2" (position 2). > In this case, printing out {column \"%s\"} is no

Re: explain analyze rows=%.0f

2025-03-06 Thread Robert Haas
On Thu, Mar 6, 2025 at 8:30 AM Matthias van de Meent wrote: > On Thu, 6 Mar 2025 at 14:18, Alena Rybakina wrote: > > > > Hi! I got a query plan with a strange number of rows. Could you please > > help me understand it? > > > > To be honest I can't understand why 0.50 number of rows here? > > Beca

Re: dblink: Add SCRAM pass-through authentication

2025-03-06 Thread Peter Eisentraut
On 11.02.25 00:19, Jacob Champion wrote: These don't seem right to me. SCRAM passthrough should be considered as_part_ of the connstr/security checks, but I think it should not _bypass_ those checks. We have to enforce the use of the SCRAM credentials on the remote for safety, similarly to GSS de

Re: making EXPLAIN extensible

2025-03-06 Thread Robert Haas
On Wed, Mar 5, 2025 at 4:38 PM Tom Lane wrote: > v4 has addressed most of my nitpicks, but you still have typedefs > for ExplainState in both header files. My bet is that at least > one buildfarm animal will complain about that. I could be wrong > though, maybe all such compilers are in disuse n

Re: Refactoring postmaster's code to cleanup after child exit

2025-03-06 Thread Heikki Linnakangas
On 05/03/2025 01:23, Michael Paquier wrote: On Tue, Mar 04, 2025 at 05:58:42PM -0500, Andres Freund wrote: On 2024-12-10 12:00:12 +0200, Heikki Linnakangas wrote: 2. Move the pgstat_bestart() call earlier in the startup sequence, so that a backend shows up in pg_stat_activity before it acquires

Re: Refactoring postmaster's code to cleanup after child exit

2025-03-06 Thread Andres Freund
Hi, On 2025-03-05 20:49:33 -0800, Noah Misch wrote: > > This behaviour makes it really hard to debug problems. It'd have been a lot > > easier to understand the problem if we'd seen psql's stderr before the test > > died. > > > > I guess that mean at the very least we'd need to put an eval {} aro

Re: ZStandard (with dictionaries) compression support for TOAST compression

2025-03-06 Thread Nikhil Kumar Veldanda
Hi, > Overall idea is great. > > I just want to mention LZ4 also have API to use dictionary. Its dictionary > will be as simple as "virtually prepended" text (in contrast to complex > ZStd dictionary format). > > I mean, it would be great if "dictionary" will be common property for > different alg

Re: Statistics Import and Export

2025-03-06 Thread Andres Freund
Hi, On 2025-03-06 14:47:08 -0500, Tom Lane wrote: > Nathan Bossart writes: > > On Thu, Mar 06, 2025 at 01:47:34PM -0500, Tom Lane wrote: > >> ... I wonder if we could just rip out pg_upgrade's support > >> for DB-level parallelism, which is not terribly pretty anyway, and > >> simply pass the -j

Re: Statistics Import and Export

2025-03-06 Thread Tom Lane
Andres Freund writes: > On 2025-03-06 13:47:34 -0500, Tom Lane wrote: >> ... I wonder if we could just rip out pg_upgrade's support >> for DB-level parallelism, which is not terribly pretty anyway, and >> simply pass the -j switch straight to pg_dump and pg_restore. > I don't think that'd work we

Re: what's going on with lapwing?

2025-03-06 Thread Tom Lane
Andrew Dunstan writes: > On 2025-03-06 Th 10:45 AM, Robert Haas wrote: >> By the way, is there a particular reason why we're keeping Debian 7 >> coverage in the buildfarm? I don't want to be in a huge rush to kill >> platforms people still care about, but it was pointed out to me >> off-list that

Re: Statistics Import and Export

2025-03-06 Thread Tom Lane
Nathan Bossart writes: > On Thu, Mar 06, 2025 at 01:47:34PM -0500, Tom Lane wrote: >> ... I wonder if we could just rip out pg_upgrade's support >> for DB-level parallelism, which is not terribly pretty anyway, and >> simply pass the -j switch straight to pg_dump and pg_restore. > That would cert

Re: Back-patch of: avoid multiple hard links to same WAL file after a crash

2025-03-06 Thread Nathan Bossart
On Thu, Mar 06, 2025 at 11:30:13AM -0800, Noah Misch wrote: > Options I see: > > 1. Make v14 and v13 skip WAL recycling and preallocation during archive >recovery, like newer branches do. I think that means back-patching the six >commits cc2c7d6~4 cc2c7d6~3 cc2c7d6~2 cc2c7d6~1 cc2c7d6 e36

Re: Showing primitive index scan count in EXPLAIN ANALYZE (for skip scan and SAOP scans)

2025-03-06 Thread Peter Geoghegan
On Thu, Mar 6, 2025 at 2:12 PM Robert Haas wrote: > Well, I think this calls the basic design into question. We discussed > putting this into IndexScanDescData as a convenient way of piping it > through to EXPLAIN, but what I think we have now discovered is that > there isn't actually convenient a

Re: Statistics Import and Export

2025-03-06 Thread Andres Freund
Hi, On 2025-03-06 13:47:34 -0500, Tom Lane wrote: > Andres Freund writes: > >And in contrast to analyzing the database in parallel, the > > pg_dump/restore > >work to restore stats afaict happens single-threaded for each database. > > In principle we should be able to do stats dump/resto

Re: ZStandard (with dictionaries) compression support for TOAST compression

2025-03-06 Thread Tom Lane
Robert Haas writes: > On Thu, Mar 6, 2025 at 12:43 AM Nikhil Kumar Veldanda > wrote: >> Notably, this is the first compression algorithm for Postgres that can make >> use of a dictionary to provide higher levels of compression, but >> dictionaries have to be generated and maintained, > I think

Re: what's going on with lapwing?

2025-03-06 Thread Tom Lane
Robert Haas writes: > On Thu, Mar 6, 2025 at 1:07 PM Tom Lane wrote: >> Maybe invent a build-farm.conf option like "newest_branch_to_build"? > Yes, that would be nice. I also think we should mandate the use of > that option for OS versions that are EOL for more than X years, for > some to-be-det

Re: dblink: Add SCRAM pass-through authentication

2025-03-06 Thread Jacob Champion
On Mon, Mar 3, 2025 at 9:01 AM Jacob Champion wrote: > I keep getting pulled away from my review of 0002 Here's a review of v3-0002: > +dblink_connstr_check(const char *connstr, bool useScramPassthrough) > { > + if (useScramPassthrough) > + { > + if (dblink_connstr_has_scram_require_a

Re: ZStandard (with dictionaries) compression support for TOAST compression

2025-03-06 Thread Robert Haas
On Thu, Mar 6, 2025 at 12:43 AM Nikhil Kumar Veldanda wrote: > Notably, this is the first compression algorithm for Postgres that can make > use of a dictionary to provide higher levels of compression, but dictionaries > have to be generated and maintained, I think that solving the problems aro

Re: Showing primitive index scan count in EXPLAIN ANALYZE (for skip scan and SAOP scans)

2025-03-06 Thread Robert Haas
On Thu, Mar 6, 2025 at 1:58 PM Peter Geoghegan wrote: > On Thu, Mar 6, 2025 at 1:54 PM Robert Haas wrote: > > Hmm, it seems weird that you can't get a hold of that structure to me. > > Why can't you just go find it in the DSM? > > Sorry, I was unclear. > > One reason is that there isn't necessari

Re: Log connection establishment timings

2025-03-06 Thread Andres Freund
Hi, On 2025-01-20 15:01:38 +, Bertrand Drouvot wrote: > Regarding the TimestampTz vs instr_time choice, we have things like: > > + TimestampTz fork_time = ((BackendStartupData *) startup_data)->fork_time; > + TimestampTz cur_time = GetCurrentTimestamp(); > + > + conn_timing.fork_duration = Ti

Re: Statistics Import and Export

2025-03-06 Thread Tom Lane
Andres Freund writes: > On 2025-03-06 13:47:51 -0500, Corey Huinker wrote: >> I'm at the same conclusion. This would mean keeping the one >> getAttributeStats query perrelation, > Why does it have to mean that? It surely would be easier with separate > queries, but I don't think there's anything

Re: Showing primitive index scan count in EXPLAIN ANALYZE (for skip scan and SAOP scans)

2025-03-06 Thread Robert Haas
On Thu, Mar 6, 2025 at 1:17 PM Peter Geoghegan wrote: > The immediate problem is that when the parallel leader doesn't > participate, there is no valid IndexScanDescData in planstate to work > off of. There isn't an obvious way to get to shared memory from the > leader process, since that all goes

Re: what's going on with lapwing?

2025-03-06 Thread Robert Haas
On Thu, Mar 6, 2025 at 1:07 PM Tom Lane wrote: > If there were some trivial way to do that, it'd be more acceptable. > Maybe invent a build-farm.conf option like "newest_branch_to_build"? > branches_to_build covers some adjacent territory, but its filtering > options go the wrong way (only branche

Re: Simplify the logic a bit (src/bin/scripts/reindexdb.c)

2025-03-06 Thread Álvaro Herrera
On 2025-Feb-28, Ranier Vilela wrote: > v2 attached, please comment if you have any further objections. I think running the tests would have been a good idea, as would checking for compiler warnings. -- Álvaro Herrera 48°01'N 7°57'E — https://www.EnterpriseDB.com/ "I love the Pos

Re: Statistics Import and Export

2025-03-06 Thread Corey Huinker
> > > The more I think about it, the less correct it seems to me to have the > statement to restore statistics tracked via ArchiveOpts->createStmt. We > use > that for DDL, but this really is data, not DDL. Because we store it in > ->createStmt it's stored in-memory for the runtime of pg_dump, wh

Re: Statistics Import and Export

2025-03-06 Thread Corey Huinker
> > Would it be appropriate to create a temp table? I wouldn't normally > expect pg_dump to create temp tables, but I can't think of a major > reason not to. > I think we can't - the db might be a replica. > > If not, did you have in mind a CTE with a large VALUES expression, or > just a giant I

Re: explain analyze rows=%.0f

2025-03-06 Thread Alena Rybakina
Hi! I got a query plan with a strange number of rows. Could you please help me understand it? create temp table ta (id int primary key, val int); create temp table tb (id int primary key, aval int); create temp table tc (id int primary key, aid int); insert into ta select id, id from generate_se

Re: Statistics Import and Export

2025-03-06 Thread Andres Freund
Hi, On 2025-03-06 13:00:07 -0500, Corey Huinker wrote: > > > > I don't follow. We already have the tablenames, schemanames and oids of the > > to-be-dumped tables/indexes collected in pg_dump, all that's needed is to > > send > > a list of those to the server to filter there? > > > > Do we have so

Re: Statistics Import and Export

2025-03-06 Thread Jeff Davis
On Thu, 2025-03-06 at 12:16 -0500, Andres Freund wrote: > I don't follow. We already have the tablenames, schemanames and oids > of the > to-be-dumped tables/indexes collected in pg_dump, all that's needed > is to send > a list of those to the server to filter there? Would it be appropriate to cre

Re: what's going on with lapwing?

2025-03-06 Thread Tom Lane
Robert Haas writes: > On Thu, Mar 6, 2025 at 12:22 PM Tom Lane wrote: >> I don't think that's the way to think about old buildfarm members. >> Sure, nobody is very likely to be putting PG 18 on a Debian 7 box, >> but the odds are much higher that they might have PG 13 on it and >> wish to update

Re: Statistics Import and Export

2025-03-06 Thread Jeff Davis
On Thu, 2025-03-06 at 12:16 -0500, Corey Huinker wrote: > > I'm strongly in favor of the choice to make it default. This is > reducing the impact of a post-upgrade There are potentially two different defaults: pg_dump and pg_upgrade. In any case, let's see what improvements we can make to memory

Re: Commitfest app release on Feb 17 with many improvements

2025-03-06 Thread Jelte Fennema-Nio
On Thu, 6 Mar 2025 at 18:10, Tom Lane wrote: > Please see if you can make it use patch(1). IME git is too > stiff-necked about slightly stale patches no matter which > subcommand you use. It was using patch(1) in the past for that reason, but with --3way I was able to get "git apply" to apply pa

Re: ZStandard (with dictionaries) compression support for TOAST compression

2025-03-06 Thread Nikhil Kumar Veldanda
Hi Yura, > So, to support "super-fast" mode you have to accept negative compression > levels. I didn't check, probably you're already support them? > The key point I want to emphasize is that both zstd compression levels and dictionary size should be configurable based on user preferences at attr

Re: Next commitfest app release is planned for March 18th

2025-03-06 Thread Jelte Fennema-Nio
On Thu, 6 Mar 2025 at 18:08, Tom Lane wrote: > > Peter Eisentraut writes: > > On 04.03.25 21:37, Jelte Fennema-Nio wrote: > >> 1. This new homepage includes open patches from*all* commitfests. And > >> there's currently no page with that information. > > > Ok, that's interesting, but I'm even les

Re: what's going on with lapwing?

2025-03-06 Thread Andres Freund
Hi, On 2025-03-06 11:57:05 -0500, Andrew Dunstan wrote: > On 2025-03-06 Th 10:45 AM, Robert Haas wrote: > > By the way, is there a particular reason why we're keeping Debian 7 > > coverage in the buildfarm? I don't want to be in a huge rush to kill > > platforms people still care about, but it was

Re: zstd failing on mipsel (PG 15.12, pg_verifybackup/t/010_client_untar.pl)

2025-03-06 Thread Christoph Berg
Re: To PostgreSQL Hackers > I cannot reproduce the problem on the Debian mipsel porter box, and > the (working) test does not leave the base.tar.zst mentioned about > behind. Since I don't have access to the machine where it is failing, I uploaded a new version that base64-encodes the "bad" base.t

Re: Statistics Import and Export

2025-03-06 Thread Andres Freund
Hi, On 2025-03-06 12:04:25 -0500, Corey Huinker wrote: > > > If there's value in freeing them, why isn't it being done already? What > > > other thing would consume this freed memory? > > > > I'm not saying that they can be freed, they can't right now. My point is > > just > > that we *already* ke

Re: what's going on with lapwing?

2025-03-06 Thread Andrew Dunstan
On 2025-03-06 Th 10:45 AM, Robert Haas wrote: On Wed, Mar 5, 2025 at 9:49 PM Julien Rouhaud wrote: FWIW I installed the client version 19.1 this morning and forced a run on HEAD and lapwing is back to green. Thanks, appreciate it. By the way, is there a particular reason why we're keeping D

Re: Next commitfest app release is planned for March 18th

2025-03-06 Thread Tom Lane
Peter Eisentraut writes: > On 04.03.25 21:37, Jelte Fennema-Nio wrote: >> 1. This new homepage includes open patches from*all* commitfests. And >> there's currently no page with that information. > Ok, that's interesting, but I'm even less sure why that should be the > default view. The whole p

Re: Statistics Import and Export

2025-03-06 Thread Corey Huinker
> > > > > Pardon my inexperience, but aren't the ArchiveEntry records needed right > up > > until the program's run? > > s/the/the end of the/? > yes > > If there's value in freeing them, why isn't it being done already? What > > other thing would consume this freed memory? > > I'm not saying th

Re: AIO v2.5

2025-03-06 Thread Andres Freund
Hi, On 2025-03-06 10:33:33 -0500, Robert Haas wrote: > On Tue, Mar 4, 2025 at 2:00 PM Andres Freund wrote: > > - pg_stat_aios currently has the IO Handle flags as dedicated columns. Not > > sure that's great? > > I don't like the name. I don't think it changes anything, but as Jakub pointed o

Re: Log connection establishment timings

2025-03-06 Thread Melanie Plageman
Thanks for the continued review! Attached v11 has a test added (passes locally but fails in CI, so I have to fix that). I still need to do some more manual testing and validation. On Thu, Mar 6, 2025 at 9:56 AM Bertrand Drouvot wrote: > <-- snip --> > what do you think about also doing? > >

Re: Statistics Import and Export

2025-03-06 Thread Robert Haas
On Thu, Mar 6, 2025 at 9:29 AM Andres Freund wrote: > But TBH, I do wonder how much the current memory usage of the statistics > dump/restore support is going to bite us. In some cases this will dramatically > increase pg_dump/pg_upgrade's memory usage, my tests were with tiny amounts of > data an

Re: JSON_VALUE() behavior when RETURNING bytea (expected base64 decoding)

2025-03-06 Thread Peter Eisentraut
On 06.03.25 11:55, Shay Rojansky wrote: For whatever it's worth, I'll note that SQL Server's OPENJSON does do this (so when a JSON string property is extracted as a binary type, base64 encoding is assumed). Other databases also have very specific documented conversion rules for JSON_VALUE RETUR

Re: new commitfest transition guidance

2025-03-06 Thread Daniel Gustafsson
> On 5 Mar 2025, at 19:32, Álvaro Herrera wrote: > > On 2025-Mar-05, Daniel Gustafsson wrote: > >> I would avoid using Google for finding content on the wiki, the search >> function >> on the wiki itself is generally more reliable. Searching for FOSDEM 2025 >> returns the following as the top

Re: Improve monitoring of shared memory allocations

2025-03-06 Thread Rahila Syed
Hi, Thank you for the review. cfbot found a few compiler warnings: > > https://cirrus-ci.com/task/6526903542087680 > [16:47:46.964] make -s -j${BUILD_JOBS} clean > [16:47:47.452] time make -s -j${BUILD_JOBS} world-bin > [16:49:10.496] lwlock.c: In function ‘CreateLWLocks’: > [16:49:10.496] lwloc

  1   2   >