Re: Synchronizing slots from primary to standby

2023-11-20 Thread Drouvot, Bertrand
Hi, On 11/21/23 6:16 AM, Amit Kapila wrote: On Mon, Nov 20, 2023 at 6:51 PM Drouvot, Bertrand wrote: As far the 'i' state here, from what I see, it is currently useful for: 1. Cascading standby to not sync slots with state = 'i' from the first standby. 2. Easily report Slots that did not

Re: Use of backup_label not noted in log

2023-11-20 Thread Laurenz Albe
On Mon, 2023-11-20 at 11:03 -0800, Andres Freund wrote: > > If we add a message for starting with "backup_label", shouldn't > > we also add a corresponding message for starting from a checkpoint > > found in the control file?  If you see that in a problem report, > > you immediately know what is

Re: remaining sql/json patches

2023-11-20 Thread Peter Eisentraut
I looked a bit at the parser additions, because there were some concerns expressed that they are quite big. It looks like the parser rules were mostly literally copied from the BNF in the SQL standard. That's probably a reasonable place to start, but now at the end, there is some room for

Re: simplehash: SH_OPTIMIZE_REPEAT for optimizing repeated lookups of the same key

2023-11-20 Thread Jeff Davis
On Mon, 2023-11-20 at 22:50 -0600, Nathan Bossart wrote: > I'm mostly thinking out loud here, but could we just always do this?  > I > guess you might want to avoid it if your SH_EQUAL is particularly > expensive > and you know repeated lookups are rare, but maybe that's uncommon > enough > that

Re: Why is hot_standby_feedback off by default?

2023-11-20 Thread John Naylor
On Tue, Nov 21, 2023 at 6:49 AM Andres Freund wrote: > > On 2023-11-20 16:34:47 +0700, John Naylor wrote: > > Sounds like a TODO? > > WFM. I don't personally use or update TODO, as I have my doubts about its > usefulness or state of maintenance. But please feel free to add this as a TODO > from

Re: Wrong rows estimations with joins of CTEs slows queries by more than factor 500

2023-11-20 Thread Richard Guo
On Tue, Nov 21, 2023 at 1:46 AM Tom Lane wrote: > * Do we really need to use make_tlist_from_pathtarget? Why isn't > the tlist of the cteplan good enough (indeed, more so)? I think you are right. The cteplan->targetlist is built for the CTE's best path by build_path_tlist(), which is almost

Typo with amtype = 's' in opr_sanity.sql

2023-11-20 Thread Michael Paquier
Hi all, While rebasing a patch from 2016 related to sequence AMs (more about that later), I've bumped on a mistake from 8586bf7ed888 in opr_sanity.sql, as of: +SELECT p1.oid, p1.amname, p2.oid, p2.proname +FROM pg_am AS p1, pg_proc AS p2 +WHERE p2.oid = p1.amhandler AND p1.amtype = 's' AND It

Re: Synchronizing slots from primary to standby

2023-11-20 Thread shveta malik
On Tue, Nov 21, 2023 at 10:02 AM Zhijie Hou (Fujitsu) wrote: > > On Friday, November 17, 2023 7:39 PM Amit Kapila > wrote: > > > > On Thu, Nov 16, 2023 at 5:34 PM shveta malik > > wrote: > > > > > > PFA v35. > > > > > > > Review v35-0002* > > == > > Thanks for the comments. > > >

Re: Simplify if/else logic of walsender CreateReplicationSlot

2023-11-20 Thread Peter Smith
On Tue, Nov 21, 2023 at 3:57 PM Michael Paquier wrote: > > On Mon, Nov 20, 2023 at 05:07:38PM +0900, Michael Paquier wrote: > > Good idea. What you are suggesting here improves the readability of > > this code, so +1. > > And applied this one, thanks! Thanks for pushing. == Kind Regards,

Re: Synchronizing slots from primary to standby

2023-11-20 Thread shveta malik
On Tue, Nov 21, 2023 at 10:01 AM Zhijie Hou (Fujitsu) wrote: > > On Saturday, November 18, 2023 6:46 PM Amit Kapila > wrote: > > > > On Fri, Nov 17, 2023 at 5:18 PM Drouvot, Bertrand > > wrote: > > > > > > On 11/17/23 2:46 AM, Zhijie Hou (Fujitsu) wrote: > > > > On Tuesday, November 14, 2023

Re: Synchronizing slots from primary to standby

2023-11-20 Thread Peter Smith
Here are some review comments for the patch v35-0001. == 0. GENERAL documentation I felt that the documentation gave details of the individual changes (e.g. GUC 'standby_slot_names' and API, CREATE SUBSCRIPTION option, and pg_replication_slots 'failover' attribute etc.) but there is nothing

Re: Synchronizing slots from primary to standby

2023-11-20 Thread Amit Kapila
On Mon, Nov 20, 2023 at 6:51 PM Drouvot, Bertrand wrote: > > On 11/20/23 11:59 AM, Amit Kapila wrote: > > On Mon, Nov 20, 2023 at 3:17 PM Drouvot, Bertrand > > wrote: > >> > >> On 11/18/23 11:45 AM, Amit Kapila wrote: > >>> On Fri, Nov 17, 2023 at 5:18 PM Drouvot, Bertrand > >>> wrote: > >

Re: Do away with a few backwards compatibility macros

2023-11-20 Thread Tom Lane
Nathan Bossart writes: > On Thu, Nov 16, 2023 at 09:46:22AM -0600, Nathan Bossart wrote: >> I'm fine with this because all of these macros are no-ops for all supported >> versions of Postgres. Even if an extension is using them today, you'll get >> the same behavior as before if you remove the

How to accurately determine when a relation should use local buffers?

2023-11-20 Thread Давыдов Виталий
Dear Hackers, I would like to clarify, what the correct way is to determine that a given relation is using local buffers. Local buffers, as far as I know, are used for temporary tables in backends. There are two functions/macros (bufmgr.c): SmgrIsTemp, RelationUsesLocalBuffers. The first

Re: Do away with a few backwards compatibility macros

2023-11-20 Thread Nathan Bossart
On Thu, Nov 16, 2023 at 09:46:22AM -0600, Nathan Bossart wrote: > On Thu, Nov 16, 2023 at 07:11:41PM +0530, Bharath Rupireddy wrote: >> After a recent commit 6a72c42f (a related discussion [1]) which >> removed MemoryContextResetAndDeleteChildren(), I think there are a >> couple of other backward

Re: Simplify if/else logic of walsender CreateReplicationSlot

2023-11-20 Thread Michael Paquier
On Mon, Nov 20, 2023 at 05:07:38PM +0900, Michael Paquier wrote: > Good idea. What you are suggesting here improves the readability of > this code, so +1. And applied this one, thanks! -- Michael signature.asc Description: PGP signature

Re: simplehash: SH_OPTIMIZE_REPEAT for optimizing repeated lookups of the same key

2023-11-20 Thread Nathan Bossart
On Mon, Nov 20, 2023 at 06:12:47PM -0800, Jeff Davis wrote: > The caller could do something similar, so this option is not necessary, > but it seems like it could be generally useful. It speeds things up for > the search_path cache (and is an alternative to another patch I have > that implements

Re: Hide exposed impl detail of wchar.c

2023-11-20 Thread Nathan Bossart
On Mon, Nov 20, 2023 at 05:14:17PM -0800, Jubilee Young wrote: > On Mon, Nov 20, 2023 at 2:52 PM Nathan Bossart > wrote: >> Does pgrx use ascii.h at all? > > We don't use utils/ascii.h, no. Alright. The next minor release isn't until February, so I'll let this one sit a little while longer in

Re: Annoying build warnings from latest Apple toolchain

2023-11-20 Thread Andres Freund
Hi, On 2023-11-20 16:20:20 -0500, Tom Lane wrote: > I'm generally still using autoconf, I only run meson builds when > somebody complains about them ;-). But yeah, I see lots of > "ld: warning: -undefined error is deprecated" when I do that. > This seems to have been installed by Andres'

RE: Synchronizing slots from primary to standby

2023-11-20 Thread Zhijie Hou (Fujitsu)
On Friday, November 17, 2023 7:39 PM Amit Kapila wrote: > > On Thu, Nov 16, 2023 at 5:34 PM shveta malik > wrote: > > > > PFA v35. > > > > Review v35-0002* > == Thanks for the comments. > 1. > As quoted in the commit message, > > > If a logical slot is invalidated on the

Re: archive modules loose ends

2023-11-20 Thread Nathan Bossart
On Mon, Nov 13, 2023 at 03:35:28PM -0800, Andres Freund wrote: > On 2023-11-13 16:42:31 -0600, Nathan Bossart wrote: >> There seems to be no interest in this patch, so I plan to withdraw it from >> the commitfest system by the end of the month unless such interest >> materializes. > > I think it

Re: Synchronizing slots from primary to standby

2023-11-20 Thread Amit Kapila
On Mon, Nov 20, 2023 at 4:28 PM Amit Kapila wrote: > > 9. > +synchronize_one_slot(WalReceiverConn *wrconn, RemoteSlot *remote_slot, > + bool *slot_updated) > { > ... > + else > + { > + TransactionId xmin_horizon = InvalidTransactionId; > + ReplicationSlot *slot; > + > +

Re: [PATCH] pgbench log file headers

2023-11-20 Thread Adam Hendel
Hello, On Mon, Nov 13, 2023 at 6:01 PM Andres Freund wrote: > Hi, > > On 2023-11-13 11:55:07 -0600, Adam Hendel wrote: > > Currently, pgbench will log individual transactions to a logfile when the > > `--log` parameter flag is provided. The logfile, however, does not > include > > column

Re: Use of backup_label not noted in log

2023-11-20 Thread Michael Paquier
On Mon, Nov 20, 2023 at 03:31:20PM -0400, David Steele wrote: > On 11/20/23 15:03, Andres Freund wrote: >> Besides the phrasing and the additional log message (I have no opinion about >> whether it should be backpatched or not), I used %u for TimelineID as >> appropriate, and added a comma before

Re: remaining sql/json patches

2023-11-20 Thread Amit Langote
On Nov 16, 2023, at 17:48, Amit Langote wrote: > On Thu, Nov 16, 2023 at 2:11 AM Andres Freund wrote: >> On 2023-11-15 22:00:41 +0900, Amit Langote wrote: This causes a nontrivial increase in the size of the parser (~5% in an optimized build here), I wonder if we can do better. >>>

Re: Add recovery to pg_control and remove backup_label

2023-11-20 Thread Michael Paquier
On Mon, Nov 20, 2023 at 03:58:55PM -0800, Andres Freund wrote: > I was thinking we'd just set it in the pg_basebackup style path, and we'd > error out if it's set and backup_label is present. But we'd still use > backup_label without the pg_control flag set. > > So it'd just provide a cross-check

Re: About #13489, array dimensions and CREATE TABLE ... LIKE

2023-11-20 Thread Tom Lane
Bruce Momjian writes: > On Mon, Nov 20, 2023 at 09:04:21PM -0500, Tom Lane wrote: >> Bruce Momjian writes: >>> An alternate approach would >>> be to remove pg_attribute.attndims so we don't even try to preserve >>> dimensionality. >> I could get behind that, perhaps. It looks like we're not

simplehash: SH_OPTIMIZE_REPEAT for optimizing repeated lookups of the same key

2023-11-20 Thread Jeff Davis
Patch attached. The caller could do something similar, so this option is not necessary, but it seems like it could be generally useful. It speeds things up for the search_path cache (and is an alternative to another patch I have that implements the same thing in the caller). Thoughts? Regards,

Re: About #13489, array dimensions and CREATE TABLE ... LIKE

2023-11-20 Thread Bruce Momjian
On Mon, Nov 20, 2023 at 09:04:21PM -0500, Tom Lane wrote: > Bruce Momjian writes: > > I would like to apply this patch to master because I think our current > > deficiencies in this area are unacceptable. > > I do not think this is a particularly good idea, because it creates > the impression in

Re: About #13489, array dimensions and CREATE TABLE ... LIKE

2023-11-20 Thread Tom Lane
Bruce Momjian writes: > I would like to apply this patch to master because I think our current > deficiencies in this area are unacceptable. I do not think this is a particularly good idea, because it creates the impression in a couple of places that we track this data, when we do not really do

Re:Re:Re: How to solve the problem of one backend process crashing and causing other processes to restart?

2023-11-20 Thread yuansong
thanks,After reconsideration, I realized that what I really want is for other connections to remain unaffected when a process crashes. This is something that a connection pool cannot solve. At 2023-11-14 09:41:03, "Thomas wen" wrote: Hi yuansong there is connnection pool path

Re: meson documentation build open issues

2023-11-20 Thread Andres Freund
Hi, On 2023-11-20 08:27:48 +0100, Peter Eisentraut wrote: > On 17.11.23 19:53, Andres Freund wrote: > > I pushed the first two commits (the selinux stuff) and worked a bit more on > > the subsequent ones. > > Patches 0001 through 0004 look good to me. Cool, I pushed them now. > Some possible

Re: pg_upgrade and logical replication

2023-11-20 Thread Michael Paquier
On Mon, Nov 20, 2023 at 09:49:41AM +0530, Amit Kapila wrote: > On Tue, Nov 14, 2023 at 7:21 AM vignesh C wrote: >> There are couple of things happening here: a) In the first part we >> take care of setting subscription relation to SYNCDONE and dropping >> the replication slot at publisher node,

Re: About #13489, array dimensions and CREATE TABLE ... LIKE

2023-11-20 Thread Bruce Momjian
On Fri, Sep 8, 2023 at 05:10:51PM -0400, Bruce Momjian wrote: > I knew we only considered the array dimension sizes to be documentation > _in_ the query, but I thought we at least properly displayed the number > of dimensions specified at creation when we described the table in psql, > but it

Re: Adding facility for injection points (or probe points?) for more advanced tests

2023-11-20 Thread Michael Paquier
On Mon, Nov 20, 2023 at 04:53:45PM +0530, Ashutosh Bapat wrote: > On Wed, Oct 25, 2023 at 9:43 AM Michael Paquier wrote: >> I have added some documentation to explain that, as well. I am not >> wedded to the name proposed in the patch, so if you feel there is >> better, feel free to propose

Re: Faster "SET search_path"

2023-11-20 Thread Jeff Davis
On Thu, 2023-11-16 at 16:46 -0800, Jeff Davis wrote: > While I considered OOM during hash key initialization, I missed some > other potential out-of-memory hazards. Attached a fixup patch 0003, > which re-introduces one list copy but it simplifies things > substantially in addition to being safer

Re: POC, WIP: OR-clause support for indexes

2023-11-20 Thread Alena Rybakina
On 20.11.2023 11:52, Andrei Lepikhov wrote: Looking into the patch, I found some trivial improvements (see attachment). Also, it is not obvious that using a string representation of the clause as a hash table key is needed here. Also, by making a copy of the node in the get_key_nconst_node(),

Re: Show WAL write and fsync stats in pg_stat_io

2023-11-20 Thread Michael Paquier
On Mon, Nov 20, 2023 at 05:43:17PM +0300, Nazir Bilal Yavuz wrote: > Yes, the timings for the writes and the syncs should work. Another > question I have in mind is the pg_stat_reset_shared() function. When > we call it with 'io' it will reset pg_stat_wal's timings and when we > call it with 'wal'

Re: Hide exposed impl detail of wchar.c

2023-11-20 Thread Andres Freund
Hi, On 2023-11-16 17:11:03 -0800, Jubilee Young wrote: > We don't directly `#include` C into Rust, but use libclang to preprocess and > compile a wrapping C header into a list of symbols Rust will look for at link > time. Our failure is in libclang and how we steer it: > - The Clang-C API

Re: Add recovery to pg_control and remove backup_label

2023-11-20 Thread Andres Freund
Hi, On 2023-11-21 08:52:08 +0900, Michael Paquier wrote: > On Mon, Nov 20, 2023 at 12:37:46PM -0800, Andres Freund wrote: > > Given that, I wonder if what we should do is to just add a new field to > > pg_control that says "error out if backup_label does not exist", that we set > > when creating

Re: Add recovery to pg_control and remove backup_label

2023-11-20 Thread Michael Paquier
On Mon, Nov 20, 2023 at 12:37:46PM -0800, Andres Freund wrote: > Given that, I wonder if what we should do is to just add a new field to > pg_control that says "error out if backup_label does not exist", that we set > when creating a streaming base backup That would mean that one still needs to

Re: Why is hot_standby_feedback off by default?

2023-11-20 Thread Andres Freund
On 2023-11-20 16:34:47 +0700, John Naylor wrote: > Sounds like a TODO? WFM. I don't personally use or update TODO, as I have my doubts about its usefulness or state of maintenance. But please feel free to add this as a TODO from my end...

Re: Add recovery to pg_control and remove backup_label

2023-11-20 Thread Andres Freund
Hi, On 2023-11-20 14:18:15 -0700, David G. Johnston wrote: > On Mon, Nov 20, 2023 at 1:37 PM Andres Freund wrote: > > > > > Given that, I wonder if what we should do is to just add a new field to > > pg_control that says "error out if backup_label does not exist", that we > > set > > when

Re: Add recovery to pg_control and remove backup_label

2023-11-20 Thread Michael Paquier
On Mon, Nov 20, 2023 at 11:11:13AM -0500, Robert Haas wrote: > I think we need more votes to make a change this big. I have a > concern, which I think I've expressed before, that we keep whacking > around the backup APIs, and that has a cost which is potentially > larger than the benefits. The

Re: PANIC serves too many masters

2023-11-20 Thread Andres Freund
Hi, On 2023-11-20 17:55:32 -0500, Tom Lane wrote: > Jeff Davis writes: > > Is the error level the right way to express what we want to happen? It > > seems like what we really want is to decide on the behavior, i.e. > > restart or not, and generate core or not. That could be done a > > different

Re: PANIC serves too many masters

2023-11-20 Thread Tom Lane
Jeff Davis writes: > Is the error level the right way to express what we want to happen? It > seems like what we really want is to decide on the behavior, i.e. > restart or not, and generate core or not. That could be done a > different way, like: > ereport(PANIC, > (errmsg("could

Re: Hide exposed impl detail of wchar.c

2023-11-20 Thread Nathan Bossart
On Mon, Nov 20, 2023 at 10:50:36AM -0800, Jubilee Young wrote: > In that case, I took a look across the codebase and saw a > utils/ascii.h that doesn't > seem to have gotten much love, but I suppose one could argue that it's > intended > to be a backend-only header file? That might work. It's

Re: Partial aggregates pushdown

2023-11-20 Thread Bruce Momjian
On Mon, Nov 20, 2023 at 03:51:33PM -0500, Robert Haas wrote: > On Mon, Nov 13, 2023 at 3:26 AM fujii.y...@df.mitsubishielectric.co.jp > wrote: > > In postgres_fdw.sql, I have corrected the output format for floating point > > numbers > > by extra_float_digits. > > Looking at this, I find that

Re: PANIC serves too many masters

2023-11-20 Thread Jeff Davis
On Mon, 2023-11-20 at 17:12 -0500, Tom Lane wrote: > I'd be inclined to keep PANIC with its current meaning, and > incrementally change call sites where we decide that's not the > best behavior.  I think those will be a minority, maybe a small > minority.  (PANIC_EXIT had darn well better be a

Re: PSQL error: total cell count of XXX exceeded

2023-11-20 Thread Tom Lane
Alvaro Herrera writes: > I think we're bound to hit this limit at some point in the future, and > it seems easy enough to solve. I propose the attached, which is pretty > much what Hongxu last submitted, with some minor changes. This bit needs more work: - content->cells =

Re: CREATE FUNCTION ... SEARCH { DEFAULT | SYSTEM | SESSION }

2023-11-20 Thread Jeff Davis
On Mon, 2023-11-20 at 15:52 -0500, Robert Haas wrote: > I agree. Not to burden you, but do you know what the overhead is now, > and do you have any plans to further reduce it? I don't believe > that's > the only thing we ought to be doing here, necessarily, but it is one > thing that we definitely

Re: PANIC serves too many masters

2023-11-20 Thread Tom Lane
Jeff Davis writes: > On Sat, 2023-11-18 at 14:29 -0800, Andres Freund wrote: >> I don't quite know what we should do. But the current situation >> decidedly >> doesn't seem great. > Agreed. +1 > Better classification is nice, but it also requires more > discipline and it might not always be

Re: Annoying build warnings from latest Apple toolchain

2023-11-20 Thread Tom Lane
I wrote: > The autoconf side seems to just be letting this option default. > I'm not sure what the default choice is, but evidently it's not > "-undefined error"? Or were they stupid enough to not allow you > to explicitly select the default behavior? Seems we are not the only project having

Re: PANIC serves too many masters

2023-11-20 Thread Jeff Davis
Hi, On Sat, 2023-11-18 at 14:29 -0800, Andres Freund wrote: > I don't quite know what we should do. But the current situation > decidedly > doesn't seem great. Agreed. Better classification is nice, but it also requires more discipline and it might not always be obvious which category something

Re: Annoying build warnings from latest Apple toolchain

2023-11-20 Thread Tom Lane
Robert Haas writes: > On Mon, Nov 20, 2023 at 3:53 PM Tom Lane wrote: >> 13.6.2? longfin's host is on 13.6.1, and the only thing Software >> Update is showing me is an option to upgrade to Sonoma. But anyway... > Uh, I guess Apple made a special version just for me? That's > definitely what

Re: Add recovery to pg_control and remove backup_label

2023-11-20 Thread David G. Johnston
On Mon, Nov 20, 2023 at 1:37 PM Andres Freund wrote: > > Given that, I wonder if what we should do is to just add a new field to > pg_control that says "error out if backup_label does not exist", that we > set > when creating a streaming base backup > > I thought this was DOA since we don't want

Re: Annoying build warnings from latest Apple toolchain

2023-11-20 Thread Robert Haas
On Mon, Nov 20, 2023 at 3:53 PM Tom Lane wrote: > 13.6.2? longfin's host is on 13.6.1, and the only thing Software > Update is showing me is an option to upgrade to Sonoma. But anyway... Uh, I guess Apple made a special version just for me? That's definitely what it says. > > [2264/2287]

Re: Annoying build warnings from latest Apple toolchain

2023-11-20 Thread Tom Lane
Robert Haas writes: > Is there still outstanding work on this thread? Because I'm just now > using a new MacBook (M2, Ventura 13.6.2) and I'm getting a lot of this > kind of thing in a meson build: 13.6.2? longfin's host is on 13.6.1, and the only thing Software Update is showing me is an

Re: CREATE FUNCTION ... SEARCH { DEFAULT | SYSTEM | SESSION }

2023-11-20 Thread Robert Haas
On Tue, Nov 14, 2023 at 11:21 PM Jeff Davis wrote: > After adding the search path cache (recent commit f26c2368dc) hopefully > that helps to make the above suggestion more reasonable performance- > wise. I think we can call that progress. I agree. Not to burden you, but do you know what the

Re: Partial aggregates pushdown

2023-11-20 Thread Robert Haas
On Mon, Nov 13, 2023 at 3:26 AM fujii.y...@df.mitsubishielectric.co.jp wrote: > In postgres_fdw.sql, I have corrected the output format for floating point > numbers > by extra_float_digits. Looking at this, I find that it's not at all clear to me how the partial aggregate function is defined.

Re: PSQL error: total cell count of XXX exceeded

2023-11-20 Thread Alvaro Herrera
On 2023-Sep-12, Tom Lane wrote: > I'm more than a bit skeptical about trying to do something about this, > simply because this range of query result sizes is far past what is > practical. The OP clearly hasn't tested his patch on actually > overflowing query results, and I don't care to either.

Re: Add recovery to pg_control and remove backup_label

2023-11-20 Thread Andres Freund
Hi, On 2023-11-20 15:56:19 -0400, David Steele wrote: > I understand this is an option -- but does it need to be? What is the > benefit of excluding the manifest? It's not free to create the manifest, particularly if checksums are enabled. Also, for external backups, there's no manifest... -

Re: Add recovery to pg_control and remove backup_label

2023-11-20 Thread Andres Freund
Hi, On 2023-11-20 11:11:13 -0500, Robert Haas wrote: > I think we need more votes to make a change this big. I have a > concern, which I think I've expressed before, that we keep whacking > around the backup APIs, and that has a cost which is potentially > larger than the benefits. +1. The

Re: Implement missing join selectivity estimation for range types

2023-11-20 Thread Schoemans Maxime
On 14/11/2023 20:46, Tom Lane wrote: > I took a brief look through this very interesting work. I concur > with Tomas that it feels a little odd that range join selectivity > would become smarter than scalar inequality join selectivity, and > that we really ought to prioritize applying these

Re: Annoying build warnings from latest Apple toolchain

2023-11-20 Thread Andres Freund
Hi, On 2023-11-20 14:46:13 -0500, Robert Haas wrote: > On Mon, Nov 20, 2023 at 2:35 PM Andres Freund wrote: > > > Is there still outstanding work on this thread? Because I'm just now > > > using a new MacBook (M2, Ventura 13.6.2) and I'm getting a lot of this > > > kind of thing in a meson

Re: Add recovery to pg_control and remove backup_label

2023-11-20 Thread David Steele
On 11/20/23 15:47, Robert Haas wrote: On Mon, Nov 20, 2023 at 2:41 PM David Steele wrote: I can't see why a backup would continue to be valid without a manifest -- that's not very standard for backup software. If you have the critical info in backup_label, you can't afford to lose that, so why

Re: Parallel CREATE INDEX for BRIN indexes

2023-11-20 Thread Matthias van de Meent
On Wed, 8 Nov 2023 at 12:03, Tomas Vondra wrote: > > Hi, > > here's an updated patch, addressing the review comments, and reworking > how the work is divided between the workers & leader etc. > > 0001 is just v2, rebased to current master > > 0002 and 0003 address most of the issues, in

Re: Add recovery to pg_control and remove backup_label

2023-11-20 Thread Robert Haas
On Mon, Nov 20, 2023 at 2:41 PM David Steele wrote: > I can't see why a backup would continue to be valid without a manifest > -- that's not very standard for backup software. If you have the > critical info in backup_label, you can't afford to lose that, so why > should backup_manifest be any

Re: Annoying build warnings from latest Apple toolchain

2023-11-20 Thread Robert Haas
On Mon, Nov 20, 2023 at 2:35 PM Andres Freund wrote: > > Is there still outstanding work on this thread? Because I'm just now > > using a new MacBook (M2, Ventura 13.6.2) and I'm getting a lot of this > > kind of thing in a meson build: > > Ventura? In that case I assume you installed newer

Re: Add recovery to pg_control and remove backup_label

2023-11-20 Thread David Steele
On 11/20/23 14:44, Robert Haas wrote: On Mon, Nov 20, 2023 at 12:54 PM David Steele wrote: Another thing we could do is explicitly error if we see backup_label in PGDATA during recovery. That's just a few lines of code so would not be a big deal to maintain. This error would only be visible on

Re: Annoying build warnings from latest Apple toolchain

2023-11-20 Thread Andres Freund
Hi, On 2023-11-20 14:14:00 -0500, Robert Haas wrote: > On Sat, Oct 7, 2023 at 12:09 PM Tom Lane wrote: > > Done that way. > > Is there still outstanding work on this thread? Because I'm just now > using a new MacBook (M2, Ventura 13.6.2) and I'm getting a lot of this > kind of thing in a meson

Re: Use of backup_label not noted in log

2023-11-20 Thread David Steele
On 11/20/23 15:03, Andres Freund wrote: On 2023-11-20 11:35:15 +0100, Laurenz Albe wrote: If we add a message for starting with "backup_label", shouldn't we also add a corresponding message for starting from a checkpoint found in the control file? If you see that in a problem report, you

Re: trying again to get incremental backup

2023-11-20 Thread Robert Haas
On Mon, Nov 20, 2023 at 2:10 PM Robert Haas wrote: > > Is this meant to support multiple timelines each with non-overlapping > > adjacent ranges, rather than multiple non-adjacent ranges? > > Correct. I don't see how non-adjacent LSN ranges could ever be a > useful thing, but adjacent ranges on

Re: Annoying build warnings from latest Apple toolchain

2023-11-20 Thread Robert Haas
On Sat, Oct 7, 2023 at 12:09 PM Tom Lane wrote: > Done that way. Is there still outstanding work on this thread? Because I'm just now using a new MacBook (M2, Ventura 13.6.2) and I'm getting a lot of this kind of thing in a meson build: [2264/2287] Linking target

Re: psql's FETCH_COUNT (cursor) is not being respected for CTEs

2023-11-20 Thread Daniel Verite
Hi, Here's a new version to improve the performance of FETCH_COUNT and extend the cases when it can be used. Patch 0001 adds a new mode in libpq to allow the app to retrieve larger chunks of results than the single row of the row-by-row mode. The maximum number of rows per PGresult is set by

Re: trying again to get incremental backup

2023-11-20 Thread Robert Haas
On Mon, Nov 20, 2023 at 2:03 PM Alvaro Herrera wrote: > That sounds good to me. Not having to parse the manifest server-side > sounds like a win, as does saving the transfer, for the cases where the > manifest is large. OK. I'll look into this next week, hopefully. > Is this meant to support

Re: Use of backup_label not noted in log

2023-11-20 Thread David Steele
On 11/20/23 14:27, Andres Freund wrote: Hi, On 2023-11-19 14:28:12 -0400, David Steele wrote: On 11/18/23 17:49, Andres Freund wrote: On 2023-11-18 10:01:42 -0800, Andres Freund wrote: Not enamored with the phrasing of the log messages, but here's a prototype: When starting up with

Re: Use of backup_label not noted in log

2023-11-20 Thread Andres Freund
Hi, On 2023-11-20 11:35:15 +0100, Laurenz Albe wrote: > On Mon, 2023-11-20 at 17:30 +0900, Michael Paquier wrote: > > + if (ControlFile->backupStartPoint != InvalidXLogRecPtr) > > + ereport(LOG, > > + (errmsg("continuing to start from base backup with redo > >

Re: trying again to get incremental backup

2023-11-20 Thread Alvaro Herrera
On 2023-Nov-16, Robert Haas wrote: > On Thu, Nov 16, 2023 at 12:23 PM Alvaro Herrera > wrote: > > I don't understand this point. Currently, the protocol is that > > UPLOAD_MANIFEST is used to send the manifest prior to requesting the > > backup. You seem to be saying that you're thinking of

Re: Hide exposed impl detail of wchar.c

2023-11-20 Thread Jubilee Young
On Fri, Nov 17, 2023 at 2:26 AM John Naylor wrote: > > On Fri, Nov 17, 2023 at 5:54 AM Nathan Bossart > wrote: > > > > It looks like is_valid_ascii() was originally added to pg_wchar.h so that > > it could easily be used elsewhere [0] [1], but that doesn't seem to have > > happened yet. > > > >

Re: Add recovery to pg_control and remove backup_label

2023-11-20 Thread Robert Haas
On Mon, Nov 20, 2023 at 12:54 PM David Steele wrote: > Another thing we could do is explicitly error if we see backup_label in > PGDATA during recovery. That's just a few lines of code so would not be > a big deal to maintain. This error would only be visible on restore, so > it presumes that

Re: Use of backup_label not noted in log

2023-11-20 Thread Andres Freund
Hi, On 2023-11-20 17:30:31 +0900, Michael Paquier wrote: > On Sat, Nov 18, 2023 at 01:49:15PM -0800, Andres Freund wrote: > > Note that the LSN in the "continuing" case is the one the backup started at, > > not where recovery will start. > > > > I've wondered whether it's worth also adding an

Re: Use of backup_label not noted in log

2023-11-20 Thread Andres Freund
Hi, On 2023-11-20 11:24:25 -0500, Robert Haas wrote: > I do also think it is worth considering how this proposal interacts > with the proposal to remove backup_label. If that proposal goes > through, then this proposal is obsolete, I believe. I think it's the opposite, if anything. Today you can

Re: Add recovery to pg_control and remove backup_label

2023-11-20 Thread David Steele
On 11/20/23 12:11, Robert Haas wrote: On Sun, Nov 19, 2023 at 8:16 PM Michael Paquier wrote: (I am not exactly sure how, but we've lost pgsql-hackers on the way when you sent v5. Now added back in CC with the two latest patches you've proposed attached.) Here is a short summary of what

Re: Wrong rows estimations with joins of CTEs slows queries by more than factor 500

2023-11-20 Thread Tom Lane
Richard Guo writes: > On Fri, Nov 17, 2023 at 11:38 AM Tom Lane wrote: >> That line of argument also leads to the conclusion that it'd be >> okay to expose info about the ordering of the CTE result to the >> upper planner. > In the light of this conclusion, I had a go at propagating the

Re: Use of backup_label not noted in log

2023-11-20 Thread David Steele
On 11/20/23 12:24, Robert Haas wrote: On Mon, Nov 20, 2023 at 5:35 AM Laurenz Albe wrote: I can accept that adding log messages to back branches is ok. Perhaps I am too nervous about things like that, because as an extension developer I have been bitten too often by ABI breaks in minor

Re: On non-Windows, hard depend on uselocale(3)

2023-11-20 Thread Tom Lane
Thomas Munro writes: > If we are sure that we'll *never* want locale-aware printf-family > functions (ie we *always* want "C" locale), then in the thought > experiment above where I suggested we supply replacement _l() > functions, we could just skip that for the printf family, but make > that

Re: should check collations when creating partitioned index

2023-11-20 Thread Tom Lane
Peter Eisentraut writes: > On 14.11.23 17:15, Tom Lane wrote: >> I don't love the patch details though. It seems entirely wrong to check >> this before we check the opclass match. > Not sure why? The order doesn't seem to matter? The case that was bothering me was if we had a non-collated

Re: Use of backup_label not noted in log

2023-11-20 Thread Robert Haas
On Mon, Nov 20, 2023 at 5:35 AM Laurenz Albe wrote: > I can accept that adding log messages to back branches is ok. > Perhaps I am too nervous about things like that, because as an extension > developer I have been bitten too often by ABI breaks in minor releases > in the past. I think that

Re: Add recovery to pg_control and remove backup_label

2023-11-20 Thread Robert Haas
On Sun, Nov 19, 2023 at 8:16 PM Michael Paquier wrote: > (I am not exactly sure how, but we've lost pgsql-hackers on the way > when you sent v5. Now added back in CC with the two latest patches > you've proposed attached.) > > Here is a short summary of what has been missed by the lists: > -

[PATCH] fix race condition in libpq (related to ssl connections)

2023-11-20 Thread Willi Mann
Hi, I've found a race condition in libpq. It is about the initialization of the my_bio_methods static variable in fe-secure-openssl.c, which is not protected by any lock. The race condition may make the initialization of the connection fail, and as an additional weird consequence, it might cause

Re: Inquiry on Generating Bitmaps from Filter Conditions in Index Scans

2023-11-20 Thread Jinjing Zhou
Thanks. Our project is at https://github.com/tensorchord/pgvecto.rs. A custom index is implemented for the vector similarity search, which implements `amgettuples` with direction support to provide candidates for the order by clause.  And we want to inject the filter condition using bitmap

Re: Inquiry on Generating Bitmaps from Filter Conditions in Index Scans

2023-11-20 Thread Jinjing Zhou
Thanks a lot! This is exactly what I'm asking. We've tried the CustomScanAPI at https://github.com/tensorchord/pgvecto.rs/pull/126, but met error with "variable not found in subplan target list". We're still investigating the root cause and thanks for your guidance! Best Jinjing Zhou > From:

Re: Show WAL write and fsync stats in pg_stat_io

2023-11-20 Thread Nazir Bilal Yavuz
Hi, Thanks for the feedback. On Mon, 20 Nov 2023 at 10:47, Michael Paquier wrote: > > On Thu, Nov 09, 2023 at 02:39:26PM +0300, Nazir Bilal Yavuz wrote: > > There are some differences between pg_stat_wal and pg_stat_io while > > collecting WAL stats. For example in the XLogWrite() function in

Re: Use of backup_label not noted in log

2023-11-20 Thread David Steele
On 11/20/23 06:35, Laurenz Albe wrote: If we add a message for starting with "backup_label", shouldn't we also add a corresponding message for starting from a checkpoint found in the control file? If you see that in a problem report, you immediately know what is going on. +1. It is easier to

Re: Use of backup_label not noted in log

2023-11-20 Thread David Steele
[Resending since I accidentally replied off-list] On 11/18/23 17:49, Andres Freund wrote: On 2023-11-18 10:01:42 -0800, Andres Freund wrote: What about adding it to the "redo starts at" message, something like redo starts at 12/12345678, taken from control file or redo starts at

Re: Add recovery to pg_control and remove backup_label

2023-11-20 Thread David Steele
On 11/19/23 21:15, Michael Paquier wrote: (I am not exactly sure how, but we've lost pgsql-hackers on the way when you sent v5. Now added back in CC with the two latest patches you've proposed attached.) Ugh, I must have hit reply instead of reply all. It's a rookie error and you hate to see

Re: Synchronizing slots from primary to standby

2023-11-20 Thread Drouvot, Bertrand
On 11/20/23 11:59 AM, Amit Kapila wrote: On Mon, Nov 20, 2023 at 3:17 PM Drouvot, Bertrand wrote: On 11/18/23 11:45 AM, Amit Kapila wrote: On Fri, Nov 17, 2023 at 5:18 PM Drouvot, Bertrand wrote: On 11/17/23 2:46 AM, Zhijie Hou (Fujitsu) wrote: On Tuesday, November 14, 2023 10:27 PM

Flushing large data immediately in pqcomm

2023-11-20 Thread Melih Mutlu
Hi hackers I've been looking into ways to reduce the overhead we're having in pqcomm and I'd like to propose a small patch to modify how socket_putmessage works. Currently socket_putmessage copies any input data into the pqcomm send buffer (PqSendBuffer) and the size of this buffer is 8K. When

Re: Inquiry on Generating Bitmaps from Filter Conditions in Index Scans

2023-11-20 Thread Matthias van de Meent
On Mon, 20 Nov 2023 at 09:30, Jinjing Zhou wrote: > > Hi hackers, > > I hope this message finds you well. I am reaching out to seek guidance on a > specific aspect of PostgreSQL's index scanning functionality. > > I am currently working on a vector search extension for postgres, where I > need

  1   2   >