Re: Slow catchup of 2PC (twophase) transactions on replica in LR

2024-07-21 Thread Amit Kapila
On Mon, Jul 22, 2024 at 8:26 AM Hayato Kuroda (Fujitsu) wrote: > > ``` > @@ -1089,6 +1089,12 @@ CheckAlterSubOption(Subscription *sub, const char > *option, > * is because both failover and two_phase options of the slot on the > * publisher cannot be modified if the slot is currently

Re: Exposing the lock manager's WaitForLockers() to SQL

2024-07-21 Thread Will Mortensen
On Thu, May 30, 2024 at 12:01 AM Will Mortensen wrote: > FWIW, another solution might be to directly expose the functions that > WaitForLockers() calls, namely GetLockConflicts() (generalized to > GetLockers() in the first patch) to identify the transactions holding > the locks, and VirtualXactLoc

Re: Add privileges test for pg_stat_statements to improve coverage

2024-07-21 Thread kuroda . keisuke
Hi Fujii-san, Thank you for your reply and comment! attach v2 fixed patch. meson.build needs to be updated as well, like the Makefile. Yes. Update 'contrib/pg_stat_statements/meson.build'. For the privileges test, should we explicitly set pg_stat_statements.track_utility at the start, as d

Re: Incremental backup from a streaming replication standby

2024-07-21 Thread Michael Paquier
On Sat, Jun 29, 2024 at 07:01:04AM +0200, Laurenz Albe wrote: > The WAL summarizer is running on the standby server, but when I try > to take an incremental backup, I get an error that I understand to mean > that WAL summarizing hasn't caught up yet. Added an open item for this one. -- Michael

Re: [BUG] Fix DETACH with FK pointing to a partitioned table fails

2024-07-21 Thread Tender Wang
Alvaro Herrera 于2024年7月19日周五 21:18写道: > Hello, > > I think the fix for the check triggers should be as the attached. Very > close to what you did, but you were skipping some operations that needed > to be kept. AFAICS this patch works correctly for the posted cases. > After applying the attach

Re: Allow logical failover slots to wait on synchronous replication

2024-07-21 Thread shveta malik
On Fri, Jul 19, 2024 at 2:52 AM John H wrote: > > Hi Shveta, > > Thanks for taking a look at the patch. > > > > will leave user no option to unlink failover-enabled logical > > > subscribers from the wait on synchronous standbys. > > That's a good point. I'm a bit biased in that I don't think ther

Re: Make reorder buffer max_changes_in_memory adjustable?

2024-07-21 Thread Jingtang Zhang
Thanks, Tomas. > Theoretically, yes, we could make max_changes_in_memory a GUC, but it's > not clear to me how would that help 12/13, because there's ~0% chance > we'd backpatch that ... What I mean is not about back-patch work. Things should happen on publisher side? Consider when the publisher

Re: Slow catchup of 2PC (twophase) transactions on replica in LR

2024-07-21 Thread Amit Kapila
On Sat, Jul 20, 2024 at 9:31 PM vignesh C wrote: > > On Thu, 18 Jul 2024 at 07:41, Hayato Kuroda (Fujitsu) > wrote: > > > > Dear Peter, > > > > Thanks for giving comments! PSA new version. > > Couple of suggestions: > 1) How will user know which all transactions should be rolled back > since the

Re: pg_verifybackup: TAR format backup verification

2024-07-21 Thread Sravan Kumar
Hi Amul, thanks for working on this. + file_name_len = strlen(relpath); > + if (file_name_len < file_extn_len || > + strcmp(relpath + file_name_len - file_extn_len, file_extn) != 0) > + { > + if (compress_algorithm == PG_COMPRESSION_NONE) > + report_backup_error(context, > + "\"%s\" is not a

RE: Slow catchup of 2PC (twophase) transactions on replica in LR

2024-07-21 Thread Hayato Kuroda (Fujitsu)
Dear Amit, > + /* > + * Do not allow changing the option if the subscription is enabled. This > + * is because both failover and two_phase options of the slot on the > + * publisher cannot be modified if the slot is currently acquired by the > + * existing walsender. > + */ > + if (sub->enabled) >

Re: Windows default locale vs initdb

2024-07-21 Thread Thomas Munro
Ertan Küçükoglu offered to try to review and test this, so here's a rebase. Some notes: * it turned out that the Turkish i/I test problem I mentioned earlier in this thread[1] was just always broken on Windows, we just didn't ever test with UTF-8 before Meson took over; it's skipped now, see comm

Re: Should consider materializing the cheapest inner path in consider_parallel_nestloop()

2024-07-21 Thread Richard Guo
On Fri, Jul 19, 2024 at 12:00 PM Alexander Lakhin wrote: > 18.07.2024 17:30, Richard Guo wrote: > > I have no idea why the underlying statistics changed, but it seems > > that this slight change is sufficent to result in a different plan. > > I think it could be caused by the same reason as [1] an

Re: documentation structure

2024-07-21 Thread jian he
1. manually change line 4 in split_func_sgml.py and run the script. 2. git apply v6-0001-all-filelist-for-directory-doc-src-sgml-func.patch now you can "ninja doc/src/sgml/html" The logic is simple, but I am using verbose variable names, that's why split_func_sgml.py size is large. adding a new d

Re: Patch bug: Fix jsonpath .* on Arrays

2024-07-21 Thread David E. Wheeler
On Jul 21, 2024, at 20:54, Michael Paquier wrote: > What I mean is that the main regression test suite did not complain on > your original patch posted here: > https://www.postgresql.org/message-id/A95346F9-6147-46E0-809E-532A485D71D6%40justatheory.com > > But the new tests showed a difference,

Re: Removing unneeded self joins

2024-07-21 Thread Andrei Lepikhov
On 20/7/2024 18:38, Alexander Korotkov wrote: On Fri, Jul 19, 2024 at 11:30 AM Dean Rasheed wrote: On Wed, 17 Jul 2024 at 01:45, Alexander Korotkov wrote: We initially didn't use ChangeVarNodes() in SJE at all. See the last patch version without it [1]. We're trying to address Tom Lane's pr

Re: pg_upgrade and logical replication

2024-07-21 Thread Michael Paquier
On Sat, Jul 20, 2024 at 09:03:07PM -0500, Nathan Bossart wrote: >> This is an extremely expensive way to perform that check, and so I'm >> wondering why we don't just do >> >> SELECT count(*) FROM pg_catalog.pg_subscription; >> >> once in count_old_cluster_subscriptions(). > > Like so... A

Re: POC, WIP: OR-clause support for indexes

2024-07-21 Thread Alexander Korotkov
On Mon, Jul 22, 2024 at 3:52 AM Alexander Korotkov wrote: > Please, check that there is still possibility to the generate BitmapOr plan. > > # explain select * from t where (b = 1 or b = 2 or a = 2 or a = 3); > QUERY PLAN > -

Re: Patch bug: Fix jsonpath .* on Arrays

2024-07-21 Thread Michael Paquier
On Fri, Jul 19, 2024 at 09:49:50AM -0400, David E. Wheeler wrote: >> It was fun to see that HEAD was silenced with the first patch of this >> thread that tweaked the behavior with arrays. > > Uh, what? Sorry I don’t follow. What I mean is that the main regression test suite did not complain on yo

Re: POC, WIP: OR-clause support for indexes

2024-07-21 Thread Alexander Korotkov
Hi, Alena! Let me answer to some of your findings. On Mon, Jul 22, 2024 at 12:53 AM Alena Rybakina wrote: > To be honest,I saw a larger problem. Look at the query bellow: > > master: > > alena@postgres=# create table t (a int not null, b int not null, c int not > null); > insert into t (select

Re: Internal error codes triggered by tests

2024-07-21 Thread Michael Paquier
On Thu, Jul 18, 2024 at 09:37:06AM +0200, Daniel Gustafsson wrote: > On 18 Jul 2024, at 09:29, Michael Paquier wrote: >> How about using a new error code in class 58, say a >> ERRCODE_FILE_NAME_TOO_LONG like in the attached? >> ERRCODE_DUPLICATE_FILE is like that; it exists just for the mapping >>

Re: xid_wraparound tests intermittent failure.

2024-07-21 Thread Thomas Munro
On Mon, Jul 22, 2024 at 8:08 AM Alexander Lakhin wrote: > https://wiki.postgresql.org/wiki/Known_Buildfarm_Test_Failures This is great. Thanks for collating all this info here! And of course all the research and reports behind it.

Re: POC, WIP: OR-clause support for indexes

2024-07-21 Thread Alena Rybakina
Hi! Thank you for your contribution to this thread! To be honest,I saw a larger problem. Look at the query bellow: master: alena@postgres=# create table t (a int not null, b int not null, c int not null); insert into t (select 1, 1, i from generate_series(1,1) i); insert into t (select i,

Re: xid_wraparound tests intermittent failure.

2024-07-21 Thread Tom Lane
Andrew Dunstan writes: > On 2024-07-21 Su 1:34 PM, Tom Lane wrote: >> Locally, I've not managed to reproduce the failure yet; so perhaps >> there is some platform dependency. What are you testing on? > Linux ub22arm 5.15.0-116-generic #126-Ubuntu SMP Mon Jul 1 10:08:40 UTC > 2024 aarch64 aarch6

Re: CI, macports, darwin version problems

2024-07-21 Thread Thomas Munro
On Mon, Jul 22, 2024 at 8:34 AM Joe Conway wrote: > Hmmm, maybe nevermind? I rebooted the mac mini and now it seems to be > working. Maybe someone can confirm. There ought to be plenty of space > available for sonoma and ventura at the same time now. Thanks for doing that. Initial results are th

Re: Vacuum ERRORs out considering freezing dead tuples from before OldestXmin

2024-07-21 Thread Peter Geoghegan
On Sun, Jul 21, 2024 at 5:04 PM Tom Lane wrote: > > There will always be a small number of extremely slow buildfarm > > animals. Optimizing for things like Raspberry pi animals with SD cards > > just doesn't seem like a good use of developer time. I really care > > about keeping the tests fast, bu

Re: Vacuum ERRORs out considering freezing dead tuples from before OldestXmin

2024-07-21 Thread Tom Lane
Peter Geoghegan writes: > On Sun, Jul 21, 2024 at 12:51 PM Tom Lane wrote: >> I do not think the answer to this is to nag the respective animal >> owners to raise PG_TEST_TIMEOUT_DEFAULT. IMV this test is simply >> not worth the cycles it takes, at least not for these machines. > Can't we just

Re: WIP: parallel GiST index builds

2024-07-21 Thread Tomas Vondra
On 7/21/24 21:31, Andrey M. Borodin wrote: > Hi Tomas! > >> On 7 Jun 2024, at 20:41, Tomas Vondra >> wrote: >> >> After looking into parallel builds for BRIN and GIN indexes, I was >> wondering if there's a way to do parallel builds for GiST too. I >> knew next to nothing about how GiST works,

Re: CI, macports, darwin version problems

2024-07-21 Thread Joe Conway
On 7/21/24 16:15, Joe Conway wrote: On 7/18/24 10:33, Joe Conway wrote: On 7/18/24 10:23, Nazir Bilal Yavuz wrote: On Thu, 18 Jul 2024 at 17:01, Joe Conway wrote: So perhaps I am back to needing more storage... You might not need more storage. Thomas knows better, but AFAIU, CFBot will pull

Re: Vacuum ERRORs out considering freezing dead tuples from before OldestXmin

2024-07-21 Thread Peter Geoghegan
On Sun, Jul 21, 2024 at 12:51 PM Tom Lane wrote: > I do not think the answer to this is to nag the respective animal > owners to raise PG_TEST_TIMEOUT_DEFAULT. IMV this test is simply > not worth the cycles it takes, at least not for these machines. Can't we just move it to PG_TEST_EXTRA? Alongs

Re: CI, macports, darwin version problems

2024-07-21 Thread Joe Conway
On 7/18/24 10:33, Joe Conway wrote: On 7/18/24 10:23, Nazir Bilal Yavuz wrote: On Thu, 18 Jul 2024 at 17:01, Joe Conway wrote: So perhaps I am back to needing more storage... You might not need more storage. Thomas knows better, but AFAIU, CFBot will pull only sonoma images after the patch i

Re: Lock-free compaction. Why not?

2024-07-21 Thread Tomas Vondra
Please don't top-post ... On 7/21/24 16:42, Ahmed Yarub Hani Al Nuaimi wrote: > That clearly explains the problem. But this got me thinking: what if we do > both index and heap optimization at the same time? > Meaning that the newly move heap tuple which is used to compact/defragment > heap pages

Re: xid_wraparound tests intermittent failure.

2024-07-21 Thread Alexander Lakhin
Hello, 21.07.2024 20:34, Tom Lane wrote: Andrew Dunstan writes: I noticed this when working on the PostgreSQL::Test::Session project I have in hand. All the tests pass except occasionally the xid_wraparound tests fail. It's not always the same test script that fails either. I tried everything

Re: WIP: parallel GiST index builds

2024-07-21 Thread Andrey M. Borodin
Hi Tomas! > On 7 Jun 2024, at 20:41, Tomas Vondra wrote: > > After looking into parallel builds for BRIN and GIN indexes, I was > wondering if there's a way to do parallel builds for GiST too. I knew > next to nothing about how GiST works, but I gave it a shot and here's > what I have - the atta

Re: xid_wraparound tests intermittent failure.

2024-07-21 Thread Andrew Dunstan
On 2024-07-21 Su 1:34 PM, Tom Lane wrote: Andrew Dunstan writes: I noticed this when working on the PostgreSQL::Test::Session project I have in hand. All the tests pass except occasionally the xid_wraparound tests fail. It's not always the same test script that fails either. I tried everythin

Re: [PATCH] TODO “Allow LISTEN on patterns”

2024-07-21 Thread Emanuel Calvo
Hi Alexander, I did a review on the new patch version and I observed that the identifier passed to the LISTEN command is handled differently between outer and inner levels. When the outer level exceeds the 64 characters limitation, the outer level of the channel name is truncated, but leaves the

Re: xid_wraparound tests intermittent failure.

2024-07-21 Thread Tom Lane
Andrew Dunstan writes: > I noticed this when working on the PostgreSQL::Test::Session project I > have in hand. All the tests pass except occasionally the xid_wraparound > tests fail. It's not always the same test script that fails either. I > tried everything but couldn't make the failure stop

Re: Vacuum ERRORs out considering freezing dead tuples from before OldestXmin

2024-07-21 Thread Tom Lane
Melanie Plageman writes: > When I run it on my machine with some added logging, the space taken > by dead items is about 330 kB more than maintenance_work_mem (which is > set to 1 MB). I could roughly double the excess by increasing the > number of inserted tuples from 40 to 60. I'll do th

xid_wraparound tests intermittent failure.

2024-07-21 Thread Andrew Dunstan
I noticed this when working on the PostgreSQL::Test::Session project I have in hand. All the tests pass except occasionally the xid_wraparound tests fail. It's not always the same test script that fails either. I tried everything but couldn't make the failure stop. So then I switched out my

Re: [BUG?] check_exclusion_or_unique_constraint false negative

2024-07-21 Thread Michail Nikolaev
Hello, Andres. Sorry to bother you, but I feel it's necessary to validate the possible issue regarding someone who can decide whether it is okay or not. The issue is reproducible with the first UPSERT implementation (your commit 168d5805e4c08bed7b95d351bf097cff7c07dd65 from 2015) and up to now. T

Re: why there is not VACUUM FULL CONCURRENTLY?

2024-07-21 Thread Pavel Stehule
Hi ne 21. 7. 2024 v 17:13 odesílatel Kirill Reshke napsal: > Hi! > I'm interested in the vacuum concurrently feature being inside the > core, so will try to review patch set and give valuable feedback. For > now, just a few little thoughts.. > > > > One more thing is about pg_squeeze background

Re: why there is not VACUUM FULL CONCURRENTLY?

2024-07-21 Thread Kirill Reshke
Hi! I'm interested in the vacuum concurrently feature being inside the core, so will try to review patch set and give valuable feedback. For now, just a few little thoughts.. > The first version is attached. The actual feature is in 0003. 0004 is probably > not necessary now, but I haven't realiz

Re: Lock-free compaction. Why not?

2024-07-21 Thread Ahmed Yarub Hani Al Nuaimi
That clearly explains the problem. But this got me thinking: what if we do both index and heap optimization at the same time? Meaning that the newly move heap tuple which is used to compact/defragment heap pages would be followed by moving the index (creating and then deleting) a new index tuple at

Re: Trying to build x86 version on windows using meson

2024-07-21 Thread Thomas Munro
On Thu, Mar 21, 2024 at 11:21 AM Andres Freund wrote: > Obviously tests can't run in that environment, but building should be > possible. I can e.g. build postgres for x86-64 windows on my linux machine, > but can't run the tests (in theory they could be run with wine, but wine isn't > complete en

Re: REINDEX not updating partition progress

2024-07-21 Thread Ilya Gladyshev
Forgot to update partition_relid in reindex_index in the second patch. Fixed in attachment. v2-0001-make-REINDEX-track-partition-progress.patch Description: Binary data v2-0002-partition_relid-column-for-create-index-progress.patch Description: Binary data 21 июля 2024 г., в 01:49, Ilya Gladyshe

Re: Make reorder buffer max_changes_in_memory adjustable?

2024-07-21 Thread Tomas Vondra
On 7/21/24 07:19, Jingtang Zhang wrote: > Hi hackers. > > Recently I came to an issue about logical replicating very big > transactions. Since we already have logical_decoding_work_mem to keep > the memory usage, there is no risk of OOM during decoding. However, the > memory usage still goes out o

Re: Lock-free compaction. Why not?

2024-07-21 Thread Tomas Vondra
On 7/21/24 04:13, Tom Lane wrote: > David Rowley writes: >> No resource links, but if you move a tuple to another page then you >> must also adjust the index. If you have no exclusive lock on the >> table, then you must assume older transactions still need the old >> tuple version, so you need to

Re: POC, WIP: OR-clause support for indexes

2024-07-21 Thread Nikolay Shaplov
В письме от среда, 17 июля 2024 г. 22:36:19 MSK пользователь Alexander Korotkov написал: Hi All! I am continue reading the patch, now it's newer version First main question: As far a I can get, the entry point for OR->ANY convertation have been moved to match_clause_to_indexcol funtion, that