Re: Lock-free compaction. Why not?

2024-07-22 Thread Robert Haas
On Mon, Jul 22, 2024 at 1:00 PM Ahmed Yarub Hani Al Nuaimi wrote: > That is a very useful thread and I'll keep on following it but it is not > exactly what I'm trying to achieve here. > You see, there is a great difference between VACUUM FULL CONCURRENTLY and > adding compaction to lazy

Re: [18] Policy on IMMUTABLE functions and Unicode updates

2024-07-22 Thread Laurenz Albe
On Mon, 2024-07-22 at 16:26 +0200, Peter Eisentraut wrote: > I propose that, going forward, we take more care with Unicode updates: > > assess the impact, provide time for comments, and consider possible > > mitigations. In other words, it would be reviewed like any other > > change. > > I

Re: Vacuum ERRORs out considering freezing dead tuples from before OldestXmin

2024-07-22 Thread Robert Haas
On Mon, Jul 22, 2024 at 11:48 AM Tom Lane wrote: > I'm a little suspicious > of using it for tests that merely take an unreasonable amount of > time --- to me, that indicates laziness on the part of the test > author. Laziness would have been not bothering to develop a TAP test for this at all.

Re: Add support to TLS 1.3 cipher suites and curves lists

2024-07-22 Thread Jacob Champion
be an invasive > change for adding suffix 's'. Can we just add an entry to map_old_guc_names to handle it? Something like (untested) static const char *const map_old_guc_names[] = { "sort_mem", "work_mem", "vacuum_mem", "maintenance_work_mem", +

Re: Add new COPY option REJECT_LIMIT

2024-07-22 Thread Kirill Reshke
Hi! Nice feature. Few comments: > + When a positive integer value is specified, COPY > limits > + the maximum tolerable number of errors while converting a column's > input > + value into its data type. If nothing is specified, then the maximum tolerable number of errors is

Re: Incremental backup from a streaming replication standby fails

2024-07-22 Thread Laurenz Albe
On Mon, 2024-07-22 at 09:37 -0400, Robert Haas wrote: > How about something like this: > > An incremental backup is only possible if replay would begin from a > later checkpoint than for the previous backup upon which it depends. > On the primary, this condition is always satisfied, because each

Re: Lock-free compaction. Why not?

2024-07-22 Thread Ahmed Yarub Hani Al Nuaimi
That is a very useful thread and I'll keep on following it but it is not exactly what I'm trying to achieve here. You see, there is a great difference between VACUUM FULL CONCURRENTLY and adding compaction to lazy vacuuming. The main factor here is resource utilization: a lot of companies have

Re: Add privileges test for pg_stat_statements to improve coverage

2024-07-22 Thread Fujii Masao
On 2024/07/22 15:23, kuroda.keis...@nttcom.co.jp wrote: Hi Fujii-san, Thank you for your reply and comment! attach v2 fixed patch. Thanks for updating the patch! +SELECT query, calls, rows FROM pg_stat_statements + WHERE queryid IS NULL ORDER BY query COLLATE "C"; Shouldn't we also

Re: Vacuum ERRORs out considering freezing dead tuples from before OldestXmin

2024-07-22 Thread Andres Freund
Hi, On 2024-07-21 12:51:51 -0400, Tom Lane wrote: > Melanie Plageman writes: > > When I run it on my machine with some added logging, the space taken > > by dead items is about 330 kB more than maintenance_work_mem (which is > > set to 1 MB). I could roughly double the excess by increasing the >

Re: xid_wraparound tests intermittent failure.

2024-07-22 Thread Tom Lane
Masahiko Sawada writes: > Looking at dodo's failures, it seems that while it passes > module-xid_wraparound-check, all failures happened only during > testmodules-install-check-C. Can we check the server logs written > during xid_wraparound test in testmodules-install-check-C? Oooh, that is

Re: [18] Policy on IMMUTABLE functions and Unicode updates

2024-07-22 Thread Jeff Davis
On Mon, 2024-07-22 at 11:14 -0400, Robert Haas wrote: > On Mon, Jul 22, 2024 at 10:26 AM Peter Eisentraut > wrote: > > I disagree with that.  We should put ourselves into the position to > > adopt new Unicode versions without fear.  Similar to updates to > > time > > zones, snowball, etc. > > >

Re: warn if GUC set to an invalid shared library

2024-07-22 Thread Justin Pryzby
On Fri, May 24, 2024 at 01:15:13PM -0400, Robert Haas wrote: > + /* Note that filename was already canonicalized */ > > I see that this comment is copied from load_libraries(), but I don't > immediately see where the canonicalization actually happens. Do you > know, or can you find out? Because

Re: xid_wraparound tests intermittent failure.

2024-07-22 Thread Masahiko Sawada
On Sun, Jul 21, 2024 at 2:36 PM Tom Lane wrote: > > Andrew Dunstan writes: > > On 2024-07-21 Su 1:34 PM, Tom Lane wrote: > >> Locally, I've not managed to reproduce the failure yet; so perhaps > >> there is some platform dependency. What are you testing on? > > > Linux ub22arm

Re: Vacuum ERRORs out considering freezing dead tuples from before OldestXmin

2024-07-22 Thread Peter Geoghegan
On Mon, Jul 22, 2024 at 11:49 AM Tom Lane wrote: > >> Andres has suggested in the past that we allow maintenance_work_mem be > >> set to a lower value or introduce some kind of development GUC so that > >> we can more easily test multiple pass index vacuuming. Do you think > >> this would be

Re: Vacuum ERRORs out considering freezing dead tuples from before OldestXmin

2024-07-22 Thread Tom Lane
Alvaro Herrera writes: > I think it's a bad idea to require buildfarm owners to edit their config > files as we add tests that depend on PG_TEST_EXTRA. AFAIR we invented > that setting so that tests that had security implications could be made > opt-in instead of opt-out; I think this was a

Re: Enhance pg_dump multi-threaded streaming (WAS: Re: filesystem full during vacuum - space recovery issues)

2024-07-22 Thread Andrew Dunstan
On 2024-07-19 Fr 9:46 AM, Thomas Simpson wrote: Hi Scott, I realize some of the background was snipped on what I sent to the hacker list, I'll try to fill in the details. Short background is very large database ran out of space during vacuum full taking down the server.  There is a

Re: Vacuum ERRORs out considering freezing dead tuples from before OldestXmin

2024-07-22 Thread Tom Lane
Peter Geoghegan writes: > On Mon, Jul 22, 2024 at 9:32 AM Melanie Plageman > wrote: >> Andres has suggested in the past that we allow maintenance_work_mem be >> set to a lower value or introduce some kind of development GUC so that >> we can more easily test multiple pass index vacuuming. Do you

Re: Vacuum ERRORs out considering freezing dead tuples from before OldestXmin

2024-07-22 Thread Tom Lane
Melanie Plageman writes: > On Sun, Jul 21, 2024 at 5:04 PM Tom Lane wrote: >> I note also that the PG_TEST_EXTRA approach has caused xid_wraparound >> to get next-to-zero buildfarm coverage. If that test is actually >> capable of revealing problems, we're unlikely to find out under the >>

Re: [18] Policy on IMMUTABLE functions and Unicode updates

2024-07-22 Thread Jeff Davis
On Mon, 2024-07-22 at 16:26 +0200, Peter Eisentraut wrote: > Unless I missed something here, all the problem examples involve > unassigned code points that were later assigned. For normalization and case mapping that's right. For regexes, a character property could change. But that's mostly a

Re: Remove dependence on integer wrapping

2024-07-22 Thread Nathan Bossart
On Fri, Jul 19, 2024 at 07:32:18PM -0400, Joseph Koshakow wrote: > On Fri, Jul 19, 2024 at 2:45 PM Nathan Bossart > wrote: >> +/* dim[i] = 1 + upperIndx[i] - lowerIndx[i]; */ >> +if (pg_add_s32_overflow(1, upperIndx[i], [i])) >> +ereport(ERROR, >> +

Re: Vacuum ERRORs out considering freezing dead tuples from before OldestXmin

2024-07-22 Thread Peter Geoghegan
On Mon, Jul 22, 2024 at 9:32 AM Melanie Plageman wrote: > Andres has suggested in the past that we allow maintenance_work_mem be > set to a lower value or introduce some kind of development GUC so that > we can more easily test multiple pass index vacuuming. Do you think > this would be worth it?

Re: [18] Policy on IMMUTABLE functions and Unicode updates

2024-07-22 Thread Robert Haas
On Mon, Jul 22, 2024 at 10:26 AM Peter Eisentraut wrote: > I disagree with that. We should put ourselves into the position to > adopt new Unicode versions without fear. Similar to updates to time > zones, snowball, etc. > > We can't be discussing the merits of the Unicode update every year. >

Re: Custom explain options

2024-07-22 Thread Pavel Stehule
Hi po 22. 7. 2024 v 17:08 odesílatel Konstantin Knizhnik napsal: > > On 16/01/2024 5:38 pm, Tomas Vondra wrote: > > By "broken" you mean that you prefetch items only from a single leaf > > page, so immediately after reading the next one nothing is prefetched. > Correct? > > > Yes, exactly. It

Re: Assertion failure with summarize_wal enabled during pg_createsubscriber

2024-07-22 Thread Robert Haas
I went ahead and committed 0001, which is pretty trivial. Hopefully, at least, it's trivial enough that I didn't mess it up. Here's a rebase of 0002 and 0003, now 0001 and 0002, with one minor fix to hopefully avoid annoying CI. I'm still hoping for some review/feedback/testing of these before I

Re: pg_upgrade and logical replication

2024-07-22 Thread Nathan Bossart
On Mon, Jul 22, 2024 at 03:45:19PM +0530, Amit Kapila wrote: > On Mon, Jul 22, 2024 at 7:35 AM Michael Paquier wrote: >> On Sat, Jul 20, 2024 at 09:03:07PM -0500, Nathan Bossart wrote: >> >> This is an extremely expensive way to perform that check, and so I'm >> >> wondering why we don't just do

Re: Cannot find a working 64-bit integer type on Illumos

2024-07-22 Thread Peter Eisentraut
On 14.07.24 16:51, Tom Lane wrote: Peter Eisentraut writes: On 04.07.24 03:55, Thomas Munro wrote: Personally, I find "PRId64" pretty unreadable. "INT64_MODIFIER" wasn't nice either, though, and following standards is good, so I'm sure I'll get used to it. Using PRId64 would be very

Re: Send duration output to separate log files

2024-07-22 Thread Alastair Turner
On Fri, 12 Jul 2024 at 15:58, Greg Sabino Mullane wrote: > On Thu, Jul 11, 2024 at 6:47 AM Alastair Turner > wrote: > >> The other category of logging which would benefit from a separate file >> is audit. It also can create massive volumes of log content. Splitting >> audit information off

Re: [18] Policy on IMMUTABLE functions and Unicode updates

2024-07-22 Thread Peter Eisentraut
On 19.07.24 21:41, Jeff Davis wrote: On Fri, 2024-07-19 at 21:06 +0200, Laurenz Albe wrote: Perhaps I should moderate my statement: if a change affects only a newly introduced code point (which is unlikely to be used in a database), and we think that the change is very important, we could

Re: Enhance pg_dump multi-threaded streaming (WAS: Re: filesystem full during vacuum - space recovery issues)

2024-07-22 Thread Scott Ribe
Do you actually have 100G networking between the nodes? Because if not, a single CPU should be able to saturate 10G. Likewise the receiving end would need disk capable of keeping up. Which brings up the question, why not write to disk, but directly to the destination rather than write locally

Re: Vacuum ERRORs out considering freezing dead tuples from before OldestXmin

2024-07-22 Thread Alvaro Herrera
On 2024-Jul-22, Melanie Plageman wrote: > On Sun, Jul 21, 2024 at 5:04 PM Tom Lane wrote: > > I note also that the PG_TEST_EXTRA approach has caused xid_wraparound > > to get next-to-zero buildfarm coverage. If that test is actually > > capable of revealing problems, we're unlikely to find out

Re: Build with LTO / -flto on macOS

2024-07-22 Thread Peter Eisentraut
On 19.07.24 12:40, Aleksander Alekseev wrote: It seems to me that the patch is not going to become any better and it doesn't need any more attention from the reviewers. Thus I changed the status of the CF entry to "Ready for Committer". I'm happy to commit this patch. I checked that for

Re: Windows default locale vs initdb

2024-07-22 Thread Ertan Küçükoglu
Andrew Dunstan , 22 Tem 2024 Pzt, 16:44 tarihinde şunu yazdı: > I have an environment I can use for testing. But what exactly am I > testing? :-) Install a few "problem" language/region settings, switch > the system and ensure initdb runs ok? > > Other than Turkish, which locales should I

Re: Windows default locale vs initdb

2024-07-22 Thread Andrew Dunstan
On 2024-07-21 Su 10:51 PM, Thomas Munro wrote: Ertan Küçükoglu offered to try to review and test this, so here's a rebase. Some notes: * it turned out that the Turkish i/I test problem I mentioned earlier in this thread[1] was just always broken on Windows, we just didn't ever test with

Re: Incremental backup from a streaming replication standby fails

2024-07-22 Thread Robert Haas
On Fri, Jul 19, 2024 at 6:07 PM Laurenz Albe wrote: > Here is a patch. > I went for both the errhint and some documentation. Hmm, the hint doesn't end up using the word "standby" anywhere. That seems like it might not be optimal? +Like a base backup, you can take an incremental backup from

Re: Vacuum ERRORs out considering freezing dead tuples from before OldestXmin

2024-07-22 Thread Melanie Plageman
On Sun, Jul 21, 2024 at 4:29 PM Peter Geoghegan wrote: > > On Sun, Jul 21, 2024 at 12:51 PM Tom Lane wrote: > > I do not think the answer to this is to nag the respective animal > > owners to raise PG_TEST_TIMEOUT_DEFAULT. IMV this test is simply > > not worth the cycles it takes, at least not

Re: Vacuum ERRORs out considering freezing dead tuples from before OldestXmin

2024-07-22 Thread Melanie Plageman
On Sun, Jul 21, 2024 at 5:04 PM Tom Lane wrote: > > Peter Geoghegan writes: > > On Sun, Jul 21, 2024 at 12:51 PM Tom Lane wrote: > >> I do not think the answer to this is to nag the respective animal > >> owners to raise PG_TEST_TIMEOUT_DEFAULT. IMV this test is simply > >> not worth the

Re: Vacuum ERRORs out considering freezing dead tuples from before OldestXmin

2024-07-22 Thread Melanie Plageman
On Sun, Jul 21, 2024 at 12:51 PM Tom Lane wrote: > > Melanie Plageman writes: > > When I run it on my machine with some added logging, the space taken > > by dead items is about 330 kB more than maintenance_work_mem (which is > > set to 1 MB). I could roughly double the excess by increasing the

Re: xid_wraparound tests intermittent failure.

2024-07-22 Thread Robert Haas
On Sun, Jul 21, 2024 at 7:28 PM Thomas Munro wrote: > On Mon, Jul 22, 2024 at 8:08 AM Alexander Lakhin wrote: > > https://wiki.postgresql.org/wiki/Known_Buildfarm_Test_Failures > > This is great. Thanks for collating all this info here! And of > course all the research and reports behind it.

Re: xid_wraparound tests intermittent failure.

2024-07-22 Thread Andrew Dunstan
On 2024-07-21 Su 4:08 PM, Alexander Lakhin wrote: Hello, 21.07.2024 20:34, Tom Lane wrote: Andrew Dunstan writes: I noticed this when working on the PostgreSQL::Test::Session project I have in hand. All the tests pass except occasionally the xid_wraparound tests fail. It's not always the

Re: Windows default locale vs initdb

2024-07-22 Thread Ertan Küçükoglu
Thomas Munro , 22 Tem 2024 Pzt, 14:00 tarihinde şunu yazdı: > Sorry, I didn't mean to put you on the spot :-) Yeah you'd need to > install a compiler, various libraries and tools to be able to build > form source with a patch. Unfortunately I'm not the best person to > explain how to do that on

Re: CI, macports, darwin version problems

2024-07-22 Thread Joe Conway
On 7/21/24 17:26, Thomas Munro wrote: On Mon, Jul 22, 2024 at 8:34 AM Joe Conway wrote: Hmmm, maybe nevermind? I rebooted the mac mini and now it seems to be working. Maybe someone can confirm. There ought to be plenty of space available for sonoma and ventura at the same time now. Thanks

Re: Lock-free compaction. Why not?

2024-07-22 Thread Michael Banck
Hi, On Mon, Jul 22, 2024 at 08:39:23AM -0400, Robert Haas wrote: > What the extensions that are out there seem to do is, as I understand > it, an online table rewrite with concurrent change capture, and then > you apply the changes to the output table afterward. That has the > problem that if the

Re: Lock-free compaction. Why not?

2024-07-22 Thread Robert Haas
On Sat, Jul 20, 2024 at 10:13 PM Tom Lane wrote: > The actually tricky part about that is that you have to ensure that > any concurrent scan will see one of the two copies --- not both, > and not neither. This is fairly hard when the concurrent query > might be using any of several scan methods,

Re: Add new COPY option REJECT_LIMIT

2024-07-22 Thread torikoshia
On Fri, Jul 19, 2024 at 11:48 PM Junwang Zhao wrote: Thanks for the comment. In patch 0002, the ratio is calculated by the already skipped/processed rows, but what if a user wants to copy 1000 rows, and he/she can tolerate 10 error rows, so he/she might set *reject_limit 0.01*, but one bad

Re: Make COPY format extendable: Extract COPY TO format implementations

2024-07-22 Thread Tomas Vondra
On 7/22/24 09:45, Sutou Kouhei wrote: > Hi Tomas, > > Thanks for joining this thread! > > In <257d5573-07da-48c3-ac07-e047e7a65...@enterprisedb.com> > "Re: Make COPY format extendable: Extract COPY TO format implementations" > on Fri, 19 Jul 2024 14

Re: Special-case executor expression steps for common combinations

2024-07-22 Thread Andreas Karlsson
On 7/4/24 6:26 PM, Daniel Gustafsson wrote: 2) We could generate functions which return void rather than NULL and therefore not have to do a return at all but I am not sure that small optimization and extra clarity would be worth the hassle. The current approach with adding Assert() is ok

Re: WIP: parallel GiST index builds

2024-07-22 Thread Andrey M. Borodin
> On 22 Jul 2024, at 14:53, Tomas Vondra wrote: > > > > On 7/22/24 13:08, Andrey M. Borodin wrote: >> >> >>> On 22 Jul 2024, at 12:26, Tomas Vondra >>> wrote: >>> >>> I don't understand how would that solve the problem, can you >>> elaborate? Which of the values are you suggesting

Re: WIP: parallel GiST index builds

2024-07-22 Thread Tomas Vondra
On 7/22/24 13:08, Andrey M. Borodin wrote: > > >> On 22 Jul 2024, at 12:26, Tomas Vondra >> wrote: >> >> I don't understand how would that solve the problem, can you >> elaborate? Which of the values are you suggesting should be >> replaced with the shared counter? lastlsn? > > I think

Re: [EXTERNAL] Re: Add non-blocking version of PQcancel

2024-07-22 Thread Alvaro Herrera
On 2024-Jul-16, Alvaro Herrera wrote: > On 2024-Jul-16, Alvaro Herrera wrote: > > > Maybe we can disable this test specifically on Cygwin. We could do that > > by creating a postgres_fdw_cancel.sql file, with the current output for > > all platforms, and a "SELECT version() ~ 'cygwin' AS

Re: WIP: parallel GiST index builds

2024-07-22 Thread Andrey M. Borodin
> On 22 Jul 2024, at 12:26, Tomas Vondra wrote: > > I don't understand how would that solve the problem, can you elaborate? > Which of the values are you suggesting should be replaced with the > shared counter? lastlsn? I think during build we should consider index unlogged and always use

Re: Windows default locale vs initdb

2024-07-22 Thread Thomas Munro
On Mon, Jul 22, 2024 at 8:04 PM Ertan Küçükoglu wrote: > I am a complete noob about PostgreSQL development. > I don't know about the PostgreSQL CI system. > I will be needing some help as to how to do the tests. > I have access to different Windows OSes (v10, Server 2022 mainly). > These systems

Re: Virtual generated columns

2024-07-22 Thread jian he
another bug? drop table gtest12v; CREATE TABLE gtest12v (a int PRIMARY KEY, b bigint, c int GENERATED ALWAYS AS (b * 2) VIRTUAL); insert into gtest12v (a,b) values (11, 22147483647); table gtest12v; insert ok, but select error: ERROR: integer out of range should insert fail? CREATE TABLE

Re: pg_upgrade and logical replication

2024-07-22 Thread Amit Kapila
On Mon, Jul 22, 2024 at 7:35 AM Michael Paquier wrote: > > On Sat, Jul 20, 2024 at 09:03:07PM -0500, Nathan Bossart wrote: > >> This is an extremely expensive way to perform that check, and so I'm > >> wondering why we don't just do > >> > >> SELECT count(*) FROM pg_catalog.pg_subscription;

Re: Windows default locale vs initdb

2024-07-22 Thread Thomas Munro
On Mon, Jul 22, 2024 at 8:38 PM Zaid Shabbir wrote: > Can you please list down some of the use cases for the patch ? Other than > Turkish, does this patch have an impact on other locales too ? Hi Zaid, Yes, initdb.exe would use BCP47 codes by default for all languages. Who knows which country

Re: Possible incorrect row estimation for Gather paths

2024-07-22 Thread Richard Guo
On Mon, Jul 22, 2024 at 4:47 PM Anthonin Bonnefoy wrote: > On Wed, Jul 17, 2024 at 3:59 AM Richard Guo wrote: > > I can reproduce this problem with the query below. > > > > explain (costs on) select * from tenk1 order by twenty; > >QUERY PLAN > >

Re: why there is not VACUUM FULL CONCURRENTLY?

2024-07-22 Thread Michael Banck
On Mon, Jul 22, 2024 at 01:23:03PM +0500, Kirill Reshke wrote: > Also, should we create a cf entry for this thread already? I was wondering about this as well, but there is one for the upcoming commitfest already: https://commitfest.postgresql.org/49/5117/ Michael

Re: Make reorder buffer max_changes_in_memory adjustable?

2024-07-22 Thread Tomas Vondra
On 7/22/24 05:28, Jingtang Zhang wrote: > Thanks, Tomas. > >> Theoretically, yes, we could make max_changes_in_memory a GUC, but it's >> not clear to me how would that help 12/13, because there's ~0% chance >> we'd backpatch that ... > > What I mean is not about back-patch work. Things should

Re: WIP: parallel GiST index builds

2024-07-22 Thread Tomas Vondra
On 7/22/24 11:00, Andrey M. Borodin wrote: > > >> On 21 Jul 2024, at 23:42, Tomas Vondra wrote: >> >>> >>> 1. Do I get it right that is_parallel argument for gistGetFakeLSN() >>> is only needed for assertion? And this assertion can be ensured just >>> by inspecting code. Is it really necessary?

Re: Slow catchup of 2PC (twophase) transactions on replica in LR

2024-07-22 Thread Peter Smith
Hi, Patch v22-0001 LGTM apart from the following nitpicks == src/sgml/ref/alter_subscription.sgml nitpick - /one needs to/you need to/ == src/backend/commands/subscriptioncmds.c CheckAlterSubOption: nitpick = "ideally we could have..." doesn't make sense because the code uses a more

Re: Conflict detection and logging in logical replication

2024-07-22 Thread shveta malik
On Fri, Jul 19, 2024 at 2:06 PM shveta malik wrote: > > On Thu, Jul 18, 2024 at 7:52 AM Zhijie Hou (Fujitsu) > wrote: > > > > Attach the V5 patch set which changed the following. > Please find last batch of comments on v5: patch001: 1) create subscription sub1 ... (detect_conflict=true); I

Re: Use read streams in CREATE DATABASE command when the strategy is wal_log

2024-07-22 Thread Nazir Bilal Yavuz
Hi, On Sat, 20 Jul 2024 at 21:14, Noah Misch wrote: > > On Sat, Jul 20, 2024 at 03:01:31PM +0300, Nazir Bilal Yavuz wrote: > > > > With the separate commit (e00c45f685), does it make sense to rename > > the smgr_persistence parameter of the ReadBuffer_common() to > > persistence? Because,

Re: WIP: parallel GiST index builds

2024-07-22 Thread Andrey M. Borodin
> On 21 Jul 2024, at 23:42, Tomas Vondra wrote: > >> >> 1. Do I get it right that is_parallel argument for gistGetFakeLSN() >> is only needed for assertion? And this assertion can be ensured just >> by inspecting code. Is it really necessary? > > Yes, in the patch it's only used for an

Re: proposal: schema variables

2024-07-22 Thread Pavel Stehule
po 22. 7. 2024 v 10:23 odesílatel Laurenz Albe napsal: > Thanks for the updated patch and the fixes! > > On Mon, 2024-07-22 at 08:37 +0200, Pavel Stehule wrote: > > > > --- a/doc/src/sgml/ref/pg_restore.sgml > > > > +++ b/doc/src/sgml/ref/pg_restore.sgml > > > > > > > + > > > > + -A

Re: Possible incorrect row estimation for Gather paths

2024-07-22 Thread Anthonin Bonnefoy
On Wed, Jul 17, 2024 at 3:59 AM Richard Guo wrote: > > I can reproduce this problem with the query below. > > explain (costs on) select * from tenk1 order by twenty; >QUERY PLAN > -

Re: pgsql: Add more SQL/JSON constructor functions

2024-07-22 Thread Amit Langote
On Thu, Jul 18, 2024 at 3:04 PM jian he wrote: > we still have problem in transformJsonBehavior > > currently transformJsonBehavior: > SELECT JSON_VALUE(jsonb '1234', '$' RETURNING bit(3) DEFAULT 010111 ON > ERROR); > ERROR: cannot cast behavior expression of type text to bit > LINE 1:

Re: Windows default locale vs initdb

2024-07-22 Thread Zaid Shabbir
Hello Thomas, Can you please list down some of the use cases for the patch ? Other than Turkish, does this patch have an impact on other locales too ? Regards, Zaid On Mon, Jul 22, 2024 at 7:52 AM Thomas Munro wrote: > Ertan Küçükoglu offered to try to review and test this, so here's a

Re: proposal: schema variables

2024-07-22 Thread Laurenz Albe
Thanks for the updated patch and the fixes! On Mon, 2024-07-22 at 08:37 +0200, Pavel Stehule wrote: > > > --- a/doc/src/sgml/ref/pg_restore.sgml > > > +++ b/doc/src/sgml/ref/pg_restore.sgml > > > > > +      > > > +      -A > > class="parameter">schema_variable > > > +      --variable= > >

Re: why there is not VACUUM FULL CONCURRENTLY?

2024-07-22 Thread Kirill Reshke
> Also, we obviously need more tests on this. Both tap-test and > regression tests I suppose. The one simple test to this patch can be done this way: 1) create test relation (call it vac_conc_r1 for example) and fill it with dead tuples (insert + update or insert + delete) 2) create injection

Re: Windows default locale vs initdb

2024-07-22 Thread Ertan Küçükoglu
Hi, I am a complete noob about PostgreSQL development. I don't know about the PostgreSQL CI system. I will be needing some help as to how to do the tests. I have access to different Windows OSes (v10, Server 2022 mainly). These systems can be set to English or Turkish locales if needed. I can

Re: Virtual generated columns

2024-07-22 Thread jian he
statistic related bug. borrow examples from https://www.postgresql.org/docs/current/sql-createstatistics.html CREATE TABLE t3 (a timestamp PRIMARY KEY, b timestamp GENERATED ALWAYS AS (a) VIRTUAL); CREATE STATISTICS s3 (ndistinct) ON b FROM t3; INSERT INTO t3(a) SELECT i FROM

Re: Make COPY format extendable: Extract COPY TO format implementations

2024-07-22 Thread Sutou Kouhei
Hi Yong, Thanks for joining this thread! In <453d52d4-2ac5-49f6-928d-79f8a4c08...@ebay.com> "Re: Make COPY format extendable: Extract COPY TO format implementations" on Mon, 22 Jul 2024 07:11:15 +, "Li, Yong" wrote: > My understanding is that the provi

Re: Make COPY format extendable: Extract COPY TO format implementations

2024-07-22 Thread Sutou Kouhei
Hi Tomas, Thanks for joining this thread! In <257d5573-07da-48c3-ac07-e047e7a65...@enterprisedb.com> "Re: Make COPY format extendable: Extract COPY TO format implementations" on Fri, 19 Jul 2024 14:40:05 +0200, Tomas Vondra wrote: > I think it'd be helpful if you could

Re: jsonpath: Inconsistency of timestamp_tz() Output

2024-07-22 Thread Jeevan Chalke
On Fri, Jul 19, 2024 at 7:35 PM David E. Wheeler wrote: > On Jul 10, 2024, at 11:19, David E. Wheeler wrote: > > > Oh, and the time and date were wrong, too, because I blindly used the > same conversion for dates as for timestamps. Fixed in v2. > > > > PR:

Re: Make COPY format extendable: Extract COPY TO format implementations

2024-07-22 Thread Li, Yong
Hi Kou, I tried to follow the thread but had to skip quite some discussions in the middle part of the thread. From what I read, it appears to me that there were a lot of back-and-forth discussions on the specific implementation details (i.e. do not touch existing format implementation),

Re: Flush pgstats file during checkpoints

2024-07-22 Thread Bertrand Drouvot
Hi, On Wed, Jul 17, 2024 at 12:52:12PM +0900, Michael Paquier wrote: > On Tue, Jul 16, 2024 at 10:37:39AM +0900, Michael Paquier wrote: > > On Fri, Jul 12, 2024 at 01:01:19PM +, Bertrand Drouvot wrote: > >> Instead of removing the stat file, should we keep it around until the > >> first call

Re: Slow catchup of 2PC (twophase) transactions on replica in LR

2024-07-22 Thread Amit Kapila
On Mon, Jul 22, 2024 at 8:26 AM Hayato Kuroda (Fujitsu) wrote: > > ``` > @@ -1089,6 +1089,12 @@ CheckAlterSubOption(Subscription *sub, const char > *option, > * is because both failover and two_phase options of the slot on the > * publisher cannot be modified if the slot is currently

Re: Exposing the lock manager's WaitForLockers() to SQL

2024-07-22 Thread Will Mortensen
On Thu, May 30, 2024 at 12:01 AM Will Mortensen wrote: > FWIW, another solution might be to directly expose the functions that > WaitForLockers() calls, namely GetLockConflicts() (generalized to > GetLockers() in the first patch) to identify the transactions holding > the locks, and

Re: Add privileges test for pg_stat_statements to improve coverage

2024-07-22 Thread kuroda . keisuke
Hi Fujii-san, Thank you for your reply and comment! attach v2 fixed patch. meson.build needs to be updated as well, like the Makefile. Yes. Update 'contrib/pg_stat_statements/meson.build'. For the privileges test, should we explicitly set pg_stat_statements.track_utility at the start, as

Re: Incremental backup from a streaming replication standby

2024-07-22 Thread Michael Paquier
On Sat, Jun 29, 2024 at 07:01:04AM +0200, Laurenz Albe wrote: > The WAL summarizer is running on the standby server, but when I try > to take an incremental backup, I get an error that I understand to mean > that WAL summarizing hasn't caught up yet. Added an open item for this one. --

Re: [BUG] Fix DETACH with FK pointing to a partitioned table fails

2024-07-21 Thread Tender Wang
Alvaro Herrera 于2024年7月19日周五 21:18写道: > Hello, > > I think the fix for the check triggers should be as the attached. Very > close to what you did, but you were skipping some operations that needed > to be kept. AFAICS this patch works correctly for the posted cases. > After applying the

Re: Allow logical failover slots to wait on synchronous replication

2024-07-21 Thread shveta malik
On Fri, Jul 19, 2024 at 2:52 AM John H wrote: > > Hi Shveta, > > Thanks for taking a look at the patch. > > > > will leave user no option to unlink failover-enabled logical > > > subscribers from the wait on synchronous standbys. > > That's a good point. I'm a bit biased in that I don't think

Re: Make reorder buffer max_changes_in_memory adjustable?

2024-07-21 Thread Jingtang Zhang
Thanks, Tomas. > Theoretically, yes, we could make max_changes_in_memory a GUC, but it's > not clear to me how would that help 12/13, because there's ~0% chance > we'd backpatch that ... What I mean is not about back-patch work. Things should happen on publisher side? Consider when the

Re: Slow catchup of 2PC (twophase) transactions on replica in LR

2024-07-21 Thread Amit Kapila
On Sat, Jul 20, 2024 at 9:31 PM vignesh C wrote: > > On Thu, 18 Jul 2024 at 07:41, Hayato Kuroda (Fujitsu) > wrote: > > > > Dear Peter, > > > > Thanks for giving comments! PSA new version. > > Couple of suggestions: > 1) How will user know which all transactions should be rolled back > since the

Re: pg_verifybackup: TAR format backup verification

2024-07-21 Thread Sravan Kumar
Hi Amul, thanks for working on this. + file_name_len = strlen(relpath); > + if (file_name_len < file_extn_len || > + strcmp(relpath + file_name_len - file_extn_len, file_extn) != 0) > + { > + if (compress_algorithm == PG_COMPRESSION_NONE) > + report_backup_error(context, > + "\"%s\" is not a

RE: Slow catchup of 2PC (twophase) transactions on replica in LR

2024-07-21 Thread Hayato Kuroda (Fujitsu)
Dear Amit, > + /* > + * Do not allow changing the option if the subscription is enabled. This > + * is because both failover and two_phase options of the slot on the > + * publisher cannot be modified if the slot is currently acquired by the > + * existing walsender. > + */ > + if (sub->enabled)

Re: Windows default locale vs initdb

2024-07-21 Thread Thomas Munro
Ertan Küçükoglu offered to try to review and test this, so here's a rebase. Some notes: * it turned out that the Turkish i/I test problem I mentioned earlier in this thread[1] was just always broken on Windows, we just didn't ever test with UTF-8 before Meson took over; it's skipped now, see

Re: Should consider materializing the cheapest inner path in consider_parallel_nestloop()

2024-07-21 Thread Richard Guo
On Fri, Jul 19, 2024 at 12:00 PM Alexander Lakhin wrote: > 18.07.2024 17:30, Richard Guo wrote: > > I have no idea why the underlying statistics changed, but it seems > > that this slight change is sufficent to result in a different plan. > > I think it could be caused by the same reason as [1]

Re: documentation structure

2024-07-21 Thread jian he
nly 2 "sect1" exists, each file has only " import subprocess import os import re os.chdir("/home/jian/Desktop/pg_src/src8/postgres/doc/src/sgml") # func_logical="func-logical.sgml" func_comparison="func-comparison.sgml" func_math="func-math.sgml

Re: Patch bug: Fix jsonpath .* on Arrays

2024-07-21 Thread David E. Wheeler
On Jul 21, 2024, at 20:54, Michael Paquier wrote: > What I mean is that the main regression test suite did not complain on > your original patch posted here: > https://www.postgresql.org/message-id/A95346F9-6147-46E0-809E-532A485D71D6%40justatheory.com > > But the new tests showed a difference,

Re: Removing unneeded self joins

2024-07-21 Thread Andrei Lepikhov
proposal to re-use more of existing tree-manipulation infrastructure [2]. I agree with you that the case with ChangeVarNodes() looks questionable. Do you have other ideas how we can re-use some more of existing tree-manipulation infrastructure in SJE? As I can see, ChangeVarNodes is dedicated

Re: pg_upgrade and logical replication

2024-07-21 Thread Michael Paquier
On Sat, Jul 20, 2024 at 09:03:07PM -0500, Nathan Bossart wrote: >> This is an extremely expensive way to perform that check, and so I'm >> wondering why we don't just do >> >> SELECT count(*) FROM pg_catalog.pg_subscription; >> >> once in count_old_cluster_subscriptions(). > > Like so...

Re: POC, WIP: OR-clause support for indexes

2024-07-21 Thread Alexander Korotkov
On Mon, Jul 22, 2024 at 3:52 AM Alexander Korotkov wrote: > Please, check that there is still possibility to the generate BitmapOr plan. > > # explain select * from t where (b = 1 or b = 2 or a = 2 or a = 3); > QUERY PLAN >

Re: Patch bug: Fix jsonpath .* on Arrays

2024-07-21 Thread Michael Paquier
On Fri, Jul 19, 2024 at 09:49:50AM -0400, David E. Wheeler wrote: >> It was fun to see that HEAD was silenced with the first patch of this >> thread that tweaked the behavior with arrays. > > Uh, what? Sorry I don’t follow. What I mean is that the main regression test suite did not complain on

Re: POC, WIP: OR-clause support for indexes

2024-07-21 Thread Alexander Korotkov
Hi, Alena! Let me answer to some of your findings. On Mon, Jul 22, 2024 at 12:53 AM Alena Rybakina wrote: > To be honest,I saw a larger problem. Look at the query bellow: > > master: > > alena@postgres=# create table t (a int not null, b int not null, c int not > null); > insert into t (select

Re: Internal error codes triggered by tests

2024-07-21 Thread Michael Paquier
On Thu, Jul 18, 2024 at 09:37:06AM +0200, Daniel Gustafsson wrote: > On 18 Jul 2024, at 09:29, Michael Paquier wrote: >> How about using a new error code in class 58, say a >> ERRCODE_FILE_NAME_TOO_LONG like in the attached? >> ERRCODE_DUPLICATE_FILE is like that; it exists just for the mapping

Re: xid_wraparound tests intermittent failure.

2024-07-21 Thread Thomas Munro
On Mon, Jul 22, 2024 at 8:08 AM Alexander Lakhin wrote: > https://wiki.postgresql.org/wiki/Known_Buildfarm_Test_Failures This is great. Thanks for collating all this info here! And of course all the research and reports behind it.

Re: POC, WIP: OR-clause support for indexes

2024-07-21 Thread Alena Rybakina
Hi! Thank you for your contribution to this thread! To be honest,I saw a larger problem. Look at the query bellow: master: alena@postgres=# create table t (a int not null, b int not null, c int not null); insert into t (select 1, 1, i from generate_series(1,1) i); insert into t (select

Re: xid_wraparound tests intermittent failure.

2024-07-21 Thread Tom Lane
Andrew Dunstan writes: > On 2024-07-21 Su 1:34 PM, Tom Lane wrote: >> Locally, I've not managed to reproduce the failure yet; so perhaps >> there is some platform dependency. What are you testing on? > Linux ub22arm 5.15.0-116-generic #126-Ubuntu SMP Mon Jul 1 10:08:40 UTC > 2024 aarch64

Re: CI, macports, darwin version problems

2024-07-21 Thread Thomas Munro
On Mon, Jul 22, 2024 at 8:34 AM Joe Conway wrote: > Hmmm, maybe nevermind? I rebooted the mac mini and now it seems to be > working. Maybe someone can confirm. There ought to be plenty of space > available for sonoma and ventura at the same time now. Thanks for doing that. Initial results are

Re: Vacuum ERRORs out considering freezing dead tuples from before OldestXmin

2024-07-21 Thread Peter Geoghegan
On Sun, Jul 21, 2024 at 5:04 PM Tom Lane wrote: > > There will always be a small number of extremely slow buildfarm > > animals. Optimizing for things like Raspberry pi animals with SD cards > > just doesn't seem like a good use of developer time. I really care > > about keeping the tests fast,

<    5   6   7   8   9   10   11   12   13   14   >