Re: [PROPOSAL] : Use of ORDER BY clause in insert.sql

2022-10-27 Thread Amul Sul
On Fri, Oct 28, 2022 at 10:43 AM Tom Lane wrote: > > David Rowley writes: > > On Fri, 28 Oct 2022 at 16:51, Amul Sul wrote: > >> If we > >> are too sure that the output usually comes in the same order then the > >> ORDER BY clause that exists in other tests seems useless. I am a bit > >>

Re: [PROPOSAL] : Use of ORDER BY clause in insert.sql

2022-10-27 Thread Amul Sul
On Fri, Oct 28, 2022 at 10:28 AM David Rowley wrote: > > On Fri, 28 Oct 2022 at 16:51, Amul Sul wrote: > > > > On Thu, Oct 27, 2022 at 6:54 PM Tom Lane wrote: > > > Please be specific about the circumstances in which the output is > > > unstable for you. With zero information to go on, it

Re: Adding doubly linked list type which stores the number of items in the list

2022-10-27 Thread David Rowley
Thank you for having a look at this. On Thu, 27 Oct 2022 at 19:32, Bharath Rupireddy wrote: > Some comments on the patch: > 1. I think it's better to just return dlist_is_empty(>dlist) && > (head->count == 0); from dclist_is_empty() and remove the assert for > better readability and safety

Re: [PROPOSAL] : Use of ORDER BY clause in insert.sql

2022-10-27 Thread Tom Lane
David Rowley writes: > On Fri, 28 Oct 2022 at 16:51, Amul Sul wrote: >> If we >> are too sure that the output usually comes in the same order then the >> ORDER BY clause that exists in other tests seems useless. I am a bit >> confused & what could be a possible bug? > You can't claim that if

Re: [PROPOSAL] : Use of ORDER BY clause in insert.sql

2022-10-27 Thread David Rowley
On Fri, 28 Oct 2022 at 16:51, Amul Sul wrote: > > On Thu, Oct 27, 2022 at 6:54 PM Tom Lane wrote: > > Please be specific about the circumstances in which the output is > > unstable for you. With zero information to go on, it seems about as > > likely that this change is masking a bug as that

Latches vs lwlock contention

2022-10-27 Thread Thomas Munro
Hi, We usually want to release lwlocks, and definitely spinlocks, before calling SetLatch(), to avoid putting a system call into the locked region so that we minimise the time held. There are a few places where we don't do that, possibly because it's not just a simple latch to hold a pointer to

Re: [PROPOSAL] : Use of ORDER BY clause in insert.sql

2022-10-27 Thread Amul Sul
On Thu, Oct 27, 2022 at 6:54 PM Tom Lane wrote: > > Nishant Sharma writes: > > We would like to share a proposal of a patch, where we have added order by > > clause in two select statements in src/test/regress/sql/insert.sql file and > > respective changes in src/test/regress/expected/insert.out

Re: Allow file inclusion in pg_hba and pg_ident files

2022-10-27 Thread Julien Rouhaud
On Fri, Oct 28, 2022 at 10:24:23AM +0900, Michael Paquier wrote: > On Thu, Oct 27, 2022 at 12:26:25PM +0800, Julien Rouhaud wrote: > > I am still not completely sure what's the best way to do things here, > so let's do the following: let's keep the patch the way you think is > better for now (I

Re: pg_recvlogical prints bogus error when interrupted

2022-10-27 Thread Bharath Rupireddy
On Fri, Oct 28, 2022 at 4:41 AM Andres Freund wrote: > > On 2022-10-24 08:15:11 +0530, Bharath Rupireddy wrote: > > > > + /* When we get SIGINT/SIGTERM, we exit */ > > + if (ready_to_exit) > > + { > > + /* > > + * Try

Re: GUC values - recommended way to declare the C variables?

2022-10-27 Thread Michael Paquier
On Fri, Oct 28, 2022 at 11:48:13AM +0900, Michael Paquier wrote: > Actually, pg_iovec.h uses WIN32 without any previous header declared, > but win32.h tells a different story as of ed9b3606, where we would > define WIN32 if it does not exist yet. Seeing all the places where pg_status.h is

Re: GUC values - recommended way to declare the C variables?

2022-10-27 Thread Michael Paquier
On Thu, Oct 27, 2022 at 07:00:26PM +1100, Peter Smith wrote: > The GUC defaults of guc_tables.c, and the modified GUC C var > declarations now share the same common #define'd value (instead of > cut/paste preprocessor code). Thanks. I have not looked at the checkup logic yet, but the central

Merging LatchWaitSet and FeBeWaitSet

2022-10-27 Thread Thomas Munro
Hi, Currently all backends have LatchWaitSet (latch.c), and most also have FeBeWaitSet (pqcomm.c). It's not the end of the world, but it's a little bit wasteful in terms of kernel resources to have two epoll/kqueue descriptors per backend. I wonder if we should consider merging them into a

Re: Make mesage at end-of-recovery less scary.

2022-10-27 Thread Justin Pryzby
rebased >From 67ce65038ae6a7d5b023b7472df9f9ca9835d5f5 Mon Sep 17 00:00:00 2001 From: Kyotaro Horiguchi Date: Thu, 7 Jul 2022 11:51:45 +0900 Subject: [PATCH] Make End-Of-Recovery error less scary When recovery in any type ends, we see a bit scary error message like "invalid record length" that

Re: Improve tab completion for ALTER FUNCTION/PROCEDURE/ROUTINE

2022-10-27 Thread Dong Wook Lee
On Fri, Oct 28, 2022 at 12:08 AM vignesh C wrote: > > Hi, > > Tab completion for ALTER FUNCTION/PROCEDURE/ROUTINE action was > missing, this patch adds the tab completion for the same. > > Regards, > Vignesh Hi, I applied your patch and did some tests. Is it okay not to consider SET and RESET

Re: Use pg_pwritev_with_retry() instead of write() in dir_open_for_write() to avoid partial writes?

2022-10-27 Thread Michael Paquier
On Thu, Oct 27, 2022 at 03:58:25PM -0700, Andres Freund wrote: > The block sizes don't need to match, do they? As long as the block is properly > aligned, we can change the iov_len of the final iov to match whatever the size > is being passed in, no? Hmm. Based on what Bharath has written

Re: Allow file inclusion in pg_hba and pg_ident files

2022-10-27 Thread Michael Paquier
On Thu, Oct 27, 2022 at 12:26:25PM +0800, Julien Rouhaud wrote: > On Thu, Oct 27, 2022 at 12:08:31PM +0900, Michael Paquier wrote: >> >> Putting things afresh, there are two different things here (sorry I >> need to see that typed ;p): >> 1) How do we want to check reliably the loading of the HBA

Re: Perform streaming logical transactions by background workers and parallel apply

2022-10-27 Thread Masahiko Sawada
On Thu, Oct 27, 2022 at 11:34 AM shiy.f...@fujitsu.com wrote: > > On Wed, Oct 26, 2022 7:19 PM Amit Kapila wrote: > > > > On Tue, Oct 25, 2022 at 8:38 AM Masahiko Sawada > > wrote: > > > > > > On Fri, Oct 21, 2022 at 6:32 PM houzj.f...@fujitsu.com > > > wrote: > > > > > > I've started to

Re: Support logical replication of DDLs

2022-10-27 Thread Peter Smith
On Fri, Oct 28, 2022 at 11:20 AM Zheng Li wrote: > > > 1. It might be useful to add this thread to the commitfest, if only so > > the cfbot can discover the latest patch set and alert about any rebase > > problems. > > There is already a commitfest entry for the thread that I added back in

Re: pg_ls_tmpdir to show directories and shared filesets (and pg_ls_*)

2022-10-27 Thread Justin Pryzby
On Fri, Dec 13, 2019 at 03:03:47PM +1300, Thomas Munro wrote: > > Actually, I tried using pg_ls_tmpdir(), but it unconditionally masks > > non-regular files and thus shared filesets. Maybe that's worth > > discussion on a new thread ? > > > > src/backend/utils/adt/genfile.c > > /*

Re: Support logical replication of DDLs

2022-10-27 Thread Zheng Li
> Hi, authors on this thread. > > The patch v32-0001 is very large, so it will take some time to review > the code in detail. Thanks for reviewing! > Meanwhile, here are some general comments about the patch: > > == > > 1. It might be useful to add this thread to the commitfest, if only so >

Re: Requiring 32 bit atomics

2022-10-27 Thread Andres Freund
Hi, On 2022-10-27 19:44:13 -0400, Tom Lane wrote: > Turns out they have a pretty cute workaround for it, on HPPA and a couple of > other atomics-less arches they still support. They've written short > sequences that have the effect of CAS and are designed to store to memory > only at the end.

Re: Requiring 32 bit atomics

2022-10-27 Thread Tom Lane
I wrote: > But wait, you say, what about mamba-nee-gaur, my HPPA dinosaur? sigh ... s/mamba/chickadee/. Got too many NetBSD machines, perhaps. regards, tom lane

Re: Requiring 32 bit atomics

2022-10-27 Thread Tom Lane
Thomas Munro writes: > We have fallback code for computers that don't have 32 bit atomic ops. > Of course all modern ISAs have 32 bit atomics, but various comments > imagine that a new architecture might be born that we don't have > support for yet, so the fallback provides a way to bring a new

Re: Proposal to use JSON for Postgres Parser format

2022-10-27 Thread Andres Freund
Hi, On 2022-09-19 22:29:15 -0400, Tom Lane wrote: > There are certainly reasons to think about changing the node tree > storage format; but if we change it, I'd like to see it go to something > more compact not more verbose. Very much seconded - the various pg_node_trees are a quite significant

Re: Support logical replication of DDLs

2022-10-27 Thread Peter Smith
Hi, authors on this thread. The patch v32-0001 is very large, so it will take some time to review the code in detail. Meanwhile, here are some general comments about the patch: == 1. It might be useful to add this thread to the commitfest, if only so the cfbot can discover the latest patch

Re: pg_recvlogical prints bogus error when interrupted

2022-10-27 Thread Andres Freund
Hi, On 2022-10-24 08:15:11 +0530, Bharath Rupireddy wrote: > I came up with the attached v2 patch, please have a look. Thanks for working on this! > + /* When we get SIGINT/SIGTERM, we exit */ > + if (ready_to_exit) > + { > + /* > +

Re: Documentation for building with meson

2022-10-27 Thread Andres Freund
Hi, On 2022-10-27 14:15:32 -0700, Jacob Champion wrote: > On Thu, Oct 27, 2022 at 1:04 AM John Naylor > wrote: > > This does not work for me in a fresh install until running > > > > meson test --suite setup > > > > In fact, we see in > > > > https://wiki.postgresql.org/wiki/Meson > > > > meson

Re: Use pg_pwritev_with_retry() instead of write() in dir_open_for_write() to avoid partial writes?

2022-10-27 Thread Andres Freund
Hi, Interestingly, I also needed something like pg_pwrite_zeros() today. Exposed via smgr, for more efficient relation extensions. On 2022-10-27 14:54:00 +0900, Michael Paquier wrote: > Regarding 0002, using pg_pwrite_zeros() as a routine name, as > suggested by Thomas, sounds good to me.

Requiring 32 bit atomics

2022-10-27 Thread Thomas Munro
Hi, We have fallback code for computers that don't have 32 bit atomic ops. Of course all modern ISAs have 32 bit atomics, but various comments imagine that a new architecture might be born that we don't have support for yet, so the fallback provides a way to bring a new system up by implementing

Re: Documentation for building with meson

2022-10-27 Thread Jacob Champion
On Thu, Oct 27, 2022 at 1:04 AM John Naylor wrote: > This does not work for me in a fresh install until running > > meson test --suite setup > > In fact, we see in > > https://wiki.postgresql.org/wiki/Meson > > meson test --suite setup --suite main (Is there a way to declare a dependency on the

Re: Have nodeSort.c use datum sorts single-value byref types

2022-10-27 Thread David Rowley
On Wed, 26 Oct 2022 at 23:35, David Rowley wrote: > I think this is a fairly trivial patch, so if nobody objects, I plan > to push it in the next few days. Pushed. David

Re: Allow single table VACUUM in transaction block

2022-10-27 Thread Justin Pryzby
On Thu, Oct 27, 2022 at 10:31:31AM +0100, Simon Riggs wrote: > Allows both ANALYZE and vacuum of toast tables, but not VACUUM FULL. Maybe I misunderstood what you meant: you said "not VACUUM FULL", but with your patch, that works: postgres=# begin; VACUUM FULL pg_class; commit; BEGIN VACUUM

Re: Reducing planning time on tables with many indexes

2022-10-27 Thread Alvaro Herrera
On 2022-Aug-19, David Geier wrote: > Beyond that I did some off-CPU profiling to precisely track down which lock > serializes execution. It turned out to be the MyProc::fpInfoLock lightweight > lock. This lock is used in the fast path of the heavyweight lock. In the > contenting case, fpInfoLock

heavily contended lwlocks with long wait queues scale badly

2022-10-27 Thread Andres Freund
Hi, I am working on posting a patch series making relation extension more scalable. As part of that I was running some benchmarks for workloads that I thought should not or just positively impacted - but I was wrong, there was some very significant degradation at very high client counts. After

Re: Code checks for App Devs, using new options for transaction behavior

2022-10-27 Thread Simon Riggs
On Thu, 27 Oct 2022 at 12:09, Simon Riggs wrote: > Comments please Update from patch tester results. -- Simon Riggshttp://www.EnterpriseDB.com/ 002_nested_xacts.v7.patch Description: Binary data 001_psql_parse_only.v1.patch Description: Binary data

Re: Allow single table VACUUM in transaction block

2022-10-27 Thread Bharath Rupireddy
On Thu, Oct 27, 2022 at 9:49 PM Simon Riggs wrote: > > On Thu, 27 Oct 2022 at 10:31, Simon Riggs > wrote: > > > Tests, docs. > > The patch tester says that a pg_upgrade test is failing on Windows, > but works for me. > > t/002_pg_upgrade.pl .. ok > > Anybody shed any light on that, much

Re: Allow single table VACUUM in transaction block

2022-10-27 Thread Simon Riggs
On Thu, 27 Oct 2022 at 10:31, Simon Riggs wrote: > Tests, docs. The patch tester says that a pg_upgrade test is failing on Windows, but works for me. t/002_pg_upgrade.pl .. ok Anybody shed any light on that, much appreciated. -- Simon Riggshttp://www.EnterpriseDB.com/

Improve tab completion for ALTER FUNCTION/PROCEDURE/ROUTINE

2022-10-27 Thread vignesh C
Hi, Tab completion for ALTER FUNCTION/PROCEDURE/ROUTINE action was missing, this patch adds the tab completion for the same. Regards, Vignesh From 4b0473799a0d6af126ccd3d7802e5d0cbb83b944 Mon Sep 17 00:00:00 2001 From: "vignesh.c" Date: Thu, 27 Oct 2022 14:00:46 +0530 Subject: [PATCH v1] Tab

Re: Proposal to use JSON for Postgres Parser format

2022-10-27 Thread Michel Pelletier
On Wed, Sep 21, 2022 at 11:04 AM Matthias van de Meent < boekewurm+postg...@gmail.com> wrote: > On Tue, 20 Sept 2022 at 17:29, Alvaro Herrera > wrote: > > > > On 2022-Sep-20, Matthias van de Meent wrote: > > > > > Allow me to add: compressability > > > > > > In the thread surrounding [0] there

Re: Reducing duplicativeness of EquivalenceClass-derived clauses

2022-10-27 Thread Zhang Mingli
Hi, On Oct 27, 2022, 21:29 +0800, Tom Lane , wrote: > Zhang Mingli writes: > > How about combine ec->ec_sources and ec->derives as one list for less codes? > > Keeping them separate is required for the broken-EC code paths. > Even if it weren't, I wouldn't merge them just to save a couple > of

Re: Reducing duplicativeness of EquivalenceClass-derived clauses

2022-10-27 Thread Tom Lane
Zhang Mingli writes: > How about combine ec->ec_sources and ec->derives as one list for less codes? Keeping them separate is required for the broken-EC code paths. Even if it weren't, I wouldn't merge them just to save a couple of lines of code --- I think it's useful to be able to tell which

Re: [PROPOSAL] : Use of ORDER BY clause in insert.sql

2022-10-27 Thread Tom Lane
Nishant Sharma writes: > We would like to share a proposal of a patch, where we have added order by > clause in two select statements in src/test/regress/sql/insert.sql file and > respective changes in src/test/regress/expected/insert.out output file. > This would help in generating output in

Re: Reducing duplicativeness of EquivalenceClass-derived clauses

2022-10-27 Thread Zhang Mingli
HI, On Oct 26, 2022, 06:09 +0800, Tom Lane , wrote: > While fooling with my longstanding outer-join variables changes > (I am making progress on that, honest), I happened to notice that > equivclass.c is leaving some money on the table by generating > redundant RestrictInfo clauses. It already

Re: [PATCHES] Post-special page storage TDE support

2022-10-27 Thread Matthias van de Meent
Hi On Mon, 24 Oct 2022, 19:56 David Christensen, < david.christen...@crunchydata.com> wrote: > > Discussion is welcome and encouraged! Did you read the related thread with related discussion from last June, "Re: better page-level checksums" [0]? In that I argued that space at the end of a page

[PROPOSAL] : Use of ORDER BY clause in insert.sql

2022-10-27 Thread Nishant Sharma
Hi, We would like to share a proposal of a patch, where we have added order by clause in two select statements in src/test/regress/sql/insert.sql file and respective changes in src/test/regress/expected/insert.out output file. This would help in generating output in consistent sequence, as

Re: pg_recvlogical prints bogus error when interrupted

2022-10-27 Thread Bharath Rupireddy
On Mon, Oct 24, 2022 at 8:15 AM Bharath Rupireddy wrote: > > On Fri, Oct 21, 2022 at 7:52 AM Kyotaro Horiguchi > wrote: > > > > > +1. How about emitting a message like its friend pg_receivewal, like > > > the attached patch? > > > > I'm not a fan of treating SIGINT as an error in this case. It

Code checks for App Devs, using new options for transaction behavior

2022-10-27 Thread Simon Riggs
In the past, developers have wondered how we can provide "--dry-run" functionality https://www.postgresql.org/message-id/15791.1450383201%40sss.pgh.pa.us This is important for application developers, especially when migrating programs to Postgres. Presented here are 3 features aimed at

Avoid using list_delete_first in simplify_or/and_arguments

2022-10-27 Thread Richard Guo
Hi hackers, While trying to measure if there is any gain from the change as discussed in [1], I happened to notice another place that is slowed down by list_delete_first. I'm using the query as below: (n=100; printf "explain (summary on) select * from t where " for ((i=1;i<$n;i++)); do

Allow single table VACUUM in transaction block

2022-10-27 Thread Simon Riggs
It is a common user annoyance to have a script fail because someone added a VACUUM, especially when using --single-transaction option. Fix, so that this works without issue: BEGIN; VACUUM (ANALYZE) vactst; COMMIT; Allows both ANALYZE and vacuum of toast tables, but not VACUUM FULL.

Re: Add LSN along with offset to error messages reported for WAL file read/write/validate header failures

2022-10-27 Thread Bharath Rupireddy
On Tue, Oct 4, 2022 at 2:58 PM Bharath Rupireddy wrote: > > On Thu, Sep 29, 2022 at 7:43 PM Bharath Rupireddy > wrote: > > > > Please see the attached v1 patch. > > FWIW, I'm attaching Nathan's patch that introduced the new function > pg_walfile_offset_lsn as 0002 in the v1 patch set. Here's the

Re: Use pg_pwritev_with_retry() instead of write() in dir_open_for_write() to avoid partial writes?

2022-10-27 Thread Bharath Rupireddy
On Thu, Oct 27, 2022 at 11:24 AM Michael Paquier wrote: > > On Thu, Sep 29, 2022 at 08:09:56PM -0700, Nathan Bossart wrote: > > Looks reasonable to me. > > 0001, to move pg_pwritev_with_retry() to a new home, seems fine, so > applied. Thanks. > Regarding 0002, using pg_pwrite_zeros() as a

Re: Documentation for building with meson

2022-10-27 Thread John Naylor
+# Run the main pg_regress and isolation tests +meson test --suite main This does not work for me in a fresh install until running meson test --suite setup In fact, we see in https://wiki.postgresql.org/wiki/Meson meson test --suite setup --suite main That was just an eyeball check from a

Re: GUC values - recommended way to declare the C variables?

2022-10-27 Thread Peter Smith
On Thu, Oct 27, 2022 at 1:33 PM Michael Paquier wrote: > >... > > Anyway, per my previous comments in my last message of this thread as > of https://www.postgresql.org/message-id/y1nnwftrnl3it...@paquier.xyz, > I don't see a need for DYNAMIC_DEFAULT from the other thread, nor do I > see a need to

Re: Simplifying our Trap/Assert infrastructure

2022-10-27 Thread Michael Paquier
On Wed, Oct 12, 2022 at 09:19:17PM +0200, Peter Eisentraut wrote: > I'm in favor of this. These variants are a distraction. Agreed, even if extensions could use these, it looks like any out-of-core code using what's removed here would also gain in clarity. This is logically fine (except for an

Re: [patch] Have psql's \d+ indicate foreign partitions

2022-10-27 Thread Alvaro Herrera
On 2022-Oct-24, Justin Pryzby wrote: > On Mon, Oct 24, 2022 at 09:44:18PM +0900, Ian Lawrence Barwick wrote: > > + else if (child_relkind == RELKIND_FOREIGN_TABLE > > && is_partitioned) > > + appendPQExpBuffer(, ", server: > >

Re: Adding doubly linked list type which stores the number of items in the list

2022-10-27 Thread Bharath Rupireddy
On Thu, Oct 27, 2022 at 9:05 AM David Rowley wrote: > > As part of the AIO work [1], there are quite a number of dlist_heads > which a counter is used to keep track on how many items are in the > list. We also have a few places in master which do the same thing. > > In order to tidy this up and