Re: BF animal malleefowl reported an failure in 001_password.pl

2023-01-13 Thread Tom Lane
"houzj.f...@fujitsu.com" writes: > I noticed one BF failure[1] when monitoring the BF for some other commit. > [1] > https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=malleefowl=2023-01-13%2009%3A54%3A51 > ... > So it seems the connection happens before pg_ident.conf is actually reloaded ?

Re: fixing CREATEROLE

2023-01-13 Thread vignesh C
On Tue, 10 Jan 2023 at 23:16, Robert Haas wrote: > > On Thu, Jan 5, 2023 at 2:53 PM Robert Haas wrote: > > On Tue, Jan 3, 2023 at 3:11 PM Robert Haas wrote: > > > Committed and back-patched 0001 with fixes for the issues that you > > > pointed out. > > > > > > Here's a trivial rebase of the

Re: Removing redundant grouping columns

2023-01-13 Thread vignesh C
On Sat, 31 Dec 2022 at 02:32, Tom Lane wrote: > > I wrote: > > Richard Guo writes: > >> While we are here, I wonder if we can do the same trick for > >> distinctClause, to cope with cases like > >> select distinct a.x, b.y from a, b where a.x = b.y; > > > We do that already, no? > > Oh, wait, I

Re: Reduce timing overhead of EXPLAIN ANALYZE using rdtsc?

2023-01-13 Thread vignesh C
On Wed, 4 Jan 2023 at 17:32, David Geier wrote: > > I fixed the compilation error on CFBot. > I missed adding instr_time.c to the Meson makefile. > New patch set attached. The patch does not apply on top of HEAD as in [1], please post a rebased patch: === Applying patches on top of PostgreSQL

Re: Pluggable toaster

2023-01-13 Thread vignesh C
On Sun, 8 Jan 2023 at 01:40, Nikita Malakhov wrote: > > Hi! > > Thank you for your attention. > I've rebased the patchset onto the latest master (from 07.01), but the second > part is still > in pre-patch shape - it is working, but tests are failing due to changes in > TOAST relations > logic -

Re: Operation log for major operations

2023-01-13 Thread vignesh C
On Mon, 5 Dec 2022 at 13:42, Dmitry Koval wrote: > > Hi! > > >I think storing this in pg_control is a bad idea. That file is > >extremely critical and if you break it, you're pretty much SOL on > >recovering your data. I suggest that this should use a separate file. > > Thanks. Operation log

Re: backup.sgml typo

2023-01-13 Thread Amit Kapila
On Sat, Jan 14, 2023 at 7:32 AM Tatsuo Ishii wrote: > > There seem to be a small typo in backup.sgml > (archive_command is unnecessarily > repeated). Attached is a patch to fix that. > LGTM. -- With Regards, Amit Kapila.

Re: Time delayed LR (WAS Re: logical replication restrictions)

2023-01-13 Thread vignesh C
On Thu, 12 Jan 2023 at 21:09, Takamichi Osumi (Fujitsu) wrote: > > On Thursday, January 12, 2023 12:04 PM Kyotaro Horiguchi > wrote: > > At Wed, 11 Jan 2023 12:46:24 +, "Hayato Kuroda (Fujitsu)" > > wrote in > > > them. Which version is better? > > > > > > Some comments by a quick loock,

Re: Add BufFileRead variants with short read and EOF detection

2023-01-13 Thread Amit Kapila
On Thu, Jan 12, 2023 at 2:44 PM Peter Eisentraut wrote: > > On 10.01.23 07:20, Amit Kapila wrote: > > Yeah, we can do that but not sure if it is worth doing any of those > > because there are already other places that don't use the exact > > context. > > Ok, updated patches attached. > Both the

Re: Decoupling antiwraparound autovacuum from special rules around auto cancellation

2023-01-13 Thread Andres Freund
Hi, On 2023-01-13 19:39:41 -0800, Peter Geoghegan wrote: > On Fri, Jan 13, 2023 at 6:09 PM Andres Freund wrote: > > I don't think the split is right. There's too much in 0001 - it's basically > > introducing the terminology of 0002 already. Could you make it a much more > > minimal change? > >

Re: Amcheck verification of GiST and GIN

2023-01-13 Thread Andrey Borodin
On Fri, Jan 13, 2023 at 7:35 PM Jose Arthur Benetasso Villanova wrote: > > Hello again. I see the change. Thanks > Thanks! I also found out that there was a CI complaint about amcheck.h not including some necessary stuff. Here's a version with a fix for that. Best regards, Andrey Borodin.

Re: Decoupling antiwraparound autovacuum from special rules around auto cancellation

2023-01-13 Thread Peter Geoghegan
On Fri, Jan 13, 2023 at 6:09 PM Andres Freund wrote: > I don't think the split is right. There's too much in 0001 - it's basically > introducing the terminology of 0002 already. Could you make it a much more > minimal change? Okay. I thought that you might say that. I really just wanted to show

Re: Amcheck verification of GiST and GIN

2023-01-13 Thread Jose Arthur Benetasso Villanova
On Fri, 13 Jan 2023, Andrey Borodin wrote: On Fri, Jan 13, 2023 at 3:46 AM Jose Arthur Benetasso Villanova wrote: The only thing that I found is the gin_index_parent_check function in docs still references the "gin_index_parent_check(index regclass, heapallindexed boolean) returns void"

Re: How to find the number of cached pages for a relation?

2023-01-13 Thread Andres Freund
Hi, On 2023-01-13 17:28:31 -0800, Amin wrote: > Before scanning a relation, in the planner stage, I want to make a call to > retrieve information about how many pages will be a hit for a specific > relation. The module pg_buffercache seems to be doing a similar thing. > Also, pg_statio_all_tables

Re: Is RecoveryConflictInterrupt() entirely safe in a signal handler?

2023-01-13 Thread Thomas Munro
On Thu, Jan 5, 2023 at 2:14 PM Tom Lane wrote: > The rcancelrequested API is something that I devised out of whole cloth > awhile ago. It's not in Tcl's copy of the code, which AFAIK is the > only other project using this regex engine. I do still have vague > hopes of someday seeing the engine

Re: Decoupling antiwraparound autovacuum from special rules around auto cancellation

2023-01-13 Thread Andres Freund
Hi, On 2023-01-13 16:13:45 -0800, Peter Geoghegan wrote: > On Fri, Jan 13, 2023 at 2:00 PM Andres Freund wrote: > > I think it'd be a good idea to split off the part of the patch that > > introduces > > AutoVacType / adds logging for what triggered. That's independently useful, > > likely

backup.sgml typo

2023-01-13 Thread Tatsuo Ishii
There seem to be a small typo in backup.sgml (archive_command is unnecessarily repeated). Attached is a patch to fix that. Best reagards, -- Tatsuo Ishii SRA OSS LLC English: http://www.sraoss.co.jp/index_en/ Japanese:http://www.sraoss.co.jp diff --git a/doc/src/sgml/backup.sgml

Re: postgres_fdw uninterruptible during connection establishment / ProcSignalBarrier

2023-01-13 Thread Thomas Munro
On Tue, Jan 10, 2023 at 3:05 PM Andres Freund wrote: > On 2023-01-03 12:05:20 -0800, Andres Freund wrote: > > The attached patch adds libpq-be-fe-helpers.h and uses it in > > libpqwalreceiver, > > dblink, postgres_fdw. > > > As I made libpq-be-fe-helpers.h handle reserving external fds, > >

How to find the number of cached pages for a relation?

2023-01-13 Thread Amin
Hi, Before scanning a relation, in the planner stage, I want to make a call to retrieve information about how many pages will be a hit for a specific relation. The module pg_buffercache seems to be doing a similar thing. Also, pg_statio_all_tables seems to be having that information, but it is

Re: pgsql: Add new GUC createrole_self_grant.

2023-01-13 Thread David G. Johnston
On Fri, Jan 13, 2023 at 4:46 PM Andres Freund wrote: > > I don't really see what that has to do with the topic at hand, unless you > want > to suggest removing the entire section about how to write secure security > definer functions? > Not remove, but I'm not seeing why the introduction of

Announcing Release 16 of the PostgreSQL Buildfarm client

2023-01-13 Thread Andrew Dunstan
Hot on the heels of Release 15 comes Release 16. This release deals with some issues that have been discovered with the check for update feature of Release 15 and the |force_every| and |trigger_exclude| features, so that it now works correctly with those features. It also features these items:

Re: Fixes required for cross version update testing

2023-01-13 Thread Andrew Dunstan
On 2023-01-13 Fr 19:33, Justin Pryzby wrote: > On Fri, Jan 13, 2023 at 05:20:41PM -0500, Andrew Dunstan wrote: >> Over at [1] there was some discussion of moving knowledge of what's >> required to be fixed from old branch repos to be able to upgrade them >> into the core code, instead of having

Re: Fixes required for cross version update testing

2023-01-13 Thread Andrew Dunstan
On 2023-01-13 Fr 19:49, Tom Lane wrote: > Andrew Dunstan writes: >> Here's a piece of WIP for that, in the form of a perl module that >> provides a function that takes an old version number / tag and provides >> the set of sql statements that need to be run to make the old repo >> upgradeable.

Re: Fixes required for cross version update testing

2023-01-13 Thread Tom Lane
Andrew Dunstan writes: > Here's a piece of WIP for that, in the form of a perl module that > provides a function that takes an old version number / tag and provides > the set of sql statements that need to be run to make the old repo > upgradeable. It still needs a good deal of polish, but it's a

Extracting cross-version-upgrade knowledge from buildfarm client

2023-01-13 Thread Tom Lane
This is a followup to the discussion at [1], in which we agreed that it's time to fix the buildfarm client so that knowledge about cross-version discrepancies in pg_dump output can be moved into the community git tree, making it feasible for people other than Andrew to fix problems when we change

Re: Fixes required for cross version update testing

2023-01-13 Thread Justin Pryzby
On Fri, Jan 13, 2023 at 05:20:41PM -0500, Andrew Dunstan wrote: > Over at [1] there was some discussion of moving knowledge of what's > required to be fixed from old branch repos to be able to upgrade them > into the core code, instead of having it reside in a buildfarm client > module. Is this

Re: Amcheck verification of GiST and GIN

2023-01-13 Thread Andrey Borodin
On Fri, Jan 13, 2023 at 3:46 AM Jose Arthur Benetasso Villanova wrote: > > The only thing that I found is the gin_index_parent_check function in docs > still references the "gin_index_parent_check(index regclass, > heapallindexed boolean) returns void" > Correct! Please find the attached fixed

Re: Decoupling antiwraparound autovacuum from special rules around auto cancellation

2023-01-13 Thread Peter Geoghegan
On Fri, Jan 13, 2023 at 2:00 PM Andres Freund wrote: > On 2023-01-12 16:08:06 -0500, Robert Haas wrote: > > Normally, the XID age of a table never reaches autovacuum_freeze_max_age in > > the first place. > > That's not at all my experience. I often see it being the primary reason for >

Re: pgsql: Add new GUC createrole_self_grant.

2023-01-13 Thread Andres Freund
Hi, On 2023-01-12 18:15:50 -0700, David G. Johnston wrote: > On Thu, Jan 12, 2023 at 8:11 AM Robert Haas wrote: > > > On Wed, Jan 11, 2023 at 7:53 PM David G. Johnston > > wrote: > > > Justed wanted to chime in and say Robert has eloquently put into words > > much of what I have been thinking

Re: pg_stat_bgwriter.buffers_backend is pretty meaningless (and more?)

2023-01-13 Thread Andres Freund
Hi, On 2023-01-13 13:38:15 -0500, Melanie Plageman wrote: > > I think that's marginally better, but I think having to define both > > FIRST and NUM is excessive and doesn't make it less fragile. Not sure > > what anyone else will say, but I'd prefer if it started at "0". The reason for using

Re: add PROCESS_MAIN to VACUUM

2023-01-13 Thread Nathan Bossart
On Fri, Jan 13, 2023 at 03:24:09PM -0800, Jeff Davis wrote: > For completeness, did you consider CLUSTER and REINDEX options as well? I have not, but I can put together patches for those as well. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: add PROCESS_MAIN to VACUUM

2023-01-13 Thread Jeff Davis
On Thu, 2022-12-29 at 16:00 -0800, Nathan Bossart wrote: > The motivation for adding this option is to make it easier to VACUUM > only a > relation's TOAST table.  At the moment, you need to find the TOAST > table by > examining a relation's reltoastrelid, and you need USAGE on the > pg_toast >

Re: allow granting CLUSTER, REFRESH MATERIALIZED VIEW, and REINDEX

2023-01-13 Thread Nathan Bossart
On Fri, Jan 13, 2023 at 02:56:26PM -0800, Nathan Bossart wrote: > Okay. Here is a new patch set. And here is a rebased patch set (c44f633 changed the same LOCK TABLE docs). -- Nathan Bossart Amazon Web Services: https://aws.amazon.com >From 48be435025ba10235c821cbd298c43763cbc5a56 Mon Sep 17

Re: allow granting CLUSTER, REFRESH MATERIALIZED VIEW, and REINDEX

2023-01-13 Thread Nathan Bossart
On Fri, Jan 13, 2023 at 01:30:28PM -0800, Jeff Davis wrote: > If we care about that use case, let's do it right and have forms of > VACUUM/CLUSTER/REINDEX that check permissions on the main table, skip > the work on the main table, and descend directly to the toast tables. > That doesn't seem

Re: Patch: Global Unique Index

2023-01-13 Thread Cary Huang
On 2022-11-30 2:30 p.m., Greg Stark wrote: On Tue, 29 Nov 2022 at 21:16, Tom Lane wrote: I actually think that that problem should be soluble with a slightly different approach. The thing that feels insoluble is that you can't do this without acquiring sufficient locks to prevent addition of

Fixes required for cross version update testing

2023-01-13 Thread Andrew Dunstan
Over at [1] there was some discussion of moving knowledge of what's required to be fixed from old branch repos to be able to upgrade them into the core code, instead of having it reside in a buildfarm client module. Here's a piece of WIP for that, in the form of a perl module that provides a

Re: Decoupling antiwraparound autovacuum from special rules around auto cancellation

2023-01-13 Thread Andres Freund
Hi, On 2023-01-12 16:08:06 -0500, Robert Haas wrote: > Normally, the XID age of a table never reaches autovacuum_freeze_max_age in > the first place. That's not at all my experience. I often see it being the primary reason for autovacuum vacuuming large tables on busy OLTP systems. Even without

Re: allow granting CLUSTER, REFRESH MATERIALIZED VIEW, and REINDEX

2023-01-13 Thread Jeff Davis
On Fri, 2023-01-13 at 12:33 -0800, Nathan Bossart wrote: > That would fix the problem in the original complaint, but it wouldn't > allow > for vacuuming toast tables directly if you only have MAINTAIN > privileges on > the main relation.  If you can vacuum the toast table indirectly via > the >

Re: Reduce timing overhead of EXPLAIN ANALYZE using rdtsc?

2023-01-13 Thread Andres Freund
Hi, On 2023-01-13 15:25:16 -0500, Tom Lane wrote: > Andres Freund writes: > > Does anybody see a reason to not move forward with this aspect? We do a fair > > amount of INSTR_TIME_ACCUM_DIFF() etc, and that gets a good bit cheaper by > > just using nanoseconds. > > Cheaper, and perhaps more

Re: real/float example for testlibpq3

2023-01-13 Thread Mark Wong
On Thu, Nov 03, 2022 at 09:55:22AM -0400, Tom Lane wrote: > Peter Eisentraut writes: > > On 01.11.22 09:15, Tom Lane wrote: > >> Agreed that the libpq manual is not the place for this, but I feel > >> like it will also be clutter in "Data Types". Perhaps we should > >> invent a new appendix or

Re: allow granting CLUSTER, REFRESH MATERIALIZED VIEW, and REINDEX

2023-01-13 Thread Nathan Bossart
On Fri, Jan 13, 2023 at 11:56:03AM -0800, Jeff Davis wrote: > I'm hesitant to add an index to pg_class just for the privilege checks > on toast tables, and I don't think we need to. I bet this index will be useful for more than just these privilege checks (e.g., autovacuum currently creates a

Re: Reduce timing overhead of EXPLAIN ANALYZE using rdtsc?

2023-01-13 Thread Tom Lane
Andres Freund writes: > On 2023-01-04 13:02:05 +0100, David Geier wrote: >> Subject: [PATCH 1/3] Change instr_time to just store nanoseconds, that's >> cheaper. > Does anybody see a reason to not move forward with this aspect? We do a fair > amount of INSTR_TIME_ACCUM_DIFF() etc, and that gets a

Re: Rework of collation code, extensibility

2023-01-13 Thread Peter Geoghegan
On Wed, Jan 11, 2023 at 3:44 PM Jeff Davis wrote: > Attached trivial rebase as v6. Some review comments for this v6. Comments on 0001-*: * I think that 0002-* can be squashed with 0001-*, since there isn't any functional reason why you'd want to commit the strcoll() and strxfrm() changes

Re: allow granting CLUSTER, REFRESH MATERIALIZED VIEW, and REINDEX

2023-01-13 Thread Jeff Davis
On Mon, 2023-01-09 at 14:51 -0800, Nathan Bossart wrote: > On Tue, Jan 03, 2023 at 03:45:49PM -0800, Nathan Bossart wrote: > > I'd like to get this fixed, but I have yet to hear thoughts on the > > suggested approach.  I'll proceed with adjusting the tests and > > documentation shortly unless

Re: Reduce timing overhead of EXPLAIN ANALYZE using rdtsc?

2023-01-13 Thread Andres Freund
Hi, On 2023-01-04 13:02:05 +0100, David Geier wrote: > From be18633d4735f680c7910fcb4e8ac90c4eada131 Mon Sep 17 00:00:00 2001 > From: David Geier > Date: Thu, 17 Nov 2022 10:22:01 +0100 > Subject: [PATCH 1/3] Change instr_time to just store nanoseconds, that's > cheaper. Does anybody see a

Re: properly sizing attnums in pg_get_publication_tables

2023-01-13 Thread Justin Pryzby
On Fri, Jan 13, 2023 at 07:37:29AM -0800, Ted Yu wrote: > Hi, > I was looking at commit b7ae03953690a1dee455ba3823cc8f71a72cbe1d . > > In `pg_get_publication_tables`, attnums is allocated with size > `desc->natts`. However, since some columns may be dropped, this size may be > larger than

Re: Transaction timeout

2023-01-13 Thread Nikolay Samokhvalov
On Fri, Jan 13, 2023 at 10:16 AM Andrey Borodin wrote: > > – it seems we could (should) have one more successful "1s wait, 3s > sleep" iteration here, ~727ms somehow wasted in a loop, quite a lot. > > I think big chunk from these 727ms were spent between "BEGIN" and > "select now(),

Re: [EXTERNAL] Re: Support load balancing in libpq

2023-01-13 Thread Jacob Champion
On Fri, Jan 13, 2023 at 9:10 AM Jelte Fennema wrote: > > > Just a quick single-issue review, but I agree with Maxim that having > > one PRNG, seeded once, would be simpler > > I don't agree that it's simpler. Because now there's a mutex you have > to manage, and honestly cross-platform threading

Re: Generate pg_stat_get_xact*() functions with Macros

2023-01-13 Thread Andres Freund
Hi, On 2023-01-13 10:36:49 +0100, Drouvot, Bertrand wrote: > > > > Although I suspect this actually hints at an architectural thing that > > > > could be > > > > fixed better: Perhaps we should replace find_tabstat_entry() with a > > > > version > > > > returning a fully "reconciled"

Re: Blocking execution of SECURITY INVOKER

2023-01-13 Thread Andres Freund
Hi, On 2023-01-13 10:04:19 -0800, Jeff Davis wrote: > However, the normal use case for expressions (whether in a default > expression or check constraint or whatever) is very simple and doesn't > even involve table access. There must be a way to satisfy those simple > cases without opening up a

Re: Transaction timeout

2023-01-13 Thread Andrey Borodin
Thanks for the review Nikolay! On Fri, Jan 13, 2023 at 8:03 AM Nikolay Samokhvalov wrote: > > 1) The current test set has only 2 simple cases – I'd suggest adding one more > (that one that didn't work in v1): > > gitpod=# set transaction_timeout to '20ms'; > SET > gitpod=# begin; select

Re: Blocking execution of SECURITY INVOKER

2023-01-13 Thread Jeff Davis
On Fri, 2023-01-13 at 00:16 -0800, Andres Freund wrote: > Just think of set_config(), pg_read_file(), lo_create(), > binary_upgrade_*(), > pg_drop_replication_slot()... Thank you for walking through the details here. I missed it from your first example because it was an extreme case -- a

Re: Add the ability to limit the amount of memory that can be allocated to backends.

2023-01-13 Thread Andres Freund
Hi, On 2023-01-13 09:15:10 -0500, Reid Thompson wrote: > On Mon, 2023-01-09 at 18:31 -0800, Andres Freund wrote: > > > Dynamic shared memory allocations are included only in the value displayed > > > for the backend that created them, they are not included in the value for > > > backends that are

Re: PL/Python: Fix return in the middle of PG_TRY() block.

2023-01-13 Thread Nathan Bossart
On Thu, Jan 12, 2023 at 09:49:00PM -0800, Andres Freund wrote: > On 2023-01-12 10:44:33 -0800, Nathan Bossart wrote: >> There's another "return" later on in this PG_TRY block. I wonder if it's >> possible to detect this sort of thing at compile time. > > Clang provides some annotations that

Re: No Callbacks on FATAL

2023-01-13 Thread Andres Freund
Hi, On 2023-01-13 16:14:11 +0300, Aleksander Alekseev wrote: > > > Hm? MemoryContextDelete() unconditionally calls the > > > callbacks. ShutdownPostgres() calls AbortOutOfAnyTransaction(). So if > > > there's > > > an ongoing transaction, we'll call the reset callbacks on > > > TopMemoryContext

Re: [EXTERNAL] Re: Support load balancing in libpq

2023-01-13 Thread Jelte Fennema
> Just a quick single-issue review, but I agree with Maxim that having > one PRNG, seeded once, would be simpler I don't agree that it's simpler. Because now there's a mutex you have to manage, and honestly cross-platform threading in C is not simple. However, I attached two additional patches

Re: Transaction timeout

2023-01-13 Thread Nikolay Samokhvalov
On Thu, Jan 12, 2023 at 11:47 AM Andrey Borodin wrote: > On Thu, Jan 12, 2023 at 11:24 AM Nathan Bossart > wrote: > > > > On Sun, Dec 18, 2022 at 12:53:31PM -0800, Andrey Borodin wrote: > > > I've rewritten this part to correctly report all timeouts that did > > > happen. However there's now a

properly sizing attnums in pg_get_publication_tables

2023-01-13 Thread Ted Yu
Hi, I was looking at commit b7ae03953690a1dee455ba3823cc8f71a72cbe1d . In `pg_get_publication_tables`, attnums is allocated with size `desc->natts`. However, since some columns may be dropped, this size may be larger than necessary. When `nattnums > 0` is false, there is no need to allocate the

Re: Fix condition in shm_toc and remove unused function shm_toc_freespace.

2023-01-13 Thread Zhang Mingli
Hi, Regards, Zhang Mingli On Jan 12, 2023, 16:54 +0800, Richard Guo , wrote: > > On Thu, Jan 12, 2023 at 2:50 PM Zhang Mingli wrote: > > On Jan 12, 2023, 14:34 +0800, Zhang Mingli , wrote: > > > Some conditions in shm_toc_insert and shm_toc_allocate are bogus, like: > > > if (toc_bytes

Re: PL/Python: Fix return in the middle of PG_TRY() block.

2023-01-13 Thread Xing Guo
Hi Nathan. On 1/13/23, Nathan Bossart wrote: > On Thu, Jan 12, 2023 at 11:19:29PM +0800, Xing Guo wrote: >> I was running static analyser against PostgreSQL and found there're 2 >> return statements in PL/Python module which is not safe. Patch is >> attached. > > Is the problem that

Re: Get relid for a relation

2023-01-13 Thread Tom Lane
Amin writes: > In CustomScan cost estimator, where PlannerInfo and RelOptInfo are passed, > I want to get access to the relation stats (for example pg_stat_all_tables) > by calling pg_stat_fetch_stat_tabentry(). However, I don't have access to > relid to pass to this function. Sure you do. The

Re: Add the ability to limit the amount of memory that can be allocated to backends.

2023-01-13 Thread Reid Thompson
On Mon, 2023-01-09 at 18:31 -0800, Andres Freund wrote: > Hi, > > On 2023-01-05 13:44:20 -0500, Reid Thompson wrote: > > This new field displays the current bytes of memory allocated to the > > backend process. It is updated as memory for the process is > > malloc'd/free'd. Memory allocated to

Re: [feature]COPY FROM enable FORCE_NULL/FORCE_NOT_NULL on all columns

2023-01-13 Thread Zhang Mingli
HI, On Dec 27, 2022, 19:02 +0800, Melih Mutlu , wrote: Hi, Having  FORCE_NULL(*) and FORCE_NOT_NULL(*) sounds good, since postgres already has FORCE_QUOTE(*). I just quickly tried out your patch. It worked for me as expected.  One little suggestion: + if

Re: Fix GUC_NO_SHOW_ALL test scenario in 003_check_guc.pl

2023-01-13 Thread Justin Pryzby
On Fri, Jan 13, 2023 at 06:15:38PM +0530, Nitin Jadhav wrote: > Hi, > > The commit 7265dbffad7feac6ea9d373828583b5d3d152e07 has added a script > in src/backend/utils/misc/check_guc that cross-checks the consistency > of the GUCs with postgresql.conf.sample, making sure that its format > is in

BF animal malleefowl reported an failure in 001_password.pl

2023-01-13 Thread houzj.f...@fujitsu.com
Hi, I noticed one BF failure[1] when monitoring the BF for some other commit. # Failed test 'authentication success for method password, connstr user=scram_role: log matches' # at t/001_password.pl line 120. # '2023-01-13 07:33:46.741 EST [243628:5] LOG: received SIGHUP,

Re: No Callbacks on FATAL

2023-01-13 Thread Aleksander Alekseev
Hi hackers, > > Hm? MemoryContextDelete() unconditionally calls the > > callbacks. ShutdownPostgres() calls AbortOutOfAnyTransaction(). So if > > there's > > an ongoing transaction, we'll call the reset callbacks on TopMemoryContext > > and > > its children. > > Hmm ... I'd forgotten that we'd

Re: PG11 to PG14 Migration Slowness

2023-01-13 Thread Aleksander Alekseev
Hi Vigneshk, > I'm migrating our existing PG instances from PG11.4 to PG14.3. I have > around 5 Million Tables in a single database. When migrating using > pg_upgrade, its taking 3 hours for the process to complete. I'm not sure if > its the intended behaviour or we're missing

Fix GUC_NO_SHOW_ALL test scenario in 003_check_guc.pl

2023-01-13 Thread Nitin Jadhav
Hi, The commit 7265dbffad7feac6ea9d373828583b5d3d152e07 has added a script in src/backend/utils/misc/check_guc that cross-checks the consistency of the GUCs with postgresql.conf.sample, making sure that its format is in line with what guc.c has. As per the commit message, the parameters which are

Re: Amcheck verification of GiST and GIN

2023-01-13 Thread Jose Arthur Benetasso Villanova
On Sun, 8 Jan 2023, Andrey Borodin wrote: On Sun, Jan 8, 2023 at 8:05 PM Andrey Borodin wrote: Please find the attached new version. In this patchset heapallindexed flag is removed from GIN checks. Uh... sorry, git-formatted wrong branch. Here's the correct version. Double checked.

Re: Fix pg_publication_tables to exclude generated columns

2023-01-13 Thread Amit Kapila
On Thu, Jan 12, 2023 at 12:33 PM shiy.f...@fujitsu.com wrote: > > On Wed, Jan 11, 2023 2:40 PM Amit Kapila wrote: > > > > On Wed, Jan 11, 2023 at 10:07 AM Tom Lane wrote: > > > > > > Amit Kapila writes: > > > >> On Mon, Jan 9, 2023 11:06 PM Tom Lane wrote: > > > >>> We could just not fix it

Re: Exit walsender before confirming remote flush in logical replication

2023-01-13 Thread Amit Kapila
On Wed, Dec 28, 2022 at 8:18 AM Hayato Kuroda (Fujitsu) wrote: > > Dear Amit, > > > > Firstly I considered 2, but I thought 1 seemed to be better. > > > PSA the updated patch. > > > > > > > I think even for logical replication we should check whether there is > > any pending WAL (via

RE: Perform streaming logical transactions by background workers and parallel apply

2023-01-13 Thread houzj.f...@fujitsu.com
On Friday, January 13, 2023 1:43 PM Masahiko Sawada wrote: > On Thu, Jan 12, 2023 at 9:34 PM houzj.f...@fujitsu.com > wrote: > > > > On Thursday, January 12, 2023 7:08 PM Amit Kapila > wrote: > > > > > > On Thu, Jan 12, 2023 at 4:21 PM shveta malik > wrote: > > > > > > > > On Thu, Jan 12,

RE: Perform streaming logical transactions by background workers and parallel apply

2023-01-13 Thread houzj.f...@fujitsu.com
On Friday, January 13, 2023 1:02 PM Masahiko Sawada wrote: > > On Fri, Jan 13, 2023 at 1:28 PM Amit Kapila wrote: > > > > On Fri, Jan 13, 2023 at 9:06 AM Amit Kapila > wrote: > > > > > > On Fri, Jan 13, 2023 at 7:56 AM Peter Smith > wrote: > > > > > > > > > > > > > > > 3. > > > > > > > >

RE: Perform streaming logical transactions by background workers and parallel apply

2023-01-13 Thread houzj.f...@fujitsu.com
On Friday, January 13, 2023 2:20 PM Peter Smith wrote: > > Here are some review comments for patch v79-0002. Thanks for your comments. > == > > General > > 1. > > I saw that earlier in this thread Hou-san [1] and Amit [2] also seemed to say > there is not much point for this patch. > >

Re: Non-decimal integer literals

2023-01-13 Thread Dean Rasheed
On Wed, 14 Dec 2022 at 05:47, Peter Eisentraut wrote: > > committed Now that we have this for integer types, I think it's worth doing for numeric as well, since the parser will now pass such things through to numeric_in() when they don't fit in an int64, and it seems plausible that at least some

Re: releasing ParallelApplyTxnHash when pa_launch_parallel_worker returns NULL

2023-01-13 Thread Amit Kapila
On Thu, Jan 12, 2023 at 8:25 AM Ted Yu wrote: > On Wed, Jan 11, 2023 at 6:54 PM Amit Kapila wrote: >> >> On Wed, Jan 11, 2023 at 9:31 AM Ted Yu wrote: >> > >> > On Tue, Jan 10, 2023 at 7:55 PM houzj.f...@fujitsu.com >> > wrote: >> >> >> >> On Wednesday, January 11, 2023 10:21 AM Ted Yu >>

Re: Generate pg_stat_get_xact*() functions with Macros

2023-01-13 Thread Drouvot, Bertrand
Hi, On 1/12/23 7:24 PM, Andres Freund wrote: Hi, On 2023-01-12 08:38:57 +0100, Drouvot, Bertrand wrote: On 1/11/23 11:59 PM, Andres Freund wrote: Now that this patch renames some fields I don't mind renaming the fields - the prefixes really don't provide anything useful. But it's not clear

Re: Minimal logical decoding on standbys

2023-01-13 Thread Drouvot, Bertrand
Hi, On 1/11/23 9:27 PM, Andres Freund wrote: Hi, On 2023-01-06 10:52:06 +0100, Drouvot, Bertrand wrote: The problem I have with that is that I saw a lot of flakiness in the tests due to the race condition. So introducing them in that order just doesn't make a whole lot of sense to me. You

Re: Get access to the whole query in CustomScan path callback

2023-01-13 Thread Amin
I cannot find any information related to other relations in the query other than the one which is being scanned in the root pointer. Is there any function call which can be used to get access to it? On Wed, Dec 21, 2022 at 9:46 AM Tom Lane wrote: > Amin writes: > > The goal is to have access

Get relid for a relation

2023-01-13 Thread Amin
Hi, In CustomScan cost estimator, where PlannerInfo and RelOptInfo are passed, I want to get access to the relation stats (for example pg_stat_all_tables) by calling pg_stat_fetch_stat_tabentry(). However, I don't have access to relid to pass to this function. For a sample relation, when I

Re: Blocking execution of SECURITY INVOKER

2023-01-13 Thread Jeff Davis
On Thu, 2023-01-12 at 19:38 -0800, Andres Freund wrote: > I.e. the default arguments where evaluated with the invoker's > permissions, not > the definer's, despite being controlled by the less privileged user. This is a very interesting case. It also involves tricking the superuser into executing

Re: [EXTERNAL] Re: [PATCH] Support using "all" for the db user in pg_ident.conf

2023-01-13 Thread Jelte Fennema
> Even if folks applying quotes > would not be able anymore to replace the pattern, the risk seems a bit > remote? Yeah I agree the risk is remote. To be clear, the main pattern I'm worried about breaking is simply "\1". Where people had put quotes around \1 for no reason. All in all, I'm fine if

Re: Blocking execution of SECURITY INVOKER

2023-01-13 Thread Andres Freund
Hi, On 2023-01-12 23:38:50 -0800, Jeff Davis wrote: > On Thu, 2023-01-12 at 19:29 -0800, Andres Freund wrote: > > superuser: > > # CREATE FUNCTION exec_su(p_sql text) RETURNS text LANGUAGE plpgsql > > SECURITY DEFINER AS $$BEGIN RAISE NOTICE 'executing %', p_sql; > > EXECUTE p_sql;RETURN

Re: Sampling-based timing for EXPLAIN ANALYZE

2023-01-13 Thread David Geier
Nice idea. On 1/6/23 10:19, Jelte Fennema wrote: Do you have some performance comparison between TIMING ON and TIMING SAMPLING? +1 to see some numbers compared to TIMING ON. Mostly I'm wondering if the sampling based approach gains us enough to be worth it, once the patch to use RDTSC