Re: Adding Support for Copy callback functionality on COPY TO api

2022-10-07 Thread Michael Paquier
On Fri, Oct 07, 2022 at 02:48:24PM -0700, Nathan Bossart wrote: > Here is an attempt at adding such a test module. Using an ereport(NOTICE) to show the data reported in the callback is fine by me. How about making the module a bit more modular, by passing as argument a regclass and building a

Re: use has_privs_of_role() for pg_hba.conf

2022-10-07 Thread Michael Paquier
On Fri, Oct 07, 2022 at 07:59:08AM -0400, Robert Haas wrote: > I hadn't noticed this thread before. > > I'm not sure whether this is properly considered a privilege check. It > could even be an anti-privilege, if the pg_hba.conf line in question > is maked "reject". > > I'm not taking the

Re: Difference between HeapTupleData and TupleTableSlot structures

2022-10-07 Thread Tom Lane
Ajay P S writes: > I am pretty new to the Postgres code base. I would like to know the > difference between HeapTupleData and TupleTableSlot structures. HeapTupleData is just a pointer to a concrete tuple. It exists mainly because it's often convenient to pass around the tuple's logical

Re: START_REPLICATION SLOT causing a crash in an assert build

2022-10-07 Thread Andres Freund
Hi, On 2022-10-07 12:00:56 -0700, Andres Freund wrote: > On 2022-10-07 15:30:43 +0900, Kyotaro Horiguchi wrote: > > The key point of this is this: > > > > +* XXX: I think there cannot actually be data from an older slot > > +* here. After a crash we throw away the old stats data and if

Difference between HeapTupleData and TupleTableSlot structures

2022-10-07 Thread Ajay P S
Hi, I am pretty new to the Postgres code base. I would like to know the difference between HeapTupleData and TupleTableSlot structures. Basically I am trying to understand some of the table access methods like heap_insert, heap_getnext, heap_getnextslot etc where some accepts Heaptuple as input

Re: Avoid mix char with bool type in comparisons

2022-10-07 Thread Ranier Vilela
Em sex., 7 de out. de 2022 às 15:45, Ranier Vilela escreveu: > Em sex., 7 de out. de 2022 às 14:44, Tom Lane > escreveu: > >> Ranier Vilela writes: >> > Anyway, it's not *true* that collectMatchesForHeapRow deals >> > only "true" and "false", once that key->entryRes is initialized with >> >

Re: Non-robustness in pmsignal.c

2022-10-07 Thread Tom Lane
Andres Freund writes: > On 2022-10-07 20:35:58 -0400, Tom Lane wrote: >> Andres Freund writes: >>> Why are we even tracking PM_CHILD_UNUSED / PM_CHILD_ASSIGNED in shared >>> memory? >> Because those flags are set by the child processes too, cf >> MarkPostmasterChildActive and

Re: Non-robustness in pmsignal.c

2022-10-07 Thread Andres Freund
Hi, On 2022-10-07 20:35:58 -0400, Tom Lane wrote: > Andres Freund writes: > > Why are we even tracking PM_CHILD_UNUSED / PM_CHILD_ASSIGNED in shared > > memory? > > Because those flags are set by the child processes too, cf > MarkPostmasterChildActive and MarkPostmasterChildInactive. Only

Re: Non-robustness in pmsignal.c

2022-10-07 Thread Tom Lane
Andres Freund writes: > Why are we even tracking PM_CHILD_UNUSED / PM_CHILD_ASSIGNED in shared memory? Because those flags are set by the child processes too, cf MarkPostmasterChildActive and MarkPostmasterChildInactive. > Are you thinking these should be backpatched? I am, but I'm not

Re: Non-robustness in pmsignal.c

2022-10-07 Thread Andres Freund
Hi, On 2022-10-07 19:57:35 -0400, Tom Lane wrote: > As I mentioned in another thread, I came across a reproducible > situation in which a memory clobber in a child backend crashes > the postmaster too, at least on FreeBSD/arm64. Needless to say, > this is Not Cool. Ugh. > I've now traced down

Non-robustness in pmsignal.c

2022-10-07 Thread Tom Lane
As I mentioned in another thread, I came across a reproducible situation in which a memory clobber in a child backend crashes the postmaster too, at least on FreeBSD/arm64. Needless to say, this is Not Cool. I've now traced down what is happening, and it's this: 1. Careless coding in aset.c

subtransaction performance

2022-10-07 Thread Zhihong Yu
Hi, I stumbled over: https://about.gitlab.com/blog/2021/09/29/why-we-spent-the-last-month-eliminating-postgresql-subtransactions/ I wonder if SAVEPOINT / subtransaction performance has been boosted since the blog was written. Cheers

Re: use has_privs_of_role() for pg_hba.conf

2022-10-07 Thread Nathan Bossart
On Fri, Oct 07, 2022 at 04:18:59PM -0400, Tom Lane wrote: > There's another problem there, which is that buildfarm animals > using -DENFORCE_REGRESSION_TEST_NAME_RESTRICTIONS will complain > about role names that don't start with "regress_". Huh, I hadn't noticed that one before. It looks like

Re: Adding Support for Copy callback functionality on COPY TO api

2022-10-07 Thread Nathan Bossart
On Fri, Oct 07, 2022 at 03:49:31PM +0900, Michael Paquier wrote: > Perhaps there should be a module in src/test/modules/ to provide a > short, still useful, example of what this could achieve? Here is an attempt at adding such a test module. -- Nathan Bossart Amazon Web Services:

Re: Nicely exiting PG_TRY and PG_CATCH

2022-10-07 Thread Tom Lane
Mikhail Gribkov writes: > Usually it's not a good idea to exit PG_TRY() block via return statement. > Otherwise it would leave PG_exception_stack global variable in a wrong > state and next ereport() will jump to some junk address. Yeah, you can't return or goto out of the PG_TRY part. >

Re: Add support for DEFAULT specification in COPY FROM

2022-10-07 Thread Israel Barth Rubio
Hello Zhihong, > For the last question, please take a look at: > > #define MemSetAligned(start, val, len) \ > > which is called by palloc0(). Oh, I totally missed that. Thanks for the heads up! I'm attaching the new patch version, which contains both the fix to the problem reported by Andres,

Re: use has_privs_of_role() for pg_hba.conf

2022-10-07 Thread Tom Lane
Nathan Bossart writes: > On Fri, Oct 07, 2022 at 03:34:51PM +0900, Michael Paquier wrote: >> Thanks. I would perhaps use names less generic than role{1,2,3} for >> the roles or "role1" for the database name, but the logic looks >> sound. > Here is a new version with more descriptive role names.

Re: use has_privs_of_role() for pg_hba.conf

2022-10-07 Thread Nathan Bossart
On Fri, Oct 07, 2022 at 03:34:51PM +0900, Michael Paquier wrote: > Thanks. I would perhaps use names less generic than role{1,2,3} for > the roles or "role1" for the database name, but the logic looks > sound. Here is a new version with more descriptive role names. -- Nathan Bossart Amazon Web

Re: Modernizing our GUC infrastructure

2022-10-07 Thread Tom Lane
I wrote: >> Attached is a patch series that attempts to modernize our GUC >> infrastructure, in particular removing the performance bottlenecks >> it has when there are lots of GUC variables. > Rebased over 0a20ff54f. Here's a v3 rebased up to HEAD. The only real change is that I added a couple

Re: Add support for DEFAULT specification in COPY FROM

2022-10-07 Thread Israel Barth Rubio
Hello Andres, > cfbot shows that tests started failing with this version: > https://cirrus-ci.com/github/postgresql-cfbot/postgresql/commitfest/39/3822 > A full backtrace is at https://api.cirrus-ci.com/v1/task/5354378189078528/logs/cores.log Thanks for pointing this out. I had initially missed

Re: Add support for DEFAULT specification in COPY FROM

2022-10-07 Thread Zhihong Yu
On Fri, Oct 7, 2022 at 12:09 PM Israel Barth Rubio wrote: > Hello Zhihong, > > > + /* attribute is NOT to be copied from input */ > > > > I think saying `is NOT copied from input` should suffice. > > > > + /* fieldno is 0-index and attnum is 1-index */ > > > > 0-index ->

Re: meson PGXS compatibility

2022-10-07 Thread Andres Freund
Hi, On 2022-10-05 13:07:10 -0700, Andres Freund wrote: > 0003: aix: Build SUBSYS.o using $(CC) -r instead of $(LD) -r > > This is the only direct use of $(LD), and xlc -r and gcc -r end up with the > same set of symbols and similar performance (noise is high, so hard to say > if >

Re: Add support for DEFAULT specification in COPY FROM

2022-10-07 Thread Israel Barth Rubio
Hello Zhihong, > + /* attribute is NOT to be copied from input */ > > I think saying `is NOT copied from input` should suffice. > > + /* fieldno is 0-index and attnum is 1-index */ > > 0-index -> 0-indexed I have applied both suggestions, thanks! I'll submit a 4th version

Re: START_REPLICATION SLOT causing a crash in an assert build

2022-10-07 Thread Andres Freund
Hi, On 2022-10-07 15:30:43 +0900, Kyotaro Horiguchi wrote: > At Fri, 7 Oct 2022 12:14:40 +0900, Masahiko Sawada > wrote in > > > What about if we go the other direction - simply remove the name from the > > > stats entry at all. I don't actually think we need it anymore. Unless I am > > >

Re: Avoid mix char with bool type in comparisons

2022-10-07 Thread Ranier Vilela
Em sex., 7 de out. de 2022 às 14:44, Tom Lane escreveu: > Ranier Vilela writes: > > Anyway, it's not *true* that collectMatchesForHeapRow deals > > only "true" and "false", once that key->entryRes is initialized with > > GIN_FALSE not false. > > Look: GinTernaryValue is carefully designed so

Re: [BUG] parenting a PK constraint to a self-FK one (Was: Self FK oddity when attaching a partition)

2022-10-07 Thread Alvaro Herrera
On 2022-Oct-05, Alvaro Herrera wrote: > Backpatching this to 12 shows yet another problem -- the topmost > relation acquires additional FK constraints, not yet sure why. I think > we must have fixed something in 13 that wasn't backpatched, but I can't > remember what it is and whether it was

Re: Avoid mix char with bool type in comparisons

2022-10-07 Thread Ranier Vilela
Em sex., 7 de out. de 2022 às 13:32, Robert Haas escreveu: > On Fri, Oct 7, 2022 at 11:40 AM Tom Lane wrote: > > FWIW, I did take a look at this code, and I don't see any bug. > > The entryRes[] array entries are indeed GinTernaryValue, but it's > > obvious by inspection that

Re: Avoid mix char with bool type in comparisons

2022-10-07 Thread Tom Lane
Ranier Vilela writes: > Anyway, it's not *true* that collectMatchesForHeapRow deals > only "true" and "false", once that key->entryRes is initialized with > GIN_FALSE not false. Look: GinTernaryValue is carefully designed so that it is representation-compatible with bool, including that

Re: Error-safe user functions

2022-10-07 Thread Tom Lane
Nikita Glukhov writes: > On 30.09.2022, Tom Lane wrote: >> I strongly recommend against having a new pg_proc column at all. > I think the only way to avoid catalog modification (adding new columns > to pg_proc or pg_type, introducing new function signatures etc.) and > to avoid adding some

Re: Mingw task for Cirrus CI

2022-10-07 Thread Melih Mutlu
Hi hackers. Andres Freund , 22 Eyl 2022 Per, 20:03 tarihinde şunu yazdı: > Now that meson has been merged, could you try to adjust this patch so it > builds with meson? > Attached patch is the adjusted version to build with meson. Add a commented-out trigger-type: manual and note in the commit

Re: Avoid mix char with bool type in comparisons

2022-10-07 Thread Ranier Vilela
Em sex., 7 de out. de 2022 às 12:40, Tom Lane escreveu: > Robert Haas writes: > > On Thu, Oct 6, 2022 at 8:35 PM Ranier Vilela > wrote: > >> No Tom, unfortunately I don't have the knowledge to create a test with > GIN_MAYBE values. > > > Well then don't post. > > > Basically what you're saying

Re: proposal: possibility to read dumped table's name from file

2022-10-07 Thread Pavel Stehule
pá 7. 10. 2022 v 16:03 odesílatel Julien Rouhaud napsal: > On Fri, Oct 07, 2022 at 07:26:08AM +0200, Pavel Stehule wrote: > > > > I checked this code again, and I don't think some refactoring is easy. > > getFiltersFromFile is not duplicated. It is just probably badly named. > > > > These

Re: Simplify event trigger support checking functions

2022-10-07 Thread Robert Haas
On Fri, Oct 7, 2022 at 8:10 AM Peter Eisentraut wrote: > The only drawback in terms of code robustness is that someone adding a > new shared object type would have to remember to add it to > EventTriggerSupportsObjectType(). This doesn't seem like a good idea to me. If the function names give

Re: Avoid mix char with bool type in comparisons

2022-10-07 Thread Robert Haas
On Fri, Oct 7, 2022 at 11:40 AM Tom Lane wrote: > FWIW, I did take a look at this code, and I don't see any bug. > The entryRes[] array entries are indeed GinTernaryValue, but it's > obvious by inspection that matchPartialInPendingList only returns > true or false, therefore

Re: Avoid mix char with bool type in comparisons

2022-10-07 Thread Tom Lane
Robert Haas writes: > On Thu, Oct 6, 2022 at 8:35 PM Ranier Vilela wrote: >> No Tom, unfortunately I don't have the knowledge to create a test with >> GIN_MAYBE values. > Well then don't post. > Basically what you're saying is that you suspect there might be a > problem with this code but you

Re: create subscription - improved warning message

2022-10-07 Thread Tom Lane
Peter Smith writes: > WARNING: tables were not subscribed, you will have to run ALTER > SUBSCRIPTION ... REFRESH PUBLICATION to subscribe the tables > When I first encountered the above CREATE SUBSCRIPTION warning message > I thought it was dubious-looking English... > On closer inspection I

Re: Convert macros to static inline functions

2022-10-07 Thread Peter Eisentraut
On 04.10.22 08:57, Amul Sul wrote: On Tue, Oct 4, 2022 at 12:00 PM Peter Eisentraut wrote: On 16.05.22 10:27, Peter Eisentraut wrote: Inspired by [0], I looked to convert more macros to inline functions. Here is another one from the same batch of work that I somehow didn't send in last

Re: proposal: possibility to read dumped table's name from file

2022-10-07 Thread Julien Rouhaud
On Fri, Oct 07, 2022 at 07:26:08AM +0200, Pavel Stehule wrote: > > I checked this code again, and I don't think some refactoring is easy. > getFiltersFromFile is not duplicated. It is just probably badly named. > > These routines are used from pg_dump, pg_dumpall and pg_restore. There are >

Re: Avoid mix char with bool type in comparisons

2022-10-07 Thread Robert Haas
On Thu, Oct 6, 2022 at 8:35 PM Ranier Vilela wrote: > No Tom, unfortunately I don't have the knowledge to create a test with > GIN_MAYBE values. Well then don't post. Basically what you're saying is that you suspect there might be a problem with this code but you don't really know that and you

Re: ps command does not show walsender's connected db

2022-10-07 Thread Fujii Masao
On 2022/10/07 18:43, bt22nakamorit wrote: 2022-10-07 16:59  Fujii Masao wrote: s/subscriber/publisher ? I did not understand what this means. Sorry for confusing wording... I was just trying to say that walsender is connected to a database of the publisher instead of subscriber.

Re: [patch] \g with multiple result sets and \watch with copy queries

2022-10-07 Thread Daniel Verite
Tom Lane wrote: > > Currently, test/regress/sql/psql.sql doesn't AFAICS write anything > > outside of stdout, but \g, \o, \copy need to write to external > > files to be tested properly. > > Yeah, I don't think we can usefully test these in psql.sql, because > file-system side effects

[PATCH] Reset single-row processing mode at end of pipeline commands queue

2022-10-07 Thread Denis Laxalde
Hello, I'm trying to make single-row mode and pipeline mode work together in Psycopg using libpq. I think there is something wrong with respect to the single-row mode flag, not being correctly reset, in some situations. The minimal case I'm considering is (in a pipeline): * send query 1, *

Simplify event trigger support checking functions

2022-10-07 Thread Peter Eisentraut
We have in event_trigger.c two functions EventTriggerSupportsObjectType() and EventTriggerSupportsObjectClass() that return whether a given object type/class supports event triggers. Maybe there was a real differentiation there once, but right now it seems to me that *all* object types/classes

Re: future of serial and identity columns

2022-10-07 Thread Magnus Hagander
On Fri, Oct 7, 2022 at 2:03 PM Vik Fearing wrote: > On 10/4/22 09:41, Peter Eisentraut wrote: > > In PostgreSQL 10, we added identity columns, as an alternative to serial > > columns (since 6.something). They mostly work the same. Identity > > columns are SQL-conforming, have some more

Re: future of serial and identity columns

2022-10-07 Thread Vik Fearing
On 10/4/22 09:41, Peter Eisentraut wrote: In PostgreSQL 10, we added identity columns, as an alternative to serial columns (since 6.something).  They mostly work the same.  Identity columns are SQL-conforming, have some more features (e.g., overriding clause), and are a bit more robust in

Re: use has_privs_of_role() for pg_hba.conf

2022-10-07 Thread Robert Haas
On Fri, Apr 1, 2022 at 6:07 PM Nathan Bossart wrote: > > 6198420 ensured that has_privs_of_role() is used for predefined roles, > which means that the role inheritance hierarchy is checked instead of mere > role membership. However, inheritance is still not respected for > pg_hba.conf.

Re: [PATCH] Use indexes on the subscriber when REPLICA IDENTITY is full on the publisher

2022-10-07 Thread Önder Kalacı
Hi Kurado Hayato, > In your patch: > > ``` > + /* Simple case, we already have a primary key or a replica > identity index */ > + idxoid = GetRelationIdentityOrPK(localrel); > + if (OidIsValid(idxoid)) > + return idxoid; > + > + /* > +* If index

Re: XID formatting and SLRU refactorings (was: Add 64-bit XIDs into PostgreSQL 15)

2022-10-07 Thread Maxim Orlov
> > This is the wrong thread / CF entry. Please see > Yep, my fault. Sorry about that. -- Best regards, Maxim Orlov.

Re: Make ON_ERROR_STOP stop on shell script failure

2022-10-07 Thread bt22nakamorit
2022-10-07 17:16 Fujii Masao wrote: The patch failed to be applied into the master cleanly. Could you rebase it? patching file src/bin/psql/common.c Hunk #1 succeeded at 94 (offset 4 lines). Hunk #2 succeeded at 104 (offset 4 lines). Hunk #3 succeeded at 133 (offset 4 lines). Hunk #4 succeeded

Re: Eliminating SPI from RI triggers - take 2

2022-10-07 Thread Amit Langote
On Fri, Oct 7, 2022 at 19:15 Alvaro Herrera wrote: > On 2022-Oct-07, Amit Langote wrote: > > > > Thanks for the heads up. Hmm, this I am not sure how to reproduce on > > > my own, so I am currently left with second-guessing what may be going > > > wrong on 32 bit machines with whichever of the

Re: Eliminating SPI from RI triggers - take 2

2022-10-07 Thread Alvaro Herrera
On 2022-Oct-07, Amit Langote wrote: > > Thanks for the heads up. Hmm, this I am not sure how to reproduce on > > my own, so I am currently left with second-guessing what may be going > > wrong on 32 bit machines with whichever of the 4 patches. > > > > For now, I'll just post 0001, which I am

Re: Eliminating SPI from RI triggers - take 2

2022-10-07 Thread Amit Langote
On Fri, Oct 7, 2022 at 6:26 PM Amit Langote wrote: > On Sun, Oct 2, 2022 at 10:24 AM Andres Freund wrote: > > On 2022-10-01 18:21:15 -0700, Andres Freund wrote: > > > On 2022-09-29 18:18:10 +0900, Amit Langote wrote: > > > > So, here's a final revision for today. Sorry for the noise. > > > > >

Re: ps command does not show walsender's connected db

2022-10-07 Thread bt22nakamorit
2022-10-07 16:59 Fujii Masao wrote: s/subscriber/publisher ? I did not understand what this means. Thanks for the patch! - + printf("fork child process\n"); + printf(" am_walsender: %d\n", am_walsender); + printf(" am_db_walsender: %d\n",

Re: Eliminating SPI from RI triggers - take 2

2022-10-07 Thread Amit Langote
On Sun, Oct 2, 2022 at 10:24 AM Andres Freund wrote: > On 2022-10-01 18:21:15 -0700, Andres Freund wrote: > > On 2022-09-29 18:18:10 +0900, Amit Langote wrote: > > > So, here's a final revision for today. Sorry for the noise. > > > > This appears to fail on 32bit systems. Seems the new test is

Re: Make ON_ERROR_STOP stop on shell script failure

2022-10-07 Thread Fujii Masao
On 2022/09/30 16:54, bt22nakamorit wrote: 2022-09-21 11:45 に Fujii Masao wrote: We can execute the shell commands via psql in various ways other than \! meta-command. For example, * `command` * \g | command * \gx | command * \o | command * \w | command * \copy ... program 'command'

Re: [PoC] Improve dead tuple storage for lazy vacuum

2022-10-07 Thread Masahiko Sawada
On Fri, Oct 7, 2022 at 2:29 PM John Naylor wrote: > > On Fri, Sep 16, 2022 at 1:01 PM Masahiko Sawada wrote: > > In addition to two patches, I've attached the third patch. It's not > > part of radix tree implementation but introduces a contrib module > > bench_radix_tree, a tool for radix tree

Re: ps command does not show walsender's connected db

2022-10-07 Thread Fujii Masao
On 2022/10/06 22:30, bt22nakamorit wrote: Hi, When walsender process is evoked for logical replication, walsender is connected to a database of the subscriber. Naturally, ones would want the name of the connected database to show in the entry of ps command for walsender. In detail, running

Nicely exiting PG_TRY and PG_CATCH

2022-10-07 Thread Mikhail Gribkov
Hi hackers, I've found some odd lines in plpython-related code. These look to me like a potential source of problems, but maybe I'm not fully aware of some nuances. Usually it's not a good idea to exit PG_TRY() block via return statement. Otherwise it would leave PG_exception_stack global

Re: Adding Support for Copy callback functionality on COPY TO api

2022-10-07 Thread Michael Paquier
On Tue, Aug 02, 2022 at 04:49:19PM -0700, Nathan Bossart wrote: > I've rebased this patch and will register it in the next commitfest > shortly. Perhaps there should be a module in src/test/modules/ to provide a short, still useful, example of what this could achieve? -- Michael signature.asc

make_ctags: use -I option to ignore pg_node_attr macro

2022-10-07 Thread Yugo NAGATA
Hi, I found that tag files generated by src/tools/make_ctags doesn't include some keywords, that were field names of node structs, for example norm_select in RestrictInfo. Such fields are defined with pg_node_attr macro that was introduced recently, like: Selectivity norm_selec

Re: use has_privs_of_role() for pg_hba.conf

2022-10-07 Thread Michael Paquier
On Thu, Oct 06, 2022 at 08:27:11PM -0700, Nathan Bossart wrote: > On Fri, Oct 07, 2022 at 11:06:47AM +0900, Michael Paquier wrote: >> Rather than putting that in a separate script, which means >> initializing a new node, etc. could it be better to put that in >> 001_password.pl instead? It would

Re: START_REPLICATION SLOT causing a crash in an assert build

2022-10-07 Thread Kyotaro Horiguchi
At Fri, 7 Oct 2022 12:14:40 +0900, Masahiko Sawada wrote in > > What about if we go the other direction - simply remove the name from the > > stats entry at all. I don't actually think we need it anymore. Unless I am > > missing something right now - entirely possible! - the danger that > >

Re: Fast COPY FROM based on batch insert

2022-10-07 Thread Etsuro Fujita
On Tue, Sep 27, 2022 at 6:03 PM Etsuro Fujita wrote: > I will review the patch a bit more, but I feel that it is > in good shape. One thing I noticed is this bit added to CopyMultiInsertBufferFlush() to run triggers on the foreign table. + /* Run AFTER ROW INSERT triggers */ +

RE: Perform streaming logical transactions by background workers and parallel apply

2022-10-07 Thread houzj.f...@fujitsu.com
On Thursday, October 6, 2022 8:40 PM Kuroda, Hayato/黒田 隼人 wrote: > > Dear Hou, > > I put comments for v35-0001. Thanks for the comments. > 01. catalog.sgml > > ``` > + Controls how to handle the streaming of in-progress transactions: > + f = disallow streaming of in-progress