Re: please update ps display for recovery checkpoint

2020-10-02 Thread Michael Paquier
On Sat, Sep 19, 2020 at 11:00:31AM -0500, Justin Pryzby wrote: > Maybe it's a bad idea if the checkpointer is continuously changing its > display. > I don't see the utility in it, since log_checkpoints does more than ps could > ever do. I'm concerned that would break things for someone using some

Re: Improve choose_custom_plan for initial partition prune case

2020-10-02 Thread Amit Langote
Hi Andy, On Fri, Oct 2, 2020 at 1:04 AM Andy Fan wrote: > > Given the plan example: > > CREATE TABLE measurement ( > city_id int not null, > logdate date not null, > peaktempint, > unitsales int > ) PARTITION BY RANGE (logdate); > > CREATE TABLE measu

Re: pgbench - refactor init functions with buffers

2020-10-02 Thread Fabien COELHO
Can you elaborate what you meant by the new "print overheads should probably be avoided" comment? Because printf is slow and this is on the critical path of data generation. Printf has to interpret the format each time just to print three ints, specialized functions could be used which woul

Re: please update ps display for recovery checkpoint

2020-10-02 Thread Justin Pryzby
On Fri, Oct 02, 2020 at 04:28:14PM +0900, Michael Paquier wrote: > > Related: I have always thought that this message meant "recovery will > > complete > > Real Soon", but I now understand it to mean "beginning the recovery > > checkpoint, > > which is flagged CHECKPOINT_IMMEDIATE" (and may take

RE: Transactions involving multiple postgres foreign servers, take 2

2020-10-02 Thread tsunakawa.ta...@fujitsu.com
From: Masahiko Sawada > You proposed the first idea > to avoid such a situation that FDW implementor can write the code > while trying to reduce the possibility of errors happening as much as > possible, for example by usingpalloc_extended(MCXT_ALLOC_NO_OOM) and > hash_search(HASH_ENTER_NULL) but

Re: Error code missing for "wrong length of inner sequence" error

2020-10-02 Thread Heikki Linnakangas
On 01/10/2020 14:21, Daniel Gustafsson wrote: On 1 Oct 2020, at 12:54, Heikki Linnakangas wrote: Most checks when converting between SQL and Python types use the PLy_elog() function, which uses the genericc ERRCODE_EXTERNAL_ROUTINE_EXCEPTION error code, but I think ERRCODE_ARRAY_SUBSCRIPT_E

a misbehavior of partition row movement (?)

2020-10-02 Thread Amit Langote
Hi, Robert forwarded me a pgsql-general thread [1] where a ON DELETE CASCADE specified on a foreign key pointing to a partitioned table is shown to cause a possibly surprising end result during an update of the partitioned table. Example from that thread: create table parent ( id serial, constra

Re: [HACKERS] make async slave to wait for lsn to be replayed

2020-10-02 Thread a . pervushina
Anna Akenteva писал 2020-04-08 22:36: On 2020-04-08 04:09, Kyotaro Horiguchi wrote: I like your suggested keywords! I think that "AFTER" + "WITHIN" sound the most natural. We could completely give up the LSN keyword for now. The final command could look something like: BEGIN AFTER ‘0/303EC60’ W

Re: Error code missing for "wrong length of inner sequence" error

2020-10-02 Thread Daniel Gustafsson
> On 2 Oct 2020, at 13:44, Heikki Linnakangas wrote: > > On 01/10/2020 14:21, Daniel Gustafsson wrote: >>> On 1 Oct 2020, at 12:54, Heikki Linnakangas wrote: >>> Most checks when converting between SQL and Python types use the PLy_elog() >>> function, which uses the genericc ERRCODE_EXTERNAL_RO

Re: Should walsernder check correctness of WAL records?

2020-10-02 Thread Konstantin Knizhnik
On 02.10.2020 3:28, Michael Paquier wrote: On Fri, Oct 02, 2020 at 12:16:25AM +, tsunakawa.ta...@fujitsu.com wrote: IIUC, walsender tries hard to send WAL as fast as possible to reduce replication lag and transaction response time, so it doesn't try to peek each WAL record. I think it's

Re: enable_incremental_sort changes query behavior

2020-10-02 Thread James Coleman
On Thu, Oct 1, 2020 at 9:10 PM James Coleman wrote: > > On Thu, Oct 1, 2020 at 6:08 PM Tomas Vondra > wrote: > > > > On Thu, Oct 01, 2020 at 09:02:57AM -0400, James Coleman wrote: > > >On Thu, Oct 1, 2020 at 3:09 AM Jaime Casanova > > > wrote: > > >> > > >> On Wed, 30 Sep 2020 at 21:21, James Col

a misbehavior of partition row movement (?)

2020-10-02 Thread David G. Johnston
On Friday, October 2, 2020, Amit Langote wrote: > > Reporter on that thread says that the last update should have failed > and I don't quite see a workable alternative to that. To be clear the OP would rather have it just work, the same as the non-row-movement version. Maybe insert the new row

Re: enable_incremental_sort changes query behavior

2020-10-02 Thread Tomas Vondra
On Fri, Oct 02, 2020 at 09:19:44AM -0400, James Coleman wrote: ... I've been able to confirm that the problem goes away if we stop adding the gather merge paths in generate_useful_gather_paths(). I'm not sure yet what conclusion that leads us to. It seems to be that the biggest clue remains th

Re: Rejecting redundant options in Create Collation

2020-10-02 Thread Daniel Verite
Michael Paquier wrote: > > Hmm ... I think that that is pretty standard behavior for a lot of > > our utility commands. Trying something at random, > > The behavior handling is a bit inconsistent. For example EXPLAIN and > VACUUM don't do that, because their parenthesized grammar got >

Re: enable_incremental_sort changes query behavior

2020-10-02 Thread James Coleman
On Fri, Oct 2, 2020 at 10:32 AM Tomas Vondra wrote: > > On Fri, Oct 02, 2020 at 09:19:44AM -0400, James Coleman wrote: > > > > ... > > > >I've been able to confirm that the problem goes away if we stop adding > >the gather merge paths in generate_useful_gather_paths(). > > > >I'm not sure yet what

Re: enable_incremental_sort changes query behavior

2020-10-02 Thread James Coleman
On Fri, Oct 2, 2020 at 10:53 AM James Coleman wrote: > > On Fri, Oct 2, 2020 at 10:32 AM Tomas Vondra > wrote: > > > > On Fri, Oct 02, 2020 at 09:19:44AM -0400, James Coleman wrote: > > > > > > ... > > > > > >I've been able to confirm that the problem goes away if we stop adding > > >the gather m

Re: Error code missing for "wrong length of inner sequence" error

2020-10-02 Thread Heikki Linnakangas
On 02/10/2020 15:06, Daniel Gustafsson wrote: On 2 Oct 2020, at 13:44, Heikki Linnakangas wrote: On 01/10/2020 14:21, Daniel Gustafsson wrote: On that note, wouldn't the dimension check errors in PLySequence_ToArray be just as well off using normal ereport()'s? Only one of them seem to error

Re: Rejecting redundant options in Create Collation

2020-10-02 Thread Tom Lane
"Daniel Verite" writes: > Assuming we agree that redundant options should consistently > raise an error for a certain class of statements, could it be handled > at the grammar level? I don't think this'd be a great idea. The grammar would have to do something pretty brute-force to check for dupl

Re: Dumping/restoring fails on inherited generated column

2020-10-02 Thread Peter Eisentraut
On 2020-09-29 18:37, Tom Lane wrote: Unfortunately this has still got a problem: it will mishandle the case of a child column that is GENERATED while its parent is not. Peter opined way upthread that we should not allow that, but according to my testing we do. Did I opine that? Commit 086ffdd

Re: buildfarm animal shoveler failing with "Illegal instruction"

2020-10-02 Thread Mark Wong
On Thu, Oct 01, 2020 at 09:12:53PM -0400, Tom Lane wrote: > Mark Wong writes: > > I'm getting Tom set up with access too, in case he has time before me to > > get a stack trace to see what's happening... > > tl;dr: it's hard to conclude that this is anything but a compiler bug. > > I was able to

Incorrect assumption in heap_prepare_freeze_tuple

2020-10-02 Thread Kuntal Ghosh
Hello hackers, In heap_prepare_freeze_tuple, we make the following assumption: * It is assumed that the caller has checked the tuple with * HeapTupleSatisfiesVacuum() and determined that it is not HEAPTUPLE_DEAD * (else we should be removing the tuple, not freezing it). Thus, when we see a co

Re: Retry Cached Remote Connections for postgres_fdw in case remote backend gets killed/goes away

2020-10-02 Thread Fujii Masao
On 2020/10/02 0:46, Bharath Rupireddy wrote: On Thu, Oct 1, 2020 at 8:10 PM Fujii Masao wrote: pg_stat_clear_snapshot() can be used to reset the entry. Thanks. I wasn't knowing it. + EXIT WHEN proccnt = 0; +END LOOP; Isn't it better to sleep here, to avoid th busy lo

Re: buildfarm animal shoveler failing with "Illegal instruction"

2020-10-02 Thread Andres Freund
On 2020-10-02 10:45:58 -0700, Mark Wong wrote: > I went ahead and stopped the jobs to run with clang 3.9. This is also > the same system that was running clang 3.8 too. I tried looking for EOL > dates, but had trouble finding anything... But I can change the > optimization flag if we want it bac

Re: enable_incremental_sort changes query behavior

2020-10-02 Thread Tomas Vondra
On Fri, Oct 02, 2020 at 10:55:14AM -0400, James Coleman wrote: On Fri, Oct 2, 2020 at 10:53 AM James Coleman wrote: On Fri, Oct 2, 2020 at 10:32 AM Tomas Vondra wrote: > > On Fri, Oct 02, 2020 at 09:19:44AM -0400, James Coleman wrote: > > > > ... > > > >I've been able to confirm that the prob

Re: Incorrect assumption in heap_prepare_freeze_tuple

2020-10-02 Thread Andres Freund
Hi, On 2020-10-02 23:26:05 +0530, Kuntal Ghosh wrote: > In heap_prepare_freeze_tuple, we make the following assumption: > > * It is assumed that the caller has checked the tuple with > * HeapTupleSatisfiesVacuum() and determined that it is not HEAPTUPLE_DEAD > * (else we should be removing the

Re: enable_incremental_sort changes query behavior

2020-10-02 Thread James Coleman
On Fri, Oct 2, 2020 at 2:25 PM Tomas Vondra wrote: > > On Fri, Oct 02, 2020 at 10:55:14AM -0400, James Coleman wrote: > >On Fri, Oct 2, 2020 at 10:53 AM James Coleman wrote: > >> > >> On Fri, Oct 2, 2020 at 10:32 AM Tomas Vondra > >> wrote: > >> > > >> > On Fri, Oct 02, 2020 at 09:19:44AM -0400,

Prepared Statements

2020-10-02 Thread Patrick REED
Hi, I am having a hard time pinning down which function creates a prepared statement. Say in some language I create a Prepared Statement and send it off. Before the first time I execute the prepared statement, which function is the one that 'creates' the prepared statement. In other words, which f

Re: Prepared Statements

2020-10-02 Thread David G. Johnston
On Fri, Oct 2, 2020 at 1:14 PM Patrick REED wrote: > Hi, > > I am having a hard time pinning down which function creates a prepared > statement. > ... > e.g. > In your favorite language: > This doesn't seem like the correct list for this question. Ponder the following and consider sending any

Re: enable_incremental_sort changes query behavior

2020-10-02 Thread Tomas Vondra
On Fri, Oct 02, 2020 at 04:12:11PM -0400, James Coleman wrote: On Fri, Oct 2, 2020 at 2:25 PM Tomas Vondra wrote: On Fri, Oct 02, 2020 at 10:55:14AM -0400, James Coleman wrote: >On Fri, Oct 2, 2020 at 10:53 AM James Coleman wrote: >> >> On Fri, Oct 2, 2020 at 10:32 AM Tomas Vondra >> wrote:

Re: enable_incremental_sort changes query behavior

2020-10-02 Thread James Coleman
On Fri, Oct 2, 2020 at 4:56 PM Tomas Vondra wrote: > > On Fri, Oct 02, 2020 at 04:12:11PM -0400, James Coleman wrote: > >On Fri, Oct 2, 2020 at 2:25 PM Tomas Vondra > > wrote: > >> > >> On Fri, Oct 02, 2020 at 10:55:14AM -0400, James Coleman wrote: > >> >On Fri, Oct 2, 2020 at 10:53 AM James Colem

Re: enable_incremental_sort changes query behavior

2020-10-02 Thread James Coleman
On Fri, Oct 2, 2020 at 4:56 PM Tomas Vondra wrote: > >And I don't see any reason why the CASE statement couldn't in theory > >(I don't know the internals enough to know when it actually happens) > >be done as part of the base relation scan (in this case, the seq > >scan). It's not dependent on any

Re: Add session statistics to pg_stat_database

2020-10-02 Thread Soumyadeep Chakraborty
On Tue, Sep 29, 2020 at 2:44 AM Laurenz Albe wrote: > > * Are we trying to capture ONLY client initiated disconnects in > > m_aborted (we are not handling other disconnects by not accounting for > > EOF..like if psql was killed)? If yes, why? > > I thought it was interesting to know how many dat

Re: enable_incremental_sort changes query behavior

2020-10-02 Thread Tomas Vondra
On Fri, Oct 02, 2020 at 05:45:52PM -0400, James Coleman wrote: On Fri, Oct 2, 2020 at 4:56 PM Tomas Vondra wrote: ... More importanly, it does not actually fix the issue - it does fix that particular query, but just replacing the DISTINCT with either ORDER BY or GROUP BY make it fail again :-

Re: enable_incremental_sort changes query behavior

2020-10-02 Thread James Coleman
On Fri, Oct 2, 2020 at 6:28 PM Tomas Vondra wrote: > > On Fri, Oct 02, 2020 at 05:45:52PM -0400, James Coleman wrote: > >On Fri, Oct 2, 2020 at 4:56 PM Tomas Vondra > > wrote: > >> > >> ... > >> > >> More importanly, it does not actually fix the issue - it does fix that > >> particular query, but

Re: Parallel copy

2020-10-02 Thread Tomas Vondra
Hello Vignesh, I've done some basic benchmarking on the v4 version of the patches (but AFAIKC the v5 should perform about the same), and some initial review. For the benchmarking, I used the lineitem table from TPC-H - for 75GB data set, this largest table is about 64GB once loaded, with another

Re: Improve choose_custom_plan for initial partition prune case

2020-10-02 Thread Andy Fan
Hi Amit: Very glad to see your comment! On Fri, Oct 2, 2020 at 4:21 PM Amit Langote wrote: > Hi Andy, > > On Fri, Oct 2, 2020 at 1:04 AM Andy Fan wrote: > > > > Given the plan example: > > > > CREATE TABLE measurement ( > > city_id int not null, > > logdate date not n

Re: a misbehavior of partition row movement (?)

2020-10-02 Thread Amit Langote
On Fri, Oct 2, 2020 at 11:32 PM David G. Johnston wrote: > On Friday, October 2, 2020, Amit Langote wrote: >> >> >> Reporter on that thread says that the last update should have failed >> and I don't quite see a workable alternative to that. > > > To be clear the OP would rather have it just work

Re: "cert" + clientcert=verify-ca in pg_hba.conf?

2020-10-02 Thread Bruce Momjian
On Fri, Sep 25, 2020 at 09:33:48AM +0900, Kyotaro Horiguchi wrote: > At Thu, 24 Sep 2020 11:43:40 -0400, Bruce Momjian wrote in > > On Thu, Sep 24, 2020 at 12:44:01PM +0900, Michael Paquier wrote: > > > On Tue, Sep 01, 2020 at 10:27:03PM -0400, Bruce Momjian wrote: > > > > OK, good. Let's wait a

Re: Logical replication CPU-bound with TRUNCATE/DROP/CREATE many tables

2020-10-02 Thread Dilip Kumar
On Fri, Oct 2, 2020 at 12:26 PM Keisuke Kuroda wrote: > > Hi Dilip, Amit, > > > > 5. Can you please once repeat the performance test done by Keisuke-San > > > to see if you have similar observations? Additionally, see if you are > > > also seeing the inconsistency related to the Truncate message r

Re: enable_incremental_sort changes query behavior

2020-10-02 Thread James Coleman
On Fri, Oct 2, 2020 at 7:07 PM James Coleman wrote: > > On Fri, Oct 2, 2020 at 6:28 PM Tomas Vondra > wrote: > > > > On Fri, Oct 02, 2020 at 05:45:52PM -0400, James Coleman wrote: > > >On Fri, Oct 2, 2020 at 4:56 PM Tomas Vondra > > > wrote: > > >> > > >> ... > > >> > > >> More importanly, it doe

Re: Resetting spilled txn statistics in pg_stat_replication

2020-10-02 Thread Masahiko Sawada
On Wed, 30 Sep 2020 at 18:10, Amit Kapila wrote: > > On Wed, Sep 30, 2020 at 1:12 PM Dilip Kumar wrote: > > > > On Fri, Sep 25, 2020 at 4:33 PM Amit Kapila wrote: > > > > > > On Thu, Sep 24, 2020 at 5:44 PM Amit Kapila > > > wrote: > > > > > > I have done some more testing of this patch especi

Re: [PATCH] Add section headings to index types doc

2020-10-02 Thread Jürgen Purtz
On 30.09.20 14:53, Heikki Linnakangas wrote: On 30/09/2020 14:25, Dagfinn Ilmari Mannsåker wrote: Michael Paquier writes: On Mon, Aug 10, 2020 at 12:52:17PM +, Jürgen Purtz wrote: The new status of this patch is: Waiting on Author This has not been answered yet, so I have marked the pa

Add primary keys to system catalogs

2020-10-02 Thread Peter Eisentraut
I saw someone ask once for a schema diagram of the system catalogs. Things like that have occasionally been produced manually, but they are not regularly updated. That made me wonder, why can't we add primary and foreign keys to system catalogs and then have existing tools produce such a schem