Re: cleaning perl code

2020-04-16 Thread Noah Misch
On Thu, Apr 16, 2020 at 09:53:46AM -0400, Tom Lane wrote: > Andrew Dunstan writes: > > On 4/15/20 11:01 PM, Noah Misch wrote: > >> It would be an unpleasant surprise to cause a perlcritic buildfarm failure > >> by > >> moving a function, verbatim, from a non-strategic file to a strategic file. >

Re: Lexer issues

2020-04-16 Thread Patrick REED
Hi Julien, Sorry for the late reply. I was able to solve the issue. It had to do with the extra syntax I had introduced in gram.y. However, since you mentioned the utility command, can you elaborate a bit more on that? Thanks, Patrick On Tue, Apr 14, 2020 at 5:00 AM Julien Rouhaud wrote: > Hel

Re: ERROR: could not open file "pg_tblspc/ issue with replication setup.

2020-04-16 Thread Rajkumar Raghuwanshi
On Fri, Apr 17, 2020 at 9:51 AM Michael Paquier wrote: > On Thu, Apr 16, 2020 at 01:56:47PM +0530, Rajkumar Raghuwanshi wrote: > > While testing for a feature I got this tablespace related error while > > running script. > > Primary and standby are running on the same host, so they would > intera

Possible cache reference leak by removeExtObjInitPriv

2020-04-16 Thread Kyotaro Horiguchi
Hello. Recently a cache reference leak was reported then fixed [1]. I happened to notice a similar possible leakage in removeEtObjInitPriv. I haven't found a way to reach the code, but can be forcibly caused by tweaking the condition. Please find the attached. regards. [1] https://www.postgre

Re: WIP/PoC for parallel backup

2020-04-16 Thread Kashif Zeeshan
On Tue, Apr 14, 2020 at 7:37 PM Asif Rehman wrote: > > > On Tue, Apr 14, 2020 at 6:32 PM Kashif Zeeshan < > kashif.zees...@enterprisedb.com> wrote: > >> Hi Asif >> >> Getting the following error on Parallel backup when --no-manifest option >> is used. >> >> [edb@localhost bin]$ >> [edb@localhost

Re: xid wraparound danger due to INDEX_CLEANUP false

2020-04-16 Thread Masahiko Sawada
On Fri, 17 Apr 2020 at 02:58, Andres Freund wrote: > > Hi, > > On 2020-04-16 16:30:02 +0900, Masahiko Sawada wrote: > > For btree indexes, IIRC skipping index cleanup could not be a cause of > > corruption, but be a cause of index bloat since it leaves recyclable > > pages which are not marked as

Re: Race condition in SyncRepGetSyncStandbysPriority

2020-04-16 Thread Kyotaro Horiguchi
At Thu, 16 Apr 2020 11:39:06 -0400, Tom Lane wrote in > Kyotaro Horiguchi writes: > > [ syncrep-fixes-4.patch ] > > I agree that we could probably improve the clarity of this code with > further rewriting, but I'm still very opposed to the idea of having > callers know that the first num_sync a

Re: 001_rep_changes.pl stalls

2020-04-16 Thread Noah Misch
On Mon, Apr 13, 2020 at 09:45:16PM -0400, Tom Lane wrote: > Noah Misch writes: > > This seems to have made the following race condition easier to hit: > > https://www.postgresql.org/message-id/flat/20200206074552.GB3326097%40rfd.leadboat.com > > https://www.postgresql.org/message-id/flat/21519.158

Re: snapshot too old issues, first around wraparound and then more.

2020-04-16 Thread Thomas Munro
On Fri, Apr 17, 2020 at 3:37 PM Thomas Munro wrote: > On Mon, Apr 13, 2020 at 5:14 PM Andres Freund wrote: > > FWIW, I think the part that is currently harder to fix is the time->xmin > > mapping and some related pieces. Second comes the test > > infrastructure. Compared to those, adding addition

Re: It is not documented that pg_promote can exit standby mode

2020-04-16 Thread Michael Paquier
On Fri, Apr 17, 2020 at 01:40:02PM +0900, Fujii Masao wrote: > Thanks for the report and the patch! It looks good to me. > Barring any objection, I will commit this patch. +1. -- Michael signature.asc Description: PGP signature

Re: It is not documented that pg_promote can exit standby mode

2020-04-16 Thread Fujii Masao
On 2020/04/17 13:11, ikedamsh wrote: Hi, The document(high-availability.sgml) says that there are only two ways to exit standby mode.  26.2.2. Standby Server Operation  Standby mode is exited and the server switches to normal operation when pg_ctl promote is run or a trigger file is found

Re: ERROR: could not open file "pg_tblspc/ issue with replication setup.

2020-04-16 Thread Michael Paquier
On Thu, Apr 16, 2020 at 01:56:47PM +0530, Rajkumar Raghuwanshi wrote: > While testing for a feature I got this tablespace related error while > running script. Primary and standby are running on the same host, so they would interact with each other as the tablespace path used by both clusters woul

It is not documented that pg_promote can exit standby mode

2020-04-16 Thread ikedamsh
Hi,  The document(high-availability.sgml) says that there are only two ways to exit standby mode.  26.2.2. Standby Server Operation  Standby mode is exited and the server switches to normal operation when pg_ctl promote is run or a trigger file is found (promote_trigger_file). But there is

Re: Do we need to handle orphaned prepared transactions in the server?

2020-04-16 Thread Hamid Akhtar
Thank you everyone for the detailed feedback. On Fri, Apr 17, 2020 at 5:40 AM Michael Paquier wrote: > On Thu, Apr 16, 2020 at 03:11:51PM -0400, Tom Lane wrote: > > Even if I liked the core idea, loading the functionality onto VACUUM > seems > > like a fairly horrid design choice. It's quite un

Re: snapshot too old issues, first around wraparound and then more.

2020-04-16 Thread Thomas Munro
On Mon, Apr 13, 2020 at 5:14 PM Andres Freund wrote: > FWIW, I think the part that is currently harder to fix is the time->xmin > mapping and some related pieces. Second comes the test > infrastructure. Compared to those, adding additional checks for old > snapshots wouldn't be too hard - although

Re: where should I stick that backup?

2020-04-16 Thread Robert Haas
On Wed, Apr 15, 2020 at 7:55 PM Robert Haas wrote: > Yeah. I think we really need to understand the performance > characteristics of pipes better. If they're slow, then anything that > needs to be fast has to work some other way (but we could still > provide a pipe-based slow way for niche uses).

Re: fixing old_snapshot_threshold's time->xid mapping

2020-04-16 Thread Thomas Munro
On Fri, Apr 17, 2020 at 5:46 AM Andres Freund wrote: > On 2020-04-16 13:34:39 -0400, Robert Haas wrote: > > On Thu, Apr 16, 2020 at 1:14 PM Andres Freund wrote: > > > I still think we need a way to test this without waiting for hours to > > > hit various edge cases. You argued against a fixed bin

Re: Making openssl_tls_init_hook OpenSSL specific

2020-04-16 Thread Michael Paquier
On Thu, Apr 16, 2020 at 02:17:33PM +0200, Daniel Gustafsson wrote: > Commit 896fcdb230e72 (sorry for chiming in too late, I missed that thread) > added a TLS init hook which is OpenSSL specific: openssl_tls_init_hook. Since > the rest of the TLS support in the backend is library agnostic, we shoul

Re: [PATHC] Fix minor memory leak in pg_basebackup

2020-04-16 Thread Michael Paquier
On Thu, Apr 16, 2020 at 10:54:09AM +, Zhang, Jie wrote: > So much the better! Thanks for confirming. Fixed, then. -- Michael signature.asc Description: PGP signature

Re: xid wraparound danger due to INDEX_CLEANUP false

2020-04-16 Thread Peter Geoghegan
On Thu, Apr 16, 2020 at 3:49 PM Andres Freund wrote: > I think we really just stop being miserly and update the xid to be > FrozenTransactionId or InvalidTransactionId when vacuum encounters one > that's from before the the xid cutoff used by vacuum (i.e. what could > become the new relfrozenxid).

Re: Allow pg_read_all_stats to read pg_stat_progress_*

2020-04-16 Thread Kyotaro Horiguchi
At Thu, 16 Apr 2020 14:46:00 +0200, Magnus Hagander wrote in > On Thu, Apr 16, 2020 at 7:05 AM Kyotaro Horiguchi > wrote: > > > At Wed, 15 Apr 2020 15:58:05 +0500, "Andrey M. Borodin" < > > x4...@yandex-team.ru> wrote in > > > > 15 апр. 2020 г., в 15:25, Magnus Hagander > > написал(а): > > >

Re: [PATCH] Incremental sort (was: PoC: Partial sort)

2020-04-16 Thread James Coleman
On Thu, Apr 16, 2020 at 1:10 PM Tom Lane wrote: > > James Coleman writes: > > On Fri, Apr 10, 2020 at 10:12 AM James Coleman wrote: > >> One thing I just noticed and had a question about: in > >> preparePresortedCols (which sets up a function call context), do we > >> need to call pg_proc_aclche

Re: sqlsmith crash incremental sort

2020-04-16 Thread Tom Lane
Tomas Vondra writes: > I think we have essentially three options: > 1) assuming there's just a single group > 2) assuming each row is a separate group > 3) something in between > If (1) and (2) are worst/best-case scenarios, maybe we should pick > something in between. We have DEFAULT_NUM_DISTINCT

Re: [PATCH] Keeps tracking the uniqueness with UniqueKey

2020-04-16 Thread Andy Fan
On Thu, Apr 16, 2020 at 8:36 PM Ashutosh Bapat wrote: > On Thu, Apr 16, 2020 at 7:47 AM Andy Fan wrote: > > > (9 rows) > > > > With this feature: > > explain analyze select a, sum(c) from grp2 group by a; > > QUERY PLAN > > > --

Re: [DOC] Document concurrent index builds waiting on each other

2020-04-16 Thread James Coleman
On Thu, Apr 16, 2020 at 6:12 PM Andres Freund wrote: > > Hi, > > On 2020-04-15 21:44:48 -0400, James Coleman wrote: > > On Wed, Apr 15, 2020 at 6:31 PM Andres Freund wrote: > > > If it's about the xmin horizon for vacuum: I think we could probably > > > avoid that using the same flag. As vacuum c

Re: sqlsmith crash incremental sort

2020-04-16 Thread James Coleman
On Thu, Apr 16, 2020 at 8:54 PM Tomas Vondra wrote: > > On Wed, Apr 15, 2020 at 11:26:12AM -0400, James Coleman wrote: > >On Wed, Apr 15, 2020 at 10:47 AM Tomas Vondra > > wrote: > >> > >> ... > >> > >> Yeah. And I'm not even sure having that information would allow good > >> estimates e.g. for UN

Re: sqlsmith crash incremental sort

2020-04-16 Thread Tomas Vondra
On Wed, Apr 15, 2020 at 11:26:12AM -0400, James Coleman wrote: On Wed, Apr 15, 2020 at 10:47 AM Tomas Vondra wrote: ... Yeah. And I'm not even sure having that information would allow good estimates e.g. for UNIONs of multiple relations etc. >> Another option is to use something as simple as

Re: Do we need to handle orphaned prepared transactions in the server?

2020-04-16 Thread Michael Paquier
On Thu, Apr 16, 2020 at 03:11:51PM -0400, Tom Lane wrote: > Even if I liked the core idea, loading the functionality onto VACUUM seems > like a fairly horrid design choice. It's quite unrelated to what that > command does. In the autovac code path, it's going to lead to multiple > autovac workers

Re: Problem with logical replication (crash with REPLICA IDENTITY FULL and cascading replication)

2020-04-16 Thread Justin Pryzby
Confirmed and added to opened items - this is a live bug back to v10. for a in data data1; do ./tmp_install/usr/local/pgsql/bin/initdb -D $a --no-sync& done; wait echo "wal_level = logical">> data/postgresql.conf; echo "port=5433" >> data1/postgresql.conf for a in data data1; do ./tmp_install/us

[PATCH] Fix possible overflow on tuplesort.c

2020-04-16 Thread Ranier Vilela
Hi, When multiplying variables, the overflow will take place anyway, and only then will the meaningless product be explicitly promoted to type int64. It is one of the operands that should have been cast instead to avoid the overflow. regards, Ranier Vilela tuplesort_avoid_possible_overflow.patc

Re: sqlsmith crash incremental sort

2020-04-16 Thread Tomas Vondra
On Thu, Apr 16, 2020 at 08:44:16PM +0200, Tomas Vondra wrote: On Thu, Apr 16, 2020 at 12:04:03PM -0400, James Coleman wrote: On Thu, Apr 16, 2020 at 8:22 AM Richard Guo wrote: On Thu, Apr 16, 2020 at 6:35 PM Richard Guo wrote: On Wed, Apr 15, 2020 at 10:47 PM Tomas Vondra wrote: Wel

Re: Poll: are people okay with function/operator table redesign?

2020-04-16 Thread Alvaro Herrera
v1 is good. I like your v2 even better. If it becomes possible to remove or soften the "inter-row" horizontal line with CSS tricks afterwards, that would be swell, but even without that, I cast my vote to using this table format. -- Álvaro Herrerahttps://www.2ndQuadrant.com/ Pos

[PATCH] Tiny optimization on nbtinsert.c

2020-04-16 Thread Ranier Vilela
Hi, Avoiding some calls and set vars, when it is not necessary. best regards, Ranier Vilela nbtinsert_tiny_optimization.patch Description: Binary data

Re: xid wraparound danger due to INDEX_CLEANUP false

2020-04-16 Thread Andres Freund
Hi, On 2020-04-16 13:28:00 -0700, Peter Geoghegan wrote: > > And, what's worse, in the INDEX_CLEANUP off case, future VACUUMs with > > INDEX_CLEANUP on might not even visit the index. As there very well > > might not be many dead heap tuples around anymore (previous vacuums with > > cleanup off wi

Re: remove_useless_groupby_columns does not need to record constraint dependencies

2020-04-16 Thread David Rowley
On Fri, 17 Apr 2020 at 02:53, Tom Lane wrote: > > David Rowley writes: > > I've reworded the comment in the attached version. > > LGTM. Thanks for reviewing. Pushed.

Re: [PATCH'] Variables assigned with values that is never used.

2020-04-16 Thread Ranier Vilela
Em sáb., 28 de mar. de 2020 às 10:33, Ranier Vilela escreveu: > Hi, > > Theses variables, are assigned with values that never is used and, can > safely have their values removed. > 1. https://github.com/postgres/postgres/commit/f0ca378d4c139eda99ef14998115c1674dac3fc5 diff --git a/src/backend/ac

Re: documenting the backup manifest file format

2020-04-16 Thread Jehan-Guillaume de Rorthais
On Wed, 15 Apr 2020 18:54:14 -0400 David Steele wrote: > On 4/15/20 6:43 PM, Jehan-Guillaume de Rorthais wrote: > > On Wed, 15 Apr 2020 12:03:28 -0400 > > Robert Haas wrote: > > > >> On Wed, Apr 15, 2020 at 11:23 AM Jehan-Guillaume de Rorthais > >> wrote: > >>> But for backup_manifest, it'

Re: [DOC] Document concurrent index builds waiting on each other

2020-04-16 Thread Andres Freund
Hi, On 2020-04-15 21:44:48 -0400, James Coleman wrote: > On Wed, Apr 15, 2020 at 6:31 PM Andres Freund wrote: > > If it's about the xmin horizon for vacuum: I think we could probably > > avoid that using the same flag. As vacuum cannot be run against a table > > that has a CIC running (although i

Re: Poll: are people okay with function/operator table redesign?

2020-04-16 Thread Andreas Karlsson
On 4/14/20 4:52 PM, Tom Lane wrote: Andreas Karlsson writes: That said, I agree with that quite many of our tables right now are ugly, but I prefer ugly to hard to read. For me the mix of having every third row split into two fields makes the tables very hard to read. I have a hard time seeing

Re: cleaning perl code

2020-04-16 Thread Mark Dilger
> On Apr 16, 2020, at 2:07 PM, Andrew Dunstan > wrote: > > > On 4/16/20 11:12 AM, Alvaro Herrera wrote: >> On 2020-Apr-16, Hamlin, Garick L wrote: >> >>> With the old expression 'something;' would be stripped away. >>> Is that an issue where this this is used? Why are we parsing >>> thes

Re: PATCH: logical_work_mem and logical streaming of large in-progress transactions

2020-04-16 Thread Tomas Vondra
On Mon, Apr 13, 2020 at 05:20:39PM +0530, Dilip Kumar wrote: On Mon, Apr 13, 2020 at 4:14 PM Kuntal Ghosh wrote: On Thu, Apr 9, 2020 at 2:40 PM Dilip Kumar wrote: > > I have rebased the patch on the latest head. I haven't yet changed > anything for xid assignment thing because it is not yet

Re: cleaning perl code

2020-04-16 Thread Andrew Dunstan
On 4/16/20 11:12 AM, Alvaro Herrera wrote: > On 2020-Apr-16, Hamlin, Garick L wrote: > >> With the old expression 'something;' would be stripped away. >> Is that an issue where this this is used? Why are we parsing >> these headers? > These are files from which bootstrap catalog data is genera

Re: xid wraparound danger due to INDEX_CLEANUP false

2020-04-16 Thread Peter Geoghegan
On Thu, Apr 16, 2020 at 11:27 AM Andres Freund wrote: > Sure, there is some pre-existing wraparound danger for individual > pages. But it's a pretty narrow corner case before INDEX_CLEANUP > off. It's a matter of degree. Hard to judge something like that. > And, what's worse, in the INDEX_CLEANU

Re: PATCH: logical_work_mem and logical streaming of large in-progress transactions

2020-04-16 Thread Kuntal Ghosh
On Tue, Apr 14, 2020 at 3:41 PM Dilip Kumar wrote: > Few review comments from 0006-Add-support-for-streaming*.patch + subxacts[nsubxacts].offset = lseek(stream_fd, 0, SEEK_END); lseek can return (-)ve value in case of error, right? + /* + * We might need to create the tablespace's tempfile dire

Re: Do we need to handle orphaned prepared transactions in the server?

2020-04-16 Thread Tom Lane
Robert Haas writes: > On Thu, Apr 16, 2020 at 2:17 PM Hamid Akhtar wrote: >> So is the concern performance overhead rather than the need for such a >> feature? > No, I don't think this would have any significant overhead. My concern > is that I think it's the wrong way to solve the problem. FW

Re: "cache reference leak" issue happened when using sepgsql module

2020-04-16 Thread Tom Lane
Michael Luo writes: > When using sepgsql module, I got warning "WARNING: cache reference leak”. > ... > The patch attached to fix this issue, please check it. Right you are, fix pushed. regards, tom lane

Re: sqlsmith crash incremental sort

2020-04-16 Thread Tomas Vondra
On Thu, Apr 16, 2020 at 12:04:03PM -0400, James Coleman wrote: On Thu, Apr 16, 2020 at 8:22 AM Richard Guo wrote: On Thu, Apr 16, 2020 at 6:35 PM Richard Guo wrote: On Wed, Apr 15, 2020 at 10:47 PM Tomas Vondra wrote: Well, yeah. The problem is the Unique simply compares the columns

Re: xid wraparound danger due to INDEX_CLEANUP false

2020-04-16 Thread Andres Freund
Hi, On 2020-04-15 18:11:40 -0700, Peter Geoghegan wrote: > On Wed, Apr 15, 2020 at 4:57 PM Robert Haas wrote: > > I seem to recall Simon raising this issue at the time that the patch > > was being discussed, and I thought that we had eventually decided that > > it was OK for some reason. But I do

Re: Do we need to handle orphaned prepared transactions in the server?

2020-04-16 Thread Robert Haas
On Thu, Apr 16, 2020 at 2:17 PM Hamid Akhtar wrote: > So is the concern performance overhead rather than the need for such a > feature? No, I don't think this would have any significant overhead. My concern is that I think it's the wrong way to solve the problem. If you need to check for prepare

Re: Do we need to handle orphaned prepared transactions in the server?

2020-04-16 Thread Hamid Akhtar
On Thu, Apr 16, 2020 at 5:20 PM Robert Haas wrote: > On Thu, Apr 16, 2020 at 4:43 AM Hamid Akhtar > wrote: > > My real question is whether vacuum should be preemptively complaining > about prepared transactions or stale replication slots rather than waiting > for transaction id to exceed the saf

Re: Race condition in SyncRepGetSyncStandbysPriority

2020-04-16 Thread Tom Lane
Fujii Masao writes: > On 2020/04/14 22:52, Tom Lane wrote: >> *Yes it does*. The existing code can deliver entirely broken results >> if some walsender exits between where we examine the priorities and >> where we fetch the WAL pointers. > IMO that the broken results can be delivered when walsen

Re: xid wraparound danger due to INDEX_CLEANUP false

2020-04-16 Thread Andres Freund
Hi, On 2020-04-16 16:30:02 +0900, Masahiko Sawada wrote: > For btree indexes, IIRC skipping index cleanup could not be a cause of > corruption, but be a cause of index bloat since it leaves recyclable > pages which are not marked as recyclable. The index bloat is the main > side effect of skipping

Re: Should we add xid_current() or a int8->xid cast?

2020-04-16 Thread Thomas Munro
On Fri, Apr 17, 2020 at 3:44 AM Mark Dilger wrote: > Hmm, I should have spoken sooner... > > src/backend/replication/walsender.c:static bool > TransactionIdInRecentPast(TransactionId xid, uint32 epoch); > src/backend/utils/adt/xid8funcs.c:TransactionIdInRecentPast(FullTransactionId > fxid, Trans

Re: fixing old_snapshot_threshold's time->xid mapping

2020-04-16 Thread Andres Freund
Hi, On 2020-04-16 13:34:39 -0400, Robert Haas wrote: > On Thu, Apr 16, 2020 at 1:14 PM Andres Freund wrote: > > I still think we need a way to test this without waiting for hours to > > hit various edge cases. You argued against a fixed binning of > > old_snapshot_threshold/100 arguing its too co

Re: fixing old_snapshot_threshold's time->xid mapping

2020-04-16 Thread Robert Haas
On Thu, Apr 16, 2020 at 1:14 PM Andres Freund wrote: > I still think we need a way to test this without waiting for hours to > hit various edge cases. You argued against a fixed binning of > old_snapshot_threshold/100 arguing its too coarse. How about a 1000 or > so? For 60 days, the current max f

Re: Race condition in SyncRepGetSyncStandbysPriority

2020-04-16 Thread Fujii Masao
On 2020/04/14 22:52, Tom Lane wrote: Kyotaro Horiguchi writes: SyncRepGetSyncStandbysPriority() is runing holding SyncRepLock so sync_standby_priority of any walsender can be changed while the function is scanning welsenders. The issue is we already have inconsistent walsender information be

Re: fixing old_snapshot_threshold's time->xid mapping

2020-04-16 Thread Andres Freund
Hi, On 2020-04-16 12:41:55 -0400, Robert Haas wrote: > I'm starting a new thread for this, because the recent discussion of > problems with old_snapshot_threshold[1] touched on a lot of separate > issues, and I think it will be too confusing if we discuss all of them > on one thread. Attached are

Re: [PATCH] Incremental sort (was: PoC: Partial sort)

2020-04-16 Thread Tom Lane
James Coleman writes: > On Fri, Apr 10, 2020 at 10:12 AM James Coleman wrote: >> One thing I just noticed and had a question about: in >> preparePresortedCols (which sets up a function call context), do we >> need to call pg_proc_aclcheck? > Background: this came up because I noticed that pg_pro

Re: Poll: are people okay with function/operator table redesign?

2020-04-16 Thread Tom Lane
Pierre Giraud writes: > Le 16/04/2020 à 16:43, Tom Lane a écrit : >> Pierre Giraud writes: >>> The screenshot attached uses a tag for the descrition/example block. >> That looks about right, perhaps, but could you be a little clearer about >> how you accomplished that? > Attached you will find

Re: Include sequence relation support in logical replication

2020-04-16 Thread Cary Huang
Hi Craig, Andres Thank you guys so much for your reviews and comments. Really helpful. Yes you guys are right, Sequence does not guarantee free of gaps and replicating sequence is useful for failover cases, then there will be no problem for a subscriber to get a future value 32 increments aft

Re: [PATCH] Incremental sort (was: PoC: Partial sort)

2020-04-16 Thread James Coleman
On Fri, Apr 10, 2020 at 10:12 AM James Coleman wrote: > > One thing I just noticed and had a question about: in > preparePresortedCols (which sets up a function call context), do we > need to call pg_proc_aclcheck? Background: this came up because I noticed that pg_proc_aclcheck is called in the

fixing old_snapshot_threshold's time->xid mapping

2020-04-16 Thread Robert Haas
Hi, I'm starting a new thread for this, because the recent discussion of problems with old_snapshot_threshold[1] touched on a lot of separate issues, and I think it will be too confusing if we discuss all of them on one thread. Attached are three patches. 0001 makes oldSnapshotControl "extern" ra

Re: sqlsmith crash incremental sort

2020-04-16 Thread James Coleman
On Thu, Apr 16, 2020 at 8:22 AM Richard Guo wrote: > > > On Thu, Apr 16, 2020 at 6:35 PM Richard Guo wrote: >> >> >> On Wed, Apr 15, 2020 at 10:47 PM Tomas Vondra >> wrote: >>> >>> >>> Well, yeah. The problem is the Unique simply compares the columns in the >>> order it sees them, and it does n

Re: Should we add xid_current() or a int8->xid cast?

2020-04-16 Thread Mark Dilger
> On Apr 6, 2020, at 5:14 PM, Thomas Munro wrote: > > On Sun, Apr 5, 2020 at 11:31 AM Thomas Munro wrote: >> On Sun, Apr 5, 2020 at 10:34 AM Mark Dilger >> wrote: >>> The "xid8_" warts are partly motivated by having a type named "xid8", which >>> is a bit of a wart in itself. >> >> Just a

Re: Race condition in SyncRepGetSyncStandbysPriority

2020-04-16 Thread Tom Lane
Kyotaro Horiguchi writes: > [ syncrep-fixes-4.patch ] I agree that we could probably improve the clarity of this code with further rewriting, but I'm still very opposed to the idea of having callers know that the first num_sync array elements are the active ones. It's wrong (or at least differen

Re: cleaning perl code

2020-04-16 Thread Alvaro Herrera
On 2020-Apr-16, Hamlin, Garick L wrote: > With the old expression 'something;' would be stripped away. > Is that an issue where this this is used? Why are we parsing > these headers? These are files from which bootstrap catalog data is generated, which is why we parse from Perl; but also where

Re: Poll: are people okay with function/operator table redesign?

2020-04-16 Thread Pierre Giraud
Le 16/04/2020 à 16:43, Tom Lane a écrit : > Pierre Giraud writes: >> Le 16/04/2020 à 00:18, Tom Lane a écrit : >>> The main disadvantage I can see to the v2 design is that we're back >>> to having two per function, which is inevitably going to result >>> in PDF builds putting page breaks betwee

"cache reference leak" issue happened when using sepgsql module

2020-04-16 Thread Michael Luo
Hi ! When using sepgsql module, I got warning "WARNING: cache reference leak”. ``` postgres=# UPDATE range_parted set c = 95 WHERE a = 'b' and b > 10 and c > 100 returning (range_parted), *; WARNING: cache reference leak: cache pg_attribute (7), tuple 38/54 has count 1 WARNING: cache reference

Re: remove_useless_groupby_columns does not need to record constraint dependencies

2020-04-16 Thread Tom Lane
David Rowley writes: > I've reworded the comment in the attached version. LGTM. regards, tom lane

Re: Poll: are people okay with function/operator table redesign?

2020-04-16 Thread Tom Lane
Pierre Giraud writes: > Le 16/04/2020 à 00:18, Tom Lane a écrit : >> The main disadvantage I can see to the v2 design is that we're back >> to having two per function, which is inevitably going to result >> in PDF builds putting page breaks between those rows. But you can't >> have everything ..

Re: cleaning perl code

2020-04-16 Thread Andrew Dunstan
On 4/16/20 10:20 AM, Hamlin, Garick L wrote: > On Thu, Apr 16, 2020 at 08:50:35AM -0400, Andrew Dunstan wrote: >> It would also be more robust using non-greedy matching: > This seems more important. > I don't know how/where this is being used, but if it has input like: > > /* one */ > something;

Re: cleaning perl code

2020-04-16 Thread Hamlin, Garick L
On Thu, Apr 16, 2020 at 08:50:35AM -0400, Andrew Dunstan wrote: > > It would also be more robust using non-greedy matching: This seems more important. I don't know how/where this is being used, but if it has input like: /* one */ something; /* two */ With the old expression 'something;' would

Re: Autovacuum on partitioned table (autoanalyze)

2020-04-16 Thread Justin Pryzby
On Thu, Apr 16, 2020 at 06:16:45PM +0900, yuzuko wrote: > > I think it ought to be possible to configure this feature such that an > > auto-analyze on any child partition would trigger analyze of the parent. I > > think that would be important for maintaining accurate stats of the > > partition >

Re: cleaning perl code

2020-04-16 Thread Tom Lane
Andrew Dunstan writes: > On 4/15/20 11:01 PM, Noah Misch wrote: >> It would be an unpleasant surprise to cause a perlcritic buildfarm failure by >> moving a function, verbatim, from a non-strategic file to a strategic file. >> Having two Perl style regimes in one tree is itself a liability. > Hon

Re: sqlsmith crash incremental sort

2020-04-16 Thread Tomas Vondra
On Thu, Apr 16, 2020 at 04:44:10PM +0800, Richard Guo wrote: On Mon, Apr 13, 2020 at 8:09 AM Tomas Vondra wrote: I've been messing with this the whole day, without much progress :-( I'm 99.% sure it's the same issue described by the quoted comment, because the plan looks like this: Ne

Re: cleaning perl code

2020-04-16 Thread Andrew Dunstan
On 4/15/20 11:01 PM, Noah Misch wrote: > On Wed, Apr 15, 2020 at 03:43:36PM -0400, Andrew Dunstan wrote: >> On 4/14/20 4:44 PM, Alvaro Herrera wrote: >>> On 2020-Apr-14, Andrew Dunstan wrote: One of the things that's a bit sad is that perlcritic doesn't generally let you apply policies

Re: Allow pg_read_all_stats to read pg_stat_progress_*

2020-04-16 Thread Magnus Hagander
On Thu, Apr 16, 2020 at 7:05 AM Kyotaro Horiguchi wrote: > At Wed, 15 Apr 2020 15:58:05 +0500, "Andrey M. Borodin" < > x4...@yandex-team.ru> wrote in > > > 15 апр. 2020 г., в 15:25, Magnus Hagander > написал(а): > > > I think that makes perfect sense. The documentation explicitly says > "can rea

Re: [PATCH] Keeps tracking the uniqueness with UniqueKey

2020-04-16 Thread Ashutosh Bapat
On Thu, Apr 16, 2020 at 7:47 AM Andy Fan wrote: > (9 rows) > > With this feature: > explain analyze select a, sum(c) from grp2 group by a; > QUERY PLAN > ---

Re: sqlsmith crash incremental sort

2020-04-16 Thread Richard Guo
On Thu, Apr 16, 2020 at 6:35 PM Richard Guo wrote: > > On Wed, Apr 15, 2020 at 10:47 PM Tomas Vondra < > tomas.von...@2ndquadrant.com> wrote: > >> >> Well, yeah. The problem is the Unique simply compares the columns in the >> order it sees them, and it does not match the column order desired by >

Re: Do we need to handle orphaned prepared transactions in the server?

2020-04-16 Thread Robert Haas
On Thu, Apr 16, 2020 at 4:43 AM Hamid Akhtar wrote: > My real question is whether vacuum should be preemptively complaining about > prepared transactions or stale replication slots rather than waiting for > transaction id to exceed the safe limit. I presume by the time safe limit is > exceeded,

Making openssl_tls_init_hook OpenSSL specific

2020-04-16 Thread Daniel Gustafsson
Commit 896fcdb230e72 (sorry for chiming in too late, I missed that thread) added a TLS init hook which is OpenSSL specific: openssl_tls_init_hook. Since the rest of the TLS support in the backend is library agnostic, we should IMO make this hook follow that pattern, else this will make a non-OpenS

RE: [PATHC] Fix minor memory leak in pg_basebackup

2020-04-16 Thread Zhang, Jie
Hi Michael so much the better! -Original Message- From: Michael Paquier [mailto:mich...@paquier.xyz] Sent: Thursday, April 16, 2020 2:31 PM To: Zhang, Jie/张 杰 Cc: pgsql-hackers@lists.postgresql.org Subject: Re: [PATHC] Fix minor memory leak in pg_basebackup On Wed, Apr 15, 2020 at 10:

Re: sqlsmith crash incremental sort

2020-04-16 Thread Richard Guo
On Wed, Apr 15, 2020 at 10:47 PM Tomas Vondra wrote: > > Well, yeah. The problem is the Unique simply compares the columns in the > order it sees them, and it does not match the column order desired by > incremental sort. But we don't push down this information at all :-( > This is a nice optimi

Re: PATCH: logical_work_mem and logical streaming of large in-progress transactions

2020-04-16 Thread Erik Rijkers
On 2020-04-16 11:33, Dilip Kumar wrote: On Tue, Apr 14, 2020 at 9:14 PM Erik Rijkers wrote: On 2020-04-14 12:10, Dilip Kumar wrote: > v14-0001-Immediately-WAL-log-assignments.patch + > v14-0002-Issue-individual-invalidations-with.patch + > v14-0003-Extend-the-outpu

Re: PATCH: logical_work_mem and logical streaming of large in-progress transactions

2020-04-16 Thread Dilip Kumar
On Tue, Apr 14, 2020 at 9:14 PM Erik Rijkers wrote: > > On 2020-04-14 12:10, Dilip Kumar wrote: > > > v14-0001-Immediately-WAL-log-assignments.patch + > > v14-0002-Issue-individual-invalidations-with.patch + > > v14-0003-Extend-the-output-plugin-API-with-stream-methods.

Re: Race condition in SyncRepGetSyncStandbysPriority

2020-04-16 Thread Kyotaro Horiguchi
At Thu, 16 Apr 2020 16:48:28 +0900, Masahiko Sawada wrote in > This is just a notice; I'm reading your latest patch but it seems to > include unrelated changes: > > $ git diff --stat > src/backend/replication/syncrep.c | 475 > +

Re: [HACKERS] advanced partition matching algorithm for partition-wise join

2020-04-16 Thread Etsuro Fujita
On Mon, Apr 6, 2020 at 8:43 PM Ashutosh Bapat wrote: > On Fri, 3 Apr 2020 at 20:45, Etsuro Fujita wrote: > But it will be good to have following addition I suggested in my patches to > make sure nparts is set to 0 for an unpartitioned relation as per the comment > on nparts in RelOptInfo. > @@

Re: pg_restore: could not close data file: Success

2020-04-16 Thread Kyotaro Horiguchi
At Thu, 16 Apr 2020 14:40:09 +0900, Michael Paquier wrote in > On Thu, Apr 16, 2020 at 12:08:09PM +0900, Kyotaro Horiguchi wrote: > > I'm surprised to find an old thread about the same issue. > > > > https://www.postgresql.org/message-id/20160307.174354.251049100.horiguchi.kyotaro%40lab.ntt.co.

Re: Autovacuum on partitioned table (autoanalyze)

2020-04-16 Thread yuzuko
Hi Justin, Thank you for commens. On Tue, Apr 7, 2020 at 12:32 PM Justin Pryzby wrote: > > Not sure if you saw my earlier message ? > I'm sorry, I didn't notice for a while. > I think it ought to be possible to configure this feature such that an > auto-analyze on any child partition would trig

Problem with logical replication

2020-04-16 Thread Dilip Kumar
While try to setup a cascading replication, I have observed that if we set the REPLICA IDENTITY to FULL on the subscriber side then there is an Assert hit. After analysis I have found that, when we set the REPLICA IDENTITY to FULL on subscriber side (because I wanted to make this a publisher for a

Re: sqlsmith crash incremental sort

2020-04-16 Thread Richard Guo
On Mon, Apr 13, 2020 at 8:09 AM Tomas Vondra wrote: > > I've been messing with this the whole day, without much progress :-( > > I'm 99.% sure it's the same issue described by the quoted comment, > because the plan looks like this: > > Nested Loop Left Join > -> Sample Scan on pg_names

Re: Do we need to handle orphaned prepared transactions in the server?

2020-04-16 Thread Hamid Akhtar
This patch actually does not discard any prepared transactions and only throws a warning for each orphaned one. So, there is no behaviour change except for getting some warnings in the log or emitting some warning to a client executing a vacuum command. I hear all the criticism which I don't disag

ERROR: could not open file "pg_tblspc/ issue with replication setup.

2020-04-16 Thread Rajkumar Raghuwanshi
Hi, While testing for a feature I got this tablespace related error while running script. The Issue is not reproducible everytime, but If I am running the same set of commands after 2-3 runs I am able to reproduce the same error. --first run - pass # master slave setup + mkdir /tmp/test_bkp/tblsp

Re: Race condition in SyncRepGetSyncStandbysPriority

2020-04-16 Thread Masahiko Sawada
On Thu, 16 Apr 2020 at 16:22, Kyotaro Horiguchi wrote: > > At Wed, 15 Apr 2020 11:31:49 -0400, Tom Lane wrote in > > Kyotaro Horiguchi writes: > > > At Tue, 14 Apr 2020 16:32:40 -0400, Tom Lane wrote in > > > + stby->is_sync_standby = true; /* might change below */ > > > > > I'm une

Re: Vacuum o/p with (full 1, parallel 0) option throwing an error

2020-04-16 Thread Masahiko Sawada
On Thu, 16 Apr 2020 at 15:02, Amit Kapila wrote: > > On Wed, Apr 15, 2020 at 9:12 AM Amit Kapila wrote: > > > > On Wed, Apr 15, 2020 at 9:03 AM Justin Pryzby wrote: > > > > > > On Wed, Apr 15, 2020 at 08:54:17AM +0530, Amit Kapila wrote: > > > > On Tue, Apr 14, 2020 at 8:55 AM Amit Kapila > >

Re: spin_delay() for ARM

2020-04-16 Thread Pavel Stehule
čt 16. 4. 2020 v 9:18 odesílatel Amit Khandekar napsal: > On Mon, 13 Apr 2020 at 20:16, Amit Khandekar > wrote: > > On Sat, 11 Apr 2020 at 04:18, Tom Lane wrote: > > > > > > I wrote: > > > > A more useful test would be to directly experiment with contended > > > > spinlocks. As I recall, we ha

Re: xid wraparound danger due to INDEX_CLEANUP false

2020-04-16 Thread Masahiko Sawada
On Thu, Apr 16, 2020 at 8:38 AM Andres Freund wrote: > > Hi, > > commit a96c41feec6b6616eb9d5baee9a9e08c20533c38 > Author: Robert Haas > Date: 2019-04-04 14:58:53 -0400 > > Allow VACUUM to be run with index cleanup disabled. > > This commit adds a new reloption, vacuum_index_cleanup, wh

Re: Race condition in SyncRepGetSyncStandbysPriority

2020-04-16 Thread Kyotaro Horiguchi
At Wed, 15 Apr 2020 11:31:49 -0400, Tom Lane wrote in > Kyotaro Horiguchi writes: > > At Tue, 14 Apr 2020 16:32:40 -0400, Tom Lane wrote in > > + stby->is_sync_standby = true; /* might change below */ > > > I'm uneasy with that. In quorum mode all running standbys are marked > >

Re: spin_delay() for ARM

2020-04-16 Thread Amit Khandekar
On Mon, 13 Apr 2020 at 20:16, Amit Khandekar wrote: > On Sat, 11 Apr 2020 at 04:18, Tom Lane wrote: > > > > I wrote: > > > A more useful test would be to directly experiment with contended > > > spinlocks. As I recall, we had some test cases laying about when > > > we were fooling with the spin