Re: PG compilation error with Visual Studio 2015/2017/2019

2020-04-06 Thread davinder singh
On Mon, Apr 6, 2020 at 8:17 PM Juan José Santamaría Flecha < juanjo.santama...@gmail.com> wrote: > > How do you reproduce this issue with Visual Studio? I see there is an > ifdef directive above IsoLocaleName(): > > #if defined(WIN32) && defined(LC_MESSAGES) > > I would expect

Re: A bug when use get_bit() function for a long bytea string

2020-04-06 Thread movead...@highgo.ca
Hello hackers, After several patch change by hacker's proposal, I think it's ready to commit, can we commit it before doing the code freeze for pg-13? Regards, Highgo Software (Canada/China/Pakistan) URL : www.highgo.ca EMAIL: mailto:movead(dot)li(at)highgo(dot)ca

Re: Parallel copy

2020-04-06 Thread vignesh C
On Thu, Mar 12, 2020 at 6:39 PM vignesh C wrote: > Existing parallel copy code flow. Copy supports copy operation from csv, txt & bin format file. For processing csv & text format, it will read 64kb chunk or lesser size if in case the file has lesser size contents in the input file. Server will

Re: doc review for parallel vacuum

2020-04-06 Thread Justin Pryzby
On Tue, Apr 07, 2020 at 09:57:46AM +0530, Amit Kapila wrote: > On Mon, Mar 23, 2020 at 10:34 AM Amit Kapila wrote: > > On Sun, Mar 22, 2020 at 7:48 AM Justin Pryzby wrote: > > > > > > Original, long thread > > >

Re: backup manifests and contemporaneous buildfarm failures

2020-04-06 Thread Tom Lane
Robert Haas writes: > Taking stock of the situation this morning, most of the buildfarm is > now green. There are three failures, on eelpout (6 hours ago), > fairywren (17 hours ago), and hyrax (3 days, 7 hours ago). fairywren has now done this twice in the pg_validatebackupCheck step: exec

Re: doc review for parallel vacuum

2020-04-06 Thread Amit Kapila
On Mon, Mar 23, 2020 at 10:34 AM Amit Kapila wrote: > > On Sun, Mar 22, 2020 at 7:48 AM Justin Pryzby wrote: > > > > Original, long thread > > https://www.postgresql.org/message-id/flat/CAA4eK1%2Bnw1FBK3_sDnW%2B7kB%2Bx4qbDJqetgqwYW8k2xv82RZ%2BKw%40mail.gmail.com#b1745ee853b137043e584b500b41300f

Re: [PATCH] Incremental sort (was: PoC: Partial sort)

2020-04-06 Thread Justin Pryzby
On Mon, Apr 06, 2020 at 09:57:22PM +0200, Tomas Vondra wrote: > I've pushed the fist part of this patch series - I've reorganized it a I scanned through this again post-commit. Find attached some suggestions. Shouldn't non-text explain output always show both disk *and* mem, including zeros ?

Re: Spurious "apparent wraparound" via SimpleLruTruncate() rounding

2020-04-06 Thread Noah Misch
On Mon, Apr 06, 2020 at 02:46:09PM -0400, Tom Lane wrote: > Noah Misch writes: > > On Wed, Mar 25, 2020 at 04:42:31PM -0400, Tom Lane wrote: > >> So I think what we're actually trying to accomplish here is to > >> ensure that instead of deleting up to half of the SLRU space > >> before the

Re: WIP/PoC for parallel backup

2020-04-06 Thread Jeevan Chalke
Asif, After recent backup manifest addition, patches needed to rebase and reconsideration of a few things like making sure that parallel backup creates a manifest file correctly or not etc. -- Jeevan Chalke Associate Database Architect & Team Lead, Product Development EnterpriseDB Corporation

Re: Autovacuum on partitioned table (autoanalyze)

2020-04-06 Thread Justin Pryzby
Not sure if you saw my earlier message ? I think it ought to be possible to configure this feature such that an auto-analyze on any child partition would trigger analyze of the parent. I think that would be important for maintaining accurate stats of the partition key column for many cases

Re: Don't try fetching future segment of a TLI.

2020-04-06 Thread Fujii Masao
On 2020/04/07 10:29, Kyotaro Horiguchi wrote: Thank you for picking this up. At Tue, 7 Apr 2020 02:43:02 +0900, Fujii Masao wrote in On 2020/03/19 22:22, Pavel Suderevsky wrote: Hi, I've tested patch provided by Kyotaro and do confirm it fixes the issue. The patch looks good to me.

Re: d25ea01275 and partitionwise join

2020-04-06 Thread Amit Langote
On Tue, Apr 7, 2020 at 2:41 AM Tom Lane wrote: > Amit Langote writes: > > which does succeed in using partitionwise join. Please see attached > > delta that applies on your v7 if that is what you'd rather have. > > I figured these queries were cheap enough that we could afford to run > both.

Re: Don't try fetching future segment of a TLI.

2020-04-06 Thread Fujii Masao
On 2020/04/07 4:04, David Steele wrote: On 4/6/20 1:43 PM, Fujii Masao wrote: On 2020/03/19 22:22, Pavel Suderevsky wrote: Hi, I've tested patch provided by Kyotaro and do confirm it fixes the issue. The patch looks good to me. Attached is the updated version of the patch. I updated

Re: [HACKERS] Restricting maximum keep segments by repslots

2020-04-06 Thread Kyotaro Horiguchi
At Mon, 6 Apr 2020 14:58:39 -0400, Alvaro Herrera wrote in > > LOG: slot rep1 is invalidated at 0/1C0 due to exceeding > > max_slot_wal_keep_size > > Sounds good. Here's a couple of further adjustments to your v24. This > passes the existing tests (pg_basebackup exception noted below),

Re: [PATCH] Incremental sort (was: PoC: Partial sort)

2020-04-06 Thread Tomas Vondra
On Mon, Apr 06, 2020 at 11:00:37PM -0400, Tom Lane wrote: Tomas Vondra writes: I came to the same conclusion (that the change in TuplesortMethod definiton is the culprit) a while ago and was about to push a fix that initialized it correctly in ExecSortInitializeDSM. But I agree reverting it

Re: [PATCH] Incremental sort (was: PoC: Partial sort)

2020-04-06 Thread Tom Lane
Tomas Vondra writes: > I came to the same conclusion (that the change in TuplesortMethod > definiton is the culprit) a while ago and was about to push a fix that > initialized it correctly in ExecSortInitializeDSM. But I agree reverting > it back to the old definition is probably better. Yeah,

Re: [PATCH] Incremental sort (was: PoC: Partial sort)

2020-04-06 Thread Tomas Vondra
On Mon, Apr 06, 2020 at 10:19:41PM -0400, Tom Lane wrote: James Coleman writes: Fair enough. Unsure if Tomas is still online to comment and/or push, but reverting SORT_TYPE_STILL_IN_PROGRESS back to 0 works for me as an initial fix. I'm guessing he went to bed, so I'll push a fix in a

[patch] some PQExpBuffer are not destroyed in pg_dump

2020-04-06 Thread Zhang, Jie
Hi al In getDefaultACLs function, some PQExpBuffer are not destroy File: src/bin/pg_dump/pg_dump.c DefaultACLInfo * getDefaultACLs(Archive *fout, int *numDefaultACLs) { .. if (fout->remoteVersion >= 90600) { PQExpBuffer acl_subquery = createPQExpBuffer();

Re: WAL usage calculation patch

2020-04-06 Thread Amit Kapila
On Mon, Apr 6, 2020 at 7:58 PM Euler Taveira wrote: > > On Mon, 6 Apr 2020 at 10:37, Julien Rouhaud wrote: >> >> On Mon, Apr 06, 2020 at 10:12:55AM -0300, Euler Taveira wrote: >> > On Mon, 6 Apr 2020 at 00:25, Amit Kapila wrote: >> > >> > > >> > > I have pushed pg_stat_statements and Explain

Re: Make MemoryContextMemAllocated() more precise

2020-04-06 Thread Jeff Davis
On Sun, 2020-04-05 at 16:48 -0700, Andres Freund wrote: > > I still think we should do something for v13, such as the > > originally- > > proposed patch[1]. It's not critical, but it simply reports a > > better > > number for memory consumption. Currently, the memory usage appears > > to > > jump,

Re: Autovacuum on partitioned table

2020-04-06 Thread yuzuko
Hi Alvaro, Thank you for your comments. > I'm confused about some error messages in the regression test when a > column is mentioned twice, that changed from mentioning the table named > in the vacuum command, to mentioning the first partition. Is that > because you changed an lappend() to

Re: User Interface for WAL usage data

2020-04-06 Thread Amit Kapila
On Mon, Apr 6, 2020 at 10:04 PM Justin Pryzby wrote: > > On Thu, Apr 02, 2020 at 08:29:31AM +0200, Julien Rouhaud wrote: > > > > "full page records" seems to be showing the number of full page > > > > images, not the record having full page images. > > > > > > I am not sure what exactly is a

Re: Make MemoryContextMemAllocated() more precise

2020-04-06 Thread Jeff Davis
On Mon, 2020-04-06 at 23:39 +1200, David Rowley wrote: > 1. The comment mentions "passthru", but you've removed that > parameter. Fixed, thank you. > 2. I don't think MemoryContextCount is the best name for this > function. When I saw: I've gone back and forth on naming a bit. The right name,

Re: [HACKERS] make async slave to wait for lsn to be replayed

2020-04-06 Thread Anna Akenteva
On 2020-04-07 00:58, Kartyshov Ivan wrote: Ok, here is a new version of patch with single LSN and TIMEOUT. I had a look at the code and did some more code cleanup, with Ivan's permission. This is what I did: - Removed "WAIT FOR" command tag from cmdtaglist.h and renamed WaitStmt to

Re: [PATCH] Incremental sort (was: PoC: Partial sort)

2020-04-06 Thread Tom Lane
James Coleman writes: > Fair enough. Unsure if Tomas is still online to comment and/or push, > but reverting SORT_TYPE_STILL_IN_PROGRESS back to 0 works for me as an > initial fix. I'm guessing he went to bed, so I'll push a fix in a moment. The patch has survived enough test cycles here now to

Re: [PATCH] Incremental sort (was: PoC: Partial sort)

2020-04-06 Thread James Coleman
On Mon, Apr 6, 2020 at 10:09 PM Tom Lane wrote: > > James Coleman writes: > > On Mon, Apr 6, 2020 at 9:46 PM Tom Lane wrote: > >> I think the correct fix is to change the enum declaration. > > > Hmm. I don't actually really like that, because it means the value > > here isn't actually

Re: WAL usage calculation patch

2020-04-06 Thread Amit Kapila
On Mon, Apr 6, 2020 at 10:01 PM Justin Pryzby wrote: > > On Mon, Apr 06, 2020 at 05:01:30PM +0200, Peter Eisentraut wrote: > > I noticed in some of the screenshots that were tweeted that for example in > > > > WAL: records=1 bytes=56 > > > > there are two spaces between pieces of data.

Re: [PATCH] Incremental sort (was: PoC: Partial sort)

2020-04-06 Thread Tom Lane
James Coleman writes: > On Mon, Apr 6, 2020 at 9:46 PM Tom Lane wrote: >> I think the correct fix is to change the enum declaration. > Hmm. I don't actually really like that, because it means the value > here isn't actually semantically correct. That is, the sort type is > not "in progress";

Re: [PATCH] Incremental sort (was: PoC: Partial sort)

2020-04-06 Thread James Coleman
On Mon, Apr 6, 2020 at 9:46 PM Tom Lane wrote: > > Tomas Vondra writes: > > I don't know, I've tried running the tests on a number of machines, > > similar to those failing. Rapsberry Pi, Fedora 31, ... and it worked > > everywhere while the failures seem consistent. > > On my machine, it

Re: [PATCH] Incremental sort (was: PoC: Partial sort)

2020-04-06 Thread Tom Lane
Tomas Vondra writes: > I don't know, I've tried running the tests on a number of machines, > similar to those failing. Rapsberry Pi, Fedora 31, ... and it worked > everywhere while the failures seem consistent. On my machine, it reproduces about one time in six with force_parallel_mode =

Re: A bug when use get_bit() function for a long bytea string

2020-04-06 Thread movead...@highgo.ca
>In existing releases, the SQL definitions are set_bit(bytea,int4,int4) >and get_bit(bytea,int4) and cannot be changed to not break the API. >So the patch meant for existing releases has to deal with a too-narrow >int32 bit number. >Internally in the C functions, you may convert that number to

Re: Don't try fetching future segment of a TLI.

2020-04-06 Thread Kyotaro Horiguchi
Thank you for picking this up. At Tue, 7 Apr 2020 02:43:02 +0900, Fujii Masao wrote in > On 2020/03/19 22:22, Pavel Suderevsky wrote: > > Hi, > > I've tested patch provided by Kyotaro and do confirm it fixes the > > issue. > > The patch looks good to me. Attached is the updated version of the

Re: [PATCH] Incremental sort (was: PoC: Partial sort)

2020-04-06 Thread Tomas Vondra
On Mon, Apr 06, 2020 at 08:42:13PM -0400, Tom Lane wrote: Tomas Vondra writes: It doesn't seem to be particularly platform-specific, but I've been unable to reproduce it so far. It seems on older gcc versions, though. It's looking kind of like an uninitialized-memory problem. Note the

Re: [PATCH] Incremental sort (was: PoC: Partial sort)

2020-04-06 Thread Tomas Vondra
On Mon, Apr 06, 2020 at 07:27:19PM -0400, James Coleman wrote: On Mon, Apr 6, 2020 at 7:09 PM James Coleman wrote: On Mon, Apr 6, 2020 at 6:13 PM Tomas Vondra wrote: > > On Mon, Apr 06, 2020 at 05:47:48PM -0400, James Coleman wrote: > >On Mon, Apr 6, 2020 at 5:40 PM Tomas Vondra > > wrote: >

Re: FETCH FIRST clause WITH TIES option

2020-04-06 Thread Alvaro Herrera
Some ruleutils.c code added by this patch is not covered by tests: 5246 : /* Add the LIMIT clause if given */ 52471115 : if (query->limitOffset != NULL) 5248 : { 5249 0 : appendContextKeyword(context, " OFFSET ",

Re: [PATCH] Incremental sort (was: PoC: Partial sort)

2020-04-06 Thread James Coleman
On Mon, Apr 6, 2020 at 7:31 PM Tomas Vondra wrote: > > On Mon, Apr 06, 2020 at 07:09:11PM -0400, James Coleman wrote: > >On Mon, Apr 6, 2020 at 6:13 PM Tomas Vondra > > wrote: > >> > >> On Mon, Apr 06, 2020 at 05:47:48PM -0400, James Coleman wrote: > >> >On Mon, Apr 6, 2020 at 5:40 PM Tomas

Re: ERROR: invalid input syntax for type circle

2020-04-06 Thread David Zhang
Hi Tom, Thanks for the review. Generated a new patch v2 (attached) following your suggestion and performed the same test again. The test results looks good including the "make check". On 2020-04-06 3:16 p.m., Tom Lane wrote: David Zhang writes: I got an error when I was trying to insert

Re: [PATCH] Incremental sort (was: PoC: Partial sort)

2020-04-06 Thread Tom Lane
Tomas Vondra writes: > It doesn't seem to be particularly platform-specific, but I've been > unable to reproduce it so far. It seems on older gcc versions, though. It's looking kind of like an uninitialized-memory problem. Note the latest from spurfowl,

Re: FETCH FIRST clause WITH TIES option

2020-04-06 Thread Alvaro Herrera
There was a minor conflict in planmain.h. Here's a refreshed version. -- Álvaro Herrerahttps://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services >From 51dee439e5165efad186c48d1a3b9119f90542e8 Mon Sep 17 00:00:00 2001 From: Alvaro Herrera

Re: proposal \gcsv

2020-04-06 Thread Tom Lane
Here's a WIP patch for the parenthesized-options route. I realized that if we make the options be single words in the form name=value, we can easily handle the shortcut forms with no value. So that's what this does. What this does *not* do is offer any solution to the question of how to put a

Re: [HACKERS] make async slave to wait for lsn to be replayed

2020-04-06 Thread Alexander Korotkov
On Tue, Apr 7, 2020 at 12:58 AM Kartyshov Ivan wrote: > On 2020-04-04 03:14, Alexander Korotkov wrote: > > I think that now we would be fine with single LSN and single TIMEOUT. > > In future we may add multiple LSNs/TIMEOUTs or/and support for > > expressions as LSNs/TIMEOUTs if we figure out

Re: [HACKERS] Restricting maximum keep segments by repslots

2020-04-06 Thread Alvaro Herrera
On 2020-Apr-07, Kyotaro Horiguchi wrote: > At Mon, 6 Apr 2020 12:54:56 -0400, Alvaro Herrera > wrote in > > Thanks for the revised version. Please note that you forgot to "git > > add" the test file, to it's not in the patch. > > Oops! I forgot that I was working after just doing patch -p1

Re: Should we add xid_current() or a int8->xid cast?

2020-04-06 Thread Thomas Munro
On Sun, Apr 5, 2020 at 11:31 AM Thomas Munro wrote: > On Sun, Apr 5, 2020 at 10:34 AM Mark Dilger > wrote: > > The "xid8_" warts are partly motivated by having a type named "xid8", which > > is a bit of a wart in itself. > > Just a thought for the future, not sure if it's a good one: would it >

Re: [HACKERS] Restricting maximum keep segments by repslots

2020-04-06 Thread Kyotaro Horiguchi
At Mon, 6 Apr 2020 12:54:56 -0400, Alvaro Herrera wrote in > Thanks for the revised version. Please note that you forgot to "git > add" the test file, to it's not in the patch. Oops! I forgot that I was working after just doing patch -p1 on my working directory. This is the version that

Re: Improving connection scalability: GetSnapshotData()

2020-04-06 Thread Andres Freund
Hi, On 2020-04-06 06:39:59 -0700, Andres Freund wrote: > > 3) Plain pgbench read-write (you already did it for sure) > > -s 100 -M prepared -T 700 > > autovacuum=off, fsync on: > clients tps master tps pgxact > 1 474 479 > 16 43564476 > 40

Re: [HACKERS] Restricting maximum keep segments by repslots

2020-04-06 Thread Alvaro Herrera
On 2020-Apr-06, Alvaro Herrera wrote: > I think there's a race condition in this: if we kill a walsender and it > restarts immediately before we (checkpoint) can acquire the slot, we > will wait for it to terminate on its own. Fixing this requires changing > the ReplicationSlotAcquire API so

Re: [PATCH] Incremental sort (was: PoC: Partial sort)

2020-04-06 Thread Tomas Vondra
On Mon, Apr 06, 2020 at 07:09:11PM -0400, James Coleman wrote: On Mon, Apr 6, 2020 at 6:13 PM Tomas Vondra wrote: On Mon, Apr 06, 2020 at 05:47:48PM -0400, James Coleman wrote: >On Mon, Apr 6, 2020 at 5:40 PM Tomas Vondra > wrote: >> >> On Mon, Apr 06, 2020 at 11:12:32PM +0200, Tomas Vondra

Re: [PATCH] Incremental sort (was: PoC: Partial sort)

2020-04-06 Thread James Coleman
On Mon, Apr 6, 2020 at 7:09 PM James Coleman wrote: > > On Mon, Apr 6, 2020 at 6:13 PM Tomas Vondra > wrote: > > > > On Mon, Apr 06, 2020 at 05:47:48PM -0400, James Coleman wrote: > > >On Mon, Apr 6, 2020 at 5:40 PM Tomas Vondra > > > wrote: > > >> > > >> On Mon, Apr 06, 2020 at 11:12:32PM

Re: [PATCH] Incremental sort (was: PoC: Partial sort)

2020-04-06 Thread James Coleman
On Mon, Apr 6, 2020 at 6:13 PM Tomas Vondra wrote: > > On Mon, Apr 06, 2020 at 05:47:48PM -0400, James Coleman wrote: > >On Mon, Apr 6, 2020 at 5:40 PM Tomas Vondra > > wrote: > >> > >> On Mon, Apr 06, 2020 at 11:12:32PM +0200, Tomas Vondra wrote: > >> >On Mon, Apr 06, 2020 at 04:54:38PM -0400,

Re: [PATCH] Incremental sort (was: PoC: Partial sort)

2020-04-06 Thread Tomas Vondra
On Mon, Apr 06, 2020 at 06:34:04PM -0400, Tom Lane wrote: Tomas Vondra writes: On Mon, Apr 06, 2020 at 05:51:43PM -0400, Tom Lane wrote: Well, it's *less* unhappy. thorntail is showing that the number of workers field is not stable; that will need to be masked. Yeah, I've already pushed a

Re: [PATCH] Incremental sort (was: PoC: Partial sort)

2020-04-06 Thread Tom Lane
Tomas Vondra writes: > On Mon, Apr 06, 2020 at 05:51:43PM -0400, Tom Lane wrote: >> Well, it's *less* unhappy. thorntail is showing that the number of >> workers field is not stable; that will need to be masked. > Yeah, I've already pushed a fix for that. But there seems to be another > failure

Re: [PATCH] Incremental sort (was: PoC: Partial sort)

2020-04-06 Thread Tomas Vondra
On Mon, Apr 06, 2020 at 05:51:43PM -0400, Tom Lane wrote: Tomas Vondra writes: OK, I've pushed a fix - this should make the buildfarm happy again. Well, it's *less* unhappy. thorntail is showing that the number of workers field is not stable; that will need to be masked. Yeah, I've

Re: ERROR: invalid input syntax for type circle

2020-04-06 Thread Tom Lane
David Zhang writes: > I got an error when I was trying to insert a circle using the syntax > (the 3rd one) specified in the latest document. Hm. Presumably, that has never worked, and we've had no complaints to date. I'm halfway inclined to treat it as a documentation bug and remove the claim

Re: [HACKERS] Restricting maximum keep segments by repslots

2020-04-06 Thread Alvaro Herrera
On 2020-Apr-06, Alvaro Herrera wrote: > Lastly, I noticed that we're now changing the slot's restart_lsn to > Invalid without being the slot's owner, which goes counter to what is > said in slot.h: > > * - Individual fields are protected by mutex where only the backend owning > * the slot is

Re: [PATCH] Incremental sort (was: PoC: Partial sort)

2020-04-06 Thread Tomas Vondra
On Mon, Apr 06, 2020 at 05:47:48PM -0400, James Coleman wrote: On Mon, Apr 6, 2020 at 5:40 PM Tomas Vondra wrote: On Mon, Apr 06, 2020 at 11:12:32PM +0200, Tomas Vondra wrote: >On Mon, Apr 06, 2020 at 04:54:38PM -0400, Alvaro Herrera wrote: >>On 2020-Apr-06, Tom Lane wrote: >> >>>Locally,

Re: Let people set host(no)ssl settings from initdb

2020-04-06 Thread Cary Huang
The following review has been posted through the commitfest application: make installcheck-world: tested, passed Implements feature: tested, passed Spec compliant: tested, passed Documentation:tested, passed Hi I applied the patch

Re: [HACKERS] make async slave to wait for lsn to be replayed

2020-04-06 Thread Kartyshov Ivan
On 2020-04-04 03:14, Alexander Korotkov wrote: I think that now we would be fine with single LSN and single TIMEOUT. In future we may add multiple LSNs/TIMEOUTs or/and support for expressions as LSNs/TIMEOUTs if we figure out it's necessary. I also think it's good to couple waiting for lsn with

Re: [PATCH] Incremental sort (was: PoC: Partial sort)

2020-04-06 Thread Tom Lane
Tomas Vondra writes: > OK, I've pushed a fix - this should make the buildfarm happy again. Well, it's *less* unhappy. thorntail is showing that the number of workers field is not stable; that will need to be masked. regards, tom lane

Re: [PATCH] Incremental sort (was: PoC: Partial sort)

2020-04-06 Thread James Coleman
On Mon, Apr 6, 2020 at 5:40 PM Tomas Vondra wrote: > > On Mon, Apr 06, 2020 at 11:12:32PM +0200, Tomas Vondra wrote: > >On Mon, Apr 06, 2020 at 04:54:38PM -0400, Alvaro Herrera wrote: > >>On 2020-Apr-06, Tom Lane wrote: > >> > >>>Locally, things pass without force_parallel_mode, but turning it on

Re: nbtree: assertion failure in _bt_killitems() for posting tuple

2020-04-06 Thread Peter Geoghegan
On Sun, Apr 5, 2020 at 5:15 PM Peter Geoghegan wrote: > I am thinking about pushing a fix along the lines of the attached > patch. This preserves the assertion, while avoiding the check in cases > where it doesn't apply, such as when a dirty snapshot is in use. Pushed. Thanks. -- Peter

Re: [PATCH] Incremental sort (was: PoC: Partial sort)

2020-04-06 Thread Tomas Vondra
On Mon, Apr 06, 2020 at 11:12:32PM +0200, Tomas Vondra wrote: On Mon, Apr 06, 2020 at 04:54:38PM -0400, Alvaro Herrera wrote: On 2020-Apr-06, Tom Lane wrote: Locally, things pass without force_parallel_mode, but turning it on produces failures that look similar to rhinoceros's (didn't examine

Re: where should I stick that backup?

2020-04-06 Thread Robert Haas
On Mon, Apr 6, 2020 at 1:32 PM Magnus Hagander wrote: > Now, if we were just talking about compression, it would actually be > interesting to implement some sort of "postgres compression API" if > you will, that is implemented by a shared library. This library could > then be used from

Re: [PATCH] Incremental sort (was: PoC: Partial sort)

2020-04-06 Thread James Coleman
On Mon, Apr 6, 2020 at 5:22 PM James Coleman wrote: > > On Mon, Apr 6, 2020 at 5:20 PM James Coleman wrote: > > > > On Mon, Apr 6, 2020 at 5:12 PM Tomas Vondra > > wrote: > > > > > > On Mon, Apr 06, 2020 at 04:54:38PM -0400, Alvaro Herrera wrote: > > > >On 2020-Apr-06, Tom Lane wrote: > > > > >

Re: [PATCH] Incremental sort (was: PoC: Partial sort)

2020-04-06 Thread James Coleman
On Mon, Apr 6, 2020 at 5:20 PM James Coleman wrote: > > On Mon, Apr 6, 2020 at 5:12 PM Tomas Vondra > wrote: > > > > On Mon, Apr 06, 2020 at 04:54:38PM -0400, Alvaro Herrera wrote: > > >On 2020-Apr-06, Tom Lane wrote: > > > > > >> Locally, things pass without force_parallel_mode, but turning it

Re: [PATCH] Incremental sort (was: PoC: Partial sort)

2020-04-06 Thread James Coleman
On Mon, Apr 6, 2020 at 5:12 PM Tomas Vondra wrote: > > On Mon, Apr 06, 2020 at 04:54:38PM -0400, Alvaro Herrera wrote: > >On 2020-Apr-06, Tom Lane wrote: > > > >> Locally, things pass without force_parallel_mode, but turning it on > >> produces failures that look similar to rhinoceros's (didn't

Re: Online verification of checksums

2020-04-06 Thread Michael Banck
Hi, Am Montag, den 06.04.2020, 16:45 -0400 schrieb Tom Lane: > I wrote: > > Another thing that's bothering me is that the patch compares page LSN > > against GetInsertRecPtr(); but that function says > > ... > > I'm not convinced that an approximation is good enough here. It seems > > like a

Re: SyncRepLock acquired exclusively in default configuration

2020-04-06 Thread Andres Freund
Hi, On 2020-04-06 11:51:06 -0700, Ashwin Agrawal wrote: > On Mon, Apr 6, 2020 at 1:52 AM Masahiko Sawada < > masahiko.saw...@2ndquadrant.com> wrote: > > On Mon, 6 Apr 2020 at 14:04, Andres Freund wrote: > > > I'm really not ok with unneccessarily adding an exclusive lock > > > acquisition to

ERROR: invalid input syntax for type circle

2020-04-06 Thread David Zhang
Hi, I got an error when I was trying to insert a circle using the syntax (the 3rd one) specified in the latest document. https://www.postgresql.org/docs/current/datatype-geometric.html#DATATYPE-CIRCLE < ( x , y ) , r > ( ( x , y ) , r )   ( x , y ) , r     x , y   , r Here is how to

Re: [PATCH] Incremental sort (was: PoC: Partial sort)

2020-04-06 Thread Tomas Vondra
On Mon, Apr 06, 2020 at 04:54:38PM -0400, Alvaro Herrera wrote: On 2020-Apr-06, Tom Lane wrote: Locally, things pass without force_parallel_mode, but turning it on produces failures that look similar to rhinoceros's (didn't examine other BF members). FWIW I looked at the eight failures there

Re: where should I stick that backup?

2020-04-06 Thread Robert Haas
On Mon, Apr 6, 2020 at 2:23 PM Stephen Frost wrote: > So, instead of talking about 'bzip2 > %f.bz2', and then writing into our > documentation that that's how this feature can be used, what about > proposing something that would actually work reliably with this > interface? Which properly

Re: [PATCH] Incremental sort (was: PoC: Partial sort)

2020-04-06 Thread Alvaro Herrera
On 2020-Apr-06, Tom Lane wrote: > Locally, things pass without force_parallel_mode, but turning it on > produces failures that look similar to rhinoceros's (didn't examine > other BF members). FWIW I looked at the eight failures there were about fifteen minutes ago and they were all identical.

Re: Improving connection scalability: GetSnapshotData()

2020-04-06 Thread Andres Freund
Hi, On 2020-04-06 06:39:59 -0700, Andres Freund wrote: > These benchmarks are on my workstation. The larger VM I used in the last > round wasn't currently available. One way to reproduce the problem at smaller connection counts / smaller machines is to take more snapshots. Doesn't fully

Re: [PATCH] Incremental sort (was: PoC: Partial sort)

2020-04-06 Thread Tom Lane
Tomas Vondra writes: > On Mon, Apr 06, 2020 at 04:14:38PM -0400, Tom Lane wrote: >> Did you ever use force_parallel_mode = regress? > Ah, not sure - probably not in this round of tests and there were some > changes in the explain code. Thanks for the hint. Locally, things pass without

Re: archive recovery fetching wrong segments

2020-04-06 Thread Grigory Smolkin
On 4/6/20 10:51 PM, David Steele wrote: On 4/6/20 3:23 PM, Grigory Smolkin wrote: On 4/6/20 9:17 PM, David Steele wrote: Hi Grigory, Hello! On 4/5/20 8:02 PM, Grigory Smolkin wrote: Hello, hackers! I`m investigating a complains from our clients about archive recovery speed been very

Re: Online verification of checksums

2020-04-06 Thread Tom Lane
I wrote: > Another thing that's bothering me is that the patch compares page LSN > against GetInsertRecPtr(); but that function says > ... > I'm not convinced that an approximation is good enough here. It seems > like a page that's just now been updated could have an LSN beyond the > current XLOG

Re: [PATCH] Incremental sort (was: PoC: Partial sort)

2020-04-06 Thread Tomas Vondra
On Mon, Apr 06, 2020 at 04:14:38PM -0400, Tom Lane wrote: Tomas Vondra writes: Hmmm, I see the buildfarm is not happy about it - a couple of animals failed, but some succeeded. The failure seems like a simple difference in explain output, but it's not clear why would it happen (and I've ran

Re: [PATCH] Incremental sort (was: PoC: Partial sort)

2020-04-06 Thread Tom Lane
Tomas Vondra writes: > Hmmm, I see the buildfarm is not happy about it - a couple of animals > failed, but some succeeded. The failure seems like a simple difference > in explain output, but it's not clear why would it happen (and I've > ran the tests many times but never seen this failure). Did

Re: [PATCH] Incremental sort (was: PoC: Partial sort)

2020-04-06 Thread Tomas Vondra
On Mon, Apr 06, 2020 at 09:57:22PM +0200, Tomas Vondra wrote: Hi, I've pushed the fist part of this patch series - I've reorganized it a bit by moving the add_partial_path changes to the end. That way I've been able to add regression test demonstrating impact of the change on plans involving

Re: backup manifests and contemporaneous buildfarm failures

2020-04-06 Thread Andrew Dunstan
On 4/6/20 7:53 AM, Robert Haas wrote: > On Sun, Apr 5, 2020 at 4:07 PM Andrew Dunstan > wrote: >> Do I need to precede those with some recursive chmod commands? > +1. > >> Perhaps >> the client should refuse to run if there is still something left after >> these. > +1 to that, too. > See

Re: Online verification of checksums

2020-04-06 Thread Tom Lane
Michael Banck writes: > [ 0001-Fix-checksum-verification-in-base-backups-for-random_V3.patch ] I noticed that the cfbot wasn't testing this because of a minor merge conflict. I rebased it over that, and also readjusted things a little bit to avoid unnecessarily reindenting existing code, in

Re: archive recovery fetching wrong segments

2020-04-06 Thread David Steele
On 4/6/20 3:23 PM, Grigory Smolkin wrote: On 4/6/20 9:17 PM, David Steele wrote: Hi Grigory, Hello! On 4/5/20 8:02 PM, Grigory Smolkin wrote: Hello, hackers! I`m investigating a complains from our clients about archive recovery speed been very slow, and I`ve noticed a really strange

Re: archive recovery fetching wrong segments

2020-04-06 Thread Grigory Smolkin
On 4/6/20 9:17 PM, David Steele wrote: Hi Grigory, Hello! On 4/5/20 8:02 PM, Grigory Smolkin wrote: Hello, hackers! I`m investigating a complains from our clients about archive recovery speed been very slow, and I`ve noticed a really strange and, I think, a very dangerous recovery

Re: DROP OWNED CASCADE vs Temp tables

2020-04-06 Thread Tom Lane
Alvaro Herrera writes: > On 2020-Jan-14, Alvaro Herrera wrote: >> Hmm, it seems to be doing it differently. Maybe it should be acquiring >> locks on all objects in that nested loop and verified them for >> existence, so that when it calls performMultipleDeletions the objects >> are already

Re: Don't try fetching future segment of a TLI.

2020-04-06 Thread David Steele
On 4/6/20 1:43 PM, Fujii Masao wrote: On 2020/03/19 22:22, Pavel Suderevsky wrote: Hi, I've tested patch provided by Kyotaro and do confirm it fixes the issue. The patch looks good to me. Attached is the updated version of the patch. I updated only comments. Barring any objection, I will

Re: [HACKERS] Restricting maximum keep segments by repslots

2020-04-06 Thread Alvaro Herrera
On 2020-Apr-06, Kyotaro Horiguchi wrote: > At Fri, 3 Apr 2020 20:14:03 -0300, Alvaro Herrera > wrote in > Agreed and thanks for the code. The patch is enough to express the > intention. I fixed some compilation errors and made a clean up of > KeepLogSeg. InvalidateObsoleteReplicationSlots

Re: SyncRepLock acquired exclusively in default configuration

2020-04-06 Thread Ashwin Agrawal
On Mon, Apr 6, 2020 at 1:52 AM Masahiko Sawada < masahiko.saw...@2ndquadrant.com> wrote: > On Mon, 6 Apr 2020 at 14:04, Andres Freund wrote: > > > > commit 48c9f4926562278a2fd2b85e7486c6d11705f177 > > Author: Simon Riggs > > Date: 2017-12-29 14:30:33 + > > > > Fix race condition when

Re: Spurious "apparent wraparound" via SimpleLruTruncate() rounding

2020-04-06 Thread Tom Lane
Noah Misch writes: > On Wed, Mar 25, 2020 at 04:42:31PM -0400, Tom Lane wrote: >> So I think what we're actually trying to accomplish here is to >> ensure that instead of deleting up to half of the SLRU space >> before the cutoff, we delete up to half-less-one-segment. >> Maybe it should be

Re: Allow CLUSTER, VACUUM FULL and REINDEX to change tablespace on the fly

2020-04-06 Thread Justin Pryzby
On Mon, Apr 06, 2020 at 08:43:46PM +0300, Alexey Kondratov wrote: > Thanks for the input, but I am afraid that the patch set became a bit messy > now. I have eyeballed it and found some inconsistencies. > > const char *name; /* name of database to reindex > */ > -

RE: Index Skip Scan

2020-04-06 Thread Floris Van Nee
> > Hm, I wasn't aware about this one, thanks for bringing this up. Btw, Floris, I > would appreciate if in the future you can make it more visible that changes > you > suggest contain some fixes. E.g. it wasn't clear for me from your previous > email > that that's the case, and it doesn't

Re: where should I stick that backup?

2020-04-06 Thread Stephen Frost
Greetings, * Magnus Hagander (mag...@hagander.net) wrote: > On Mon, Apr 6, 2020 at 4:45 PM Stephen Frost wrote: > > * Noah Misch (n...@leadboat.com) wrote: > > > On Fri, Apr 03, 2020 at 10:19:21AM -0400, Robert Haas wrote: > > > > What I'm thinking about is: suppose we add an option to

Re: where should I stick that backup?

2020-04-06 Thread Stephen Frost
Greetings, * Robert Haas (robertmh...@gmail.com) wrote: > On Mon, Apr 6, 2020 at 10:45 AM Stephen Frost wrote: > > For my 2c, at least, introducing more shell commands into critical parts > > of the system is absolutely the wrong direction to go in. > > archive_command continues to be a mess

Re: archive recovery fetching wrong segments

2020-04-06 Thread David Steele
Hi Grigory, On 4/5/20 8:02 PM, Grigory Smolkin wrote: Hello, hackers! I`m investigating a complains from our clients about archive recovery speed been very slow, and I`ve noticed a really strange and, I think, a very dangerous recovery behavior. When running multi-timeline archive

Re: Allow cluster owner to bypass authentication

2020-04-06 Thread David Steele
On 4/5/20 6:15 AM, Peter Eisentraut wrote: On 2020-03-27 15:58, David Steele wrote: Hi Peter, On 12/27/19 3:22 PM, Stephen Frost wrote: * Tom Lane (t...@sss.pgh.pa.us) wrote: I think it'd be great if this behavior could be implemented within the notation, because we could then just set up a

Re: Allow CLUSTER, VACUUM FULL and REINDEX to change tablespace on the fly

2020-04-06 Thread Alexey Kondratov
On 2020-04-03 21:27, Justin Pryzby wrote: On Wed, Apr 01, 2020 at 08:08:36AM -0500, Justin Pryzby wrote: Or maybe you'd want me to squish my changes into yours and resend after any review comments ? I didn't hear any feedback, so I've now squished all "parenthesized" and "fix" commits.

Re: Don't try fetching future segment of a TLI.

2020-04-06 Thread Fujii Masao
On 2020/03/19 22:22, Pavel Suderevsky wrote: Hi, I've tested patch provided by Kyotaro and do confirm it fixes the issue. The patch looks good to me. Attached is the updated version of the patch. I updated only comments. Barring any objection, I will commit this patch. Any chance it will

Re: d25ea01275 and partitionwise join

2020-04-06 Thread Tom Lane
Amit Langote writes: > which does succeed in using partitionwise join. Please see attached > delta that applies on your v7 if that is what you'd rather have. I figured these queries were cheap enough that we could afford to run both. With that and some revision of the comments (per attached),

Re: pg_stat_statements issue with parallel maintenance (Was Re: WAL usage calculation patch)

2020-04-06 Thread Peter Geoghegan
On Mon, Apr 6, 2020 at 2:21 AM Amit Kapila wrote: > AFAIU, it uses heapam_index_build_range_scan but for writing to index, > it doesn't use buffer manager. Right. It doesn't need to use the buffer manager to write to the index, unlike (say) GIN's CREATE INDEX. -- Peter Geoghegan

Using the rr debugging tool to debug Postgres

2020-04-06 Thread Peter Geoghegan
rr is a tool that makes gdb much more useful by supporting recording and replaying of the program being debugged. I highly recommend trying rr if you're somebody that regularly uses gdb to debug Postgres. rr implements a gdbserver under the hood, so it's very easy to start using once you're

Re: where should I stick that backup?

2020-04-06 Thread Magnus Hagander
On Mon, Apr 6, 2020 at 4:45 PM Stephen Frost wrote: > > Greetings, > > * Noah Misch (n...@leadboat.com) wrote: > > On Fri, Apr 03, 2020 at 10:19:21AM -0400, Robert Haas wrote: > > > What I'm thinking about is: suppose we add an option to pg_basebackup > > > with a name like --pipe-output. This

Re: [HACKERS] Restricting maximum keep segments by repslots

2020-04-06 Thread Alvaro Herrera
On 2020-Apr-06, Kyotaro Horiguchi wrote: > > * Andres complained that the "distance" column was not a great value to > > expose (20171106132050.6apzynxrqrzgh...@alap3.anarazel.de). That's > > right: it changes both by the insertion LSN as well as the slot's > > consumption. Maybe we can

  1   2   >