Re: [HACKERS] CLUSTER command progress monitor

2019-09-15 Thread Tattsu Yama
Hi Michael, >> > Attached file is WIP patch.In my patch, I added "command id" to all >> APIs of >> > progress reporting to isolate commands. Therefore, it doesn't allow to >> > cascade updating system views. And my patch is on WIP so it needs >> clean-up >> > and test. >> > I share it anyway. :)

Re: [HACKERS] [PATCH] pageinspect function to decode infomasks

2019-09-15 Thread Amit Kapila
On Sat, Sep 14, 2019 at 3:00 PM Michael Paquier wrote: > > On Sat, Sep 14, 2019 at 03:03:57PM +0900, Michael Paquier wrote: > > On Sat, Sep 14, 2019 at 11:18:37AM +0530, Amit Kapila wrote: > >> Won't 'Lateral' clause be helpful here as the patch contains it in one > >> of its tests? > > > > Ah tru

Re: POC: Cleaning up orphaned files using undo logs

2019-09-15 Thread Thomas Munro
On Mon, Sep 16, 2019 at 5:27 AM Kuntal Ghosh wrote: > While testing zheap over undo apis, we've found the following > issues/scenarios that might need some fixes/discussions: Thanks! > 1. In UndoLogAllocateInRecovery, when we find the current log number > from the list of registered blocks, we d

Re:

2019-09-15 Thread nilsocket
Extremely, sorry forgot to add attachment. On Mon, Sep 16, 2019 at 11:11 AM nilsocket wrote: > We want to export data from PG to Kafka, > We can't rely on extension which we have written as there could be any > problems which we are not aware of and PG might break. > We don't want our master to

[no subject]

2019-09-15 Thread nilsocket
We want to export data from PG to Kafka, We can't rely on extension which we have written as there could be any problems which we are not aware of and PG might break. We don't want our master to go down because of the extension we have written. So, we are okay with having a new PG instance whose w

Re: Leakproofness of texteq()/textne()

2019-09-15 Thread Tom Lane
Stephen Frost writes: > * Robert Haas (robertmh...@gmail.com) wrote: >> On Thu, Sep 12, 2019 at 5:19 PM Tom Lane wrote: >>> However, if there is some character C that makes ICU misbehave like >>> that, we are going to have problems with indexing strings containing C, >>> whether we think varstr_c

Re: Leakproofness of texteq()/textne()

2019-09-15 Thread Stephen Frost
Greetings, * Robert Haas (robertmh...@gmail.com) wrote: > On Thu, Sep 12, 2019 at 5:19 PM Tom Lane wrote: > > However, if there is some character C that makes ICU misbehave like > > that, we are going to have problems with indexing strings containing C, > > whether we think varstr_cmp is leaky or

Re: range test for hash index?

2019-09-15 Thread Paul A Jungwirth
On Sat, Sep 14, 2019 at 5:13 AM Amit Kapila wrote: > In general, the hash_range is covered by some of the existing test, > but I don't which test. See the code coverage report here: > https://coverage.postgresql.org/src/backend/utils/adt/rangetypes.c.gcov.html Thanks! I did some experimenting, a

Re: block-level incremental backup

2019-09-15 Thread Robert Haas
On Thu, Sep 12, 2019 at 9:13 AM Jeevan Chalke wrote: > I had a look over this issue and observed that when the new database is > created, the catalog files are copied as-is into the new directory > corresponding to a newly created database. And as they are just copied, > the LSN on those pages are

Re: block-level incremental backup

2019-09-15 Thread Robert Haas
On Fri, Sep 13, 2019 at 1:08 PM vignesh C wrote: > Instead of reading the whole file here, we can read the file page by page. > There is a possibility of data inconsistency if data is not read page by > page, data will be consistent if read page by page as full page write will be > enabled at t

Re: [PATCH] Incremental sort (was: PoC: Partial sort)

2019-09-15 Thread James Coleman
On Sat, Jul 20, 2019 at 11:25 AM Tomas Vondra wrote: > >> ... > >> > >> I think this may be a thinko, as this plan demonstrates - but I'm not > >> sure about it. I wonder if this might be penalizing some other types of > >> plans (essentially anything with limit + gather). > >> > >> Attached is a

Re: [PATCH] Improve performance of NOTIFY over many databases (v2)

2019-09-15 Thread Tom Lane
Martijn van Oosterhout writes: > On Sat, 14 Sep 2019 at 17:08, Tom Lane wrote: >> None of this seems to respond to my point: it looks to me like it would >> work fine if you simply dropped the patch's additions in PreCommit_Notify >> and ProcessCompletedNotifies, because there is already enough l

Re: (Re)building index using itself or another index of the same table

2019-09-15 Thread Tomas Vondra
On Thu, Sep 12, 2019 at 11:08:28AM -0400, Tom Lane wrote: Arseny Sher writes: A problem of similar nature can be reproduced with the following stripped-down scenario: CREATE TABLE pears(f1 int primary key, f2 int); INSERT INTO pears SELECT i, i+1 FROM generate_series(1, 100) i; CREATE OR REP

Re: Implementing Incremental View Maintenance

2019-09-15 Thread Paul Draper
As I understand it, the current patch performs immediate IVM using AFTER STATEMENT trigger transition tables. However, multiple tables can be modified *before* AFTER STATEMENT triggers are fired. CREATE TABLE example1 (a int); CREATE TABLE example2 (a int); CREATE INCREMENTAL MATERIALIZED VIEW m

Re: log spam with postgres_fdw

2019-09-15 Thread Jeff Janes
On Sun, Sep 15, 2019 at 11:14 AM Tom Lane wrote: > Jeff Janes writes: > > When closing the local session which had used postgres_fdw over an ssl > > connection, I get log spam on the foreign server saying: > > LOG: could not receive data from client: Connection reset by peer > > It is easy to r

Re: POC: Cleaning up orphaned files using undo logs

2019-09-15 Thread Kuntal Ghosh
Hello Thomas, While testing zheap over undo apis, we've found the following issues/scenarios that might need some fixes/discussions: 1. In UndoLogAllocateInRecovery, when we find the current log number from the list of registered blocks, we don't check whether the block->in_use flag is true or no

Re: Primary keepalive message not appearing in Logical Streaming Replication

2019-09-15 Thread Tomas Vondra
On Sun, Sep 15, 2019 at 09:44:14AM -0600, Michael Loftis wrote: On Sun, Sep 15, 2019 at 08:36 Virendra Negi wrote: Oh I miss the documentation link there you go https://www.postgresql.org/docs/9.5/protocol-replication.html On Sun, Sep 15, 2019 at 8:05 PM Virendra Negi wrote: Agreed but why

Re: Primary keepalive message not appearing in Logical Streaming Replication

2019-09-15 Thread Jeff Janes
On Sun, Sep 15, 2019 at 11:44 AM Michael Loftis wrote: > > > On Sun, Sep 15, 2019 at 08:36 Virendra Negi wrote: > >> Oh I miss the documentation link there you go >> https://www.postgresql.org/docs/9.5/protocol-replication.html >> >> On Sun, Sep 15, 2019 at 8:05 PM Virendra Negi >> wrote: >> >>

Re: Efficient output for integer types

2019-09-15 Thread David Fetter
On Sun, Sep 15, 2019 at 02:06:29PM +0500, Andrey Borodin wrote: > > 15 сент. 2019 г., в 12:18, David Fetter написал(а): > > > > Please find attached a couple of patches intended to $subject. > > > > This patch set cut the time to copy ten million rows of randomly sized > > int8s (10 of them) by

Re: Primary keepalive message not appearing in Logical Streaming Replication

2019-09-15 Thread Michael Loftis
On Sun, Sep 15, 2019 at 08:36 Virendra Negi wrote: > Oh I miss the documentation link there you go > https://www.postgresql.org/docs/9.5/protocol-replication.html > > On Sun, Sep 15, 2019 at 8:05 PM Virendra Negi > wrote: > >> Agreed but why is there a message specification for it describe in th

Re: log spam with postgres_fdw

2019-09-15 Thread Tom Lane
Jeff Janes writes: > When closing the local session which had used postgres_fdw over an ssl > connection, I get log spam on the foreign server saying: > LOG: could not receive data from client: Connection reset by peer > It is easy to reproduce, but you must be using ssl to do so. > On searching,

Re: Primary keepalive message not appearing in Logical Streaming Replication

2019-09-15 Thread Virendra Negi
Oh I miss the documentation link there you go https://www.postgresql.org/docs/9.5/protocol-replication.html On Sun, Sep 15, 2019 at 8:05 PM Virendra Negi wrote: > Agreed but why is there a message specification for it describe in the > documentation and it ask to client reply back if a particul

log spam with postgres_fdw

2019-09-15 Thread Jeff Janes
I'm sending this to hackers, because it is not exactly a bug, and it can't be addressed from userland. I think it is a coding issue, although I haven't identified the exact code. When closing the local session which had used postgres_fdw over an ssl connection, I get log spam on the foreign serve

Fwd: Extending range type operators to cope with elements

2019-09-15 Thread Esteban Zimanyi
> So yes, I've had a need for those operators in the past. What I don't know is whether adding these functions will be worth the catalog clutter. The operators are tested and running within MobilityDB. It concerns lines 231-657 for the C code in file https://github.com/ULB-CoDE-WIT/MobilityDB/blob

Re: Primary keepalive message not appearing in Logical Streaming Replication

2019-09-15 Thread Virendra Negi
Agreed but why is there a message specification for it describe in the documentation and it ask to client reply back if a particular *bit* is set.(1 means that the client should reply to this message as soon as possible, to avoid a timeout disconnect. 0 otherwise) Primary keepalive message (B) B

Re: pg_rewind docs correction

2019-09-15 Thread James Coleman
On Sun, Sep 15, 2019 at 10:25 AM Michael Paquier wrote: > > On Sat, Sep 14, 2019 at 07:00:54PM -0400, James Coleman wrote: > > Updated (plus some additional wordsmithing). > > +The rewind operation is not expected to result in a consistent data > +directory state either internally to the n

Re: Create collation reporting the ICU locale display name

2019-09-15 Thread Michael Paquier
On Sat, Sep 14, 2019 at 03:51:03PM +0200, Daniel Verite wrote: > What about DEBUG1 as the level? > Surely we can draw a line somewhere beyond which the benefit of > getting that information surpasses the annoyance factor that > you're foreseeing? DEBUG1 is even more chatty. I agree with the other

Re: pg_rewind docs correction

2019-09-15 Thread Michael Paquier
On Sat, Sep 14, 2019 at 07:00:54PM -0400, James Coleman wrote: > Updated (plus some additional wordsmithing). +The rewind operation is not expected to result in a consistent data +directory state either internally to the node or with respect to the rest +of the cluster. Instead the res

Re: Primary keepalive message not appearing in Logical Streaming Replication

2019-09-15 Thread Michael Loftis
On Fri, Sep 13, 2019 at 07:12 Virendra Negi wrote: > Implemented the Logical Streaming Replication thing are working fine I see > the XLogData message appearing and I'm able to parse them. > > But I haven't see any "Primary Keepalive message" yet. I had tried > setting the *tcp_keepalive_interva

Re: BF failure: could not open relation with OID XXXX while querying pg_views

2019-09-15 Thread Dean Rasheed
On Sun, 15 Sep 2019 at 12:20, Tomas Vondra wrote: > > On Sun, Sep 15, 2019 at 11:27:19AM +0100, Dean Rasheed wrote: > >On Sun, 15 Sep 2019 at 11:11, Tomas Vondra > >wrote: > >> > >> On Sun, Sep 15, 2019 at 10:16:30AM +0100, Dean Rasheed wrote: > >> > > >> >Ah sorry, I missed this thread before.

Re: BF failure: could not open relation with OID XXXX while querying pg_views

2019-09-15 Thread Tomas Vondra
On Sun, Sep 15, 2019 at 11:27:19AM +0100, Dean Rasheed wrote: On Sun, 15 Sep 2019 at 11:11, Tomas Vondra wrote: On Sun, Sep 15, 2019 at 10:16:30AM +0100, Dean Rasheed wrote: > >Ah sorry, I missed this thread before. As author of that commit, it's >really on me to fix it, and the cause seems pr

Re: pglz performance

2019-09-15 Thread Oleg Bartunov
On Wed, Sep 4, 2019 at 12:22 PM Andrey Borodin wrote: > > Hi, Peter! Thanks for looking into this. > > > 4 сент. 2019 г., в 14:09, Peter Eisentraut > > написал(а): > > > > On 2019-06-24 10:44, Andrey Borodin wrote: > >>> 18 мая 2019 г., в 11:44, Andrey Borodin написал(а): > >>> > >> Hi! > >> He

Re: [HACKERS] [PROPOSAL] Effective storage of duplicates in B-tree index.

2019-09-15 Thread Oleg Bartunov
On Tue, Sep 1, 2015 at 12:33 PM Alexander Korotkov wrote: > > Hi, Tomas! > > On Mon, Aug 31, 2015 at 6:26 PM, Tomas Vondra > wrote: >> >> On 08/31/2015 09:41 AM, Anastasia Lubennikova wrote: >>> >>> I'm going to begin work on effective storage of duplicate keys in B-tree >>> index. >>> The main

Re: BF failure: could not open relation with OID XXXX while querying pg_views

2019-09-15 Thread Dean Rasheed
On Sun, 15 Sep 2019 at 11:11, Tomas Vondra wrote: > > On Sun, Sep 15, 2019 at 10:16:30AM +0100, Dean Rasheed wrote: > > > >Ah sorry, I missed this thread before. As author of that commit, it's > >really on me to fix it, and the cause seems pretty clear-cut, so I'll > >aim to get that done today. >

Re: [PATCH] Improve performance of NOTIFY over many databases (v2)

2019-09-15 Thread Martijn van Oosterhout
On Sat, 14 Sep 2019 at 17:08, Tom Lane wrote: > Martijn van Oosterhout writes: > > On Fri, 13 Sep 2019 at 22:04, Tom Lane wrote: > >> But, really ... do we need the backendTryAdvanceTail flag at all? > None of this seems to respond to my point: it looks to me like it would > work fine if you si

Re: BF failure: could not open relation with OID XXXX while querying pg_views

2019-09-15 Thread Dean Rasheed
On Sat, 14 Sep 2019 at 05:25, Tom Lane wrote: > > Tomas Vondra writes: > > On Wed, Aug 14, 2019 at 05:24:26PM +1200, Thomas Munro wrote: > >> On Wed, Aug 14, 2019 at 5:06 PM Tom Lane wrote: > >>> Oh, hmm --- yeah, that should mean it's safe. Maybe somebody incautiously > >>> changed one of the

Re: BF failure: could not open relation with OID XXXX while querying pg_views

2019-09-15 Thread Tomas Vondra
On Sun, Sep 15, 2019 at 10:16:30AM +0100, Dean Rasheed wrote: On Sat, 14 Sep 2019 at 05:25, Tom Lane wrote: Tomas Vondra writes: > On Wed, Aug 14, 2019 at 05:24:26PM +1200, Thomas Munro wrote: >> On Wed, Aug 14, 2019 at 5:06 PM Tom Lane wrote: >>> Oh, hmm --- yeah, that should mean it's safe

Re: Efficient output for integer types

2019-09-15 Thread Andrey Borodin
> 15 сент. 2019 г., в 12:18, David Fetter написал(а): > > Please find attached a couple of patches intended to $subject. > > This patch set cut the time to copy ten million rows of randomly sized > int8s (10 of them) by about a third, so at least for that case, it's > pretty decent. Hi! Look

Efficient output for integer types

2019-09-15 Thread David Fetter
Folks, Please find attached a couple of patches intended to $subject. This patch set cut the time to copy ten million rows of randomly sized int8s (10 of them) by about a third, so at least for that case, it's pretty decent. Thanks to Andrew Gierth for lots of patient help. Best, David. -- Dav