Re: [HACKERS] Logical Replication WIP

2017-01-23 Thread Petr Jelinek
On 23/01/17 17:19, Fujii Masao wrote: > On Sat, Jan 21, 2017 at 1:39 AM, Petr Jelinek > <petr.jeli...@2ndquadrant.com> wrote: >> On 20/01/17 17:33, Jaime Casanova wrote: >>> On 20 January 2017 at 11:25, Petr Jelinek <petr.jeli...@2ndquadrant.com> >>>

Re: [HACKERS] Logical Replication WIP

2017-01-22 Thread Petr Jelinek
On 20/01/17 22:30, Petr Jelinek wrote: > Since it's not exactly straight forward to find when these need to be > initialized based on commands, I decided to move the initialization code > to exec_replication_command() since that's always called before anything > so that makes it muc

Re: [HACKERS] Logical replication failing when foreign key present

2017-01-22 Thread Petr Jelinek
> 16408 (PID 19201) exited with exit code 1 > > Hi, thanks for report. Looks like I missed AfterTriggerBeginQuery/AfterTriggerEndQuery when moving the executor stuff around. Attached should fix it. -- Petr Jelinek http://www.2ndQuadrant.com/

Re: [HACKERS] Checksums by default?

2017-01-21 Thread Petr Jelinek
On 21/01/17 18:46, Stephen Frost wrote: > * Petr Jelinek (petr.jeli...@2ndquadrant.com) wrote: >> As we don't know the performance impact is (there was no benchmark done >> on reasonably current code base) I really don't understand how you can >> judge if it's worth it or no

Re: [HACKERS] Checksums by default?

2017-01-21 Thread Petr Jelinek
On 21/01/17 18:15, Stephen Frost wrote: > * Petr Jelinek (petr.jeli...@2ndquadrant.com) wrote: >> On 21/01/17 17:51, Stephen Frost wrote: >>> I'm quite sure that the performance numbers for the CREATE TABLE + COPY >>> case with wal_level=minimal would have been *far* be

Re: [HACKERS] Checksums by default?

2017-01-21 Thread Petr Jelinek
On 21/01/17 17:51, Stephen Frost wrote: > * Petr Jelinek (petr.jeli...@2ndquadrant.com) wrote: >> On 21/01/17 17:31, Stephen Frost wrote: >>> This is just changing the *default*, not requiring checksums to always >>> be enabled. We do not hold the same standard

Re: [HACKERS] Checksums by default?

2017-01-21 Thread Petr Jelinek
On 21/01/17 17:31, Stephen Frost wrote: > Petr, > > * Petr Jelinek (petr.jeli...@2ndquadrant.com) wrote: >> On 21/01/17 16:40, Stephen Frost wrote: >>> * Petr Jelinek (petr.jeli...@2ndquadrant.com) wrote: >>>> On 21/01/17 11:39, Magnus Hagander wrote:

Re: [HACKERS] Checksums by default?

2017-01-21 Thread Petr Jelinek
On 21/01/17 16:40, Stephen Frost wrote: > Petr, > > * Petr Jelinek (petr.jeli...@2ndquadrant.com) wrote: >> On 21/01/17 11:39, Magnus Hagander wrote: >>> Is it time to enable checksums by default, and give initdb a switch to >>> turn it off instead? >> &

Re: [HACKERS] Checksums by default?

2017-01-21 Thread Petr Jelinek
On 21/01/17 11:39, Magnus Hagander wrote: > Is it time to enable checksums by default, and give initdb a switch to > turn it off instead? I'd like to see benchmark first, both in terms of CPU and in terms of produced WAL (=network traffic) given that it turns on logging of hint bits. --

Re: [HACKERS] remote_apply for logical replication?

2017-01-21 Thread Petr Jelinek
_commit = off where flush position is behind. The xinfo & XACT_COMPLETION_APPLY_FEEDBACK does not affect logical replication though as it does not have access to that information (subscriber does not receive raw WAL and logical decoding does not decode this). -- Petr Jelinek http://

Re: [HACKERS] Logical Replication WIP

2017-01-20 Thread Petr Jelinek
On 20/01/17 17:23, Petr Jelinek wrote: > On 20/01/17 15:08, Peter Eisentraut wrote: >> On 1/19/17 5:01 PM, Petr Jelinek wrote: >>> There were some conflicting changes committed today so I rebased the >>> patch on top of them. >>> >>> Other than that not

Re: [HACKERS] Logical Replication WIP

2017-01-20 Thread Petr Jelinek
On 20/01/17 17:33, Jaime Casanova wrote: > On 20 January 2017 at 11:25, Petr Jelinek <petr.jeli...@2ndquadrant.com> > wrote: >> On 20/01/17 17:05, Fujii Masao wrote: >>> On Fri, Jan 20, 2017 at 11:08 PM, Peter Eisentraut >>> <peter.eisentr...@2ndquadrant.co

Re: [HACKERS] Logical Replication WIP

2017-01-20 Thread Petr Jelinek
On 20/01/17 17:05, Fujii Masao wrote: > On Fri, Jan 20, 2017 at 11:08 PM, Peter Eisentraut > <peter.eisentr...@2ndquadrant.com> wrote: >> On 1/19/17 5:01 PM, Petr Jelinek wrote: >>> There were some conflicting changes committed today so I rebased the >>> patc

Re: [HACKERS] Logical Replication WIP

2017-01-20 Thread Petr Jelinek
On 20/01/17 15:08, Peter Eisentraut wrote: > On 1/19/17 5:01 PM, Petr Jelinek wrote: >> There were some conflicting changes committed today so I rebased the >> patch on top of them. >> >> Other than that nothing much has changed, I removed the separate sync >>

Re: [HACKERS] Logical Replication WIP - FailedAssertion, File: "array_typanalyze.c", Line: 340

2017-01-19 Thread Petr Jelinek
fine for me. -- Petr Jelinek http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services diff --git a/src/include/catalog/pg_subscription.h b/src/include/catalog/pg_subscription.h index ccb8880..0ad7b0e 100644 --- a/src/include/catalog/pg_subs

Re: [HACKERS] [PATCH] ALTER DEFAULT PRIVILEGES with GRANT/REVOKE ON SCHEMAS

2017-01-18 Thread Petr Jelinek
know him in > person). Translations are not handled by patch author but by translation project so no need. > > Attached a rebased version and with the docs update pointed by Ashutosh > Sharma. > The patch looks good, the only thing I am missing is tab completion support for psql. -- Petr

Re: [HACKERS] Logical Replication WIP

2017-01-18 Thread Petr Jelinek
On 17/01/17 22:43, Robert Haas wrote: > On Tue, Jan 17, 2017 at 11:15 AM, Petr Jelinek > <petr.jeli...@2ndquadrant.com> wrote: >>> Is there anything stopping anyone from implementing it? >> >> No, just didn't seem priority for the functionality right now. > &g

Re: [HACKERS] Logical replication existing data copy

2017-01-18 Thread Petr Jelinek
On 18/01/17 17:35, Erik Rijkers wrote: > On 2017-01-18 14:46, Petr Jelinek wrote: > >> 0001-Logical-replication-support-for-initial-data-copy-v2.patch > > Applies and builds fine on top of the previous 5 patches. > > Two problems: > > 1. alter_subscription.sg

Re: [HACKERS] Logical Replication WIP

2017-01-17 Thread Petr Jelinek
On 17/01/17 17:11, Peter Eisentraut wrote: > On 1/15/17 1:48 PM, Petr Jelinek wrote: >> It's meant to decouple the synchronous commit setting for logical >> replication workers from the one set for normal clients. Now that we >> have owners for subscription and subscripti

Re: [HACKERS] Logical Replication WIP

2017-01-17 Thread Petr Jelinek
oblem. >> >> Because we don't have ALTER SUBSCRIPTION RENAME currently. Maybe should >> be Assert? > > Is there anything stopping anyone from implementing it? > No, just didn't seem priority for the functionality right now. -- Petr Jelinek http://www.2n

Re: [HACKERS] Logical Replication WIP

2017-01-15 Thread Petr Jelinek
On 15/01/17 23:57, Erik Rijkers wrote: > On 2017-01-15 23:20, Petr Jelinek wrote: > >> 0001-Add-PUBLICATION-catalogs-and-DDL-v18.patch >> 0002-Add-SUBSCRIPTION-catalog-and-DDL-v18.patch >> 0003-Define-logical-replication-protocol-and-output-plugi-v18.patch >> 0004-A

Re: [HACKERS] Logical Replication WIP

2017-01-15 Thread Petr Jelinek
On 15/01/17 20:20, Euler Taveira wrote: > On 15-01-2017 15:13, Petr Jelinek wrote: >> I am not quite sure I agree with this. Either it's system object and we >> don't replicate it (which I would have considered to be anything with >> Oid < FirstNormalObjectId) or it's user

Re: [HACKERS] Logical Replication WIP

2017-01-15 Thread Petr Jelinek
tion is not confirmed by responding to COMMIT but by reporting LSNs of various replication stages. Perhaps the logical_replication_synchronous_commit should only be boolean that would translate to 'off' and 'local' for the real synchronous_commit. -- Petr Jelinek http://www.2ndQuadran

Re: [HACKERS] Logical Replication WIP

2017-01-15 Thread Petr Jelinek
On 11/01/17 22:30, Peter Eisentraut wrote: > On 1/11/17 3:35 PM, Petr Jelinek wrote: >> On 11/01/17 18:32, Peter Eisentraut wrote: >>> On 1/11/17 3:29 AM, Petr Jelinek wrote: >>>> Okay, looking into my notes, I originally did this because we did not >>

Re: [HACKERS] Logical Replication WIP

2017-01-11 Thread Petr Jelinek
On 11/01/17 18:27, Peter Eisentraut wrote: > On 1/11/17 3:11 AM, Petr Jelinek wrote: >> That will not help, issue is that we consider names for origins to be >> unique across cluster while subscription names are per database so if >> there is origin per subscrip

Re: [HACKERS] Logical Replication WIP

2017-01-11 Thread Petr Jelinek
On 11/01/17 18:32, Peter Eisentraut wrote: > On 1/11/17 3:29 AM, Petr Jelinek wrote: >> Okay, looking into my notes, I originally did this because we did not >> allow adding tables without pkeys to publications which effectively >> prohibited FOR ALL TABLES publication

Re: [HACKERS] Logical Replication WIP

2017-01-11 Thread Petr Jelinek
On 10/01/17 15:06, Peter Eisentraut wrote: > On 1/3/17 5:23 PM, Petr Jelinek wrote: >> I got this remark about IsCatalogClass() from Andres offline as well, >> but it's not true, it only checks for FirstNormalObjectId for objects in >> pg_catalog and toast schemas, not anyw

Re: [HACKERS] Logical Replication WIP

2017-01-11 Thread Petr Jelinek
On 10/01/17 14:52, Peter Eisentraut wrote: > On 1/2/17 8:32 AM, Petr Jelinek wrote: >> On 02/01/17 05:23, Steve Singer wrote: >>> but I can't drop the subscription either >>> >>> >>> test_b=# drop subscription mysub; >>> ERROR: could not d

Re: [HACKERS] Logical Replication WIP

2017-01-04 Thread Petr Jelinek
ill apply fine on top of it. -- Petr Jelinek http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services 0001-Add-PUBLICATION-catalogs-and-DDL-v17.patch.gz Description: application/gzip -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgr

Re: [HACKERS] Logical Replication WIP

2017-01-03 Thread Petr Jelinek
it? > I am actually thinking this should correspond to MAXNUMMESSAGES (4096) as that's the limit on buffer size. I didn't find it the first time around when I was looking for good number. -- Petr Jelinek http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7

Re: [HACKERS] Logical Replication WIP

2017-01-02 Thread Petr Jelinek
On 02/01/17 05:23, Steve Singer wrote: > On 12/30/2016 05:53 AM, Petr Jelinek wrote: >> Hi, >> >> I rebased this for the changes made to inheritance and merged in the >> fixes that I previously sent separately. >> >> >> > > > I'm not

Re: [HACKERS] multivariate statistics (v19)

2016-12-30 Thread Petr Jelinek
t; + > +void > +update_mv_stats(Oid mvoid, MVNDistinct ndistinct, > + int2vector *attrs, VacAttrStats **stats) ... > +static double > +ndistinct_for_combination(double totalrows, int numrows, HeapTuple *rows, > +int2vector *attrs, VacAttrSt

Re: [HACKERS] multivariate statistics (v19)

2016-12-30 Thread Petr Jelinek
invasive than I'm > comfortable with (but maybe that'd be OK). > In theory it seems like possibly reasonable approach to me, mainly because mv statistics are user defined objects. I guess we'd have to see at least some PoC to see how invasive it is. But I ultimately think that fee

Re: [HACKERS] [PATCH] Fix minor race in commit_ts SLRU truncation vs lookups

2016-12-28 Thread Petr Jelinek
unc.oldestXid); This should probably say oldestXid instead of xid in the text description. About back-patching, I wonder if standby could be modified to move oldestXid based on pageno reverse calculation, but it's going to be slightly ugly. -- Petr Jelinek http://www.2ndQua

Re: [HACKERS] Logical decoding on standby

2016-12-20 Thread Petr Jelinek
On 21/12/16 04:06, Craig Ringer wrote: > On 20 December 2016 at 15:03, Petr Jelinek <petr.jeli...@2ndquadrant.com> > wrote: > >>> The biggest change in this patch, and the main intrusive part, is that >>> procArray->replication_slot_catalog_xmin is no longer

Re: [HACKERS] Logical decoding on standby

2016-12-20 Thread Petr Jelinek
On 21/12/16 04:31, Craig Ringer wrote: > On 20 December 2016 at 15:03, Petr Jelinek <petr.jeli...@2ndquadrant.com> > wrote: > >> But in 0003 I don't understand following code: >>> + if (endpos != InvalidXLogRecPtr && !do_start_slot) >&g

Re: [HACKERS] Logical Replication WIP

2016-12-20 Thread Petr Jelinek
On 20/12/16 10:56, Erik Rijkers wrote: > On 2016-12-20 10:48, Petr Jelinek wrote: > > Here is another small thing: > > $ psql -d testdb -p 6972 > psql (10devel_logical_replication_20161220_1008_db80acfc9d50) > Type "help" for help. > > testdb=# dr

Re: [HACKERS] Logical Replication WIP

2016-12-20 Thread Petr Jelinek
On 20/12/16 10:41, Erik Rijkers wrote: > On 2016-12-20 09:43, Petr Jelinek wrote: > >> Thanks, this was very useful. We had wrong attribute index arithmetics >> in the place where we verify that replica identities match well enough. > > Well, I spent a lot of time on the

Re: [HACKERS] Logical decoding on standby

2016-12-19 Thread Petr Jelinek
option to specify behavior). And the last one 0009 is the implementation discussed above, which I think needs rework. IMHO 0007 and 0009 should be ultimately merged. I think parts of this could be committed separately and are ready for committer IMHO, but there is no way in CF application to mark only part of patc

Re: [HACKERS] Logical Replication WIP

2016-12-19 Thread Petr Jelinek
On 19/12/16 15:39, Steve Singer wrote: > On 12/18/2016 09:04 PM, Petr Jelinek wrote: >> On 18/12/16 19:02, Steve Singer wrote: >> >>> pg_dump is also generating warnings >>> >>> pg_dump: [archiver] WARNING: don't know how to set owner for object type >

Re: [HACKERS] Logical replication existing data copy

2016-12-19 Thread Petr Jelinek
On 19/12/16 18:25, Peter Eisentraut wrote: > On 12/19/16 4:30 AM, Petr Jelinek wrote: >> So existing table data can be copied once subscription is created, but >> also new tables can be added and their data copied. This is where the >> REFRESH PUBLICATION comes in

Re: [HACKERS] Logical Replication WIP

2016-12-19 Thread Petr Jelinek
On 19/12/16 08:04, Erik Rijkers wrote: > On 2016-12-18 11:12, Petr Jelinek wrote: > > (now using latest: patchset:) > > 0001-Add-PUBLICATION-catalogs-and-DDL-v14.patch > 0002-Add-SUBSCRIPTION-catalog-and-DDL-v14.patch > 0003-Define-logical-replication-protocol-and-output-p

Re: [HACKERS] Logical Replication WIP

2016-12-18 Thread Petr Jelinek
On 17/12/16 18:34, Steve Singer wrote: > On 12/16/2016 07:49 AM, Petr Jelinek wrote: >> Hi, >> >> attached is version 13 of the patch. >> >> I merged in changes from PeterE. And did following changes: >> - fixed the ownership error messages for both pro

Re: [HACKERS] Logical Replication WIP

2016-12-18 Thread Petr Jelinek
On 17/12/16 13:37, Erik Rijkers wrote: > On 2016-12-16 13:49, Petr Jelinek wrote: >> >> version 13 of the patch. >> >> 0001-Add-PUBLICATION-catalogs-and-DDL-v13.patch.gz (~32 KB) >> 0002-Add-SUBSCRIPTION-catalog-and-DDL-v13.patch.gz (~28 KB) >> 0003-Define

Re: [HACKERS] pg_basebackups and slots

2016-12-16 Thread Petr Jelinek
ing. I'm > willing to say there are *many* more who benefit from an automatic slot. > I think the issue with slots taking over disk space should be fixed by making it possible to cut off slots when necessary (ie some GUC that would say how much behind slot can be at maximum, with something like

Re: [HACKERS] Logical Replication WIP

2016-12-15 Thread Petr Jelinek
On 15/12/16 13:06, Craig Ringer wrote: > On 15 Dec. 2016 18:19, "Petr Jelinek" <petr.jeli...@2ndquadrant.com > <mailto:petr.jeli...@2ndquadrant.com>> wrote: > > On 13/12/16 21:42, Peter Eisentraut wrote: > > On 12/10/16 2:48 AM, Petr Jelinek wrote:

Re: [HACKERS] Logical Replication WIP

2016-12-15 Thread Petr Jelinek
On 13/12/16 21:42, Peter Eisentraut wrote: > On 12/10/16 2:48 AM, Petr Jelinek wrote: >> Attached new version with your updates and rebased on top of the current >> HEAD (the partitioning patch produced quite a few conflicts). > > I have attached a few more "fixup&

Re: [HACKERS] Logical Replication WIP

2016-12-15 Thread Petr Jelinek
On 13/12/16 01:33, Andres Freund wrote: > > On 2016-12-12 09:18:48 -0500, Peter Eisentraut wrote: >> On 12/8/16 4:10 PM, Petr Jelinek wrote: >>> On 08/12/16 20:16, Peter Eisentraut wrote: >>>> On 12/6/16 11:58 AM, Peter Eisentraut wrote: >>>>>

Re: [HACKERS] Logical Replication WIP

2016-12-13 Thread Petr Jelinek
On 13/12/16 21:42, Peter Eisentraut wrote: > On 12/10/16 2:48 AM, Petr Jelinek wrote: >> Attached new version with your updates and rebased on top of the current >> HEAD (the partitioning patch produced quite a few conflicts). > > I have attached a few more "fixup&

Re: [HACKERS] Logical Replication WIP

2016-12-13 Thread Petr Jelinek
e filtering so the publications still have to reset relcache too. -- Petr Jelinek http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscrip

Re: [HACKERS] Logical Replication WIP

2016-12-13 Thread Petr Jelinek
utput something? Right now this doesn't test that >> much... > > This test was added because an earlier version of the patch would crash > on this. > I did improve the test as part of the tests improvements that were sent to committers list btw. -- Petr Jelinek h

Re: [HACKERS] Logical Replication WIP

2016-12-12 Thread Petr Jelinek
On 13/12/16 03:26, Petr Jelinek wrote: > On 13/12/16 02:41, Andres Freund wrote: >> On 2016-12-10 08:48:55 +0100, Petr Jelinek wrote: >> >>> +static List * >>> +OpenTableList(List *tables) >>> +{ >>> + List *relids = NIL; >

Re: [HACKERS] background sessions

2016-12-12 Thread Petr Jelinek
ode just to call heap_multi_insert() or whatever, do you? > You can hand-roll whatever you need there. > You do if source of your data is already in COPY (or csv) format. I do have similar usecase in logical replication followup patch that I plan to submit to Jan CF, so maybe that will be int

Re: [HACKERS] Logical Replication WIP

2016-12-12 Thread Petr Jelinek
On 13/12/16 02:41, Andres Freund wrote: > On 2016-12-10 08:48:55 +0100, Petr Jelinek wrote: > >> diff --git a/src/backend/catalog/pg_publication.c >> b/src/backend/catalog/pg_publication.c >> new file mode 100644 >> index 000..e3560b7 >> --- /d

Re: [HACKERS] PATCH: two slab-like memory allocators

2016-12-12 Thread Petr Jelinek
On 13/12/16 01:45, Tomas Vondra wrote: > On 12/12/2016 11:39 PM, Tomas Vondra wrote: >> On 12/12/2016 05:05 AM, Petr Jelinek wrote: >>> >>> I'd be happy with this patch now (as in committer ready) except that it >>> does have some merge conflicts after the rec

Re: [HACKERS] Logical Replication WIP

2016-12-12 Thread Petr Jelinek
On 13/12/16 01:33, Andres Freund wrote: > HJi, > > On 2016-12-12 09:18:48 -0500, Peter Eisentraut wrote: >> On 12/8/16 4:10 PM, Petr Jelinek wrote: >>> On 08/12/16 20:16, Peter Eisentraut wrote: >>>> On 12/6/16 11:58 AM, Peter Eisentraut wrote: >>

Re: [HACKERS] snapbuild woes

2016-12-12 Thread Petr Jelinek
On 12/12/16 23:33, Andres Freund wrote: > On 2016-12-12 23:27:30 +0100, Petr Jelinek wrote: >> On 12/12/16 22:42, Andres Freund wrote: >>> Hi, >>> >>> On 2016-12-10 23:10:19 +0100, Petr Jelinek wrote: >>>> Hi, >>>> First on

Re: [HACKERS] snapbuild woes

2016-12-12 Thread Petr Jelinek
On 12/12/16 22:42, Andres Freund wrote: > Hi, > > On 2016-12-10 23:10:19 +0100, Petr Jelinek wrote: >> Hi, >> First one is outright bug, which has to do with how we track running >> transactions. What snapbuild basically does while doing initial snapshot >> is

Re: [HACKERS] PATCH: two slab-like memory allocators

2016-12-11 Thread Petr Jelinek
On 01/12/16 03:26, Tomas Vondra wrote: > > Dne 11/27/2016 v 11:02 PM Andres Freund napsal(a): >> On 2016-11-27 22:21:49 +0100, Petr Jelinek wrote: >>> On 27/11/16 21:47, Andres Freund wrote: >>>> Hi, >>>> >>>>>> +typedef stru

Re: [HACKERS] snapbuild woes

2016-12-10 Thread Petr Jelinek
On 10/12/16 23:10, Petr Jelinek wrote: > > The attached 0002-Skip-unnecessary-snapshot-builds.patch changes this > behavior so that we don't make snapshots for transactions that we seen > wholly and know that they didn't make catalog changes even if we didn't > reach consist

[HACKERS] snapbuild woes

2016-12-10 Thread Petr Jelinek
-builds.patch changes this behavior so that we don't make snapshots for transactions that we seen wholly and know that they didn't make catalog changes even if we didn't reach consistency yet. -- Petr Jelinek http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training

Re: [HACKERS] Logical Replication WIP

2016-12-09 Thread Petr Jelinek
commit-control-for-logical--v11.patch # PJ > > Could (one of) you give me a hint? > I just sent in a rebased patch that includes all of it. -- Petr Jelinek http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services -- Se

Re: [HACKERS] Logical Replication WIP

2016-12-08 Thread Petr Jelinek
On 08/12/16 20:16, Peter Eisentraut wrote: > On 12/6/16 11:58 AM, Peter Eisentraut wrote: >> On 12/5/16 6:24 PM, Petr Jelinek wrote: >>> I think that the removal of changes to ReplicationSlotAcquire() that you >>> did will result in making it impossible to reacquire

Re: [HACKERS] Logical Replication WIP

2016-12-05 Thread Petr Jelinek
lotAcquire() that you did will result in making it impossible to reacquire temporary slot once you switched to different one in the session as the if (active_pid != 0) will always be true for temp slot. -- Petr Jelinek http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Su

Re: [HACKERS] Logical Replication WIP

2016-12-02 Thread Petr Jelinek
On 02/12/16 02:55, Thomas Munro wrote: > On Fri, Dec 2, 2016 at 2:32 PM, Peter Eisentraut > <peter.eisentr...@2ndquadrant.com> wrote: >> On 11/30/16 8:06 PM, Petr Jelinek wrote: >>> On 30/11/16 22:37, Peter Eisentraut wrote: >>>> I have taken the libpq

Re: [HACKERS] Logical Replication WIP

2016-11-30 Thread Petr Jelinek
good to me, do you plan to commit this soon or would you rather me to resubmit the patches rebased on top of this (and including this) first? -- Petr Jelinek http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services -- Sent via pgsql-hacke

Re: [HACKERS] Tackling JsonPath support

2016-11-29 Thread Petr Jelinek
On 29/11/16 17:28, Nico Williams wrote: > On Tue, Nov 29, 2016 at 05:18:17PM +0100, Petr Jelinek wrote: >> Just to add to this, the SQL/JSON proposals I've seen so far, and what >> Oracle, MSSQL and Teradata chose to implement already is basically >> subset of jso

Re: [HACKERS] Tackling JsonPath support

2016-11-29 Thread Petr Jelinek
mplementations also include lax/strict prefix keyword on top of that). I think that should give us some hint on what the base functionality should look like. -- Petr Jelinek http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services

Re: [HACKERS] Tackling JsonPath support

2016-11-28 Thread Petr Jelinek
, there is not much difference between those except contrib is included in distribution. Extensions that provide just functions are easy to integrate into core (that's how some of the existing json functions were added in the past as well). -- Petr Jelinek http://www.2ndQuadrant.com

Re: [HACKERS] Logical Replication WIP

2016-11-27 Thread Petr Jelinek
On 27/11/16 23:42, Erik Rijkers wrote: > On 2016-11-27 19:57, Petr Jelinek wrote: >> On 22/11/16 18:42, Erik Rijkers wrote: >>> On 2016-11-20 19:02, Petr Jelinek wrote: >>> >>>> 0001-Add-support-for-TE...cation-slots-v8.patch.gz (~8 KB) >>>>

Re: [HACKERS] PATCH: two slab-like memory allocators

2016-11-27 Thread Petr Jelinek
ed -> or . since you don't know whether > it's a pointer or the actual object. All to save a * in parameter and > variable declaration?... > FWIW I don't like that pattern either although it's used in many parts of our code-base. -- Petr Jelinek http://www.2ndQuadrant

Re: [HACKERS] Logical decoding on standby

2016-11-27 Thread Petr Jelinek
deeper detail as this is still very WIP. I go agree with the general design though. This also replaces the previous timeline following and decoding threads/CF entries so maybe those should be closed in CF? -- Petr Jelinek http://www.2ndQuadrant.com/ PostgreSQL Developm

Re: [HACKERS] Logical Replication WIP

2016-11-27 Thread Petr Jelinek
On 22/11/16 18:42, Erik Rijkers wrote: > On 2016-11-20 19:02, Petr Jelinek wrote: > >> 0001-Add-support-for-TE...cation-slots-v8.patch.gz (~8 KB) >> 0002-Refactor-libpqwalreceiver-v8.patch.gz (~9 KB) >> 0003-Add-PUBLICATION-catalogs-and-DDL-v8.patch.gz (~30 KB) >>

Re: [HACKERS] PATCH: two slab-like memory allocators

2016-11-27 Thread Petr Jelinek
p s0start: SELECT data FROM pg_logical_slot_get_changes('isolation_slot', NULL, NULL, 'include-xids', 'false'); data Also, let's just rename the Gen to Generation. -- Petr Jelinek http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Servic

Re: [HACKERS] Logical Replication WIP

2016-11-20 Thread Petr Jelinek
On 13/11/16 23:02, Steve Singer wrote: > On 10/31/2016 06:38 AM, Petr Jelinek wrote: >> On 31/10/16 00:52, Steve Singer wrote: >> There are some fundamental issues with initial sync that need to be >> discussed on list but this one is not known. I'll try to convert this >&g

Re: [HACKERS] Logical Replication WIP

2016-11-14 Thread Petr Jelinek
ppens). And that won't change if we convert this into array. Besides, shouldn't the code then anyway check that we only have expected data in that array otherwise we might miss corruption? So if the main reason for turning this into array is extendability for other providers then I am -1

Re: [HACKERS] Logical Replication WIP

2016-11-12 Thread Petr Jelinek
On 12/11/16 20:19, Andres Freund wrote: > On 2016-11-10 23:31:27 +0100, Petr Jelinek wrote: >> On 04/11/16 13:15, Andres Freund wrote: >>> >>> /* Prototypes for private functions */ >>> -static bool libpq_select(int timeout_ms); >>>

Re: [HACKERS] Why PostgreSQL doesn't implement a semi sync replication?

2016-11-11 Thread Petr Jelinek
servers die (or are in problems)" than "we lose data when 1 server dies" and it's also more acceptable to say "we stop answering when we lose 2 servers" but not "we stop answering when we lose 1 server", and semisync replication works for combination of these tw

Re: [HACKERS] Logical Replication WIP

2016-11-11 Thread Petr Jelinek
context management. But RelationData appears to favor rich data > structures, so maybe that is fine. > Thanks for these, some of it is result of various rebases that I did (the sync patch makes rebasing bit complicated as it touches everything) and it's easy for me to overlook it at this poin

Re: [HACKERS] Logical Replication WIP

2016-11-11 Thread Petr Jelinek
here as that would make it impossible to drop > + * subscription when publisher is down. > + */ > > I'm not convinced. Leaving a slot around without a "record" of it on > the creating side isn't nice either. Maybe a FORCE flag or something? > I would

Re: [HACKERS] Logical Replication WIP

2016-11-11 Thread Petr Jelinek
ike it should be in initial version. > I also wonder if we want an easier to > extend form of pubinsert/update/delete (say to add pubddl, pubtruncate, > pub ... without changing the schema). > So like, text array that's then parsed everywhere (I am not doing bitmask/int definitely)? >

Re: [HACKERS] Logical Replication WIP

2016-11-11 Thread Petr Jelinek
mainly because it complicates the interaction with ReplicationSlotDropPtr and it seems to me that locking there is not really needed there as other backends will never change active_pid to our backend pid and then the ReplicationSlotDropPtr does exclusive lock when resetting it. -- Petr Jelin

Re: [HACKERS] Logical Replication WIP

2016-11-10 Thread Petr Jelinek
perspective but I don't have better idea here. -- Petr Jelinek http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Logical Replication WIP

2016-11-03 Thread Petr Jelinek
On 02/11/16 17:22, Peter Eisentraut wrote: > On 10/24/16 9:22 AM, Petr Jelinek wrote: >> I added one more prerequisite patch (the first one) which adds ephemeral >> slots (or well implements UI on top of the code that was mostly already >> there). The ephemeral slots are diff

Re: [HACKERS] Fast Default WIP patch for discussion

2016-10-26 Thread Petr Jelinek
or complex) patches or patches where there is code cooperation between multiple people. Wiki page for patch sometimes contains basically what your first email said. BTW it's customary to bottom post, not top post on this list. -- Petr Jelinek http://www.2ndQuadrant.com/ Postgr

Re: [HACKERS] [COMMITTERS] pgsql: Remove extra comma at end of enum list

2016-10-24 Thread Petr Jelinek
>> rule of // for line-ending comments and /* for all other uses. We'd >> have to teach pgindent about that, and I dunno how hard that is. > > I'm not sure it's worth the resulting mess. > I think the current commenting style stylistically nicer. -- Petr Jelinek

Re: [HACKERS] [PATCH] Logical decoding timeline following take II

2016-10-24 Thread Petr Jelinek
should be committed. Yeah as I said above I split out the test harness for this reason. The good thing is that when the followup patches get in the test harness should be easy to removed as the changes are very localized. -- Petr Jelinek http://www.2ndQuadrant.com/ PostgreSQL

Re: [HACKERS] [PATCH] Send catalog_xmin separately in hot standby feedback

2016-10-24 Thread Petr Jelinek
don't retain it. I could also convert the current boolean > param ignoreVacuum into a flags argument instead of adding another > boolean. No real preference from me. > I would honestly prefer the change to GetOldestXmin to return the catalog_xmin. It seems both cleaner and does less

Re: [HACKERS] PATCH: two slab-like memory allocators

2016-10-23 Thread Petr Jelinek
ice, I'd prefer proper freeing to 5% performance gain as I seen walsenders taking GBs of memory dues to reoderbuffer allocations that are never properly freed. About your actual patch. I do like both the Slab and the Gen allocators and think that we should proceed with them for the moment. Yo

Re: [HACKERS] Indirect indexes

2016-10-20 Thread Petr Jelinek
On 20/10/16 17:24, Bruce Momjian wrote: > On Thu, Oct 20, 2016 at 05:14:51PM +0200, Petr Jelinek wrote: >>> Also, it seems indirect indexes would be useful for indexing columns >>> that are not updated frequently on tables that are updated frequently, >>> and wh

Re: [HACKERS] Indirect indexes

2016-10-20 Thread Petr Jelinek
s not have performance problem by 1% is very different from helping user who has performance problem by 50% even if she needs to think about the solution a bit. WARM can do WARM update 50% of time, indirect index can do HOT update 100% of time (provided the column is not changed), I do

Re: [HACKERS] PATCH: two slab-like memory allocators

2016-10-18 Thread Petr Jelinek
ptimized algorithm for fixed sized objects in our current allocator. The advantage of that would be that we could for example use that for things like ListCell easily (memory management of which I see quite often in profiles). -- Petr Jelinek http://www.2ndQuadrant.com/ Pos

Re: [HACKERS] logical replication connection information management

2016-10-14 Thread Petr Jelinek
Yes, superuser in the beginning, especially for subscriptions as it will be bit harder to do proper checks without loss of performance. The good part is that if we do superuser initially we can always add some GRANT or ROLE later to lift the limitation so we don't have to build the whole role m

Re: [HACKERS] autonomous transactions

2016-10-12 Thread Petr Jelinek
On 11/10/16 21:54, Merlin Moncure wrote: > On Tue, Oct 11, 2016 at 10:06 AM, Petr Jelinek <p...@2ndquadrant.com> wrote: >> On 10/10/16 16:44, Merlin Moncure wrote: >>> On Thu, Oct 6, 2016 at 3:53 PM, Simon Riggs <si...@2ndquadrant.com> wrote: >>>> On 6 Oc

Re: [HACKERS] autonomous transactions

2016-10-11 Thread Petr Jelinek
uming we >> would ever agree that just one exists. > > -1 on implementing both autonomous and background transactions. This > will confuse everyone. > I personally care much more about having background transactions than autonomous ones (as I only ever had use-cases f

Re: [HACKERS] autonomous transactions

2016-10-06 Thread Petr Jelinek
gest we commit Background Transactions first and then in a > fairly thin commit, implement Autonomous Transactions on top of it for > now and if we get a better one, switch it over. > +1 to this -- Petr Jelinek http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support,

Re: [HACKERS] Declarative partitioning - another take

2016-10-04 Thread Petr Jelinek
. > You wouldn't like it if I told you to document "note that if a > partition being detached is unlogged, it will still be unlogged". > I think this is bit different though, it basically means you are detaching whole branch from the rest of the partitioning tree, not just tha

Re: [HACKERS] PATCH: two slab-like memory allocators

2016-10-04 Thread Petr Jelinek
y GenSlab directly instead of using child contexts. Then all the calls would go to GenSlab which could decide what to do (and move the chunks between the allocators). But honestly given the usecases for GenSlab, I would at the moment prefer just to have predictable error as it can be done more cleanly and n

Re: [HACKERS] Handling dropped attributes in pglogical_proto

2016-09-28 Thread Petr Jelinek
ogical). Also the attmap should never contain attisdropped attributes. -- Petr Jelinek http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subsc

Re: [HACKERS] psql casts aspersions on server reliability

2016-09-28 Thread Petr Jelinek
gt; message even though the server had been shut down quite cleanly. The >> message got into a logfile and induced minor panic. Fortunately, I >> have not seen this happen lately. > > +1 for making this error message less frightening. I have also had to > explain it away on occasion. > +1 I've seen this being misleading way too often. -- Petr Jelinek http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

<    1   2   3   4   5   6   7   8   9   10   >