Re: [HACKERS] walsender & parallelism

On 25/04/17 01:25, Andres Freund wrote: > Hi, > > On 2017-04-24 07:31:18 +0200, Petr Jelinek wrote: >> The previous coding tried to run the unknown string throur lexer which >> could fail for some valid SQL statements as the replication command >> parser

Re: [HACKERS] snapbuild woes

On 25/04/17 00:59, Andres Freund wrote: > Hi, > > On 2017-04-15 05:18:49 +0200, Petr Jelinek wrote: >> Hi, here is updated patch (details inline). > > I'm not yet all that happy, sorry: > > Looking at 0001: > - GetOldestSafeDecodingTransactionId() only guaran

Re: [HACKERS] walsender & parallelism

On 24/04/17 20:00, Andres Freund wrote: > On 2017-04-24 18:29:51 +0200, Petr Jelinek wrote: >> On 24/04/17 07:42, Andres Freund wrote: >>> >>> >>> On April 23, 2017 10:31:18 PM PDT, Petr Jelinek >>> <petr.jeli...@2ndquadrant.com> wrote: >>&

Re: [HACKERS] Interval for launching the table sync worker

UBREL_STATE_SYNCWAIT to SUBREL_STATE_READY in a hash_seq_search loop > the table sync worker which is changed to SUBREL_STATE_READY by the > apply worker before updating the subrel_local_state could be remained > in the hash table. I think that we should scan pg_subscription_rel by &

Re: [HACKERS] walsender & parallelism

On 24/04/17 07:42, Andres Freund wrote: > > > On April 23, 2017 10:31:18 PM PDT, Petr Jelinek > <petr.jeli...@2ndquadrant.com> wrote: >> On 24/04/17 04:31, Petr Jelinek wrote: >> So actually maybe running regression tests through it might be >> reasonabl

Re: [HACKERS] walsender & parallelism

want to rerun whole regression suite against walsender given the time it would take in normal tests (although I could see doing that optionally somehow) but then what to pick from there. -- Petr Jelinek http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training &a

Re: [HACKERS] walsender & parallelism

6b2df48db4bf1738aece5551f7a47 > Okay, but why call both SetLatch and latch_sigusr1_handler? What does that buy us? -- Petr Jelinek http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services -- Sent via pgsql-hackers mailing list (pgsql-h

Re: [HACKERS] walsender & parallelism

On 24/04/17 01:59, Andres Freund wrote: > Hi, > > On 2017-04-22 17:53:19 +0200, Petr Jelinek wrote: >> Here is patch. I changed both SIGINT and SIGUSR1 handlers, afaics it >> does not break anything for existing walsender usage. > >> diff --git a/src/backend/replic

Re: [HACKERS] valgrind error in subscription code

On 22/04/17 21:16, Andres Freund wrote: > Hi, > > On 2017-04-22 21:08:18 +0200, Petr Jelinek wrote: >> Thanks, here is patch to fix that - I also removed the individual >> settings of everything to NULL/0/InvalidOid etc and just replaced it all >> with memset. >

Re: [HACKERS] logical replication and PANIC during shutdown checkpoint in publisher

On 21/04/17 06:11, Michael Paquier wrote: > On Fri, Apr 21, 2017 at 12:29 AM, Peter Eisentraut > <peter.eisentr...@2ndquadrant.com> wrote: >> On 4/20/17 07:52, Petr Jelinek wrote: >>> On 20/04/17 05:57, Michael Paquier wrote: >>>> 2nd thought

[HACKERS] Remove dead interfaces added by mistake in 7c4f52409

Hi, Seems like couple of declarations for couple of functions we never actually implemented and are not used got past review of logical replication support for initial copy path (commit 7c4f52409). Attached patch gets rid of them. -- Petr Jelinek http://www.2ndQuadrant.com

[HACKERS] Time based lag tracking for logical replication

not implement it at all. This seems like cleaner way of doing it. Thoughts? -- Petr Jelinek http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services Add-support-for-time-based-lag-tracking-to-logical-r.patch Description: binary/octet-stream --

Re: [HACKERS] tablesync patch broke the assumption that logical rep depends on?

On 21/04/17 16:31, Petr Jelinek wrote: > On 21/04/17 16:23, Peter Eisentraut wrote: >> On 4/21/17 10:11, Petr Jelinek wrote: >>> On 21/04/17 16:09, Peter Eisentraut wrote: >>>> On 4/20/17 14:29, Petr Jelinek wrote: >>>>> + /* Find unused w

Re: [HACKERS] valgrind error in subscription code

atch to fix that - I also removed the individual settings of everything to NULL/0/InvalidOid etc and just replaced it all with memset. -- Petr Jelinek http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services From 4993487f5e8750b708e76181bb78e

Re: [HACKERS] walsender & parallelism

On 21/04/17 04:37, Petr Jelinek wrote: > On 21/04/17 04:32, Craig Ringer wrote: >> On 21 April 2017 at 10:20, Petr Jelinek <petr.jeli...@2ndquadrant.com> wrote: >>> On 21/04/17 03:40, Andres Freund wrote: >>>> >>>> Since [1] walsender (not recei

Re: [HACKERS] tablesync patch broke the assumption that logical rep depends on?

On 21/04/17 16:23, Peter Eisentraut wrote: > On 4/21/17 10:11, Petr Jelinek wrote: >> On 21/04/17 16:09, Peter Eisentraut wrote: >>> On 4/20/17 14:29, Petr Jelinek wrote: >>>> + /* Find unused worker slot. */ >>>> + if (!w->in_use) >

Re: [HACKERS] tablesync patch broke the assumption that logical rep depends on?

On 21/04/17 16:09, Peter Eisentraut wrote: > On 4/20/17 14:29, Petr Jelinek wrote: >> +/* Find unused worker slot. */ >> +if (!w->in_use) >> { >> -worker = >workers[slot]; >> -br

Re: [HACKERS] Interval for launching the table sync worker

On 21/04/17 10:33, Kyotaro HORIGUCHI wrote: > Hello, > > At Thu, 20 Apr 2017 13:21:14 +0900, Masahiko Sawada > wrote in

Re: [HACKERS] Interval for launching the table sync worker

On 21/04/17 04:38, Masahiko Sawada wrote: > On Thu, Apr 20, 2017 at 8:43 PM, Petr Jelinek > <petr.jeli...@2ndquadrant.com> wrote: >> On 20/04/17 06:21, Masahiko Sawada wrote: >>> On Thu, Apr 20, 2017 at 12:30 AM, Petr Jelinek >>> <petr.jeli...@2ndquad

Re: [HACKERS] walsender & parallelism

On 21/04/17 04:32, Craig Ringer wrote: > On 21 April 2017 at 10:20, Petr Jelinek <petr.jeli...@2ndquadrant.com> wrote: >> On 21/04/17 03:40, Andres Freund wrote: >>> >>> Since [1] walsender (not receiver as commit message says) can execute >>> SQL q

Re: [HACKERS] DROP SUBSCRIPTION, query cancellations and slot handling

On 20/04/17 23:30, Peter Eisentraut wrote: > On 4/20/17 10:19, Petr Jelinek wrote: >> Hmm well since this only affects the synchronization of table >> states/names, I guess we could just simply do that before we create the >> slot as there is no expectancy of consistency betwe

Re: [HACKERS] walsender & parallelism

ugh (they both end up calling sendSelfPipeByte() eventually)? -- Petr Jelinek http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription:

Re: [HACKERS] tablesync patch broke the assumption that logical rep depends on?

On 20/04/17 18:58, Peter Eisentraut wrote: > On 4/18/17 22:13, Petr Jelinek wrote: >> So my idea was to add some kind of inuse flag. This turned out to be bit >> more complicated in terms of how to clean it than I would have hoped. >> This is due to the fact that there is no

Re: [HACKERS] DROP SUBSCRIPTION, query cancellations and slot handling

On 20/04/17 14:41, Michael Paquier wrote: > On Thu, Apr 20, 2017 at 8:47 PM, Petr Jelinek > <petr.jeli...@2ndquadrant.com> wrote: >> Or you can drop the slot manually on upstream. > > Sure, but the point here is that if for example users have > client_min_messa

Re: [HACKERS] Use sync commit for logical replication apply in TAP tests

On 20/04/17 14:57, Peter Eisentraut wrote: > On 4/18/17 22:25, Petr Jelinek wrote: >> The commit 887227a1c changed the defaults for subscriptions to do async >> commit. But since the tests often wait for disk flush and there is no >> concurrent activity this has increased the

Re: [HACKERS] logical replication and PANIC during shutdown checkpoint in publisher

command is not running when SIGUSR2 is > received as the shutdown checkpoint may have already begun. Here is an > idea: add a new state in WalSndState, say WALSNDSTATE_STOPPING, and > the shutdown checkpoint does not run as long as all WAL senders still > running do not reach such

Re: [HACKERS] DROP SUBSCRIPTION, query cancellations and slot handling

eplication slot "mysub" already exists > LOCATION: libpqrcv_create_slot, libpqwalreceiver.c:776 > CREATE SUBSCRIPTION mysub CONNECTION 'port=5432' PUBLICATION mypub, insert_only WITH(NOCREATE SLOT); Or you can drop the slot manually on upstream. -- Petr Jelinek http:

Re: [HACKERS] DROP SUBSCRIPTION, query cancellations and slot handling

ither, so there > is nothing that the user can do except drop manually the slot on the > publisher. It seems to me that the moment where the slot is created > should be a point of no-return: the subcription has to be dropped on > the replication slot is dropped on the remote. >

Re: [HACKERS] Interval for launching the table sync worker

On 20/04/17 06:21, Masahiko Sawada wrote: > On Thu, Apr 20, 2017 at 12:30 AM, Petr Jelinek > <petr.jeli...@2ndquadrant.com> wrote: >> On 19/04/17 15:57, Masahiko Sawada wrote: >>> On Wed, Apr 19, 2017 at 10:07 PM, Petr Jelinek >>> <petr.jeli...@2ndquad

Re: [HACKERS] snapbuild woes

er (or anything similar that would give us control over timing) unless we expose a lot of guts of xlog/xact as a UDFs as well. So I think simple function that does what you said and pgbench are reasonable solutions. I guess you plan to make that as one of the test/modules or something similar (the

Re: [HACKERS] Interval for launching the table sync worker

On 19/04/17 15:57, Masahiko Sawada wrote: > On Wed, Apr 19, 2017 at 10:07 PM, Petr Jelinek > <petr.jeli...@2ndquadrant.com> wrote: >> On 19/04/17 14:42, Masahiko Sawada wrote: >>> On Wed, Apr 19, 2017 at 5:12 PM, Kyotaro HORIGUCHI >>> <horiguchi.kyot...@la

Re: [HACKERS] Interval for launching the table sync worker

On 19/04/17 14:42, Masahiko Sawada wrote: > On Wed, Apr 19, 2017 at 5:12 PM, Kyotaro HORIGUCHI > <horiguchi.kyot...@lab.ntt.co.jp> wrote: >> At Tue, 18 Apr 2017 18:40:56 +0200, Petr Jelinek >> <petr.jeli...@2ndquadrant.com> wrote in >> <f64d87d1-bef3

Re: [HACKERS] Logical replication ApplyContext bloat

On 19/04/17 12:46, Stas Kelvich wrote: > >> On 19 Apr 2017, at 13:34, Simon Riggs <si...@2ndquadrant.com> wrote: >> >> On 19 April 2017 at 11:24, Petr Jelinek <petr.jeli...@2ndquadrant.com> wrote: >> >>> I'd still like you to add comment to the

Re: [HACKERS] Logical replication ApplyContext bloat

On 19/04/17 11:55, Stas Kelvich wrote: > >> On 19 Apr 2017, at 12:37, Petr Jelinek <petr.jeli...@2ndquadrant.com> wrote: >> >> On 18/04/17 13:45, Stas Kelvich wrote: >>> Hi, >>> >>> currently logical replication worker uses ApplyContext to d

Re: [HACKERS] Logical replication ApplyContext bloat

idn’t spotted any measurable difference in speed. > > Problem spotted by Mikhail Shurutov. > Hmm we also do replication protocol handling inside the ApplyContext (LogicalRepApplyLoop), are you sure this change is safe in that regard? -- Petr Jelinek http://www.2ndQuad

Re: [HACKERS] some review comments on logical rep code

On 19/04/17 10:45, Kyotaro HORIGUCHI wrote: > At Wed, 19 Apr 2017 17:43:17 +0900 (Tokyo Standard Time), Kyotaro HORIGUCHI > <horiguchi.kyot...@lab.ntt.co.jp> wrote in > <20170419.174317.114509231.horiguchi.kyot...@lab.ntt.co.jp> >> At Wed, 19 Apr 2017 10:33:29 +0200,

Re: [HACKERS] some review comments on logical rep code

On 19/04/17 10:25, Kyotaro HORIGUCHI wrote: > At Wed, 19 Apr 2017 04:18:18 +0200, Petr Jelinek > <petr.jeli...@2ndquadrant.com> wrote in > <c2cfda3b-9335-2b57-e9ee-a55a8646a...@2ndquadrant.com> >> On 18/04/17 19:27, Fujii Masao wrote: >>> On Wed, A

[HACKERS] Use sync commit for logical replication apply in TAP tests

to use sync commit which improves performance there (because we also replicate only very few transactions in the tests). -- Petr Jelinek http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services From a4cebdf95d1f8a58d12f2f3824dffaae1dbf435d Mon Se

Re: [HACKERS] some review comments on logical rep code

On 18/04/17 19:27, Fujii Masao wrote: > On Wed, Apr 19, 2017 at 1:35 AM, Petr Jelinek > <petr.jeli...@2ndquadrant.com> wrote: >> Thank you for working on this! >> >> On 18/04/17 10:16, Masahiko Sawada wrote: >>> On Tue, Apr 18, 2017 at 12:24 PM

Re: [HACKERS] tablesync patch broke the assumption that logical rep depends on?

cross processes instead of having to reinvent it every time. -- Petr Jelinek http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services From 6d32379cda7d9f69c42185b9684d32570554f4e3 Mon Sep 17 00:00:00 2001 From: Petr Jelinek <pjmo...@pjmodos.net>

Re: [HACKERS] Why does logical replication launcher set application_name?

On 18/04/17 19:18, Tom Lane wrote: > Petr Jelinek <petr.jeli...@2ndquadrant.com> writes: >> On 18/04/17 18:24, Peter Eisentraut wrote: >>> I don't see why we need to do that. It is showing the correct >>> information, isn't it? > >> It does, but it's a

Re: [HACKERS] Why does logical replication launcher set application_name?

On 18/04/17 18:24, Peter Eisentraut wrote: > On 4/18/17 12:13, Petr Jelinek wrote: >> We can definitely easily detect that the bgworker is internal one by >> library_name equals 'postgres' so we can easily remove the usesysid and >> usename based on that. > > I

Re: [HACKERS] Interval for launching the table sync worker

On 18/04/17 18:14, Peter Eisentraut wrote: > On 4/18/17 11:59, Petr Jelinek wrote: >> Hmm if we create hashtable for this, I'd say create hashtable for the >> whole table_states then. The reason why it's list now was that it seemed >> unnecessary to have hashtable when it

Re: [HACKERS] some review comments on logical rep code

t;>>>>> MyLogicalRepWorker->relstate = >>>>>>> GetSubscriptionRelState(MyLogicalRepWorker->subid, >>>>>>> MyLogicalRepWorker->relid, >>>>>>> >relstate_lsn, >>>>>>

Re: [HACKERS] Why does logical replication launcher set application_name?

On 16/04/17 22:27, Petr Jelinek wrote: > On 16/04/17 18:47, Tom Lane wrote: >> Craig Ringer <cr...@2ndquadrant.com> writes: >>> On 12 April 2017 at 13:34, Kuntal Ghosh <kuntalghosh.2...@gmail.com> wrote: >>>> For backend_type=backgrou

Re: [HACKERS] Logical replication launcher uses wal_retrieve_retry_interval

needed because default tcp keepalive settings are usually too generous. As for apply_worker_launch_interval, I think we want different name so that it can be used for tablesync rate limiting as well. -- Petr Jelinek http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Sup

Re: [HACKERS] Interval for launching the table sync worker

Hmm if we create hashtable for this, I'd say create hashtable for the whole table_states then. The reason why it's list now was that it seemed unnecessary to have hashtable when it will be empty almost always but there is no need to have both hashtable + list IMHO. -- Petr Jelinek

Re: [HACKERS] logical replication and PANIC during shutdown checkpoint in publisher

On 17/04/17 18:02, Andres Freund wrote: > On 2017-04-15 02:33:59 +0900, Fujii Masao wrote: >> On Fri, Apr 14, 2017 at 10:33 PM, Petr Jelinek >> <petr.jeli...@2ndquadrant.com> wrote: >>> On 12/04/17 15:55, Fujii Masao wrote: >>>> Hi, >>>>

Re: [HACKERS] Re: logical replication and PANIC during shutdown checkpoint in publisher

advance of shipping v10. Consequently, I will appreciate your efforts > toward speedy resolution. Thanks. > Just FYI this is not new in 10, the issue exists since the 9.4 introduction of logical replication slots. -- Petr Jelinek http://www.2ndQuadrant.

Re: [HACKERS] tablesync patch broke the assumption that logical rep depends on?

nd recreate it it works. > No that's a result of how logical decoding/slots work. We see catalogs as what they looked like at the specific point in time. If you create slot (by creating subscription) it will not see publication that was created after until decoding on that slot reaches poin

Re: [HACKERS] Why does logical replication launcher set application_name?

are reported more like other builtin processes. We could also try to add api for bgworker processes to change how they are reported so that any future workers (and all the external workers) can be reported properly as well, but that seems better fit for v11 at this point since it would be good if w

Re: [HACKERS] Logical replication - TRAP: FailedAssertion in pgstat.c

dual commit, do things work again? > > - Andres > Yeah it is, it needs to be fenced to happen only after commit, which is not guaranteed at the point of code, we probably need to put the pgstat_report_stat() inside the if above after the CommitTransactionCommand() (that will make it report stats for change

Re: [HACKERS] logical replication launcher crash on buildfarm

On 15/04/17 06:01, Tom Lane wrote: > Petr Jelinek <petr.jeli...@2ndquadrant.com> writes: >> So this is what I came up with on plane. Generalized the loading >> functionality into load_library_function which that can load either >> known postgres functions or call load_e

Re: [HACKERS] snapbuild woes

Hi, here is updated patch (details inline). On 13/04/17 20:00, Petr Jelinek wrote: > Thanks for looking at this! > > On 13/04/17 02:29, Andres Freund wrote: >> Hi, >> On 2017-03-03 01:30:11 +0100, Petr Jelinek wrote: >> >>> From 7d5b48c8cb80e7c867b2096c999d0

Re: [HACKERS] Different table schema in logical replication crashes

On 14/04/17 17:33, Peter Eisentraut wrote: > On 4/14/17 08:49, Petr Jelinek wrote: >>> Are we prepared to support different schemas in v10? Or should we >>> disallow it for v10 and add a TODO? >>> >> >> Ah nuts, yes it's supposed to be supported, we see

Re: [HACKERS] tablesync patch broke the assumption that logical rep depends on?

of that comment than the fact that logicalrep_worker_launch isn't concurrency safe. We need in_use marker for the workers and update it as needed instead of relying on pgproc. I'll write up something over the weekend. -- Petr Jelinek http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] logical replication and PANIC during shutdown checkpoint in publisher

On 14/04/17 21:05, Peter Eisentraut wrote: > On 4/14/17 14:23, Petr Jelinek wrote: >> Ah yes looking at the code, it does exactly that (on master only). Means >> that backport will be necessary. > > I think these two sentences are contradicting each other. > Heh

Re: [HACKERS] logical replication and PANIC during shutdown checkpoint in publisher

On 14/04/17 19:33, Fujii Masao wrote: > On Fri, Apr 14, 2017 at 10:33 PM, Petr Jelinek > <petr.jeli...@2ndquadrant.com> wrote: >> On 12/04/17 15:55, Fujii Masao wrote: >>> Hi, >>> >>> When I shut down the publisher while I repeated creating and dr

Re: [HACKERS] logical replication worker and statistics

se didn't happen (because tablesync was faster than apply) then the commit handler is never called so all the changes made by copy would be forgotten. Also the tablesync worker might exit from process_syncing_tables() call which is called before we report stats in the commit handler so again some changes migh

Re: [HACKERS] Different table schema in logical replication crashes

On 14/04/17 17:33, Peter Eisentraut wrote: > On 4/14/17 08:49, Petr Jelinek wrote: >>> Are we prepared to support different schemas in v10? Or should we >>> disallow it for v10 and add a TODO? >>> >> >> Ah nuts, yes it's supposed to be supported, we see

Re: [HACKERS] Why does logical replication launcher set application_name?

column? >> >> It seems to me that the logic behind that is to be able to identify >> easily the logical replication launcher in pg_stat_activity, so using >> the query field instead sounds fine for me as we need another way than >> backend_type to guess what is

Re: [HACKERS] logical replication and PANIC during shutdown checkpoint in publisher

) but I think it's important to identify what exactly causes the WAL activity in your case - if it's one of the replication commands and not SQL then we'll need to backpatch any solution we come up with to 9.4, if it's not replication command, we only need to fix 10. -- Petr Jelinek

Re: [HACKERS] Logical replication launcher uses wal_retrieve_retry_interval

On 14/04/17 14:30, Michael Paquier wrote: > On Fri, Apr 14, 2017 at 9:19 PM, Petr Jelinek > <petr.jeli...@2ndquadrant.com> wrote: >> I am not quite sure adding more GUCs is all that great option. When >> writing the patches I was wondering if we should perhaps rename the

Re: [HACKERS] Different table schema in logical replication crashes

te->range_table externally. I wonder if tablesync should just construct CopyStmt instead of calling the lower level API. -- Petr Jelinek http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services -- Sent via pgsql-hackers mailing list (pgsql-hack

Re: [HACKERS] Logical replication and inheritance

On 14/04/17 06:14, Amit Langote wrote: > On 2017/04/14 10:57, Petr Jelinek wrote: >> I don't think inheritance and partitioning should behave the same in >> terms of logical replication. > > I see. > >> >> For me the current behavior with inherited tables

Re: [HACKERS] Logical replication launcher uses wal_retrieve_retry_interval

s rename the wal_receiver_timeout and wal_retrieve_retry_interval to something that makes more sense for both physical and logical replication though. -- Petr Jelinek http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services -- Sent via pgsq

Re: [HACKERS] Interval for launching the table sync worker

On 14/04/17 12:18, Masahiko Sawada wrote: > On Fri, Apr 14, 2017 at 7:09 AM, Petr Jelinek > <petr.jeli...@2ndquadrant.com> wrote: >> On 13/04/17 12:23, Masahiko Sawada wrote: >>> On Thu, Apr 13, 2017 at 11:53 AM, Masahiko Sawada <sawada.m...@gmail.com> >>&

Re: [HACKERS] Logical replication and inheritance

of the partitions would be sent as if the change was for that partitioned table so that the partitioning scheme on subscriber does not need to be same as on publisher. That's nontrivial to do though probably. -- Petr Jelinek http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support

Re: [HACKERS] SUBSCRIPTIONS and pg_upgrade

dumped by default, just like we have --no-blobs, --no-owner, > --no-password, --no-privileges, --no-acl, --no-tablespaces, and > --no-security-labels. It seems like there is probably a fairly large > use case for excluding subscriptions even if you have sufficient > permissions to dump them. >

Re: [HACKERS] Interval for launching the table sync worker

me way for the main apply workers as well. It would have the disadvantage that if some tables were consistently failing, no other tables could get synchronized as the amount of workers is limited. OTOH the approach chosen in the patch will first try all tables and only then start rate limiting, not quit

Re: [HACKERS] logical replication apply to run with sync commit off by default

On 12/04/17 06:10, Peter Eisentraut wrote: > On 3/24/17 10:49, Petr Jelinek wrote: >> Rebase after table copy patch got committed. > > You could please sent another rebase of this, perhaps with some more > documentation, as discussed downthread. > > Also, I wonder why

Re: [HACKERS] Interval for launching the table sync worker

State is taken over to new list. > > The right target of "This" below is found at the following URL. > > https://www.postgresql.org/message-id/CAD21AoBt_XUdppddFak661_LBM2t3CfK52aLKHG%2Bekd7SkzLmg%40mail.gmail.com > >> This resolves the problem but, if

Re: [HACKERS] snapbuild woes

teering; I'll do that shortly. Please observe the >> policy on >> open item ownership[1] and send a status update within three calendar >> days of >> this message. Include a date for your subsequent status update. >> >> [1] >> https://www.postgresql.org/messa

Re: [HACKERS] snapbuild woes

Thanks for looking at this! On 13/04/17 02:29, Andres Freund wrote: > Hi, > On 2017-03-03 01:30:11 +0100, Petr Jelinek wrote: > >> From 7d5b48c8cb80e7c867b2096c999d08feda50b197 Mon Sep 17 00:00:00 2001 >> From: Petr Jelinek <pjmo...@pjmodos.net> >> Date

Re: [HACKERS] logical replication and SIGHUP

On 10/04/17 14:40, Tom Lane wrote: > Petr Jelinek <petr.jeli...@2ndquadrant.com> writes: >> Looks good to me. Just as a note, we'll have to handle this newly >> supported config rereads in the async commit patch where we override >> synchronous_commit GUC, but the config

Re: [HACKERS] max_sync_workers_per_subscription is missing in postgresql.conf

h, but it's actually taken from max_logical_replication_workers so the patch should look more like attached IMHO. -- Petr Jelinek http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services diff --git a/src/backend/utils/misc/postgresql.conf.sample

Re: [HACKERS] logical replication and SIGHUP

-safe. I think as well > that for correctness errno should be saved as SetLatch() is called and > restored afterwards. Please find attached a patch to address all that. > Looks good to me. Just as a note, we'll have to handle this newly supported config rereads in the async commit patch where

Re: [HACKERS] Interval for launching the table sync worker

a. We need a way to either detect if we are launching same worker that was already launched before, or alternatively if we are launching crashed worker and only then apply the delay. -- Petr Jelinek http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training &

Re: [HACKERS] Interval for launching the table sync worker

On 06/04/17 16:44, Masahiko Sawada wrote: > On Thu, Apr 6, 2017 at 9:06 PM, Kyotaro HORIGUCHI > wrote: >> At Thu, 06 Apr 2017 17:02:14 +0900 (Tokyo Standard Time), Kyotaro HORIGUCHI >> wrote in >>

Re: [HACKERS] subscription worker doesn't start immediately on eabled

on't be a matter. > Makes sense, I think this got lost in all the refactoring, thanks. -- Petr Jelinek http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresq

Re: [HACKERS] PoC plpgsql - possibility to force custom or generic plan

the plancache at user level. > The heuristics it uses certainly need work, That's an understatement, there are thousands of plpgsql functions in large installations of PostgreSQL which use EXECUTE everywhere just to avoid current behavior (and that's just what I've seen). -- Petr Jelin

Re: [HACKERS] Quorum commit for multiple synchronous replication.

ay with many new features including quorum-based syncrep. >> Then if many of them complain about (1) and (3), we can change the code >> at that timing. So we need more time that users can try the feature. > > I've moved (1) to a new section for things to revisit during beta. If

Re: [HACKERS] logical replication launcher crash on buildfarm

to share what I have so far. -- Petr Jelinek http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services diff --git a/src/backend/access/transam/README.parallel b/src/backend/access/transam/README.parallel index db9ac3d..b360887 100644 --- a/src/bac

Re: [HACKERS] PATCH: two slab-like memory allocators

Hi, this patch is marked as committed in CF application but the second part (generational allocator) was AFAICS never committed. Does anybody plan to push this forward? -- Petr Jelinek http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Serv

Re: [HACKERS] logical replication launcher crash on buildfarm

On 01/04/17 04:19, Andres Freund wrote: > On 2017-03-31 21:30:17 -0400, Robert Haas wrote: >> On Fri, Mar 31, 2017 at 9:26 PM, Petr Jelinek >> <petr.jeli...@2ndquadrant.com> wrote: >>>> Hmm, I don't know if there's any good reason not to just use strcmp(), >&

Re: [HACKERS] logical replication launcher crash on buildfarm

On 01/04/17 03:44, Andres Freund wrote: > On 2017-04-01 03:26:01 +0200, Petr Jelinek wrote: >> On 01/04/17 02:53, Robert Haas wrote: >>> On Fri, Mar 31, 2017 at 8:32 PM, Petr Jelinek >>> <petr.jeli...@2ndquadrant.com> wrote: >>>> Right, changed to BGW

Re: [HACKERS] logical replication launcher crash on buildfarm

On 01/04/17 02:53, Robert Haas wrote: > On Fri, Mar 31, 2017 at 8:32 PM, Petr Jelinek > <petr.jeli...@2ndquadrant.com> wrote: >> Right, changed to BGW_MAXLEN. > > Hmm, I don't know if there's any good reason not to just use strcmp(), > but sure, OK. Committed and back

Re: [HACKERS] logical replication launcher crash on buildfarm

On 31/03/17 15:42, Robert Haas wrote: > On Tue, Mar 28, 2017 at 7:39 PM, Petr Jelinek > <petr.jeli...@2ndquadrant.com> wrote: >> Sigh, forgot git add for the docs, so one more try... > > +if (strncmp(worker->bgw_library_name, "postgres", 8) != 0) > +

Re: [HACKERS] Somebody has not thought through subscription locking considerations

On 01/04/17 01:57, Petr Jelinek wrote: > On 01/04/17 01:20, Tom Lane wrote: >> Petr Jelinek <petr.jeli...@2ndquadrant.com> writes: >>> But the pg_subscription_rel is also not accessed on heap_open, the >>> problematic code is called from heap_drop_with_catalog. A

Re: [HACKERS] Somebody has not thought through subscription locking considerations

On 01/04/17 01:57, Petr Jelinek wrote: > On 01/04/17 01:20, Tom Lane wrote: >> Petr Jelinek <petr.jeli...@2ndquadrant.com> writes: >>> But the pg_subscription_rel is also not accessed on heap_open, the >>> problematic code is called from heap_drop_with_catalog. A

Re: [HACKERS] Somebody has not thought through subscription locking considerations

On 01/04/17 01:20, Tom Lane wrote: > Petr Jelinek <petr.jeli...@2ndquadrant.com> writes: >> But the pg_subscription_rel is also not accessed on heap_open, the >> problematic code is called from heap_drop_with_catalog. And VACUUM FULL >> pg_class calls heap_drop_with_ca

Re: [HACKERS] Somebody has not thought through subscription locking considerations

On 01/04/17 00:52, Tom Lane wrote: > Petr Jelinek <petr.jeli...@2ndquadrant.com> writes: >> On 31/03/17 21:00, Tom Lane wrote: >>> Looking at dependency info isn't going to fix this, it only moves the >>> unsafe catalog access somewhere else (ie pg_depend instea

Re: [HACKERS] Somebody has not thought through subscription locking considerations

On 31/03/17 21:00, Tom Lane wrote: > Petr Jelinek <petr.jeli...@2ndquadrant.com> writes: >> On 31/03/17 20:23, Tom Lane wrote: >>> No, the problematic part is that there is any heap_open happening at >>> all. That open could very easily result in a recur

Re: [HACKERS] Somebody has not thought through subscription locking considerations

On 31/03/17 20:23, Tom Lane wrote: > Petr Jelinek <petr.jeli...@2ndquadrant.com> writes: >> On 31/03/17 19:35, Tom Lane wrote: >>> At the very least, it would be a good idea to exclude the system >>> catalogs from logical replication, wouldn't it? > >> T

Re: [HACKERS] Somebody has not thought through subscription locking considerations

On 31/03/17 19:35, Tom Lane wrote: > Masahiko Sawada <sawada.m...@gmail.com> writes: >> On Fri, Mar 31, 2017 at 9:53 AM, Petr Jelinek >> <petr.jeli...@2ndquadrant.com> wrote: >>> On 30/03/17 07:25, Tom Lane wrote: >>>> I await with inter

Re: [HACKERS] Somebody has not thought through subscription locking considerations

_subscription_rel though, I'll have to dig into that. I can see that locking for example pg_trigger or pg_depend in ShareRowExclusiveLock will also block VACUUM FULL on pg_class (and other related tables like pg_attribute). So maybe we just need to be careful about not taking such a strong lock...

Re: [HACKERS] Schedule and Release Management Team for PG10

d of the current CF. > +1, the conference makes it bit inconvenient to have freeze on exactly 31st. -- Petr Jelinek http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

there. I mean I've seen the problem WARM tries to solve mostly on timestamp or boolean values and sometimes counters so it would still be helpful to quite a lot of people even if we didn't do TOAST and compressed values in v1. It's not like not doing WARM sometimes is somehow terrible, we'll just f

Re: [HACKERS] logical replication access control patches

create new objects in the database which standard CREATE privilege would allow. So I think it makes sense to push this approach. -- Petr Jelinek http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Logical replication existing data copy

ut and keepalive ping, perhaps there's > (still) something amiss there? Just a guess, I don't know ) > > As I said, I saw no more failures with the higher 3 minute setting, with > one exception: the one test that straddled the DST change (saterday 24 > march 02:00 h). I am h

<    1   2   3   4   5   6   7   8   9   10   >