Re: finding changed blocks using WAL scanning

2019-04-19 Thread Stephen Frost
Greetings, * Robert Haas (robertmh...@gmail.com) wrote: > On Fri, Apr 19, 2019 at 8:39 PM Stephen Frost wrote: > > While I do think we should at least be thinking about the load caused > > from scanning the WAL to generate a list of blocks that are changed, the > > load I was more concerned with

Re: finding changed blocks using WAL scanning

2019-04-19 Thread Robert Haas
On Thu, Apr 18, 2019 at 8:38 PM Michael Paquier wrote: > On Thu, Apr 18, 2019 at 03:51:14PM -0400, Robert Haas wrote: > > I was thinking that a dedicated background worker would be a good > > option, but Stephen Frost seems concerned (over on the other thread) > > about how much load that would ge

Re: block-level incremental backup

2019-04-19 Thread Stephen Frost
Greetings, * Robert Haas (robertmh...@gmail.com) wrote: > What I'm NOT willing to > do is build a whole bunch of infrastructure that will help pgbackrest > do amazing things but will not provide a simple and convenient way of > taking incremental backups using only core tools. I do care about > h

Re: finding changed blocks using WAL scanning

2019-04-19 Thread Robert Haas
On Fri, Apr 19, 2019 at 8:39 PM Stephen Frost wrote: > While I do think we should at least be thinking about the load caused > from scanning the WAL to generate a list of blocks that are changed, the > load I was more concerned with in the other thread is the effort > required to actually merge al

Re: finding changed blocks using WAL scanning

2019-04-19 Thread Robert Haas
On Thu, Apr 18, 2019 at 5:47 PM Bruce Momjian wrote: > How would the modblock file record all the modified blocks across > restarts and crashes? I assume that 1G of WAL would not be available > for scanning. I suppose that writing a modblock file to some PGDATA > location when WAL is removed wou

Re: block-level incremental backup

2019-04-19 Thread Robert Haas
On Thu, Apr 18, 2019 at 6:39 PM Stephen Frost wrote: > Where is the client going to get the threshold LSN from? > > If it doesn't have access to the old backup, then I'm a bit confused as > to how a incremental backup would be possible? Isn't that a requirement > here? I explained this in the ve

Re: Pathological performance when inserting many NULLs into a unique index

2019-04-19 Thread Peter Geoghegan
On Thu, Apr 18, 2019 at 8:13 PM Peter Geoghegan wrote: > It just occurred to me that the final patch will need to be more > careful about non-key attributes in INCLUDE indexes. It's not okay for > it to avoid calling _bt_check_unique() just because a non-key > attribute was NULL. It should only do

Re: [PATCH v20] GSSAPI encryption support

2019-04-19 Thread Stephen Frost
Greetings, * Michael Paquier (mich...@paquier.xyz) wrote: > On Mon, Apr 15, 2019 at 08:24:52AM -0400, Stephen Frost wrote: > > The tests are really fast enough with one KDC that I don't think it > > makes sense to have two independent tests. > > Perhaps you should add a comment about the need of

Re: finding changed blocks using WAL scanning

2019-04-19 Thread Stephen Frost
Greetings, * Robert Haas (robertmh...@gmail.com) wrote: > On Mon, Apr 15, 2019 at 11:45 PM Michael Paquier wrote: > > Any caller of XLogWrite() could switch to a new segment once the > > current one is done, and I am not sure that we would want some random > > backend to potentially slow down to

Re: block-level incremental backup

2019-04-19 Thread Stephen Frost
Greetings, * Andres Freund (and...@anarazel.de) wrote: > > > Wow. I have to admit that I feel completely opposite of that- I'd > > > *love* to have an independent tool (which ideally uses the same code > > > through the common library, or similar) that can be run to apply WAL. > > > > > > In othe

Re: Idea for fixing parallel pg_dump's lock acquisition problem

2019-04-19 Thread Tom Lane
Julien Rouhaud writes: > On Fri, Apr 19, 2019 at 7:17 PM Tom Lane wrote: >> My thought was that we'd like this to work without requiring any new >> server-side facilities, so that pg_dump could use it against any server >> version that supports parallel dump. > Couldn't we use LOCKTAG_USERLOCK f

Re: Idea for fixing parallel pg_dump's lock acquisition problem

2019-04-19 Thread Julien Rouhaud
On Fri, Apr 19, 2019 at 7:17 PM Tom Lane wrote: > > Robert Haas writes: > > On Wed, Apr 17, 2019 at 11:34 AM Tom Lane wrote: > >> ... If there are user applications > >> running that also use advisory locks, there could be unwanted > >> interference. One easy improvement is to use pg_try_adviso

Re: ExecForceStoreMinimalTuple leaks memory like there's no tomorrow

2019-04-19 Thread Andres Freund
Hi, On 2019-04-18 19:04:09 -0700, Andres Freund wrote: > On 2019-04-15 22:46:56 -0400, Tom Lane wrote: > > Using HEAD, > > > > create table t1 as select generate_series(1,4000) id; > > vacuum analyze t1; > > explain select * from t1, t1 t1b where t1.id = t1b.id; > > -- should indicate a hash

Re: Idea for fixing parallel pg_dump's lock acquisition problem

2019-04-19 Thread Tom Lane
Robert Haas writes: > On Wed, Apr 17, 2019 at 11:34 AM Tom Lane wrote: >> ... If there are user applications >> running that also use advisory locks, there could be unwanted >> interference. One easy improvement is to use pg_try_advisory_lock(k) in >> step 2, and just choose a different k if the

Re: Idea for fixing parallel pg_dump's lock acquisition problem

2019-04-19 Thread Robert Haas
On Wed, Apr 17, 2019 at 11:34 AM Tom Lane wrote: > While thinking about that, it occurred to me that we could close the > gap if the server somehow understood that the master was waiting for > the worker. And it's actually not that hard to make that happen: > we could use advisory locks. Conside

Re: TM format can mix encodings in to_char()

2019-04-19 Thread Tom Lane
=?UTF-8?Q?Juan_Jos=C3=A9_Santamar=C3=ADa_Flecha?= writes: > The problem is that the locale 'tr_TR' uses the encoding ISO-8859-9 (LATIN5), > while the test runs in UTF8. So the following code will raise an error: > SET lc_time TO 'tr_TR'; > SELECT to_char(date '2010-02-01', 'DD TMMON '); > ER

Re: POC: Cleaning up orphaned files using undo logs

2019-04-19 Thread Robert Haas
On Fri, Apr 19, 2019 at 6:16 AM Dilip Kumar wrote: > Currently, undo branch[1] contain an older version of the (undo > interface + some fixup). Now, I have merged the latest changes from > the zheap branch[2] to the undo branch[1] > which can be applied on top of the undo storage commit[3]. For

Re: clean up docs for v12

2019-04-19 Thread Justin Pryzby
Thanks for committing those portions. On Fri, Apr 19, 2019 at 05:00:26PM +0900, Michael Paquier wrote: > I am particularly referring to patches 0005 > (publications use "a superuser" in error messages as well which could > be fixed as well?), I deliberately avoided changing thesee "errhint" messa

Re: POC: Cleaning up orphaned files using undo logs

2019-04-19 Thread Dilip Kumar
On Tue, Mar 12, 2019 at 6:51 PM Thomas Munro wrote: > > On Sun, Feb 3, 2019 at 11:09 PM Andres Freund wrote: > > On 2018-12-03 18:43:04 +1300, Thomas Munro wrote: > > > Sorry for my silence... I got stuck on a design problem with the lower > > > level undo log management code that I'm now close t

Re: Unhappy about API changes in the no-fsm-for-small-rels patch

2019-04-19 Thread Amit Kapila
On Fri, Apr 19, 2019 at 1:17 PM John Naylor wrote: > On Fri, Apr 19, 2019 at 10:38 AM Amit Kapila wrote: > > > > I think if we go this route, then > > we might need to revisit it in the future to optimize it, but maybe > > that is the best alternative as of now. > > It's a much lighter-weight API

Re: TM format can mix encodings in to_char()

2019-04-19 Thread Kyotaro HORIGUCHI
Hello. At Fri, 12 Apr 2019 18:45:51 +0200, Juan José Santamaría Flecha wrote in > Hackers, > > I will use as an example the code in the regression test > 'collate.linux.utf8'. > There you can find: > > SET lc_time TO 'tr_TR'; > SELECT to_char(date '2010-04-01', 'DD TMMON '); >to_char

Re: Runtime pruning problem

2019-04-19 Thread Amit Langote
On 2019/04/19 17:00, Amit Langote wrote: > On 2019/04/19 2:25, Tom Lane wrote: >> Amit Langote writes: >>> Another idea is to teach explain.c about this special case of run-time >>> pruning having pruned all child subplans even though appendplans contains >>> one element to cater for targetlist ac

Re: Runtime pruning problem

2019-04-19 Thread Amit Langote
On 2019/04/19 3:13, Robert Haas wrote: > On Tue, Apr 16, 2019 at 10:49 PM Amit Langote > wrote: >> Maybe, not show them? That may be a bit inconsistent, because the point >> of VERBOSE is to the targetlist among other things, but maybe the users >> wouldn't mind not seeing it on such empty Append

Re: Runtime pruning problem

2019-04-19 Thread Amit Langote
On 2019/04/19 2:25, Tom Lane wrote: > Amit Langote writes: >> Another idea is to teach explain.c about this special case of run-time >> pruning having pruned all child subplans even though appendplans contains >> one element to cater for targetlist accesses. That is, Append will be >> displayed w

Re: clean up docs for v12

2019-04-19 Thread Michael Paquier
On Mon, Apr 08, 2019 at 09:18:28AM -0500, Justin Pryzby wrote: > Find attached updated patches for v12 docs. Thanks for taking the time to dig into such things. > Note that Alvaro applied an early patch for log_statement_sample_rate, but > unfortunately I hadn't sent a v2 patch with additional ch

Re: Unhappy about API changes in the no-fsm-for-small-rels patch

2019-04-19 Thread John Naylor
On Fri, Apr 19, 2019 at 10:38 AM Amit Kapila wrote: > > On Thu, Apr 18, 2019 at 2:10 PM John Naylor > wrote: > > Agreed. I suspect the most realistic way to address most of the > > objections in a short amount of time would be to: > > > > 1. rip out the local map > > 2. restore hio.c to only che

Re: pgsql: Fix plan created for inherited UPDATE/DELETE with all tables exc

2019-04-19 Thread Amit Langote
On 2019/04/19 4:41, Tom Lane wrote: > Amit Langote writes: >> On 2019/02/23 2:23, Tom Lane wrote: >>> Fix plan created for inherited UPDATE/DELETE with all tables excluded. > >> I noticed that we may have forgotten to fix one more thing in this commit >> -- nominalRelation may refer to a range ta