Re: [HACKERS] MERGE SQL Statement for PG11

2018-03-22 Thread Simon Riggs
If there are any items you believe are still open, please say so now or mention any other objections you have. Thanks for all of your detailed comments, -- Simon Riggshttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: [HACKERS] MERGE SQL Statement for PG11

2018-03-22 Thread Simon Riggs
SERT or UPDATE outcome in READ COMMITTED mode. > > BTW, the docs should be clear on the fact that "INSERT ... ON > CONFLICT" isn't a statement. INSERT is. ON CONFLICT is a clause. I think it would be better if you wrote a separate additional doc patch to explain all of this, pe

Re: Faster inserts with mostly-monotonically increasing values

2018-03-14 Thread Simon Riggs
On 14 March 2018 at 13:33, Pavan Deolasee wrote: > > > On Wed, Mar 14, 2018 at 4:53 PM, Simon Riggs > wrote: >> >> On 14 March 2018 at 04:36, Pavan Deolasee >> wrote: >> > I wonder if the shortened code path actually leads to >> > heavier con

Re: Faster inserts with mostly-monotonically increasing values

2018-03-14 Thread Simon Riggs
e optimization if uncontended. This makes sense because in many cases if the block is locked it is filling up and the RHS block is more likely to change anyway. -- Simon Riggshttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: SQL/JSON: functions

2018-03-13 Thread Simon Riggs
the purpose of a patch submission is to allow a committer to review and commit without needing to edit anything. It shouldn't be like assembling flat pack furniture while wearing a blindfold. -- Simon Riggshttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Re

Re: Faster inserts with mostly-monotonically increasing values

2018-03-06 Thread Simon Riggs
out if DESC > indexes are actually stored in the reverse order. I am gonna look at that > too. No, I meant that you were testing whether the value was higher (> 0), whereas it should be lower (< 0) on DESC indexes. -- Simon Riggshttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: [HACKERS] make async slave to wait for lsn to be replayed

2018-03-06 Thread Simon Riggs
have) > > > I have a (possibly) dumb question: if we have specified several events, > should WAIT finish if only one of them triggered? It's not immediately > obvious if we're waiting for ALL of them to trigger, or just one will > suffice (ANY). IMO the syntax could be ext

Re: chained transactions

2018-03-05 Thread Simon Riggs
t; for v11 starts. Therefore I think it should just be moved to the next > fest, it doesn't seems realistic to attempt to get this into v11. Looks like a useful patch that adds fairly minor syntax that follows the SQL Standard. It introduces no new internal infrastructure, so I can't

Re: [HACKERS] Runtime Partition Pruning

2018-02-23 Thread Simon Riggs
TE with partitions is significantly different, so its not just an oversight its a different branch of the project. We need SELECT to work first and then we have a chance of making UPDATE/DELETE work. -- Simon Riggshttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Re

Re: committing inside cursor loop

2018-02-22 Thread Simon Riggs
holdable cursor, we just make Rollback do that also. Obviously only for pinned portals, i.e. the query/ies whose results we are currently looping through. -- Simon Riggshttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: Allow workers to override datallowconn

2018-02-22 Thread Simon Riggs
that should mean it is easy/default for background workers to ignore datallowcon. -- Simon Riggshttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: Contention preventing locking

2018-02-20 Thread Simon Riggs
On 20 February 2018 at 16:07, Konstantin Knizhnik wrote: > > > On 20.02.2018 14:26, Simon Riggs wrote: >> >> Try locking the root tid rather than the TID, that is at least unique >> per page for a chain of tuples, just harder to locate. >> > As far as I underst

Re: committing inside cursor loop

2018-02-20 Thread Simon Riggs
ever rewind. So it would be good if we could pass a parameter that skips the rewind since it will never be needed and causes a performance hit. What I imagine is we can just persist the as-yet unfetched portion of the cursor from the current row onwards, rather than rewind and store the w

Re: Contention preventing locking

2018-02-20 Thread Simon Riggs
On 20 February 2018 at 13:19, Konstantin Knizhnik wrote: > > > On 20.02.2018 14:26, Simon Riggs wrote: >> >> On 15 February 2018 at 16:00, Konstantin Knizhnik >> wrote: >> >>> So in heap_acquire_tuplock all competing transactions are waiting fo

Re: Contention preventing locking

2018-02-20 Thread Simon Riggs
; handling different types of PK, including compound keys, PK updates,... Try locking the root tid rather than the TID, that is at least unique per page for a chain of tuples, just harder to locate. -- Simon Riggshttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: [HACKERS] MERGE SQL Statement for PG11

2018-02-04 Thread Simon Riggs
atever we decide for concurrent behavior will affect how we support them. We can't move forwards on them until we have that nailed down. I could give a longer technical commentary but I will be unavailable now for some time, so unable to give further discussion. -- Simon Riggs

Re: [HACKERS] MERGE SQL Statement for PG11

2018-02-04 Thread Simon Riggs
; > It is not clear to me what is exactly your concern if we try to follow > #2? To me, #2 seems like a natural choice. At first, but it gives an anomaly so is not a good choice. The patch does behavior #5, it rechecks the conditions with the latest row. Otherwise WHEN MATCHED AND a=0 THEN UPD

Re: [HACKERS] MERGE SQL Statement for PG11

2018-02-04 Thread Simon Riggs
On 3 February 2018 at 23:17, Peter Geoghegan wrote: > On Sat, Feb 3, 2018 at 2:15 PM, Simon Riggs wrote: >>> I started looking at SQL Server's MERGE to verify that it also does >>> not impose any restrictions on subselects in a MERGE UPDATE's >>> targetli

Re: [HACKERS] MERGE SQL Statement for PG11

2018-02-03 Thread Simon Riggs
happened to have those values set equal. Minor bug, thanks for spotting. -- Simon Riggshttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: [HACKERS] MERGE SQL Statement for PG11

2018-02-03 Thread Simon Riggs
NOT MATCHED > AND cast(null as int2) = pg_catalog.lastval() >THEN DO NOTHING; > > -- ERROR: unrecognized node type: 114 > MERGE INTO public.found_test_tbl as target_0 > USING public.itest7a as ref_0 > ON target_0.a = ref_0.a > WHEN NOT MATCHED >THEN INSERT VALUES ((select a from public.rtest_t3 limit 1 offset 6)); -- Simon Riggshttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: [HACKERS] MERGE SQL Statement for PG11

2018-02-03 Thread Simon Riggs
e for the > patch. We can get to this stuff later. There are no concurrency issues. The patch gives the correct answer in all cases, or an error to avoid problems. We've agreed that it is desirable we remove some of those if possible, though they are there as a result of our earlier discussions. > Note that I consider cleaning > up the Query and plan representations to be prerequisite to commit. You'll need to be more specific. Later patches do move some things. -- Simon Riggshttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: [HACKERS] make async slave to wait for lsn to be replayed

2018-02-02 Thread Simon Riggs
On 2 February 2018 at 18:46, Robert Haas wrote: > On Fri, Feb 2, 2018 at 3:46 AM, Simon Riggs wrote: >> In PG11, I propose the following command, sticking mostly to Ants' >> syntax, and allowing to wait for multiple events before it returns. It >> doesn't hold snap

Re: [HACKERS] Surjective functional indexes

2018-02-02 Thread Simon Riggs
On 1 February 2018 at 09:32, Simon Riggs wrote: > OK, thanks. Just checking my understanding and will add to the > comments in the patch. I'm feeling good about ths now, but for personal reasons won't be committing this until late Feb/early March. -- Simon Riggs

Re: [HACKERS] make async slave to wait for lsn to be replayed

2018-02-02 Thread Simon Riggs
arch and I will shepherd this to commit. Thomas, I suggest we also do what Robert suggested elsewhere which was to have an connection option that returns xid or lsn (or both) via the protocol, so we can use that with the WAIT command and you can have the overall causal consistency feature

Re: Changing WAL Header to reduce contention during ReserveXLogInsertLocation()

2018-02-01 Thread Simon Riggs
much just 2^-nbits. The full 8 bytes of the xl_prev pointer aren't working for a living. More isn't better, always. If we really can't persuade you of that, it doesn't sink the patch. We can have the WAL pointer itself - it wouldn't save space but it would at least alleviate the spinlock. -- Simon Riggshttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: [HACKERS] MERGE SQL Statement for PG11

2018-02-01 Thread Simon Riggs
On 1 February 2018 at 12:45, Simon Riggs wrote: > I think it would be very helpful if we could discuss everything with > direct relevance to v14, so this becomes a patch review, not just a > debate. > i.e. which isolation test would we like to change from ERROR to > success? or

Re: [HACKERS] make async slave to wait for lsn to be replayed

2018-02-01 Thread Simon Riggs
hat WAIT LSN is good syntax because this allows us to wait for something else in future. Without having reviewed the patch, I think we want this feature in PG11. -- Simon Riggshttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: MCV lists for highly skewed distributions

2018-02-01 Thread Simon Riggs
Author. Dean and John's results show that different algorithms work better for different cases. How about we make ANALYZE's MCV algorithm pluggable? And then include, say, 2 additional algorithms. -- Simon Riggshttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: [HACKERS] MERGE SQL Statement for PG11

2018-02-01 Thread Simon Riggs
would we like to change from ERROR to success? or which new test would you like to add? Thanks -- Simon Riggshttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: [HACKERS] MERGE SQL Statement for PG11

2018-02-01 Thread Simon Riggs
rently throws an error, as requested. It looks to be possible, but it would require some variation of EvalPlanQual. My prototype of that doesn't yet work, so I can't yet confirm whether it is even possible. If it is, I will submit as an option for PG11. -- Simon Riggshttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: [HACKERS] Surjective functional indexes

2018-02-01 Thread Simon Riggs
On 1 February 2018 at 08:49, Konstantin Knizhnik wrote: > > > On 01.02.2018 03:10, Simon Riggs wrote: >> >> On 10 January 2018 at 09:54, Konstantin Knizhnik >> wrote: >> >>> (new version attached) >> >> Why this comment? >> >>

Re: [HACKERS] Surjective functional indexes

2018-01-31 Thread Simon Riggs
Old + New for check = 2 plus calculate again in index = 3 ? -- Simon Riggshttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: [HACKERS] MERGE SQL Statement for PG11

2018-01-30 Thread Simon Riggs
ow agree with doing that and are happy that there are no dangers, then I'm happy we now have consensus again and we can continue implementing MERGE for PG11. This is a good outcome, thanks, our users will be happy. -- Simon Riggshttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: [HACKERS] MERGE SQL Statement for PG11

2018-01-30 Thread Simon Riggs
ou >> change your mind on that. > > You're mistaken. Nothing I've said here is inconsistent with my > previous remarks on how we deal with concurrency. Please see here https://www.postgresql.org/message-id/20171102191636.GA27644%40marmot On 2 November 2017 at 19:16, Peter Geoghegan

Re: [HACKERS] MERGE SQL Statement for PG11

2018-01-29 Thread Simon Riggs
On 29 January 2018 at 20:41, Peter Geoghegan wrote: > On Mon, Jan 29, 2018 at 6:11 AM, Simon Riggs wrote: >> New patch attached that correctly handles all known concurrency cases, >> with expected test output. > > This revision, v13, seems much improved in this area. >

Re: [HACKERS] MERGE SQL Statement for PG11

2018-01-29 Thread Simon Riggs
On 29 January 2018 at 17:35, Peter Geoghegan wrote: > On Mon, Jan 29, 2018 at 8:51 AM, Simon Riggs wrote: >> On 29 January 2018 at 16:44, Bruce Momjian wrote: >> >>> I think the question is how does it handle cases it doesn't support? >>> Does it give

Re: [HACKERS] MERGE SQL Statement for PG11

2018-01-29 Thread Simon Riggs
On 29 January 2018 at 16:23, Chapman Flack wrote: > On 01/29/2018 11:13 AM, Simon Riggs wrote: >> On 29 January 2018 at 15:44, Bruce Momjian wrote: >>> Uh, if we know we are going to get question on this, the patch had >>> better have an explanation of when to use

Re: [HACKERS] MERGE SQL Statement for PG11

2018-01-29 Thread Simon Riggs
On 29 January 2018 at 16:50, Tom Lane wrote: > Bruce Momjian writes: >> On Mon, Jan 29, 2018 at 04:34:48PM +0000, Simon Riggs wrote: >>> ... If unfinished means it has caveats >>> that is different to unfinished meaning crappy, risky, contentious >>> etc.. &

Re: [HACKERS] MERGE SQL Statement for PG11

2018-01-29 Thread Simon Riggs
everything benefits from further review. -- Simon Riggshttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: [HACKERS] MERGE SQL Statement for PG11

2018-01-29 Thread Simon Riggs
On 29 January 2018 at 16:06, Tom Lane wrote: > Simon Riggs writes: >> On 29 January 2018 at 15:07, Robert Haas wrote: >>> An argument could be made that this patch is already too late for PG >>> 11, because it's a major feature that was not submitted in relativ

Re: [HACKERS] MERGE SQL Statement for PG11

2018-01-29 Thread Simon Riggs
On 29 January 2018 at 15:44, Bruce Momjian wrote: > On Mon, Jan 29, 2018 at 03:12:23PM +0000, Simon Riggs wrote: >> On 29 January 2018 at 14:55, Pavel Stehule wrote: >> >> > My note was not against MERGE or INSERT ON CONFLICT. If I understand to >> > this >

Re: [HACKERS] MERGE SQL Statement for PG11

2018-01-29 Thread Simon Riggs
Overall, I'm following the style of development process you have yourself used a number of times now. Waiting for mega-patches to be complete is not as useful as phased development. -- Simon Riggshttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: [HACKERS] MERGE SQL Statement for PG11

2018-01-29 Thread Simon Riggs
ocumentation about recommended use cases. Probably it will be very often > question. That is more qualitative assessment of each, which I think I will defer on. This patch is about implementing the SQL Standard compliant MERGE command which is widely used in other databases and by various tools.

Re: [HACKERS] MERGE SQL Statement for PG11

2018-01-29 Thread Simon Riggs
cases also. MERGE as submitted here follows all MVCC rules similar to an UPDATE join. If it hits a problem with concurent activity it throws ERROR: could not serialize access due to concurrent update to make sure there is no ambiguity (as described directly above). As we discussed earlier, rem

Re: Logical Decoding and HeapTupleSatisfiesVacuum assumptions

2018-01-29 Thread Simon Riggs
On 29 January 2018 at 14:13, Tomas Vondra wrote: > 4) inspect the new row (which we still have in reorderbuffer) > > 5) Kabooom! The row has column "c" which we don't see in the catalog. We don't use caches? Why does a cache miss cause it to explode? -- Simon Ri

Re: Logical Decoding and HeapTupleSatisfiesVacuum assumptions

2018-01-29 Thread Simon Riggs
omments were based on the idea that 2PC patch would simply block aborts, which of course will not work for streaming, hence the difference) -- Simon Riggshttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: Logical Decoding and HeapTupleSatisfiesVacuum assumptions

2018-01-29 Thread Simon Riggs
d the abort path is not considered one we need to optimize. -- Simon Riggshttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: Logical Decoding and HeapTupleSatisfiesVacuum assumptions

2018-01-29 Thread Simon Riggs
ort hook?). That call will just return when and if it is OK to abort. Any waiting or throwing of ERRORs can occur inside the plugin. The plugin can provide a function to allow an abort that type of abort if it wishes, but no new core functionality required for that. If needed, we would wait on a latch the plugin provides for that. -- Simon Riggshttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: Fix permissions check on pg_stat_get_wal_senders

2018-01-24 Thread Simon Riggs
On 24 January 2018 at 13:15, Feike Steenbergen wrote: > On 5 January 2018 at 15:19, Simon Riggs wrote: >> Yup, I got this. > > I saw the commit in the master branch but not in the REL_10_STABLE branch, > I'm totally not up-to-date with the backpatching process, but I was

Re: [HACKERS] MERGE SQL Statement for PG11

2018-01-23 Thread Simon Riggs
On 24 January 2018 at 01:35, Peter Geoghegan wrote: > On Tue, Jan 23, 2018 at 5:51 AM, Simon Riggs wrote: >>> Not yet working >>> * Partitioning >>> * RLS >>> >>> Based on this successful progress I imagine I'll be looking to commit >>&

Re: Logical Decoding and HeapTupleSatisfiesVacuum assumptions

2018-01-23 Thread Simon Riggs
g to be unlocked. While it is locked the 2PCxact cannot be aborted or committed. There is no danger of accidental abort because the prepare has persistent state. -- Simon Riggshttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: Rangejoin rebased

2018-01-23 Thread Simon Riggs
On 23 January 2018 at 05:08, Jeff Davis wrote: > On Fri, Jan 19, 2018 at 2:07 AM, Simon Riggs wrote: >> err... that isn't correct. An empty range matches nothing, so can be >> ignored in joins. >> >> So probably best to explain some more, please. > > The sem

Re: Logical Decoding and HeapTupleSatisfiesVacuum assumptions

2018-01-19 Thread Simon Riggs
saction and a concurrent abort is requested then decoding should be interrupted at the next sensible point. Allowing the two actions to occur without interlock is an issue, so I suggest we just don't do it, rather than allow it and fix subsequent breakage, which is what I understand this pa

Re: [HACKERS] Transaction control in procedures

2018-01-19 Thread Simon Riggs
also prevent SAVEPOINTs, though that seems not to be intended. The two cases are dissimilar and it would be possible to block the former but allow the latter. It's not documented or tested that SET LOCAL would work or not work. Does it work? -- Simon Riggshttp://www.2ndQuadrant

Re: [HACKERS] Transaction control in procedures

2018-01-19 Thread Simon Riggs
nd ROLLBACK in cursor loop bodies. But I think people will >> eventually want it. > > The may want it, but silently promoting all cursors to held ones is > not the way to give it to them, unless we narrow it down the the > 'for-loop derived cursor' only. I don'

Re: [PATCH] session_replication_role = replica with TRUNCATE

2018-01-19 Thread Simon Riggs
ust wasted cycles. The subscriber side already has various requirements, such as manually created DDL, so requiring sensible FKs is not so bad. This situation will improve over next few years. -- Simon Riggshttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: Rangejoin rebased

2018-01-19 Thread Simon Riggs
On 19 January 2018 at 08:25, Simon Riggs wrote: > On 17 January 2018 at 05:49, Jeff Davis wrote: >> On Wed, Jan 10, 2018 at 7:49 AM, Simon Riggs wrote: >>> Do we optimize for TIMESTAMP <@ RANGE as well? >> >> Not currently. It requires a little extra comple

Re: Rangejoin rebased

2018-01-19 Thread Simon Riggs
On 17 January 2018 at 05:49, Jeff Davis wrote: > On Wed, Jan 10, 2018 at 7:49 AM, Simon Riggs wrote: >> Do we optimize for TIMESTAMP <@ RANGE as well? > > Not currently. It requires a little extra complexity because empty > ranges will match anything and need special

Re: MCV lists for highly skewed distributions

2018-01-18 Thread Simon Riggs
te too many? Which would be a problem in increased planning time. I have a slight reservaton about whether 1.25x is still a sensible heuristic. Should we add a parameter for that to allow testing during beta? Marking as Ready For Committer. -- Simon Riggshttp://www.2ndQuadrant.com

Re: Typo in slotfuncs.c

2018-01-18 Thread Simon Riggs
On 19 January 2018 at 05:38, Masahiko Sawada wrote: > Hi, > > Attached patch for fixing a typo in slotfuncs.c > > s/possition/position/ Fixed, thanks. -- Simon Riggshttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: [PATCH] Logical decoding of TRUNCATE

2018-01-18 Thread Simon Riggs
; which relations it opened and only close those and the rest should be > closed by caller? That should also remove the other ugly part which is > that the ExecuteTruncateGuts modifies the input list. What if caller > wanted to use those relations it sent as parameter later? Agreed --

Re: Faster inserts with mostly-monotonically increasing values

2018-01-18 Thread Simon Riggs
k-of-an-envelope estimate is 9 billion index tuples. If they did, the gains would be even bigger. But as you say, a unique index with many duplicates could be a problem, and index fragmentation would make the situation worse. -- Simon Riggshttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: Logical decoding fast-forward and slot advance

2018-01-18 Thread Simon Riggs
On 14 January 2018 at 23:15, Petr Jelinek wrote: > Updated version attached. Applied, thanks -- Simon Riggshttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: [HACKERS] Surjective functional indexes

2018-01-18 Thread Simon Riggs
On 18 January 2018 at 08:59, Konstantin Knizhnik wrote: > > > On 18.01.2018 11:38, Simon Riggs wrote: >> >> On 10 January 2018 at 09:54, Konstantin Knizhnik >> wrote: >> >>> Sorry, issue with documentation is fixed. >> >> OK, thanks. >&g

Re: [HACKERS] Surjective functional indexes

2018-01-18 Thread Simon Riggs
t)} {substring(Columname, Constants...)} It would be a shame if people had to remember to use this for the common and obvious cases. -- Simon Riggshttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: [HACKERS] PATCH: enabling parallel execution for cursors explicitly (experimental)

2018-01-17 Thread Simon Riggs
quot;when"? If a parallel cursor is requested, we could simply prevent other intermediate commands other than FETCH (next). -- Simon Riggshttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: [HACKERS] PATCH: enabling parallel execution for cursors explicitly (experimental)

2018-01-17 Thread Simon Riggs
the patch as proposed is about the easiest 3-5% of what would actually > be required to cover all the holes. Maybe so. At present, one major use of Cursors is in postgres_fdw. In that usage, the query executes and returns all the rows. No other side execution is possible. How do we make parallel

Re: [HACKERS] Replication status in logical replication

2018-01-16 Thread Simon Riggs
On 16 January 2018 at 06:21, Michael Paquier wrote: > On Tue, Jan 16, 2018 at 10:40:43AM +0900, Masahiko Sawada wrote: >> On Sun, Jan 14, 2018 at 12:43 AM, Simon Riggs wrote: >>> On 9 January 2018 at 04:36, Masahiko Sawada wrote: >>> We're not talking about stand

Re: let's not complain about harmless patch-apply failures

2018-01-16 Thread Simon Riggs
ple complain... asking them not to is unlikely to get anywhere. We must encourage people to speak up if they see an improvement or a lack of quality. I have benefited from such comments and they are not often intended negatively. Every complaint is not a hard blocker and complainers can also be w

Re: [HACKERS] Replication status in logical replication

2018-01-13 Thread Simon Riggs
;> >> Thoughts on better wording? >> > > I think that this DEBUG1 message appears when sent any WAL after > caught up even without this patch. This patch makes this message > appear at a properly timing. Or am I missing something? We're not talking about standby

Re: Changing WAL Header to reduce contention during ReserveXLogInsertLocation()

2018-01-13 Thread Simon Riggs
e can use that to match against in the case of error. XLogPageHeaderData can be extended by 2 bytes without increasing its size when using 8 byte alignment. The new 2 byte value is the same anywhere in the file, so that works quickly and easily. And it doesn't increase the size

Re: let's make the list of reportable GUCs configurable (was Re: Add %r substitution for psql prompts to show recovery status)

2018-01-10 Thread Simon Riggs
On 10 January 2018 at 16:08, Robert Haas wrote: > I think that we really need to think about allowing clients to tell > the server which GUCs they'd like reported, instead of having a single > list to which everyone is bound. +1 -- Simon Riggshttp://www.2

Re: Rangejoin rebased

2018-01-10 Thread Simon Riggs
per-optimization.html#id-1.8.3.17.9 https://www.postgresql.org/docs/devel/static/planner-optimizer.html -- Simon Riggshttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: Rangejoin rebased

2018-01-10 Thread Simon Riggs
On 10 January 2018 at 04:24, Jeff Davis wrote: > On Sat, Jan 6, 2018 at 10:38 AM, Simon Riggs wrote: >> For this to be useful, it needs to include some details of how to use >> it when people have NOT used range datatypes in their tables. > > Good idea. I added an example th

Re: [HACKERS] Statement-level rollback

2018-01-07 Thread Simon Riggs
On 6 November 2017 at 12:36, MauMau wrote: > when I submit the next revision of my patch. When will the next version be posted? -- Simon Riggshttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: Logical decoding fast-forward and slot advance

2018-01-07 Thread Simon Riggs
ere to make sure we understand and avoid breakage. In pg_replication_slot_advance() the moveto variable is set to PG_GETARG_LSN(1) and then unconditionally overwritten before it is used for anything. Why? -- Simon Riggshttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: [HACKERS] Replication status in logical replication

2018-01-07 Thread Simon Riggs
this removed (v4alt2) I've removed the edit that fusses over English grammar: both ways are correct. > I think this patch can be > back-patched to 9.4 as Simon mentioned. This patch appears to cause this DEBUG1 message "standby \"%s\" has now caught up with primary&q

Re: [HACKERS] [PROPOSAL] Temporal query processing with range types

2018-01-07 Thread Simon Riggs
ADME explains the ALIGN operation step-by-step with a > TEMPORAL LEFT OUTER JOIN example. That is, we start from a query > input, show how we rewrite it during parser stage, and show how the > final execution generates result tuples. Sorry, this was too complex for me. Can we get a muc

Re: [HACKERS] [PROPOSAL] Temporal query processing with range types

2018-01-06 Thread Simon Riggs
as needed So the idea is we enable Postgres to allow major new functionality, as was done for PostGIS so successfully. We can adopt syntax into the main parser later once SQLStandard accepts this, or some munged version of it. -- Simon Riggshttp://www.2ndQuadrant.com/ PostgreSQL D

Re: Rangejoin rebased

2018-01-06 Thread Simon Riggs
e want this, it might be better for the EXPLAIN to say "Time Range Join" when the ranges being joined are Time Ranges, and for other cases to just say "Range Join". The use of the word Merge doesn't help much there. -- Simon Riggshttp://www.2ndQuadrant

Re: unique indexes on partitioned tables

2018-01-06 Thread Simon Riggs
. I agree we want 0004 also, but it would be simpler to just push 0002 and 0003 now and come back later for 0004. That would also avoid any confusion over patch credits. > This serves as basis to build foreign keys on top; I'll post that > separately. -- Simon Riggshttp://w

Re: FOR EACH ROW triggers on partitioned tables

2018-01-06 Thread Simon Riggs
n make pg_dump do the right things. That's all mostly legwork, I > think. > > Also, does ALTER TABLE ... ENABLE/DISABLE TRIGGER do the right things on > partitioned tables? Not sure I care about that, since it just breaks FKs and other things, but we can add it later. -- S

Re: [HACKERS] Creating backup history files for backups taken from standbys

2018-01-05 Thread Simon Riggs
re >> still written to disk, so my take on the matter is to keep the code >> simple. > > I'm OK with that. I'm not. If we want to do this why not only do it in the modes that have meaning? i.e. put an if() test in for archive_mode == always Which also makes it a smaller and clearer patch -- Simon Riggshttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: Fix permissions check on pg_stat_get_wal_senders

2018-01-05 Thread Simon Riggs
y for > committer. Simon, as the original committer of 25fff407, could you look > at what is proposed here? Yup, I got this. -- Simon Riggshttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: [HACKERS] Timeline ID in backup_label file

2018-01-05 Thread Simon Riggs
ood so I will mark it ready for committer. Sounds good. No tests? No docs/extended explanatory comments? -- Simon Riggshttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: [HACKERS] MERGE SQL Statement for PG11

2018-01-04 Thread Simon Riggs
On 4 January 2018 at 17:29, Robert Haas wrote: > On Sat, Dec 30, 2017 at 6:01 AM, Simon Riggs wrote: >> Patch uses mechanism as agreed previously with Peter G et al. on this thread. > > I'm not sure that an agreement was reached, or what the substance of > that agreemen

Re: [PATCH] Logical decoding of TRUNCATE

2018-01-04 Thread Simon Riggs
On 29 December 2017 at 19:55, Andres Freund wrote: > Hi, > > On 2017-12-29 14:15:22 +0100, Marco Nenciarini wrote: >> This patch implements support for TRUNCATE statements >> in logical replication. The work has mainly done by Simon Riggs then >> finished by

Changing WAL Header to reduce contention during ReserveXLogInsertLocation()

2017-12-30 Thread Simon Riggs
run a multi-CPU workload with heavy WAL inserts to show benefits. [1] pg_rewind is the only exception which uses xl_prev to find the previous checkpoint record. But we can always start from the beginning of the WAL segment and read forward until we find the checkpoint record. The patch does just the

Re: [HACKERS] Commits don't block for synchronous replication

2017-12-29 Thread Simon Riggs
On 1 December 2017 at 02:50, Michael Paquier wrote: > On Fri, Nov 24, 2017 at 11:38 PM, Simon Riggs wrote: >> On 23 November 2017 at 11:11, Michael Paquier >> wrote: >> >>> This is older than the bug report of this thread. All those >>> indications

Re: Add hint about replication slots when nearing wraparound

2017-12-29 Thread Simon Riggs
prepared > transactions, or drop stale replication slots."))); > > I am not convinced that you need this bit. autovacuum_freeze_max_age can > be set to lower to even lower values than the default. > > Still, those are minor comments, so I am marking this patc

Re: [HACKERS] Surjective functional indexes

2017-12-13 Thread Simon Riggs
On 4 December 2017 at 15:35, Konstantin Knizhnik wrote: > On 30.11.2017 05:02, Michael Paquier wrote: >> >> On Wed, Sep 27, 2017 at 4:07 PM, Simon Riggs >> wrote: >>> >>> On 15 September 2017 at 16:34, Konstantin Knizhnik >>> wrote: >>> &

Re: [HACKERS] logical decoding of two-phase transactions

2017-12-12 Thread Simon Riggs
x27;s not enough. Not sure what that means. I think we would need to fire invalidations at COMMIT PREPARED, yet logically decode them at PREPARE. -- Simon Riggshttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: Rethinking MemoryContext creation

2017-12-11 Thread Simon Riggs
On 11 December 2017 at 17:38, Tom Lane wrote: > Simon Riggs writes: >> On 11 December 2017 at 16:27, Tom Lane wrote: >>> For a *very* large majority of the callers of AllocSetContextCreate, >>> the context name is a simple C string constant, so we could just store >

Re: Rethinking MemoryContext creation

2017-12-11 Thread Simon Riggs
l in that case? -- Simon Riggshttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: Postgres with pthread

2017-12-06 Thread Simon Riggs
0 in both cases, so why bother optimising for 1000 connections at all? In which case we should read the benefit at the 100 connections line, where it shows the lower 28% gain, closer to the gain your colleague reported. So I think we don't yet have enough to make a decision. -- Simon Riggs

Re: [HACKERS] SQL procedures

2017-12-01 Thread Simon Riggs
round and change all mentions of "function" > like that. That might just confuse people. Agreed \dfn shows functions only so we might want to consider having \df say "List of functions and procedures" \dfn say "List of functions" and a new option to list procedures

Re: [HACKERS] INSERT ON CONFLICT and partitioned tables

2017-11-30 Thread Simon Riggs
me cases. Note that it will > only check conflicts for the individual leaf partitions using whatever > constraint-enforcing indexes they might have. So we should have 2 patches. One for now that does DO NOTHING and another that adds the change that depends upon Alvaro's work. -- Sim

Re: [HACKERS] Issues with logical replication

2017-11-30 Thread Simon Riggs
the > code directly into SnapBuildWaitSnapshot(). This will still fail an Assert in TransactionIdIsInProgress() when snapshots are overflowed. -- Simon Riggshttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: [HACKERS] GUC for cleanup indexes threshold.

2017-11-29 Thread Simon Riggs
needless cleanup scans. So this leaves us in the situation that 1. Masahiko's patch has unresolved problems 2. Yura's patch works and is useful Unless there is disagreement on the above, it seems we should apply Yura's patch (an edited version, perhaps). -- Simon Riggs

Re: [HACKERS] SQL procedures

2017-11-28 Thread Simon Riggs
On 29 November 2017 at 02:03, Peter Eisentraut wrote: > Here is a new patch that addresses the previous review comments. > > If there are no new comments, I think this might be ready to go. Is ERRCODE_INVALID_FUNCTION_DEFINITION still appropriate? Other than that, looks ready to go.

<    2   3   4   5   6   7   8   >