Re: [HACKERS] Proposal for changes to recovery.conf API

2016-09-06 Thread Simon Riggs
n to proceed? I guess we could keep the old parameters and make them PGC_POSTMASTER, but also provide a new parameter called recovery_target that simplifies the API and is PGC_SIGHUP. That way we resolve the annoyance of handling the current ones but keep compatibility for those who can

Re: [HACKERS] Speedup twophase transactions

2016-09-06 Thread Simon Riggs
On 6 September 2016 at 02:41, Michael Paquier <michael.paqu...@gmail.com> wrote: > After review the result is attached. Perhaps a committer could get a look at > it? Yes, will do, but it will be a few more days yet. -- Simon Riggshttp://www.2ndQuadrant.com

Re: [HACKERS] Vacuum: allow usage of more than 1GB of work mem

2016-09-06 Thread Simon Riggs
new patch also. This whole idea of backwards scanning to confirm truncation seems wrong. What we want is an O(1) solution. Thinking. -- Simon Riggshttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services -- Sent via pgsq

Re: [HACKERS] Vacuum: allow usage of more than 1GB of work mem

2016-09-05 Thread Simon Riggs
On 5 September 2016 at 15:50, Claudio Freire <klaussfre...@gmail.com> wrote: > On Sun, Sep 4, 2016 at 3:46 AM, Simon Riggs <si...@2ndquadrant.com> wrote: >> On 3 September 2016 at 04:25, Claudio Freire <klaussfre...@gmail.com> wrote: >>> The patch also make

Re: [HACKERS] new autovacuum criterion for visible pages

2016-09-05 Thread Simon Riggs
every minute. We need to record something that will tell us how many VM bits have been cleared, which will then allow autovac to do a simple SELECT to decide what needs vacuuming. Vik's proposal to keep track of the rows inserted seems like the best approach to this issue. -- Simon Riggs

Re: [HACKERS] pg_sequence catalog

2016-09-05 Thread Simon Riggs
broken situation. I vote in favour of applying Peter's idea/patch, though if that is not acceptable I don't think its worth spending further time on for any of us, so if thats the case lets Return with Feedback and move on. -- Simon Riggshttp://www.2ndQuadrant.com/ PostgreS

Re: [HACKERS] INSERT .. SET syntax

2016-09-05 Thread Simon Riggs
On 3 July 2016 at 20:36, Marko Tiikkaja <ma...@joh.to> wrote: > Here's a patch for $SUBJECT. I'll probably work on the docs a bit more > before the next CF, but I thought I'd post it anyway. I think this should be Returned With Feedback. -- Simon Riggs

Re: [HACKERS] Assert(LWLockHeldByMeInMode(lock, LW_EXCLUSIVE))

2016-09-05 Thread Simon Riggs
ired. > > What do you think about something like the attached? See also an > example of use. I will add this to the next commitfest. Committed, thanks. -- Simon Riggshttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: [HACKERS] Optimization for lazy_scan_heap

2016-09-05 Thread Simon Riggs
3.anarazel.de If we have a freeze map now, surely tables will no longer be entirely frozen? What performance difference does this make, in a realistic use case? How would we test this to check it is exactly correct? -- Simon Riggshttp://www.2ndQuadrant.com/ PostgreSQL Devel

Re: [HACKERS] Logical decoding slots can go backwards when used from SQL, docs are wrong

2016-09-05 Thread Simon Riggs
n annoyance > not a serious data risking bug. I've committed to HEAD only. We can discuss backpatching elsewhere also. -- Simon Riggshttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services -- Sent via pgsql-hackers mailing l

Re: [HACKERS] LSN as a recovery target

2016-09-05 Thread Simon Riggs
On 5 September 2016 at 06:55, Michael Paquier <michael.paqu...@gmail.com> wrote: > On Sun, Sep 4, 2016 at 11:30 PM, Simon Riggs <si...@2ndquadrant.com> wrote: >> I noticed we don't mention what LSN is anywhere, so I'd like to apply >> the following doc patch also. >

Re: [HACKERS] LSN as a recovery target

2016-09-04 Thread Simon Riggs
e recovery >> parameter project, please remind me to commit that as well. > > I'm aware of this, and will adjust accordingly in the GUC patch. Thanks > for the heads up. I noticed we don't mention what LSN is anywhere, so I'd like to apply the following doc patch also. -- Simo

Re: [HACKERS] Vacuum: allow usage of more than 1GB of work mem

2016-09-04 Thread Simon Riggs
he OS more cache, especially if work > mem is configured very high to avoid multiple index scans. How long does that part ever take? Is there any substantial gain from this? Lets discuss that as a potential second patch. > Tested with pgbench scale 4000 after deleting the whole > pgbench_ac

Re: [HACKERS] LSN as a recovery target

2016-09-04 Thread Simon Riggs
On 4 September 2016 at 04:50, Michael Paquier <michael.paqu...@gmail.com> wrote: > On Sun, Sep 4, 2016 at 8:05 AM, Michael Paquier > <michael.paqu...@gmail.com> wrote: >> On Sun, Sep 4, 2016 at 1:57 AM, Simon Riggs <si...@2ndquadrant.com> wrote: >>> On 2

Re: [HACKERS] LSN as a recovery target

2016-09-03 Thread Simon Riggs
On 24 August 2016 at 05:50, Michael Paquier <michael.paqu...@gmail.com> wrote: >>> Everything else looks in good order. Committed. Thanks. -- Simon Riggshttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: [HACKERS] pg_hba_file_settings view patch

2016-09-03 Thread Simon Riggs
ould live longer than the concept of pg_hba.conf, which seems likely to become part of ALTER SYSTEM in future, so we wouldn't really want the word "file" in there. I've not seen anything yet to make me think a commit for this wouldn't happen once we've worked the detail. Than

Re: [HACKERS] autonomous transactions

2016-09-03 Thread Simon Riggs
d have better infrastructure later, e.g. LISTEN/NOTIFY. I would very much like to see ATs in v10 and this is a viable approach with working code. -- Simon Riggshttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services -- Sent via

Re: [HACKERS] [PATCH] Transaction traceability - txid_status(bigint)

2016-09-02 Thread Simon Riggs
On 2 September 2016 at 13:16, Craig Ringer <cr...@2ndquadrant.com> wrote: > So I've moved it to xlog.c... I'm pretty sure it shouldn't live in xlog.c, but there may be some good reason I can't see yet. -- Simon Riggshttp://www.2ndQuadrant.com/ PostgreSQL Developm

Re: [HACKERS] LOCK TABLE .. DEFERRABLE

2016-09-01 Thread Simon Riggs
On 5 April 2016 at 18:34, Rod Taylor <rod.tay...@gmail.com> wrote: > > > On Tue, Apr 5, 2016 at 1:10 PM, Simon Riggs <si...@2ndquadrant.com> wrote: >>> >>> If a lock is successfully obtained on one table, but not on all tables, >>> it releases

Re: [HACKERS] Speedup twophase transactions

2016-09-01 Thread Simon Riggs
ything still works on current HEAD for the new CF, so review can start. Thanks -- Simon Riggshttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make change

Re: [HACKERS] WAL consistency check facility

2016-09-01 Thread Simon Riggs
will give people a very safe feeling. I think the primary use for an rmgr filter might well be PostgreSQL developers. -- Simon Riggshttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services -- Sent via pgsql-hackers mailing list (

Re: [HACKERS] Choosing parallel_degree

2016-09-01 Thread Simon Riggs
On 12 April 2016 at 14:11, Simon Riggs <si...@2ndquadrant.com> wrote: > On 12 April 2016 at 13:53, Robert Haas <robertmh...@gmail.com> wrote: >> >> On Mon, Apr 11, 2016 at 5:45 PM, Simon Riggs <si...@2ndquadrant.com> >> wrote: >> > On 8 April 2

Re: [HACKERS] Logical decoding slots can go backwards when used from SQL, docs are wrong

2016-09-01 Thread Simon Riggs
> >> Yes. Both those patches are still pending and should be considered for >> commit in the next CF. (Of course, I think they should just be >> committed, but I would, wouldn't I?) >> >> > > I think the doc one should definitely go in and possibly be back-patched all >

Re: [HACKERS] WAL consistency check facility

2016-09-01 Thread Simon Riggs
On 1 September 2016 at 11:16, Robert Haas <robertmh...@gmail.com> wrote: > On Wed, Aug 31, 2016 at 7:02 PM, Simon Riggs <si...@2ndquadrant.com> wrote: >> I'd prefer a solution that was not dependent upon RmgrID at all. >> >> If there are various special case

Re: [HACKERS] Proposal for changes to recovery.conf API

2016-09-01 Thread Simon Riggs
h). Rather than trying > to untangle that right now, I'm posting what I have as-is, and I'll post > an updated version tomorrow. Thanks. archive_cleanup_command no longer needs to be in shmem. Checkpointer will have its own copy of the value. -- Simon Riggshttp://www.2ndQua

Re: [HACKERS] Proposal for changes to recovery.conf API

2016-09-01 Thread Simon Riggs
On 1 September 2016 at 06:34, Michael Paquier <michael.paqu...@gmail.com> wrote: > On Thu, Sep 1, 2016 at 1:15 AM, Simon Riggs <si...@2ndquadrant.com> wrote: >> This is a summary of proposed changes to the recovery.conf API for >> v10. These are based in part on earlier

Re: [HACKERS] pg_basebackup wish list

2016-09-01 Thread Simon Riggs
gress information\n")); >> >> Registered this patch to CF1. > > +1 for the idea. Looking at the patch it is taking a sane approach. Apart from this one liner change we look good to go. -- Simon Riggshttp://www.2ndQuadrant.com/ PostgreSQL Development,

Re: [HACKERS] autonomous transactions

2016-08-31 Thread Simon Riggs
audited would ever get executed. Not sure I see why you would want to turn off execution for only some functions. What happens if your function calls some other function with side-effects? How would you roll that back? How would you mark functions for the general case? Functions with side effects c

[HACKERS] Proposal for changes to recovery.conf API

2016-08-31 Thread Simon Riggs
promote" * Remove user configurable "trigger_file" mechanism - use "promote.trigger" for all cases * Remove Fallback promote mechanism, so all promotion is now "fast" in xlog.c * Rename CheckForStandbyTrigger() to CheckForPromoteTrigger() for clarity (Patch: recovery

Re: [HACKERS] WAL consistency check facility

2016-08-31 Thread Simon Riggs
not dependent upon RmgrID at all. If there are various special cases that we need to cater for, ISTM they would be flaws in the existing WAL implementation rather than anything we would want to perpetuate. I hope we'll spend time fixing them rather than add loads of weird code to work arou

Re: [HACKERS] [COMMITTERS] pgsql: Fix pg_receivexlog --synchronous

2016-08-31 Thread Simon Riggs
On 29 August 2016 at 12:34, Tom Lane <t...@sss.pgh.pa.us> wrote: > Simon Riggs <si...@2ndquadrant.com> writes: >> Fix pg_receivexlog --synchronous > > The buildfarm says you broke the 9.5 branch. > > In general, pushing inessential patches just a few hours before

Re: [HACKERS] pg_receivexlog does not report flush position with --synchronous

2016-08-29 Thread Simon Riggs
On 23 August 2016 at 14:57, Michael Paquier <michael.paqu...@gmail.com> wrote: > On Tue, Aug 23, 2016 at 9:48 PM, Gabriele Bartolini > <gabriele.bartol...@2ndquadrant.it> wrote: >> Hi Simon and Michael, >> >> 2016-08-23 10:39 GMT+02:00 Simon Riggs <si...@2n

Re: [HACKERS] WAL consistency check facility

2016-08-27 Thread Simon Riggs
On 27 August 2016 at 07:36, Amit Kapila <amit.kapil...@gmail.com> wrote: > On Fri, Aug 26, 2016 at 9:26 PM, Simon Riggs <si...@2ndquadrant.com> wrote: >> >> I think you should add this as part of the default testing for both >> check and installcheck. I can't ima

Re: [HACKERS] Renaming of pg_xlog and pg_clog

2016-08-26 Thread Simon Riggs
ake space. That helps the few people who made such mistakes, but doesn't cause massive change as a result. -- Simon Riggshttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@p

Re: [HACKERS] Renaming some binaries

2016-08-26 Thread Simon Riggs
ous users will be issuing more than one DDL command at a time, so they'll be writing a script anyway. -- Simon Riggshttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@po

Re: [HACKERS] WAL consistency check facility

2016-08-26 Thread Simon Riggs
't assume anything. > 5. Generalize the page type identification technique. Why not do this first? There are some coding guideline stuff to check as well. Thanks -- Simon Riggshttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Serv

Re: [HACKERS] Renaming of pg_xlog and pg_clog

2016-08-26 Thread Simon Riggs
doing it rather than assume it is self evident, cos it certainly isn't. -- Simon Riggshttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make cha

Re: [HACKERS] increasing the default WAL segment size

2016-08-25 Thread Simon Riggs
on. If we do have the pain of change, should we also consider making WAL files variable length? What do we gain by having the files all the same size? ISTM better to have WAL files that vary in length up to 1GB in size. (This is all about XLOG_SEG_SIZE; I presume XLOG_BLCKSZ can stay as it i

Re: [HACKERS] WAL consistency check facility

2016-08-24 Thread Simon Riggs
On 22 August 2016 at 16:56, Simon Riggs <si...@2ndquadrant.com> wrote: > On 22 August 2016 at 13:44, Kuntal Ghosh <kuntalghosh.2...@gmail.com> wrote: > >> Please let me know your thoughts on this. > > Do the regression tests pass with this option enabled?

Re: [HACKERS] LSN as a recovery target

2016-08-23 Thread Simon Riggs
and in the messages generated prior to starting to search. Everything else looks in good order. -- Simon Riggshttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To

Re: [HACKERS] pg_receivexlog does not report flush position with --synchronous

2016-08-23 Thread Simon Riggs
tch. Agreed, but I'd move all the comments above the block. -- Simon Riggshttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services pg-receivexlog-not-report-sync-flush.v2.patch Description: Binary data -- Sent via pgsql-hackers ma

Re: [HACKERS] WAL consistency check facility

2016-08-22 Thread Simon Riggs
On 22 August 2016 at 13:44, Kuntal Ghosh <kuntalghosh.2...@gmail.com> wrote: > Please let me know your thoughts on this. Do the regression tests pass with this option enabled? -- Simon Riggshttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA,

Re: [HACKERS] Pluggable storage

2016-08-18 Thread Simon Riggs
quite concerned about that as well. This objection would apply to all other proposals as well, FDW etc.. Do you see some way to add flexibility yet without adding a branch point in the code? -- Simon Riggshttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA,

Re: [HACKERS] Why we lost Uber as a user

2016-08-17 Thread Simon Riggs
by DRBD and all other block-level HA solutions, and also used by other databases, such as Oracle. Corruption on the master would often cause errors that would prevent writes and therefore those changes wouldn't even be made, let alone be replicated. -- Simon Riggshttp://www.2ndQu

Re: [HACKERS] Heap WARM Tuples - Design Draft

2016-08-10 Thread Simon Riggs
ium term plan. And this is kind of independent from > the core idea. +1 That seems like a good idea. It would allow us to produce a bitmap scan in blocksorted order. > So I'll go ahead and write a patch that implements the core > idea and some basic optimizations. +1 > We can then try differ

Re: [HACKERS] Small issues in syncrep.c

2016-08-10 Thread Simon Riggs
's suggestion. Looks good to apply immediately. 14e8803f1 was only a partial patch for the syncrep code, so I don't see any reason to keep the code as it currently is in 9.5/9.6. Any objections to backpatching this to 9.5 and 9.6? -- Simon Riggshttp://www.2ndQuadrant.com/ PostgreS

Re: [HACKERS] Logical Replication WIP

2016-08-05 Thread Simon Riggs
but > after that I plan to spend a fair amount of time helping to review > etc. this. Have a good one. -- Simon Riggshttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services -- Sent via pgsql-hackers mailing list (pgsql-hacke

Re: [HACKERS] PostgreSQL 10 Roadmaps

2016-08-05 Thread Simon Riggs
On 5 August 2016 at 04:40, Etsuro Fujita <fujita.ets...@lab.ntt.co.jp> wrote: > On 2016/08/02 23:54, Simon Riggs wrote: >> >> https://wiki.postgresql.org/wiki/PostgreSQL10_Roadmap > > > Thank you for creating the wiki page! > > I added a link to the NTT road

Re: [HACKERS] Lossy Index Tuple Enhancement (LITE)

2016-08-04 Thread Simon Riggs
es. I described that... "The same situation can occur if we have many INSERTs with same values on the same block. This could lead to a reduction in size of the btree for indexes with many duplicate values." -- Simon Riggshttp://www.2ndQuadrant.com/ PostgreSQL Developme

Re: [HACKERS] Heap WARM Tuples - Design Draft

2016-08-04 Thread Simon Riggs
On 4 August 2016 at 18:17, Andres Freund <and...@anarazel.de> wrote: > On 2016-08-04 18:11:17 +0100, Simon Riggs wrote: >> On 4 August 2016 at 17:31, Andres Freund <and...@anarazel.de> wrote: >> > Hi, >> > >> > On 2016-08-04 16:29:09 +0530, Pav

Re: [HACKERS] Heap WARM Tuples - Design Draft

2016-08-04 Thread Simon Riggs
ou would need to > set HEAP_RECHECK_REQUIRED earlier than that. Hmm. Mostly there will be one, so this is just for the first update after any VACUUM. Adding a new linepointer just to hold this seems kludgy and could mean we run out of linepointers. -- Simon Riggshttp://www.2ndQu

Re: [HACKERS] Heap WARM Tuples - Design Draft

2016-08-04 Thread Simon Riggs
mart way to > figure that out? Hmm, sorry, I did think of that point and I thought I had added it to the doc. So, yes, I agree - re-locating the root is the biggest downside to this idea. Perhaps Pavan has other thoughts? I think its doable, but it will be fiddly. -- Simon Riggs

Re: [HACKERS] Lossy Index Tuple Enhancement (LITE)

2016-08-04 Thread Simon Riggs
n made worse by non-HOT updates. So, we have a choice of which disadvantage to accept. > Also, why not use this bitmap for all indexes, not just update chains? I don't understand where you get this update chains thing from. The bitmap can apply to multiple tuples on one page, which is described.

Re: [HACKERS] Lossy Index Tuple Enhancement (LITE)

2016-08-04 Thread Simon Riggs
On 3 August 2016 at 20:37, Claudio Freire <klaussfre...@gmail.com> wrote: > On Wed, Aug 3, 2016 at 4:20 AM, Simon Riggs <si...@2ndquadrant.com> wrote: >> == IndexScan == >> >> Note that the executor code for IndexScan appears identical between >> the tw

Re: [HACKERS] Design for In-Core Logical Replication

2016-08-04 Thread Simon Riggs
to get this right, since this is much more about UI than system architecture. -- Simon Riggshttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make change

Re: [HACKERS] Lossy Index Tuple Enhancement (LITE)

2016-08-03 Thread Simon Riggs
existing index entry for the same values and heap page? That seems > quite complicated. The insertion algorithm is described. Doesn't seem complicated to me. -- Simon Riggshttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services -- Sent

Re: [HACKERS] Why we lost Uber as a user

2016-08-03 Thread Simon Riggs
ex you could > possibly get a similar index, but this might allow such an index to > still be usable for index-only scans, which a BRIN index will never be > able to provide. This idea is vaguely similar to the concepts I've been working on, based on earlier work in 2007. I'm starting a new po

[HACKERS] Lossy Index Tuple Enhancement (LITE)

2016-08-03 Thread Simon Riggs
. There are no changes to WAL records or replay. The details are all in the way we insert and interpret index tuples. == Other designs == This is related in a few ways to Grouped Item Tuple indexes, a patch Heikki worked on in 2007, but is not that close. -- Simon Riggshttp://www.2

Re: [HACKERS] Why we lost Uber as a user

2016-08-02 Thread Simon Riggs
it and it works, yay!... but Pavan has managed to shoot it down with some accurate observations about it leading to an annoying accumulation of root pointers and complex logic to remove them. So I'm not pursuing it further at this stage. I'm writing up my conclusions around what we should do no

[HACKERS] PostgreSQL 10 Roadmaps

2016-08-02 Thread Simon Riggs
https://wiki.postgresql.org/wiki/PostgreSQL10_Roadmap -- Simon Riggshttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscrip

Re: [HACKERS] PostgreSQL 10 kick-off

2016-08-01 Thread Simon Riggs
On 1 August 2016 at 17:04, Tom Lane <t...@sss.pgh.pa.us> wrote: > Simon Riggs <si...@2ndquadrant.com> writes: >> Can we confirm/refute these details now to make sure we are all in tune? > > See the other thread I started about that; please reserve this thread > for

Re: [HACKERS] PostgreSQL 10 kick-off

2016-08-01 Thread Simon Riggs
ll change to #define PG_VERSION_NUM 11 effectively allowing many future versions. Can we confirm/refute these details now to make sure we are all in tune? -- Simon Riggshttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services -

Re: [HACKERS] Design for In-Core Logical Replication

2016-07-22 Thread Simon Riggs
> And we don't want handle events on third table which was added to publication. Not sure what you mean for that part. Could you explain a little more? (I'm on leave for some days, so I don't respond quickly I will respond eventually) -- Simon Riggshttp://www.2ndQuadrant.com/ P

Re: [HACKERS] Design for In-Core Logical Replication

2016-07-20 Thread Simon Riggs
ormal even if only 2 nodes. > Old slow moving data almost always got different treatment than fast-moving > data; even if only defining which set needs to hit the other node first and > which set can trickle through later. > Agreed -- Simon Riggshttp://www.2ndQu

Re: [HACKERS] Design for In-Core Logical Replication

2016-07-20 Thread Simon Riggs
ider node has one or more Databases, each of which can publish its data in zero, one or more PUBLICATIONs. A Subscribing node can receive data in the form of zero, one or more SUBSCRIBERs, where each SUBSCRIBER may bring together data from one or more PUBLICATIONs. -- Simon Riggshttp://www.2ndQuadrant.com/ <http://www.2ndquadrant.com/> PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

[HACKERS] Design for In-Core Logical Replication

2016-07-20 Thread Simon Riggs
will start the replication process which synchronizes the initial table contents of users and departments tables and then starts replicating incremental changes to those tables. -- Simon Riggshttp://www.2ndQuadrant.com/ <http://www.2ndquadrant.com/> PostgreSQL D

Re: [HACKERS] One process per session lack of sharing

2016-07-17 Thread Simon Riggs
s, so care must be taken. I think its clear that threading is out, but it is far from being the only solution to reducing the memory overhead of sharing. Analysing the overhead and suggesting practical ways forward may help. -- Simon Riggshttp://www.2ndQuadrant.com/ <

Re: [HACKERS] pg_xlogdump follow into the future

2016-07-14 Thread Simon Riggs
can argue it's a bugfix > though, since the usecase simply did not work... > Good thinking. Patch looks good, but not tested. Needs comment on the second chunk. -- Simon Riggshttp://www.2ndQuadrant.com/ <http://www.2ndquadrant.com/> PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: [HACKERS] Simplifying the interface of UpdateMinRecoveryPoint

2016-07-13 Thread Simon Riggs
e cleaned up as we go. -- Simon Riggshttp://www.2ndQuadrant.com/ <http://www.2ndquadrant.com/> PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: [HACKERS] A Modest Upgrade Proposal

2016-07-08 Thread Simon Riggs
criptions), I definitely think we should dump replication sets and > their membership info for example. Agreed -- Simon Riggshttp://www.2ndQuadrant.com/ <http://www.2ndquadrant.com/> PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: [HACKERS] A Modest Upgrade Proposal

2016-07-08 Thread Simon Riggs
standard operating procedure. > Agreed, but it has always been considered to be something we should consider when making DDL work. -- Simon Riggshttp://www.2ndQuadrant.com/ <http://www.2ndquadrant.com/> PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: [HACKERS] A Modest Upgrade Proposal

2016-07-07 Thread Simon Riggs
nto syntax isn't warranted at this early stage. Please lets discuss those important things first, then return to whether DDL makes sense or not; it may do, or may not, or more likely which parts of it need DDL and which do not. -- Simon Riggshttp://www.2ndQuadrant.com/ <http://www.2ndquadrant.com/> PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: [HACKERS] A Modest Upgrade Proposal

2016-07-07 Thread Simon Riggs
On 8 July 2016 at 02:41, Robert Haas <robertmh...@gmail.com> wrote: > On Thu, Jul 7, 2016 at 7:15 PM, Simon Riggs <si...@2ndquadrant.com> wrote: > > Yes, I ran the unconference session. It was a shame you weren't able to > stay > > for the whole discussion. > &g

Re: [HACKERS] A Modest Upgrade Proposal

2016-07-07 Thread Simon Riggs
On 8 July 2016 at 01:47, Joshua D. Drake <j...@commandprompt.com> wrote: > * Long running transaction > And of course you can't run any transactions at all during pg_upgrade, not just long running ones. -- Simon Riggshttp://www.2ndQuadrant.com/

Re: [HACKERS] pg_xlogfile_name_offset() et al and recovery

2016-07-07 Thread Simon Riggs
On 8 July 2016 at 00:43, Michael Paquier <michael.paqu...@gmail.com> wrote: > On Thu, Jul 7, 2016 at 6:26 PM, Simon Riggs <si...@2ndquadrant.com> wrote: > > The behaviour of that function is defined in backbranches, so I suggest > we > > should not alter that now.

Re: [HACKERS] A Modest Upgrade Proposal

2016-07-07 Thread Simon Riggs
vers. Anyway, I'm cool if you don't want to use it, for while or never. Options are good. -- Simon Riggshttp://www.2ndQuadrant.com/ <http://www.2ndquadrant.com/> PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: [HACKERS] A Modest Upgrade Proposal

2016-07-07 Thread Simon Riggs
ical replication fully integrated >> into core today. Which we by no means do. >> > > I would much rather see more brain power put into pg_upgrade or in place > upgrades than logical replication (as a upgrade solution). Why is that? -- Simon Riggshttp://

Re: [HACKERS] A Modest Upgrade Proposal

2016-07-07 Thread Simon Riggs
asier to separate out that as a use-case so we can be sure we get that in 10.0, even if nothing else lands. -- Simon Riggshttp://www.2ndQuadrant.com/ <http://www.2ndquadrant.com/> PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: [HACKERS] A Modest Upgrade Proposal

2016-07-07 Thread Simon Riggs
DDL, such as backup, advisory locks and some features that use DDL that don't really need to such as LISTEN/NOTIFY, full text search etc.. Also note that both Oracle and SQLServer have moved away from DDL in favour of function APIs, most NoSQL databases and almost all languages prefer func

Re: [HACKERS] MVCC overheads

2016-07-07 Thread Simon Riggs
anged row versions into a separate data structure, so would be much more amenable to offload than PostgreSQL, in its current form. Maybe look at SLRUs (clog etc) as a place to offload something? -- Simon Riggshttp://www.2ndQuadrant.com/ <http://www.2ndquadrant.com/> PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: [HACKERS] MVCC overheads

2016-07-07 Thread Simon Riggs
e costs associated with inconsistent and incorrect results in applications, or other architectural restrictions imposed to make that possible. It's easy to make assumptions in the lab that don't work in the real world. -- Simon Riggshttp://www.2ndQuadrant.com/ <http://www.2ndquadrant.com

Re: [HACKERS] pg_xlogfile_name_offset() et al and recovery

2016-07-07 Thread Simon Riggs
g_stop_backup(). We would like to do something like this... select pg_xlogfile_name_offset(l.lsn, l.tli) from pg_stop_backup(false) l; So I suggest we add another column to the output of pg_stop_backup() to return the tli value directly. -- Simon Riggshttp://www.2ndQuadrant.com/ <http://

Re: [HACKERS] Rethinking representation of partial-aggregate steps

2016-06-23 Thread Simon Riggs
tly > far outweigh those considerations. > Agreed I can imagine plans where a useful aggregation occurs before every join, so N > 2 is easily possible. -- Simon Riggshttp://www.2ndQuadrant.com/ <http://www.2ndquadrant.com/> PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: [HACKERS] New design for FK-based join selectivity estimation

2016-06-13 Thread Simon Riggs
On 13 June 2016 at 19:16, Tom Lane <t...@sss.pgh.pa.us> wrote: > Simon Riggs <si...@2ndquadrant.com> writes: > > So a simple change is to make RelationGetFKeyList() only retrieve FKs > with > > nKeys>1. Rename to RelationGetMultiColumnFKeyList(). That greatly re

Re: [HACKERS] New design for FK-based join selectivity estimation

2016-06-13 Thread Simon Riggs
hat greatly reduces the scope for increased planning time. -- Simon Riggshttp://www.2ndQuadrant.com/ <http://www.2ndquadrant.com/> PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: [HACKERS] Does people favor to have matrix data type?

2016-05-25 Thread Simon Riggs
ase, it might help understand whether to include it or not. Multi-dimensionality of arrays isn't always useful, so this could be good. -- Simon Riggshttp://www.2ndQuadrant.com/ <http://www.2ndquadrant.com/> PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: [HACKERS] memory layouts for binary search in nbtree

2016-05-18 Thread Simon Riggs
marks. I think its a good area of work. -- Simon Riggshttp://www.2ndQuadrant.com/ <http://www.2ndquadrant.com/> PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: [HACKERS] HeapTupleSatisfiesToast() busted? (was atomic pin/unpin causing errors)

2016-05-10 Thread Simon Riggs
On 10 May 2016 at 17:20, Andres Freund <and...@anarazel.de> wrote: > On 2016-05-10 12:28:57 +0200, Simon Riggs wrote: > > On 10 May 2016 at 09:05, Andres Freund <and...@anarazel.de> wrote: > > > > > > > Is anybody ready with a good defense for SatisfiesT

Re: [HACKERS] HeapTupleSatisfiesToast() busted? (was atomic pin/unpin causing errors)

2016-05-10 Thread Simon Riggs
't commit it, I just used it to check the patch which changed btree vacuum replay. -- Simon Riggshttp://www.2ndQuadrant.com/ <http://www.2ndquadrant.com/> PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services toast_recheck.v1.patch Description: Binary

Re: [HACKERS] asynchronous and vectorized execution

2016-05-09 Thread Simon Riggs
the minimal number > of cache lines. > Team is about 2 years into research and coding prototype on those topics at this point, with agreed time for further work over next 2 years. I'll let my colleagues chime in with details since I'm not involved at that level any more. -- Simon Riggs

Re: [HACKERS] pg9.6 segfault using simple query (related to use fk for join estimates)

2016-05-08 Thread Simon Riggs
oblem. There's no point putting years of effort into parallel query if we can't work out when to use it sensibly. This patch makes major gains in that area. -- Simon Riggshttp://www.2ndQuadrant.com/ <http://www.2ndquadrant.com/> PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: [HACKERS] minor message improvement

2016-05-08 Thread Simon Riggs
On 8 May 2016 at 12:48, Simon Riggs <si...@2ndquadrant.com> wrote: > On 8 May 2016 at 03:53, Euler Taveira <eu...@timbira.com.br> wrote: > > Attached is a patch that turn it into one. >> > > Applied, by Tom. Thanks > Sorry about that; Tom's message only

Re: [HACKERS] minor message improvement

2016-05-08 Thread Simon Riggs
On 8 May 2016 at 03:53, Euler Taveira <eu...@timbira.com.br> wrote: Attached is a patch that turn it into one. > Applied, by Tom. Thanks -- Simon Riggshttp://www.2ndQuadrant.com/ <http://www.2ndquadrant.com/> PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: [HACKERS] ALTER TABLE lock downgrades have broken pg_upgrade

2016-05-07 Thread Simon Riggs
On 7 May 2016 at 16:49, Tom Lane <t...@sss.pgh.pa.us> wrote: > Simon Riggs <si...@2ndquadrant.com> writes: > > On 3 May 2016 at 18:07, Tom Lane <t...@sss.pgh.pa.us> wrote: > >> Or at least, it did until Simon decided that ALTER TABLE RESET >

Re: [HACKERS] pg_dump broken for non-super user

2016-05-07 Thread Simon Riggs
On 7 May 2016 at 16:21, Stephen Frost <sfr...@snowman.net> wrote: > * Simon Riggs (si...@2ndquadrant.com) wrote: > > On 7 May 2016 at 16:14, Stephen Frost <sfr...@snowman.net> wrote: > > > > If we don't lock it then we will have a inconsistent dump that will >

Re: [HACKERS] pg_dump broken for non-super user

2016-05-07 Thread Simon Riggs
consistent, as Tom pointed out. The catalog tables > are read using a repeatable read transaction, which will be consistent. The scan is consistent, yes, but the results would not be. -- Simon Riggshttp://www.2ndQuadrant.com/ <http://www.2ndquadrant.com/> PostgreSQL Developm

Re: [HACKERS] pg_dump broken for non-super user

2016-05-07 Thread Simon Riggs
dumped while an object is being dropped. Do we want an inconsistent dump? For what reason are we changing existing behaviour? There is no bug here, as Stephen explained. So this is a behaviour change after freeze with uncertain purpose. -- Simon Riggshttp://www.2ndQuadrant.com/

Re: [HACKERS] pg9.6 segfault using simple query (related to use fk for join estimates)

2016-05-07 Thread Simon Riggs
nalyzing the effect of new patches. I'll continue to look at this and comment further. -- Simon Riggshttp://www.2ndQuadrant.com/ <http://www.2ndquadrant.com/> PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: [HACKERS] pg9.6 segfault using simple query (related to use fk for join estimates)

2016-05-07 Thread Simon Riggs
lease. Yes, the GUC was for testing, as mentioned on the commit message. > Can't check right now with Simon, though, as he's is out of office this > week. > Am back and reading. -- Simon Riggshttp://www.2ndQuadrant.com/ <http://www.2ndquadrant.com/> PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: [HACKERS] ALTER TABLE lock downgrades have broken pg_upgrade

2016-05-07 Thread Simon Riggs
think that needs a specific command, just a function. I accept that it is my bug and should fix it. -- Simon Riggshttp://www.2ndQuadrant.com/ <http://www.2ndquadrant.com/> PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

<    1   2   3   4   5   6   7   8   9   10   >