Re: [HACKERS] WIP: Failover Slots

2017-08-08 Thread Craig Ringer
On 3 August 2017 at 04:35, Robert Haas wrote: > On Tue, Jul 25, 2017 at 8:44 PM, Craig Ringer > wrote: > > No. The whole approach seems to have been bounced from core. I don't > agree > > and continue to think this functionality is desirable but I don't get to >

Re: [HACKERS] [TRAP: FailedAssertion] causing server to crash

2017-08-06 Thread Craig Ringer
isedb.com> > I think so - specifically, that it's a leftover from a revision where the xid limit was advanced before clog truncation. I'll be finding time in the next couple of days to look more closely and ensure that's all it is. -- Craig Ringer http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services

Re: [HACKERS] Effect of dropping a partitioned table's column over time

2017-08-06 Thread Craig Ringer
ntal > materialized views (at least in one proposal) which is why I asked > about it on this list recently[2]. > Can we instead create the new partitions with the same dropped columns? Ensure that every partition, parent and child, has the same column-set? -- Craig Ringer http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services

[HACKERS] Persistent wait event sets and socket changes

2017-07-30 Thread Craig Ringer
event set when the fd-set changes. So I'm posting mostly to confirm that it's not supposed to work, and ask if anyone thinks I should submit a comment patch to latch.c documenting it. -- Craig Ringer http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services

Re: [HACKERS] On Complex Source Code Reading Strategy

2017-07-27 Thread Craig Ringer
//wiki.postgresql.org/wiki/Developer_FAQ (some of which need to be added to the "developer information" wiki page I think) -- Craig Ringer http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services

Re: [HACKERS] Syncing sql extension versions with shared library versions

2017-07-25 Thread Craig Ringer
tension function definitions can >> fail at runtime if funcs are removed or change signature, but won't fail at >> startup or load. >> >> So we let the C extension detect when it's newer than the loaded SQL ext >> during its startup and run an ALTER EXTENSION

Re: [HACKERS] TAP: allow overriding PostgresNode in get_new_node

2017-07-25 Thread Craig Ringer
er > (if at all). > > Pushed to 9.6 and HEAD. > Thanks. An upvote from our resident Perl wizard certainly does help :) -- Craig Ringer http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services

Re: [HACKERS] WIP: Failover Slots

2017-07-25 Thread Craig Ringer
On 26 July 2017 at 00:16, Thom Brown wrote: > On 8 April 2016 at 07:13, Craig Ringer wrote: > > On 6 April 2016 at 22:17, Andres Freund wrote: > > > >> > >> Quickly skimming 0001 in [4] there appear to be a number of issues: > >> * LWLockHeldBy

[HACKERS] Re: [SQL] Postgresql “alter column type” creates an event which contains “temp_table_xxx”

2017-07-25 Thread Craig Ringer
in-core logical rep doesn't natively handle truncation yet, and this is one of the things it'd be good to do for pg11, especially if more people get interested in contributing. In the mean time, logical decoding clients can special case "pg_temp_" relation names in thei

Re: [HACKERS] Syncing sql extension versions with shared library versions

2017-07-22 Thread Craig Ringer
On 22 Jul. 2017 04:19, "Mat Arye" wrote: Hi All, I am developing the TimescaleDB extension for postgres ( https://github.com/timescale/timescaledb) and have some questions about versioning. First of all, I have to say that the versioning system on the sql side is wonderful. It's really simple to

Re: [HACKERS] [PATCH] A hook for session start

2017-07-21 Thread Craig Ringer
On 21 Jul. 2017 21:58, "Yugo Nagata" wrote: On Fri, 21 Jul 2017 10:31:57 -0300 Fabrízio de Royes Mello wrote: > On Fri, Jul 21, 2017 at 9:35 AM, Yugo Nagata wrote: > > > > On Fri, 21 Jul 2017 09:53:19 +0800 > > Craig Ringer wrote: > > > > >

Re: [HACKERS] [PATCH] pageinspect function to decode infomasks

2017-07-20 Thread Craig Ringer
inition, the argument name used is > 'include_combined' whereas in documentation you have mentioned > 'show_combined'. > Fixed, thanks. I want to find time to expand the tests on this some more and look more closely, but here it is for now. -- Craig Ringer

Re: [HACKERS] xlogfilename

2017-07-20 Thread Craig Ringer
multiple physical files of 16MB which are called > WAL segments. The second 8 characters indicate the id of the logical > xlog file, and the last 8 characters indicate the sequencial number of > the segment in this xlog. > <http://www.postgresql.org/mailpref/pgsql-hackers> > You missed the timeline ID, which is the first 8 digits. -- Craig Ringer http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services

Re: [HACKERS] Definitional questions for pg_sequences view

2017-07-20 Thread Craig Ringer
but if it's a view to help users out exposing that would seem sensible. -- Craig Ringer http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services

Re: [HACKERS] More optimization effort?

2017-07-20 Thread Craig Ringer
7;ve originally matched? I'm not sure it's a blocker, but it bears consideration, and Pg might have to do more work on partial index matching too. -- Craig Ringer http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services

Re: [HACKERS] Better error message for trying to drop a DB with open subscriptions?

2017-07-20 Thread Craig Ringer
prepared xacts separately already, and errdetail_busy_db uses that to report the two separately. Since we have slot attachment data I expect reporting attached replication slots would not be hard either; you might be able to prep a patch for that in a few hours. -- Craig Ringer http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services

Re: [HACKERS] [PATCH] A hook for session start

2017-07-20 Thread Craig Ringer
l replication code in pg11 for similar reasons, and so features like DDL replication can be prototyped as extensions more practically). That said, isn't ExecutorStart_hook + ProcessUtility_hook able to serve the same job as a session-start hook, albeit at slightly higher overhead? You can just te

Re: [HACKERS] [PATCH] pageinspect function to decode infomasks

2017-07-20 Thread Craig Ringer
On 20 Jul. 2017 19:09, "Ashutosh Sharma" wrote: I had a quick look into this patch and also tested it and following are my observations. Thanks very much. I'll expand the tests to cover various normal and nonsensical masks and combinations and fix the identified issues. This was a quick morni

Re: [HACKERS] [TRAP: FailedAssertion] causing server to crash

2017-07-20 Thread Craig Ringer
to be, you might get faster results by using a custom pgbench script for one or more workers that just runs txid_current() a whole lot. Or jump the server's xid space forward. I've got a few other things on right now but I'll keep an eye out and hope for a core dump. -- Craig Ringer

Re: [HACKERS] [TRAP: FailedAssertion] causing server to crash

2017-07-19 Thread Craig Ringer
ur script assume a newly initdb'd instance with no custom configuration? If not, what setup steps/configuration precede your script run? > well short of the 2-million mark. > Er, billion. -- Craig Ringer http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services

Re: [HACKERS] [TRAP: FailedAssertion] causing server to crash

2017-07-19 Thread Craig Ringer
01000 )) - try to start pg, note the missing clog segment it complains about - dd if=/dev/zero bs=1 count=262144 of=datadir/pg_clog/$MISSINGSEGNAME - start Pg That should put you about 1000 txn's from the 1 million xid limit, assuming I got my maths right (don't assume that), and assuming

Re: [HACKERS] Dealing with logical replication

2017-07-19 Thread Craig Ringer
corner case. If we had a hook in the logical apply worker's insert or wal-message routines it'd be possible to write an extension to do this for pg10, but AFAICS we don't. So schema changes in logical replication currently require more care than in physical replication. -- Craig Ringer http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services

Re: [HACKERS] [PATCH] pageinspect function to decode infomasks

2017-07-19 Thread Craig Ringer
On 20 July 2017 at 11:33, Craig Ringer wrote: > Hi > > Whenever I'm debugging some kind of corruption incident, possible > visibility bug, etc, I always land up staring at integer infomasks or using > a SQL helper function to decode them. > > That's silly, so here

[HACKERS] [PATCH] pageinspect function to decode infomasks

2017-07-19 Thread Craig Ringer
D_ONLY or HEAP_LOCKED_UPGRADED, and filtering them out could be just as confusing as leaving them in. The infomask2 natts mask is ignored. You can bitwise-and it out in SQL pretty easily if needed. I could output it here as a constructed text datum, but it seems mostly pointless. -- Craig R

Re: [HACKERS] [GENERAL] huge RAM use in multi-command ALTER of table heirarchy

2017-07-18 Thread Craig Ringer
he v11 cycle opens, unless someone can show an example > of non-broken coding that requires it. (And if so, there ought to > be a regression test incorporating that.) Just FYI, I believe Simon's currently on holiday, so may not notice this discussion as promptly as usual. -- Craig Ringer http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services

Re: [HACKERS] building libpq.a static library

2017-07-12 Thread Craig Ringer
On 13 July 2017 at 10:58, Craig Ringer wrote: > On 12 July 2017 at 23:46, Jeroen Ooms wrote: > >> On Wed, Jul 12, 2017 at 5:11 PM, Tom Lane wrote: >> > Jeroen Ooms writes: >> >> I maintain static libraries for libpq for the R programming language >> >

Re: [HACKERS] building libpq.a static library

2017-07-12 Thread Craig Ringer
inux, use $ORIGIN in your rpath. Beware of quoting issues with $ORIGIN though. I'm not trying to block support for a static libpq, I just don't see the point. -- Craig Ringer http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services

Re: [HACKERS] New partitioning - some feedback

2017-07-09 Thread Craig Ringer
gt; > (7 rows) > > Would showing relispartition=tru tables only in \d+ fix this? > <http://www.postgresql.org/mailpref/pgsql-hackers> > I think so. I'd like to add a flag of some kind to \d column output that marks a table as having partitions, but I can't think of anything narrow enough and still useful. -- Craig Ringer http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services

Re: [HACKERS] Postgres process invoking exit resulting in sh-QUIT core

2017-07-07 Thread Craig Ringer
ations. > > I have attached output for 2 such instance. > > You seem to be missing debug symbols. Install appropriate debuginfo packages. -- Craig Ringer http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services

Re: [HACKERS] Postgres process invoking exit resulting in sh-QUIT core

2017-07-07 Thread Craig Ringer
would get signalled too. Can't immediately explain the exit code, and SIGQUIT should _not_ generate a core file. Can you show the result of attaching 'gdb' to the core file and running 'bt full' ? -- Craig Ringer http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services

Re: [HACKERS] Postgres process invoking exit resulting in sh-QUIT core

2017-07-05 Thread Craig Ringer
On 3 Jul. 2017 23:01, "K S, Sandhya (Nokia - IN/Bangalore)" < sandhya@nokia.com> wrote: Hi Craig, Thanks for the response. Scenario tried here is restart of the system multiple times. sh-QUIT core is generated when Postgres is invoking the shell to exit and may not be due to kernel or file s

Re: [BUGS] [HACKERS] Segmentation fault in libpq

2017-07-02 Thread Craig Ringer
tead of our code. > > Usually using a tool like valgrind is quite helpful to find issues like > that, because it'll show you the call-stack accessing the memory and > *also* the call-stack that lead to the memory being freed. Yep, huge help. BTW, on Windows, the free tool

Re: [HACKERS] Race-like failure in recovery/t/009_twophase.pl

2017-07-02 Thread Craig Ringer
B7E8" with normal timing, but with > enough delay in there, you get "|physical|||t|11542|||" which > triggers split's default behavior of ignoring the trailing empty > fields. It looks like the way to get split to not do that is > to pass it a "limit"

Re: [HACKERS] Postgres process invoking exit resulting in sh-QUIT core

2017-06-30 Thread Craig Ringer
> startup process waiting for 000102EB Looks like an archive_command or restore_command . If 'sh' is dumping core, you probably have issues at a low level in the kernel, file system, etc. Check dmesg. -- Craig Ringer http://www.2ndQuadrant.com/

Re: protocol version negotiation (Re: [HACKERS] Libpq PGRES_COPY_BOTH - version compatibility)

2017-06-29 Thread Craig Ringer
initial response to the startup message is > anything other than a ServerProtocolVersion message, the client should > assume it's talking to a 3.0 server. (To make this work, we would > back-patch a change into existing releases to allow any 3.x protocol > version and ignore any pg

Re: [HACKERS] Start logical decoding from any lsn present in the wal segment

2017-06-29 Thread Craig Ringer
running xacts are precisely known; see xl_running_xacts and the snapshot builder. -- Craig Ringer http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: protocol version negotiation (Re: [HACKERS] Libpq PGRES_COPY_BOTH - version compatibility)

2017-06-28 Thread Craig Ringer
On 29 June 2017 at 12:23, Craig Ringer wrote: > It does. But I don't see anywhere that extra round trips have been discussed. Ah, right, they're implied by having the server respond with some downversion message and ignore input until the client sends a new startup message. That&#

Re: protocol version negotiation (Re: [HACKERS] Libpq PGRES_COPY_BOTH - version compatibility)

2017-06-28 Thread Craig Ringer
On 29 June 2017 at 10:27, Tom Lane wrote: > Craig Ringer writes: >> On 29 June 2017 at 03:01, Robert Haas wrote: >>> It wouldn't be >>> so bad if unrecognized parameters were just ignored; the client would >>> know from the ServerProtocolVersion (or Par

Re: protocol version negotiation (Re: [HACKERS] Libpq PGRES_COPY_BOTH - version compatibility)

2017-06-28 Thread Craig Ringer
On 29 June 2017 at 09:44, Craig Ringer wrote: > I > can't personally think of much right away that wouldn't work pretty > well in a follow-on message. Actually, I take that back, there's one thing that's bugged me for a while that wouldn't work well this

Re: protocol version negotiation (Re: [HACKERS] Libpq PGRES_COPY_BOTH - version compatibility)

2017-06-28 Thread Craig Ringer
an just not ask for them. Capabilities will make startup messages bigger. Personally I don't care much about that, as on modern networks it's all about latency not message size. We'd use abbreviated capability names I expect. If the list gets too big we could always roll up capabi

Re: [HACKERS] Logical decoding on standby

2017-06-26 Thread Craig Ringer
is correct or not.In any case please someone clarify. That's a reasonable thing to do, and again, I thought I did it in a later revision, but apparently not (?). I've been working on other things and have lost track of progress here a bit. I'll check more closely. -- Craig Ringer

Re: [HACKERS] Logical decoding on standby

2017-06-26 Thread Craig Ringer
istoric is true when state->currTLI is less than > ThisTimeLineID. Correct, that was a bug. I thought it got fixed upthread though. -- Craig Ringer http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services -- Sent via pgsql-hackers m

Re: [HACKERS] Timing-sensitive case in src/test/recovery TAP tests

2017-06-25 Thread Craig Ringer
27;s > pg_replication_slot on slotname_2? It would really seem better to make > the nullness check conditional in get_slot_xmins instead. Sorry for > changing opinion here. I'm not sure I understand this. -- Craig Ringer http://www.2ndQuadrant.com/ Post

Re: [HACKERS] Timing-sensitive case in src/test/recovery TAP tests

2017-06-25 Thread Craig Ringer
On 26 June 2017 at 10:09, Tom Lane wrote: > Michael Paquier writes: >> On Mon, Jun 26, 2017 at 10:48 AM, Craig Ringer wrote: >>> $node_standby_1->poll_query_until('postgres', q[SELECT xmin IS NULL >>> from pg_replication_slots WHERE slot_name = ']

Re: [HACKERS] Timing-sensitive case in src/test/recovery TAP tests

2017-06-25 Thread Craig Ringer
ng poll_query_until? This should do the trick: $node_standby_1->poll_query_until('postgres', q[SELECT xmin IS NULL from pg_replication_slots WHERE slot_name = '] . $slotname_2 . q[']); -- Craig Ringer http://www.2ndQuadrant.com/ PostgreSQL Development,

Re: [HACKERS] SQL MERGE patches for PostgreSQL Versions

2017-06-22 Thread Craig Ringer
st entry in the 'index' is the git commit hash of the base commit, IIRC. -- Craig Ringer http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make chang

Re: [HACKERS] SQL MERGE patches for PostgreSQL Versions

2017-06-22 Thread Craig Ringer
patch. Otherwise you have to rely on what's in the email thread. -- Craig Ringer http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to you

Re: [HACKERS] PATCH: Batch/pipelining support for libpq

2017-06-21 Thread Craig Ringer
On 22 June 2017 at 09:07, Andres Freund wrote: > On 2017-06-22 09:03:05 +0800, Craig Ringer wrote: >> On 22 June 2017 at 08:29, Andres Freund wrote: >> >> > I.e. we're doing tiny write send() syscalls (they should be coalesced) >> >> That's l

Re: [HACKERS] PATCH: Batch/pipelining support for libpq

2017-06-21 Thread Craig Ringer
we don't and shouldn't), and the syscall overhead is IMO not worth worrying about just yet. > and then completely unnecessarily call recv() over and over again > without polling. To me it looks very much like we're just doing either > exactly once per command... Yeah, t

Re: [HACKERS] PATCH: Batch/pipelining support for libpq

2017-06-21 Thread Craig Ringer
On 22 Jun. 2017 07:40, "Andres Freund" wrote: On 2017-06-20 17:51:23 +0200, Daniel Verite wrote: > Andres Freund wrote: > > > FWIW, I still think this needs a pgbench or similar example integration, > > so we can actually properly measure the benefits. > > Here's an updated version of the p

Re: [HACKERS] REPLICA IDENTITY FULL

2017-06-19 Thread Craig Ringer
On 20 June 2017 at 09:47, Andres Freund wrote: > On 2017-06-20 09:45:27 +0800, Craig Ringer wrote: >> I frequently want to be able to use REPLICA IDENTITY DEFAULT, but also >> record the whole old tuple not just keys, so they can be used in >> conflict processing etc. &g

Re: [HACKERS] REPLICA IDENTITY FULL

2017-06-19 Thread Craig Ringer
the whole old tuple not just keys, so they can be used in conflict processing etc. -- Craig Ringer http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] PATCH: Batch/pipelining support for libpq

2017-06-19 Thread Craig Ringer
I'm glad folks are interested, it's not something I can dedicate much time to. Especially with a 6-week-old baby now > FWIW, I still think this needs a pgbench or similar example integration, > so we can actually properly measure the benefits. I agree. I originally wanted to patch

[HACKERS] PATCH: Making constant StringInfo

2017-06-18 Thread Craig Ringer
f call for a StringInfo that can start with an external buffer and append to it until it runs out of room, then copy it only if needed. Patch for constrant StringInfo attached. -- Craig Ringer http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Ser

Re: [HACKERS] logical replication read-only slave

2017-06-15 Thread Craig Ringer
m design idea here and following up with a patch if you get a reasonable approximation of consensus. -- Craig Ringer http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@pos

Re: [HACKERS] Decimal64 and Decimal128

2017-06-15 Thread Craig Ringer
any references to that anymore. POWER6 and POWER7 has it, which is great, but hardly justifies a push for getting it into the core Pg. Some of the discussion on https://software.intel.com/en-us/articles/intel-decimal-floating-point-math-library?page=1 suggests that doing it fully in hardware i

Re: [HACKERS] ICU support on Windows

2017-06-13 Thread Craig Ringer
that Program Files (x86) was added to punish people who fail to handle paths with spaces properly and finally make sure that everything got fixed. Because apparently "Program Files" wasn't annoying enough already. Ha, as if. People hard-code PROGRA~1 (the DOS shortname). And

Re: [HACKERS] pg_receivewal and messages printed in non-verbose mode

2017-06-13 Thread Craig Ringer
ge to me that at least the first one is written to the user as > that's not an error after promoting a standby. I agree. At least the first should be --verbose only. -- Craig Ringer http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Ser

Re: [HACKERS] tap tests on older branches fail if concurrency is used

2017-06-07 Thread Craig Ringer
es sense now, and if we decide to backpatch PostgresNode (and I get the time to do it) we can clobber that fix quite happily with the full backport. Thanks Michael for the workaround. -- Craig Ringer http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training &

Re: [HACKERS] Use of snapshot in logical replication

2017-06-07 Thread Craig Ringer
ped) or known not to be (so they'll be streamed out on the slot). See snapbuild.c etc. -- Craig Ringer http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.or

Re: [HACKERS] Challenges preventing us moving to 64 bit transaction id (XID)?

2017-06-06 Thread Craig Ringer
On 6 June 2017 at 12:38, Ashutosh Bapat wrote: > On Tue, Jun 6, 2017 at 10:00 AM, Tom Lane wrote: >> Ashutosh Bapat writes: >>> On Tue, Jun 6, 2017 at 9:48 AM, Craig Ringer wrote: >>>> Storing an epoch implies that rows can't have (xmin,xmax) different by >

Re: [HACKERS] Challenges preventing us moving to 64 bit transaction id (XID)?

2017-06-05 Thread Craig Ringer
o if you're updating/deleting an extremely old tuple you'll presumably have to set xmin to FrozenTransactionId if it isn't already, so you can set a new epoch and xmax. -- Craig Ringer http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training &

Re: [HACKERS] pg_dump issues

2017-06-05 Thread Craig Ringer
rg/support/professional_support/ . (Note, I work for one of them). -- Craig Ringer http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscripti

Re: [HACKERS] Do we need the gcc feature "__builtin_expect" to promote the branches prediction?

2017-06-05 Thread Craig Ringer
this job. Compilers are already pretty good at this. https://en.wikipedia.org/wiki/Program_optimization -- Craig Ringer http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org

Re: [HACKERS] PG10 transition tables, wCTEs and multiple operations on the same table

2017-06-05 Thread Craig Ringer
ts (row changes). If there are any cases where it's safe, they'll take some careful thought. It's only standard CTEs (SELECT-based) that I think matter for the optimisation fence behaviour. -- Craig Ringer http://www.2ndQuadrant.com/ PostgreSQL Development

Re: [HACKERS] Do we need the gcc feature "__builtin_expect" to promote the branches prediction?

2017-06-02 Thread Craig Ringer
On 2 Jun. 2017 16:42, "Hao Lee" wrote: Hi all, There is a lot of "if statement" in system, and GCC provides a feature,"__builtin_expect", which let compilers know which branch is mostly run. Compilers and CPUs are really good at guessing this. Humans are wrong about it more than we'd l

Re: [HACKERS] proposal: PLpgSQL parallel assignemnt

2017-06-02 Thread Craig Ringer
On 2 June 2017 at 15:51, Pavel Stehule wrote: > a, b := fx(); > > Comments, notes, ideas? I'd be pretty happy to have (a, b) = (x, y); (a, b) = f(x); which is SQL-esque. But what, if anything, does Ada do? -- Craig Ringer http://www.2ndQuadrant.co

Re: [HACKERS] walsender & parallelism

2017-05-31 Thread Craig Ringer
han I'd really like to stop walsenders doing things they can't safely do during shutdown. -- Craig Ringer http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgr

Re: [HACKERS] tap tests on older branches fail if concurrency is used

2017-05-31 Thread Craig Ringer
to fix it all up, this seems like a good argument for backporting the updated suite from 9.6 or pg10, with PostgresNode etc. I already have a working tree with that done to use src/test/recovery in 9.5, but haven't updated src/bin/scripts etc yet. -- Craig Ringer http://w

Re: [HACKERS] TAP: allow overriding PostgresNode in get_new_node

2017-05-31 Thread Craig Ringer
On 31 May 2017 at 08:43, Craig Ringer wrote: > Hi all > > More and more I'm finding it useful to extend PostgresNode for project > specific helper classes. But PostgresNode::get_new_node is a factory > that doesn't provide any mechanism for overriding, so you have to >

Re: [HACKERS] Replication origins and timelines

2017-05-31 Thread Craig Ringer
maintain slots on them. Some kind of solution that runs entirely on the standby will be needed. It's more a question of whether it's something built-in, easy, and nice, or some out of tree extension. -- Craig Ringer http://www.2ndQuadrant.com/ PostgreSQL Develop

Re: [HACKERS] Replication origins and timelines

2017-05-31 Thread Craig Ringer
astics for getting logical replication to actually use it to support physical failover for pg10, so it was always going to land in pg11. This is very much a "how do we do it right when we do do it" topic, not a pg10 issue. -- Craig Ringer http://www.2ndQuadra

Re: [HACKERS] Replication origins and timelines

2017-05-31 Thread Craig Ringer
On 1 June 2017 at 09:23, Andres Freund wrote: > Hi, > > On 2017-06-01 09:12:04 +0800, Craig Ringer wrote: >> TL;DR: replication origins track LSN without timeline. This is >> ambiguous when physical failover is present since / >> can now represent

Re: [HACKERS] Replication origins and timelines

2017-05-31 Thread Craig Ringer
On 1 June 2017 at 09:27, Stephen Frost wrote: > Craig, > > * Craig Ringer (cr...@2ndquadrant.com) wrote: >> TL;DR: replication origins track LSN without timeline. This is >> ambiguous when physical failover is present since / >> can now represent

[HACKERS] Replication origins and timelines

2017-05-31 Thread Craig Ringer
ll, but by default we'll detect the error. But we can't do that unless replication origins on the downstream can track the timeline. -- Craig Ringer http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services -- Sent via pgsql-hacker

Re: [HACKERS] tap tests on older branches fail if concurrency is used

2017-05-31 Thread Craig Ringer
t appears to readily reproduce on > several machines... I'll take a look at what's changed and why it's happening and get back to you. -- Craig Ringer http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services -- Sent via pg

Re: [HACKERS] TAP backpatching policy

2017-05-31 Thread Craig Ringer
#x27;d have a Makefile that finds and clobbers the in-tree PostgresNode.pm etc. So it's a hassle, but not the end of the world. I just suspect we'll all benefit from making it easier to write tests that work across more releases, and that updating the test modules in back branches is

Re: [HACKERS] pg_config --version-num

2017-05-30 Thread Craig Ringer
On 31 May 2017 9:36 am, "Michael Paquier" wrote: On Tue, May 30, 2017 at 6:14 PM, Craig Ringer wrote: > Attached is a small patch to teach pg_config how to output a --version-num > > With Pg 10, parsing versions got more annoying. Especially with > "10beta1",

[HACKERS] pg_config --version-num

2017-05-30 Thread Craig Ringer
le pre-formatted one at no cost to us. Personally I'd like to backpatch this into supported back branches, but just having it in pg 10 would be a help. -- Craig Ringer http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support,

[HACKERS] TAP backpatching policy

2017-05-30 Thread Craig Ringer
s harder) this would make it a lot more practical to do nontrivial tests in extensions - which really matters since we introduced bgworkers. Thoughts? Backpatch new TAP methods, etc, into back branches routinely? -- Craig Ringer http://www.2ndQuadrant.com/ PostgreSQL Devel

[HACKERS] TAP: allow overriding PostgresNode in get_new_node

2017-05-30 Thread Craig Ringer
r adding --version-num to pg_config. Any objections? Will submit patch if none. -- Craig Ringer http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services From 8a19793c89b165c25c88cd7149650e20ef27bd55 Mon Sep 17 00:00:00 2001 From: Craig Ringer

Re: [HACKERS] pg_resetwal is broken if run from v10 against older version of PG data directory

2017-05-29 Thread Craig Ringer
On 30 May 2017 at 00:00, Tom Lane wrote: > I think it's just horribly dangerous to run any version of > pg_resetxlog/pg_resetwal You can pretty much stop there ;) -- Craig Ringer http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training

Re: [HACKERS] Logical replication & corrupted pages recovery

2017-05-26 Thread Craig Ringer
On 27 May 2017 01:03, "Aleksander Alekseev" wrote: Hi Konstantin, > May be it is possible to somehow optimize it, by checking ranges of primary > key values It's possible. An optimization you are looking for is called Merkle tree [1]. Particularly it's used in Riak [2]. [1] https://en.wikipedi

Re: [HACKERS] Latest Data::Dumper breaks hstore_plperl regression test

2017-05-13 Thread Craig Ringer
On 14 May 2017 11:33, "Robert Haas" wrote: On Sat, May 13, 2017 at 6:22 PM, Tom Lane wrote: > Or at least, that's what I surmise from the fact that buildfarm critter > caiman has been failing that test for the last day or so, with symptoms > indicating whitespace changes in Data::Dumper output.

Re: [HACKERS] idea: custom log_line_prefix components besides application_name

2017-05-10 Thread Craig Ringer
On 10 May 2017 10:44 am, "Chapman Flack" wrote: On 05/09/17 18:48, Mark Dilger wrote: > I don't have any positive expectation that the postgres community will go > along with any of this, but just from my point of view, the cleaner way to > do what you are proposing is something like setting a s

Re: [HACKERS] Detecting schema changes during logical replication

2017-05-07 Thread Craig Ringer
On 8 May 2017 05:56, "Daniele Varrazzo" wrote: On Sun, May 7, 2017 at 8:04 PM, Andres Freund wrote: > Hi, > > On 2017-05-07 19:27:08 +0100, Daniele Varrazzo wrote: >> I'm putting together a replication system based on logical >> replication. > > Interesting. If you very briefly could recap what

Re: [HACKERS] MSVC odd TAP test problem

2017-05-06 Thread Craig Ringer
On 7 May 2017 4:24 am, "Andrew Dunstan" wrote: I have been working on enabling the remaining TAP tests on MSVC build in the buildfarm client, but I have come across an odd problem. The bin tests all run fine, but the recover tests crash and in such a way as to crash the buildfarm client itself a

Re: [HACKERS] CTE inlining

2017-05-04 Thread Craig Ringer
On 5 May 2017 at 08:17, Joe Conway wrote: > On 05/04/2017 05:03 PM, Craig Ringer wrote: >> On 5 May 2017 02:52, "Tom Lane" wrote: >> I haven't been keeping close tabs either, but surely we still have >> to have >> the optimization fence in (

Re: [HACKERS] CTE inlining

2017-05-04 Thread Craig Ringer
On 5 May 2017 06:04, "Andreas Karlsson" wrote: On 05/04/2017 06:22 PM, Andrew Dunstan wrote: > I wrote this query: > > select (json_populate_record(null::mytype, myjson)).* > from mytable; > > > It turned out that this was an order of magnitude faster: > > with r as > ( >

Re: [HACKERS] CTE inlining

2017-05-04 Thread Craig Ringer
On 5 May 2017 02:52, "Tom Lane" wrote: Tomas Vondra writes: > On 5/4/17 8:03 PM, Joe Conway wrote: >>> I haven't been able to follow this incredibly long thread, so please >>> excuse me if way off base, but are we talking about that a CTE would be >>> silently be rewritten as an inline expressio

Re: [HACKERS] Function to move the position of a replication slot

2017-05-04 Thread Craig Ringer
On 4 May 2017 at 20:45, Magnus Hagander wrote: > On Thu, May 4, 2017 at 2:42 PM, Craig Ringer wrote: >> >> On 4 May 2017 at 20:05, Magnus Hagander wrote: >> > PFA a patch that adds a new function, pg_move_replication_slot, that >> > makes >> > it poss

Re: [HACKERS] Function to move the position of a replication slot

2017-05-04 Thread Craig Ringer
l fine with the name, since I plan to add that capability in pg11 by running through logical decoding and ReorderBufferSkip()ing each xact until we reach the target lsn. -- Craig Ringer http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Servi

Re: [HACKERS] Re: [BUGS] BUG #14634: On Windows pg_basebackup should write tar to stdout in binary mode

2017-05-02 Thread Craig Ringer
s for reporting the issue. > With the attached patch, I was able to extract the tar file that gets > generated when the tar file is written into stdout. I tested the > the compressed tar also. > > This bug needs to be fixed in back branches also. We should do the same for pg_dump in -F

Re: [HACKERS] CTE inlining

2017-05-02 Thread Craig Ringer
nted and unintentional query-specific regressions let alone documented and relnoted ones. So a sad -1 to me for a GUC. Anyone big enough to be significantly upset by this planner change will have a QA/staging deployment system anyway. Or should, because we make enough other changes in a major rel

Re: [HACKERS] Regarding B-Tree Lookup

2017-05-02 Thread Craig Ringer
On 2 May 2017 7:34 pm, "Michael Paquier" wrote: On Tue, May 2, 2017 at 6:12 PM, Mahi Gurram wrote: > I'm building some custom extension on top of postgres 9.6.1. As part of > that, I would like to read Heap Tuple directly from my extension using > Primary Key. > > By default, postgres table inde

Re: [HACKERS] Potential hot-standby bug around xacts committed but in xl_running_xacts

2017-05-01 Thread Craig Ringer
ounds like a sensible solution to me. It avoids the need for a rather undesirable interlock between xlog and shmem commit. -- Craig Ringer http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services -- Sent via pgsql-hackers mailing list (pgs

Re: [HACKERS] CTE inlining

2017-05-01 Thread Craig Ringer
On 2 May 2017 at 10:45, Craig Ringer wrote: > If we want fence behaviour, we should require people to declare their > desire for fence behaviour, rather than treating it as a sort of > hint-as-a-bug that we grandfather in because we're so desperate not to > admit we have hint

Re: [HACKERS] CTE inlining

2017-05-01 Thread Craig Ringer
uot; and wonder why we haven't fixed this limitation yet, viewing it as a bug not a feature. The same logic being applied here should've prevented us from ever introducing: * inlining of SQL functions * inlining of views * inlining of subqueries ... but somehow, this one is different

Re: [HACKERS] CTE inlining

2017-05-01 Thread Craig Ringer
t's what we do right now so we can pretend we don't have query hints while still having query hints. -- Craig Ringer http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgr

Re: [HACKERS] CTE inlining

2017-05-01 Thread Craig Ringer
query hint. And yes, that's what it is, because we'd only inline when we could produce semantically equivalent results anyway. -- Craig Ringer http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services -- Sent via pgsql-hackers mailing l

<    1   2   3   4   5   6   7   8   9   10   >