[HACKERS] Handle PGRES_COPY_BOTH in psql for logical replication?

2015-06-05 Thread Shulgin, Oleksandr
Hello Hackers, At the moment one can open up a replication connection using psql and use all of the commands available on this special type of connection, except for START_REPLICATION: $ psql -d dbname=test replication=database psql (9.4.1) Type help for help. test=# IDENTIFY_SYSTEM;

Re: [HACKERS] Multixid hindsight design

2015-06-05 Thread Simon Riggs
On 1 June 2015 at 20:53, Thomas Munro thomas.mu...@enterprisedb.com wrote: On Tue, May 12, 2015 at 9:20 AM, Heikki Linnakangas hlinn...@iki.fi wrote: The beauty of this would be that the TED entries can be zapped at restart, just like pg_subtrans, and pg_multixact before 9.3. It doesn't

[HACKERS] Re: [GENERAL] 9.4.1 - 9.4.2 problem: could not access status of transaction 1

2015-06-05 Thread Noah Misch
On Thu, Jun 04, 2015 at 05:29:51PM -0400, Robert Haas wrote: Here's a new version with some more fixes and improvements: I read through this version and found nothing to change. I encourage other hackers to study the patch, though. The surrounding code is challenging. With this version, I'm

Re: [HACKERS] Handle PGRES_COPY_BOTH in psql for logical replication?

2015-06-05 Thread Shulgin, Oleksandr
On Fri, Jun 5, 2015 at 10:22 AM, Andres Freund and...@anarazel.de wrote: Maybe I'm missing something, which functions do you have in mind exactly? pg_logical_slot_get_changes() etc? Oh, totally forgot about these. However there are two significant differences between using the functions

Re: [HACKERS] [CORE] Restore-reliability mode

2015-06-05 Thread Simon Riggs
On 3 June 2015 at 18:21, Josh Berkus j...@agliodbs.com wrote: I would argue that if we delay 9.5 in order to do a 100% manual review of code, without adding any new automated tests or other non-manual tools for improving stability, then it's a waste of time; we might as well just release the

Re: [HACKERS] RFC: Remove contrib entirely

2015-06-05 Thread Jim Nasby
On 6/4/15 3:11 PM, David E. Wheeler wrote: There is no indication of what versions of pg any of PGXN modules are tested on, or even if there are tests that can be run to prove the module works correctly with a particular version of pg. Yeah, I’ve been meaning to

Re: [HACKERS] Handle PGRES_COPY_BOTH in psql for logical replication?

2015-06-05 Thread Andres Freund
On 2015-06-05 09:20:41 +0200, Shulgin, Oleksandr wrote: Hello Hackers, At the moment one can open up a replication connection using psql and use all of the commands available on this special type of connection, except for START_REPLICATION: $ psql -d dbname=test replication=database psql

Re: [HACKERS] Restore-reliability mode

2015-06-05 Thread Simon Riggs
On 3 June 2015 at 14:50, Noah Misch n...@leadboat.com wrote: Subject changed from Re: [CORE] postpone next week's release. On Sat, May 30, 2015 at 10:48:45PM -0400, Bruce Momjian wrote: Well, I think we stop what we are doing, focus on restructuring, testing, and reviewing areas that

Re: [HACKERS] Handle PGRES_COPY_BOTH in psql for logical replication?

2015-06-05 Thread Shulgin, Oleksandr
On Fri, Jun 5, 2015 at 9:26 AM, Andres Freund and...@anarazel.de wrote: On 2015-06-05 09:20:41 +0200, Shulgin, Oleksandr wrote: Hello Hackers, At the moment one can open up a replication connection using psql and use all of the commands available on this special type of connection,

Re: [HACKERS] Handle PGRES_COPY_BOTH in psql for logical replication?

2015-06-05 Thread Andres Freund
On 2015-06-05 09:43:45 +0200, Shulgin, Oleksandr wrote: How would that look like? It's just binary data you'd get back? And what would be the feedback messages and such look like, and what would be the content? It would be whatever the logical decoding plugin sends. Makes perfect sense

Re: [HACKERS] Multixid hindsight design

2015-06-05 Thread Andres Freund
On 2015-06-05 10:45:09 +0100, Simon Riggs wrote: On 1 June 2015 at 20:53, Thomas Munro thomas.mu...@enterprisedb.com wrote: On Tue, May 12, 2015 at 9:20 AM, Heikki Linnakangas hlinn...@iki.fi wrote: The beauty of this would be that the TED entries can be zapped at restart, just like

Re: [HACKERS] [idea] more aggressive join pushdown on postgres_fdw

2015-06-05 Thread Shigeru HANADA
2015/06/05 6:43、Robert Haas robertmh...@gmail.com のメール: On Sat, May 30, 2015 at 9:03 PM, Kouhei Kaigai kai...@ak.jp.nec.com wrote: Neat idea. This ties into something I've thought about and mentioned before: what if the innerrel is local, but there's a replicated copy on the remote server?

Re: [HACKERS] Multixid hindsight design

2015-06-05 Thread Simon Riggs
On 11 May 2015 at 22:20, Heikki Linnakangas hlinn...@iki.fi wrote: I'd like to discuss how we should've implemented the infamous 9.3 multixid/row-locking stuff, and perhaps still should in 9.6. Hindsight is always 20/20 - I'll readily admit that I didn't understand the problems until well

Re: [HACKERS] RFC: Remove contrib entirely

2015-06-05 Thread Robert Haas
On Thu, Jun 4, 2015 at 2:33 PM, Joshua D. Drake j...@commandprompt.com wrote: My argument was (after some preliminary discussion): 1. Review contrib 2. All modules that are core worthy install by default 3. Push all other contrib out into the wild So above, I said that we keep adding to

[HACKERS] pg_stat_*_columns?

2015-06-05 Thread Joel Jacobson
Would others find it useful to see per column statistics about accesses to specific columns? Two possible use-cases: (maybe there are more?) 1. I think it would be helpful for DBAs to better understand their own system. Finding unused *tables* is today easy thanks to pg_stat_*_tables, but

Re: [HACKERS] nested loop semijoin estimates

2015-06-05 Thread Robert Haas
On Tue, Jun 2, 2015 at 10:10 PM, Tom Lane t...@sss.pgh.pa.us wrote: What it seems like we should do, if we want to back-patch this, is apply it without the add_path_precheck changes. Then as an independent HEAD-only patch, change add_path_precheck so that it's behaving as designed. It looks

Re: [HACKERS] Multixid hindsight design

2015-06-05 Thread Simon Riggs
On 5 June 2015 at 11:02, Andres Freund and...@anarazel.de wrote: On 2015-06-05 10:45:09 +0100, Simon Riggs wrote: On 1 June 2015 at 20:53, Thomas Munro thomas.mu...@enterprisedb.com wrote: On Tue, May 12, 2015 at 9:20 AM, Heikki Linnakangas hlinn...@iki.fi wrote: The beauty of

[HACKERS] Re: [GENERAL] 9.4.1 - 9.4.2 problem: could not access status of transaction 1

2015-06-05 Thread Thomas Munro
On Fri, Jun 5, 2015 at 1:47 PM, Thomas Munro thomas.mu...@enterprisedb.com wrote: On Fri, Jun 5, 2015 at 11:47 AM, Thomas Munro thomas.mu...@enterprisedb.com wrote: On Fri, Jun 5, 2015 at 9:29 AM, Robert Haas robertmh...@gmail.com wrote: Here's a new version with some more fixes and

Re: [HACKERS] RFC: Remove contrib entirely

2015-06-05 Thread Simon Riggs
On 29 May 2015 at 02:50, Peter Eisentraut pete...@gmx.net wrote: On 5/28/15 3:35 PM, Stephen Frost wrote: What we would need for this is an 'extensions' directory, or similar, and a clear definition of what the requirements are around getting into it are. With that, we could decide for

Re: [HACKERS] RFC: Remove contrib entirely

2015-06-05 Thread Robert Haas
On Fri, Jun 5, 2015 at 8:32 AM, Stephen Frost sfr...@snowman.net wrote: Painting it as the unilateral actions of one committer is uncharitable, at best. As I see it, it is just stating the facts. There were several emails from other committers on the pg_audit thread not that long before it was

[HACKERS] Re: [GENERAL] 9.4.1 - 9.4.2 problem: could not access status of transaction 1

2015-06-05 Thread Robert Haas
On Fri, Jun 5, 2015 at 2:20 AM, Noah Misch n...@leadboat.com wrote: On Thu, Jun 04, 2015 at 05:29:51PM -0400, Robert Haas wrote: Here's a new version with some more fixes and improvements: I read through this version and found nothing to change. I encourage other hackers to study the patch,

Re: [HACKERS] RFC: Remove contrib entirely

2015-06-05 Thread Stephen Frost
Robert, * Robert Haas (robertmh...@gmail.com) wrote: On Thu, Jun 4, 2015 at 2:33 PM, Joshua D. Drake j...@commandprompt.com wrote: 1. 15 years of the same argument (current source: pg_audit) The argument about pg_audit has little to do with contrib. It is primarily about code quality,

Re: [HACKERS] Memory leak with XLogFileCopy since de768844 (WAL file with .partial)

2015-06-05 Thread Fujii Masao
On Fri, Jun 5, 2015 at 12:39 PM, Michael Paquier michael.paqu...@gmail.com wrote: On Thu, Jun 4, 2015 at 10:40 PM, Fujii Masao masao.fu...@gmail.com wrote: On Mon, Jun 1, 2015 at 4:24 PM, Michael Paquier michael.paqu...@gmail.com wrote: On Thu, May 28, 2015 at 9:09 PM, Michael Paquier

Re: [HACKERS] [CORE] Restore-reliability mode

2015-06-05 Thread Robert Haas
On Fri, Jun 5, 2015 at 2:50 AM, Simon Riggs si...@2ndquadrant.com wrote: Agreed. Cleanup can occur while we release code for public testing. The code is available for public testing right now. Stamping it a beta implies that we think it's something fairly stable that we'd be pretty happy to

Re: [HACKERS] Multixid hindsight design

2015-06-05 Thread Tom Lane
Simon Riggs si...@2ndquadrant.com writes: On 11 May 2015 at 22:20, Heikki Linnakangas hlinn...@iki.fi wrote: So the lesson here is that having a permanent pg_multixact is not nice, and we should get rid of it. Here's how to do that: An alternate proposal: 1. Store only the Locking xids in

Re: [HACKERS] [CORE] Restore-reliability mode

2015-06-05 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: On Fri, Jun 5, 2015 at 2:50 AM, Simon Riggs si...@2ndquadrant.com wrote: Agreed. Cleanup can occur while we release code for public testing. The code is available for public testing right now. Only to people who have the time and ability to pull the

Re: [HACKERS] RFC: Remove contrib entirely

2015-06-05 Thread Joshua D. Drake
On 06/05/2015 04:56 AM, Robert Haas wrote: somewhere else. At least not that I can see. 4. Eliminate the EGO of saying I have a contrib module in core I've got multiple major features in core. Any ego I may have about my PostgreSQL contributions is not based on pg_prewarm. This was

Re: [HACKERS] Multixid hindsight design

2015-06-05 Thread Robert Haas
On Fri, Jun 5, 2015 at 6:17 AM, Simon Riggs si...@2ndquadrant.com wrote: I think we should think back to exactly what we are trying to store, why and for how long. A clear definition of what we are trying to achieve is essential to solving the problem. In my understanding we need to store *

Re: [HACKERS] [CORE] Restore-reliability mode

2015-06-05 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: I don't maintain that end-user testing is unuseful at this point. I do maintain that it would be better to (1) finish fixing the known multixact bugs and (2) clean up some of the open items before we make a big push in that direction. For example,

Re: [HACKERS] [CORE] Restore-reliability mode

2015-06-05 Thread Alvaro Herrera
Michael Paquier wrote: On Fri, Jun 5, 2015 at 8:53 AM, Craig Ringer cr...@2ndquadrant.com wrote: In terms of specific testing improvements, things I think we need to have covered and runnable on the buildfarm are: * pg_dump and pg_restore testing (because it's scary we don't do this)

Re: [HACKERS] gcc -ansi versus SSE4.2 detection

2015-06-05 Thread Heikki Linnakangas
On 06/05/2015 09:27 PM, Tom Lane wrote: [ this is a bit roundabout, bear with me ] I noticed that, contrary to project policy, a //-style comment snuck into pg_regress.c a month or two back. I had had the idea that buildfarm member pademelon would complain about such comments, given its

Re: [HACKERS] [GENERAL] 9.4.1 - 9.4.2 problem: could not access status of transaction 1

2015-06-05 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: On Fri, Jun 5, 2015 at 12:00 PM, Andres Freund and...@anarazel.de wrote: On 2015-06-05 11:43:45 -0400, Tom Lane wrote: So where are we on this? Are we ready to schedule a new set of back-branch releases? If not, what issues remain to be looked at?

Re: [HACKERS] [GENERAL] 9.4.1 - 9.4.2 problem: could not access status of transaction 1

2015-06-05 Thread Andres Freund
On 2015-06-05 14:33:12 -0400, Tom Lane wrote: Robert Haas robertmh...@gmail.com writes: 1. The problem that we might truncate an SLRU members page away when it's in the buffers, but not drop it from the buffers, leading to a failure when we try to write it later. I've got a fix for this,

[HACKERS] Re: [GENERAL] 9.4.1 - 9.4.2 problem: could not access status of transaction 1

2015-06-05 Thread Alvaro Herrera
Robert Haas wrote: On Fri, Jun 5, 2015 at 2:20 AM, Noah Misch n...@leadboat.com wrote: On Thu, Jun 04, 2015 at 05:29:51PM -0400, Robert Haas wrote: Here's a new version with some more fixes and improvements: I read through this version and found nothing to change. I encourage other

Re: [HACKERS] [GENERAL] 9.4.1 - 9.4.2 problem: could not access status of transaction 1

2015-06-05 Thread Steve Kehlet
On Fri, Jun 5, 2015 at 11:47 AM Andres Freund and...@anarazel.de wrote: But I'd definitely like some independent testing for it, and I'm not sure if that's doable in time for the wrap. I'd be happy to test on my database that was broken, for however much that helps. It's a VM so I can easily

Re: [HACKERS] gcc -ansi versus SSE4.2 detection

2015-06-05 Thread Tom Lane
Heikki Linnakangas hlinn...@iki.fi writes: On 06/05/2015 09:27 PM, Tom Lane wrote: ... However I found out that adding -ansi also caused configure to stop selecting -msse4.2, which seemed odd, since that switch has no bearing on C language conformance. And it fell back to the slicing-by-8

Re: [HACKERS] Further issues with jsonb semantics, documentation

2015-06-05 Thread Petr Jelinek
On Fri, Jun 5, 2015 at 8:32 , Alvaro Herrera alvhe...@2ndquadrant.com wrote: Andrew Dunstan wrote: 'some jsonb value' - '{foo,bar}' is already ambiguous - the RH operand could be a single text datum or a text array. Hmm, but that's not in 9.4, so we can still tweak it if necessary.

[HACKERS] gcc -ansi versus SSE4.2 detection

2015-06-05 Thread Tom Lane
[ this is a bit roundabout, bear with me ] I noticed that, contrary to project policy, a //-style comment snuck into pg_regress.c a month or two back. I had had the idea that buildfarm member pademelon would complain about such comments, given its stone-age C compiler, but evidently not. After

Re: [HACKERS] Further issues with jsonb semantics, documentation

2015-06-05 Thread Alvaro Herrera
Andrew Dunstan wrote: On 06/04/2015 03:16 PM, Alvaro Herrera wrote: I'm just skimming here, but if a jsonb_path type is being proposed, perhaps it would be better not to have operators that take text or text[] as second argument. We can provide that functionality with just functions. For

Re: [HACKERS] [CORE] Restore-reliability mode

2015-06-05 Thread Josh Berkus
On 06/05/2015 07:23 AM, Tom Lane wrote: So let's call it an alpha, or some other way of setting expectations appropriately. But I think it's silly to maintain that the code is not in a state where end-user testing is useful. They just have to understand that they can't trust it with

Re: [HACKERS] [CORE] Restore-reliability mode

2015-06-05 Thread Peter Geoghegan
On Fri, Jun 5, 2015 at 8:51 AM, Andres Freund and...@anarazel.de wrote: 4. Arguable RLS security bug, EvalPlanQual() paranoia - This seems like another question of what the expectations around RLS actually are. In the end that's minor from the end user's perspective. I think that depends on

Re: [HACKERS] gcc -ansi versus SSE4.2 detection

2015-06-05 Thread Heikki Linnakangas
On 06/05/2015 10:07 PM, Tom Lane wrote: It looks like the actual reason that we aren't using the runtime-check CRC implementation is that we can't find a way to do cpuid on this old version of OS X. Not sure if it's worth the time to look for one; modern versions of OS X do have __get_cpuid().

Re: [HACKERS] [GENERAL] 9.4.1 - 9.4.2 problem: could not access status of transaction 1

2015-06-05 Thread Alvaro Herrera
Tom Lane wrote: Robert Haas robertmh...@gmail.com writes: There are at least two other known issues that seem like they should be fixed before we release: 1. The problem that we might truncate an SLRU members page away when it's in the buffers, but not drop it from the buffers, leading

Re: [HACKERS] [CORE] Restore-reliability mode

2015-06-05 Thread Jim Nasby
On 6/4/15 11:28 PM, Michael Paquier wrote: list of things to test * More configuration variations; ./configure, initdb options, and *.conf * More edge-case testing. (ie: what happens to each varlena as it approaches 1GB? 1B tables test. Etc.) * More race-condition testing, like the tool Peter

Re: [HACKERS] RFC: Remove contrib entirely

2015-06-05 Thread David E. Wheeler
On Jun 5, 2015, at 12:34 AM, Jim Nasby jim.na...@bluetreble.com wrote: A number of modules also run Travis-CI. Might be worth having a way for a module to provide it's status .png. Right. Just stick it in your README. http://blog.pgxn.org/post/116087351668/badges David smime.p7s

Re: [HACKERS] [CORE] Restore-reliability mode

2015-06-05 Thread Andres Freund
On 2015-06-05 11:05:14 -0400, Bruce Momjian wrote: To release 9.5 beta would be to get back into that cycle, and I am not sure we are ready for that. I think the fact we have multiple people all reviewing the multi-xact code now (and not dealing with 9.5) is a good sign. If we were focused

Re: [HACKERS] [CORE] Restore-reliability mode

2015-06-05 Thread Simon Riggs
On 5 June 2015 at 16:05, Bruce Momjian br...@momjian.us wrote: Please address some of the specific issues I mentioned. I can discuss them but not because I am involved directly. I take responsibility as a committer and have an interest from that perspective. In my role at 2ndQuadrant, I

[HACKERS] Is it possible to have a fast-write Index?

2015-06-05 Thread deavid
There are several use cases where I see useful an index, but adding it will slow too much inserts and updates. For example, when we have 10 million rows on a table, and it's a table which has frequent updates, we need several index to speed up selects, but then we'll slow down updates a lot,

Re: [HACKERS] [CORE] Restore-reliability mode

2015-06-05 Thread Jim Nasby
On 6/5/15 10:39 AM, Tom Lane wrote: The other side of that coin is that we might get useful comments from testers on how the feature ought to work. I don't agree with the notion that all feature details must be graven on stone tablets before we start trying to get feedback from people outside

Re: [HACKERS] [CORE] Restore-reliability mode

2015-06-05 Thread Bruce Momjian
On Fri, Jun 5, 2015 at 05:36:41PM +0200, Andres Freund wrote: I don't think anything as localized as 'do nothing but bugfixes for a while and then carry on' actually will solve the problem. We need to find and reallocate resources to put more emphasis on review, robustness and refactoring in

Re: [HACKERS] [CORE] Restore-reliability mode

2015-06-05 Thread Andres Freund
On 2015-06-05 11:20:52 -0400, Robert Haas wrote: I don't maintain that end-user testing is unuseful at this point. Unless I misunderstand you, and you're not saying that user level testing wouldn't be helpful right now, I'm utterly baffled. There's loads of user-exposed features that desperately

Re: [HACKERS] [GENERAL] 9.4.1 - 9.4.2 problem: could not access status of transaction 1

2015-06-05 Thread Andres Freund
On 2015-06-05 11:43:45 -0400, Tom Lane wrote: Robert Haas robertmh...@gmail.com writes: On Fri, Jun 5, 2015 at 2:20 AM, Noah Misch n...@leadboat.com wrote: I read through this version and found nothing to change. I encourage other hackers to study the patch, though. The surrounding code

Re: [HACKERS] RFC: Remove contrib entirely

2015-06-05 Thread Neil Tiffin
On Jun 4, 2015, at 3:11 PM, David E. Wheeler da...@justatheory.com wrote: On Jun 4, 2015, at 11:53 AM, Neil Tiffin ne...@neiltiffin.com wrote: I have looked at PGXN and would never install anything from it. Why? Because it is impossible to tell, without inside knowledge or a lot of

Re: [HACKERS] [GENERAL] 9.4.1 - 9.4.2 problem: could not access status of transaction 1

2015-06-05 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: On Fri, Jun 5, 2015 at 2:20 AM, Noah Misch n...@leadboat.com wrote: I read through this version and found nothing to change. I encourage other hackers to study the patch, though. The surrounding code is challenging. Andres tested this and discovered

Re: [HACKERS] [CORE] Restore-reliability mode

2015-06-05 Thread Simon Riggs
On 5 June 2015 at 15:00, Robert Haas robertmh...@gmail.com wrote: On Fri, Jun 5, 2015 at 2:50 AM, Simon Riggs si...@2ndquadrant.com wrote: Agreed. Cleanup can occur while we release code for public testing. The code is available for public testing right now. People test when they get the

Re: [HACKERS] [CORE] Restore-reliability mode

2015-06-05 Thread Robert Haas
On Fri, Jun 5, 2015 at 10:23 AM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: On Fri, Jun 5, 2015 at 2:50 AM, Simon Riggs si...@2ndquadrant.com wrote: Agreed. Cleanup can occur while we release code for public testing. The code is available for public testing

Re: [HACKERS] [CORE] Restore-reliability mode

2015-06-05 Thread Simon Riggs
On 5 June 2015 at 15:00, Robert Haas robertmh...@gmail.com wrote: I do agree that an indefinite development freeze with unclear parameters for resuming development and unclear goals is a bad plan. But I think giving ourselves a little more time to, say, turn the buildfarm consistently green,

Re: [HACKERS] RFC: Remove contrib entirely

2015-06-05 Thread David Fetter
On Thu, Jun 04, 2015 at 01:11:21PM -0700, David E. Wheeler wrote: On Jun 4, 2015, at 11:53 AM, Neil Tiffin ne...@neiltiffin.com wrote: I have looked at PGXN and would never install anything from it. Why? Because it is impossible to tell, without inside knowledge or a lot of work, what is

Re: [HACKERS] Further issues with jsonb semantics, documentation

2015-06-05 Thread Alvaro Herrera
Jim Nasby wrote: On 6/5/15 2:08 PM, Petr Jelinek wrote: That's a good point, and it won't get any better if/when we add the json point support in 9.6 since the syntax would be something like select jsonb '{a:1, b:2, c: {a: 2}}' - '/c/a'; and we will again silently do nothing. That's going to

Re: [HACKERS] Further issues with jsonb semantics, documentation

2015-06-05 Thread Tom Lane
Andrew Dunstan and...@dunslane.net writes: Yeah, Good point. Actually, if my memory serves me correctly (always a dubious bet), the avoidance of that kind of ambiguity is why we introduced the # and # operators in the first place, after going round and round for a while on what the API

[HACKERS] could not truncate directory pg_subtrans: apparent wraparound

2015-06-05 Thread Dan Langille
I noticed this today on my 9.4.2 server running on FreeBSD 10.1: Jun 5 18:59:40 slocum postgres[986]: [3957-1] LOG: could not truncate directory pg_subtrans: apparent wraparound Looking at a post from 2010, Tom Lane suggest this information was useful:

Re: [HACKERS] [GENERAL] 9.4.1 - 9.4.2 problem: could not access status of transaction 1

2015-06-05 Thread Joshua D. Drake
On 06/05/2015 01:56 PM, Tom Lane wrote: If we have confidence that we can ship something on Monday that is materially more trustworthy than the current releases, then let's aim to do that; but let's ship only patches we are confident in. We can do another set of releases later that

Re: [HACKERS] [GENERAL] 9.4.1 - 9.4.2 problem: could not access status of transaction 1

2015-06-05 Thread Alvaro Herrera
Joshua D. Drake wrote: I believe there are likely quite a few parties willing to help test, if we knew how? The code involved is related to checkpoints, pg_basebackups that take a long time to run, and multixact freezing and truncation. If you can set up test servers that eat lots of

Re: [HACKERS] Further issues with jsonb semantics, documentation

2015-06-05 Thread Andrew Dunstan
On 06/05/2015 02:32 PM, Alvaro Herrera wrote: 'some jsonb value' - '{foo,bar}' is already ambiguous - the RH operand could be a single text datum or a text array. Hmm, but that's not in 9.4, so we can still tweak it if necessary. Consider this jsonb datum. Nobody in their right mind would

Re: [HACKERS] [GENERAL] 9.4.1 - 9.4.2 problem: could not access status of transaction 1

2015-06-05 Thread Robert Haas
On Fri, Jun 5, 2015 at 2:47 PM, Andres Freund and...@anarazel.de wrote: On 2015-06-05 14:33:12 -0400, Tom Lane wrote: Robert Haas robertmh...@gmail.com writes: 1. The problem that we might truncate an SLRU members page away when it's in the buffers, but not drop it from the buffers, leading

Re: [HACKERS] [GENERAL] 9.4.1 - 9.4.2 problem: could not access status of transaction 1

2015-06-05 Thread Robert Haas
On Fri, Jun 5, 2015 at 2:36 PM, Alvaro Herrera alvhe...@2ndquadrant.com wrote: Tom Lane wrote: Robert Haas robertmh...@gmail.com writes: There are at least two other known issues that seem like they should be fixed before we release: 1. The problem that we might truncate an SLRU members

Re: [HACKERS] [GENERAL] 9.4.1 - 9.4.2 problem: could not access status of transaction 1

2015-06-05 Thread Tom Lane
Andres Freund and...@anarazel.de writes: On June 5, 2015 10:02:37 PM GMT+02:00, Robert Haas robertmh...@gmail.com wrote: I think we would be foolish to rush that part into the tree. We probably got here in the first place by rushing the last round of fixes too much; let's try not to double

Re: [HACKERS] Further issues with jsonb semantics, documentation

2015-06-05 Thread Jim Nasby
On 6/5/15 2:08 PM, Petr Jelinek wrote: That's a good point, and it won't get any better if/when we add the json point support in 9.6 since the syntax would be something like select jsonb '{a:1, b:2, c: {a: 2}}' - '/c/a'; and we will again silently do nothing. That's going to cause bugs in

Re: [HACKERS] [CORE] Restore-reliability mode

2015-06-05 Thread Peter Geoghegan
On Fri, Jun 5, 2015 at 7:00 AM, Robert Haas robertmh...@gmail.com wrote: I do agree that an indefinite development freeze with unclear parameters for resuming development and unclear goals is a bad plan. But I think giving ourselves a little more time to, say, turn the buildfarm consistently

Re: [HACKERS] RFC: Remove contrib entirely

2015-06-05 Thread Jeff Janes
On Fri, Jun 5, 2015 at 7:42 AM, Joshua D. Drake j...@commandprompt.com wrote: On 06/05/2015 04:56 AM, Robert Haas wrote: somewhere else. At least not that I can see. 4. Eliminate the EGO of saying I have a contrib module in core I've got multiple major features in core. Any ego I may

Re: [HACKERS] [GENERAL] 9.4.1 - 9.4.2 problem: could not access status of transaction 1

2015-06-05 Thread Andres Freund
On June 5, 2015 10:02:37 PM GMT+02:00, Robert Haas robertmh...@gmail.com wrote: On Fri, Jun 5, 2015 at 2:47 PM, Andres Freund and...@anarazel.de wrote: On 2015-06-05 14:33:12 -0400, Tom Lane wrote: Robert Haas robertmh...@gmail.com writes: 1. The problem that we might truncate an SLRU members

Re: [HACKERS] could not truncate directory pg_subtrans: apparent wraparound

2015-06-05 Thread Thomas Munro
On Sat, Jun 6, 2015 at 10:12 AM, Thomas Munro thomas.mu...@enterprisedb.com wrote: On Sat, Jun 6, 2015 at 9:45 AM, Dan Langille d...@langille.org wrote: I noticed this today on my 9.4.2 server running on FreeBSD 10.1: Jun 5 18:59:40 slocum postgres[986]: [3957-1] LOG: could not truncate

Re: [HACKERS] Further issues with jsonb semantics, documentation

2015-06-05 Thread Peter Geoghegan
On Fri, Jun 5, 2015 at 10:51 AM, Andrew Dunstan and...@dunslane.net wrote: Also, what about negative array subscripting (making the 9.4-era operator jsonb - integer operator support that for consistency with the new operator jsonb - integer operator)? Should I write the patch? Will you commit

Re: [HACKERS] could not truncate directory pg_subtrans: apparent wraparound

2015-06-05 Thread Thomas Munro
On Sat, Jun 6, 2015 at 9:45 AM, Dan Langille d...@langille.org wrote: I noticed this today on my 9.4.2 server running on FreeBSD 10.1: Jun 5 18:59:40 slocum postgres[986]: [3957-1] LOG: could not truncate directory pg_subtrans: apparent wraparound Looking at a post from 2010, Tom Lane

Re: [HACKERS] [GENERAL] 9.4.1 - 9.4.2 problem: could not access status of transaction 1

2015-06-05 Thread Robert Haas
On Fri, Jun 5, 2015 at 4:40 PM, Andres Freund and...@anarazel.de wrote: I think we would be foolish to rush that part into the tree. We probably got here in the first place by rushing the last round of fixes too much; let's try not to double down on that mistake. My problem with that approach

Re: [HACKERS] Further issues with jsonb semantics, documentation

2015-06-05 Thread Peter Geoghegan
On Fri, Jun 5, 2015 at 1:05 PM, Andrew Dunstan and...@dunslane.net wrote: So probably the least invasive change would be to rename the text[] variant operator to something like #- and rename the corresponding function to jsonb_delete_path. We could also decide not to keep an operator at all,

Re: [HACKERS] Is it possible to have a fast-write Index?

2015-06-05 Thread Gavin Flower
On 06/06/15 04:07, deavid wrote: There are several use cases where I see useful an index, but adding it will slow too much inserts and updates. For example, when we have 10 million rows on a table, and it's a table which has frequent updates, we need several index to speed up selects, but then

Re: [HACKERS] Is it possible to have a fast-write Index?

2015-06-05 Thread deavid
Thanks to everybody for answering. I wasn't expecting this attention; this is a great community :-) Jim asked me about something real. Well, the problem is this showed up more than five years ago, and keeps popping from time to time since in different circumstances. I solved them in different

Re: [HACKERS] could not truncate directory pg_subtrans: apparent wraparound

2015-06-05 Thread Thomas Munro
On Sat, Jun 6, 2015 at 1:25 PM, Alvaro Herrera alvhe...@2ndquadrant.com wrote: Thomas Munro wrote: My idea was that if I could get oldestXact == next XID in TruncateSUBSTRANS, then TransactionIdToPage(oldestXact) for a value of oldestXact that happens to be immediately after a page boundary

Re: [HACKERS] [CORE] Restore-reliability mode

2015-06-05 Thread Michael Paquier
On Sat, Jun 6, 2015 at 12:05 AM, Alvaro Herrera wrote: Michael Paquier wrote: What happened with the extension tests patches you submitted? They seemed valuable to me, but I lost track. Those ones are registered in the queue of 9.6: https://commitfest.postgresql.org/5/187/ And this is the

Re: [HACKERS] could not truncate directory pg_subtrans: apparent wraparound

2015-06-05 Thread Alvaro Herrera
Thomas Munro wrote: My idea was that if I could get oldestXact == next XID in TruncateSUBSTRANS, then TransactionIdToPage(oldestXact) for a value of oldestXact that happens to be immediately after a page boundary (so that xid % 2048 == 0) might give page number that is = latest_page_number,

Re: [HACKERS] Re: [COMMITTERS] pgsql: Map basebackup tablespaces using a tablespace_map file

2015-06-05 Thread Amit Kapila
On Fri, Jun 5, 2015 at 10:51 AM, Amit Kapila amit.kapil...@gmail.com wrote: On Fri, Jun 5, 2015 at 9:57 AM, Andrew Dunstan and...@dunslane.net wrote: On 06/04/2015 11:35 PM, Amit Kapila wrote: Theoretically, I don't see much problem by changing the checks way you have done in patch, but

Re: [HACKERS] [CORE] Restore-reliability mode

2015-06-05 Thread Bruce Momjian
On Fri, Jun 5, 2015 at 04:54:56PM +0100, Simon Riggs wrote: On 5 June 2015 at 16:05, Bruce Momjian br...@momjian.us wrote: Please address some of the specific issues I mentioned.  I can discuss them but not because I am involved directly. I take responsibility as a committer and

Re: [HACKERS] Warn about using single user + standby_mode

2015-06-05 Thread Amit Kapila
On Fri, Jun 5, 2015 at 9:59 PM, Andres Freund and...@anarazel.de wrote: Hi, When primary_conninfo is configured and standby_mode is enabled single user mode starts up, but at some point will just wait in a latch for WAL to show up. Which will obviously never happen as walreceiver isn't up.

Re: [HACKERS] Multixid hindsight design

2015-06-05 Thread Robert Haas
On Fri, Jun 5, 2015 at 10:27 AM, Tom Lane t...@sss.pgh.pa.us wrote: That way if we need to make Offsets SLRU persistent it won't bloat. We then leave the Members SLRU as non-persistent, just as it was 9.3 I don't think you can do that, because it supposes that locking XIDs need not be

Re: [HACKERS] [CORE] Restore-reliability mode

2015-06-05 Thread Bruce Momjian
On Fri, Jun 5, 2015 at 07:50:31AM +0100, Simon Riggs wrote: On 3 June 2015 at 18:21, Josh Berkus j...@agliodbs.com wrote:   I would argue that if we delay 9.5 in order to do a 100% manual review of code, without adding any new automated tests or other non-manual tools for

Re: [HACKERS] RFC: Remove contrib entirely

2015-06-05 Thread Andrew Dunstan
On 06/05/2015 03:34 AM, Jim Nasby wrote: On 6/4/15 3:11 PM, David E. Wheeler wrote: There is no indication of what versions of pg any of PGXN modules are tested on, or even if there are tests that can be run to prove the module works correctly with a particular version of pg. Yeah, I’ve been

Re: [HACKERS] RFC: Remove contrib entirely

2015-06-05 Thread Bruce Momjian
On Fri, Jun 5, 2015 at 02:42:45PM +0100, Simon Riggs wrote: On 29 May 2015 at 02:50, Peter Eisentraut pete...@gmx.net wrote: On 5/28/15 3:35 PM, Stephen Frost wrote: What we would need for this is an 'extensions' directory, or similar, and a clear definition of what the