Re: [HACKERS] Exclude pg_internal.init from base backup

2017-11-07 Thread David Steele
On 11/7/17 11:03 AM, Simon Riggs wrote: > On 5 November 2017 at 11:55, Magnus Hagander wrote: >> >> So +1 for documenting the difference in how these are handled, as this is >> important to know for somebody writing an external tool for it. > > Changes made, moving to commit

Re: [HACKERS] WIP: Restricting pg_rewind to data/wal dirs

2017-10-31 Thread David Steele
On 10/30/17 6:36 AM, Michael Paquier wrote: > On Mon, Oct 30, 2017 at 10:15 AM, Chris Travers >> >> How does rep mgr or other programs using pg_rewind know what to exclude? > > Good question. Answers could come from folks such as David Steele > (pgBackRest) or Marco (bar

Re: [HACKERS] Timeline ID in backup_label file

2017-10-26 Thread David Steele
On 10/25/17 10:10 PM, Craig Ringer wrote: > On 26 October 2017 at 02:50, Michael Paquier > wrote: >> >> I would find interesting to add at the bottom of the backup_label file >> a new field called "START TIMELINE: %d" to put this information in a >> more understandable

Re: [HACKERS] OpenFile() Permissions Refactor

2017-09-24 Thread David Steele
On 9/23/17 10:22 AM, Peter Eisentraut wrote: > On 9/13/17 10:26, David Steele wrote: >> Here's a new patch based on your review. Where I had a question I made >> a choice as described below: > > I have committed the changes to the file APIs and a fix for the umask > sa

Re: [HACKERS] Show backtrace when tap tests fail

2017-09-20 Thread David Steele
On 9/20/17 6:26 AM, Dagfinn Ilmari Mannsåker wrote: > Craig Ringer <cr...@2ndquadrant.com> writes: > >> On 20 September 2017 at 06:36, David Steele <da...@pgmasters.net> wrote: >> >>> >>> I just use: >>> >>> $SIG{__DIE__} = su

Re: [HACKERS] Show backtrace when tap tests fail

2017-09-19 Thread David Steele
On 9/19/17 5:25 PM, Tom Lane wrote: Andres Freund writes: On 2017-09-19 17:15:21 -0400, Tom Lane wrote: Meh --- Carp::Always isn't standard either, so I think this is just extra complication with little value-add. Let's just do the Devel::Confess incantation as Dagfinn

Re: [HACKERS] Creating backup history files for backups taken from standbys

2017-09-18 Thread David Steele
On 9/18/17 7:26 PM, Michael Paquier wrote: > On Tue, Sep 19, 2017 at 8:14 AM, David Steele <da...@pgmasters.net> wrote: >> On 8/31/17 11:56 PM, Michael Paquier wrote: >>> Here is an updated patch with refreshed documentation, as a result of >>> 449338c which

Re: [HACKERS] Creating backup history files for backups taken from standbys

2017-09-18 Thread David Steele
Hi Michael, On 8/31/17 11:56 PM, Michael Paquier wrote: > Here is an updated patch with refreshed documentation, as a result of > 449338c which was discussed in thread > https://www.postgresql.org/message-id/d4d951b9-89c0-6bc1-b6ff-d0b2dd5a8...@pgmasters.net. > I am just outlining the fact that

Re: [HACKERS] additional contrib test suites

2017-09-16 Thread David Steele
On 9/15/17 6:52 PM, Michael Paquier wrote: > On Sat, Sep 16, 2017 at 5:15 AM, Tom Lane wrote: >> >> Noting that mandrill is showing yet a different failure, one that I think >> is inherent to chkpass: >> >> CREATE TABLE test (i int, p chkpass); >> INSERT INTO test VALUES

Re: [HACKERS] additional contrib test suites

2017-09-14 Thread David Steele
On 9/8/17 1:32 PM, Peter Eisentraut wrote: > > Yes, some of the error messages had changed. Fixed patches attached. Patches apply and all tests pass. A few comments: * [PATCH v2 1/7] adminpack: Add test suite There are no regular tests for pg_logdir_ls(). It looks like TAP tests would be

Re: [HACKERS] OpenFile() Permissions Refactor

2017-09-13 Thread David Steele
Hi Peter, Here's a new patch based on your review. Where I had a question I made a choice as described below: On 9/1/17 1:58 PM, David Steele wrote: > On 9/1/17 1:15 PM, Peter Eisentraut wrote: >> On 8/29/17 12:15, David Steele wrote: >> >> I wonder whether we even need

[HACKERS] Exclude pg_internal.init from base backup

2017-09-02 Thread David Steele
Hackers, The cache in pg_internal.init was reused in days of yore but has been rebuilt on postmaster startup since v8.1. It appears there is no reason for this file to be backed up. I also moved the RELCACHE_INIT_FILENAME constant to relcache.h to avoid duplicating the string. I'll add this to

Re: [HACKERS] Rename RECOVERYXLOG to RECOVERYWAL?

2017-09-02 Thread David Steele
On 9/1/17 7:53 PM, Michael Paquier wrote: > On Sat, Sep 2, 2017 at 3:06 AM, Robert Haas wrote: >> I don't think this really buys us anything. If we'd applied it to v10 >> maybe, but what do we get out of whacking it around now? >> >> "Consistency", I hear you cry! Fair

Re: [HACKERS] Rename RECOVERYXLOG to RECOVERYWAL?

2017-09-01 Thread David Steele
On 9/1/17 2:06 PM, Robert Haas wrote: > On Fri, Sep 1, 2017 at 12:57 PM, David Steele <da...@pgmasters.net> wrote: >> I searched the various threads on the xlog -> wal rename and I couldn't >> find any specific mention of why this was not renamed. >> >

Re: [HACKERS] OpenFile() Permissions Refactor

2017-09-01 Thread David Steele
On 9/1/17 1:15 PM, Peter Eisentraut wrote: > On 8/29/17 12:15, David Steele wrote: > > I wonder whether we even need that much flexibility. We already set a > global umask, so we could just open all files with 0666 and let the > umask sort it out. Then we don't need all the *

[HACKERS] Rename RECOVERYXLOG to RECOVERYWAL?

2017-09-01 Thread David Steele
I searched the various threads on the xlog -> wal rename and I couldn't find any specific mention of why this was not renamed. I have attached a patch in case it was an oversight rather than left as-is on purpose. Thanks, -- -David da...@pgmasters.net diff --git

Re: [HACKERS] Update low-level backup documentation to match actual behavior

2017-08-31 Thread David Steele
On 8/31/17 4:04 PM, Robert Haas wrote: > On Wed, Aug 30, 2017 at 8:37 PM, Michael Paquier > wrote: >> Thanks for the new version. This looks fine to me. > > Committed to REL9_6_STABLE with minor wordsmithing. The edits look good to me. Thanks, Robert! -- -David

Re: [HACKERS] Update low-level backup documentation to match actual behavior

2017-08-30 Thread David Steele
Hi Michael, Thanks for reviewing! On 8/29/17 9:44 PM, Michael Paquier wrote: > On Tue, Aug 29, 2017 at 10:59 PM, David Steele <da...@pgmasters.net> wrote: >> >> Attached is the 9.6 patch. It required a bit more work in func.sgml >> than I was expecting so have a cl

[HACKERS] OpenFile() Permissions Refactor

2017-08-29 Thread David Steele
Hackers, While working on the patch to allow group reads of $PGDATA I refactored the various OpenFile() functions to use default/global permissions rather than permissions defined in each call. I think the patch stands on its own regardless of whether we accept the patch to allow group

Re: [HACKERS] Update low-level backup documentation to match actual behavior

2017-08-29 Thread David Steele
Hi Robert, On 8/25/17 4:03 PM, David Steele wrote: > On 8/25/17 3:26 PM, Robert Haas wrote: >> On Fri, Aug 25, 2017 at 3:21 PM, David Steele <da...@pgmasters.net> >> wrote: >>> No problem.  I'll base it on your commit to capture any changes you >>> m

Re: [HACKERS] Update low-level backup documentation to match actual behavior

2017-08-25 Thread David Steele
On 8/25/17 3:26 PM, Robert Haas wrote: On Fri, Aug 25, 2017 at 3:21 PM, David Steele <da...@pgmasters.net> wrote: No problem. I'll base it on your commit to capture any changes you made. Thanks, but you incorporated everything I wanted in response to my first review -- so I didn't

Re: [HACKERS] Update low-level backup documentation to match actual behavior

2017-08-25 Thread David Steele
On 8/25/17 3:13 PM, Robert Haas wrote: > On Fri, Aug 25, 2017 at 3:10 PM, David Steele <da...@pgmasters.net> wrote: >> >> Robert said he would commit this so I expect he'll do that if he doesn't >> have any objections to the changes. >> >> Robert, if you w

Re: [HACKERS] Update low-level backup documentation to match actual behavior

2017-08-25 Thread David Steele
On 8/24/17 7:36 PM, Michael Paquier wrote: > > True as well. The patch looks good to me. If a committer does not show > up soon, it may be better to register that in the CF and wait. I am > not sure that adding an open item is suited, as docs have the same > problem on 9.6. Robert said he would

Re: [HACKERS] Update low-level backup documentation to match actual behavior

2017-08-24 Thread David Steele
Hi Michael, Thanks for reviewing! Sorry for the late response, those eclipses don't just chase themselves... On 8/20/17 10:22 PM, Michael Paquier wrote: > On Fri, Aug 18, 2017 at 3:35 AM, David Steele <da...@pgmasters.net> wrote: > > + Prior to PostgreSQL 9.6, this &g

Re: [HACKERS] Update low-level backup documentation to match actual behavior

2017-08-18 Thread David Steele
On 8/18/17 3:00 PM, Robert Haas wrote: > > If you update the patch I'll apply it to 11 and 10. Attached is the updated patch. I didn't like the vague "there can be some issues on the server if it crashes during the backup" so I added a new paragraph at the appropriate step to give a more

Re: [HACKERS] Update low-level backup documentation to match actual behavior

2017-08-18 Thread David Steele
Robert, Thanks for reviewing! On 8/18/17 2:45 PM, Robert Haas wrote: > - the next WAL segment. The reason for the switch is to arrange for > + the next WAL segment when run on a primary. On a standby you can call > + pg_switch_wal on the primary to perform a manual > + switch.

[HACKERS] Update low-level backup documentation to match actual behavior

2017-08-17 Thread David Steele
As discussed in [1] our low-level backup documentation does not quite match the actual behavior of the functions on primary vs. standby. Since it appears we have decided that the remaining behavioral differences after 52f8a59dd953c68 are bugs in the documentation, the attached is a first pass at

Re: [HACKERS] pg_stop_backup(wait_for_archive := true) on standby server

2017-07-24 Thread David Steele
On 7/24/17 3:28 PM, David Steele wrote: Yes, and this is another behavioral change to consider -- one that is more likely to impact users than the change to pg_stop_backup(). If pg_basebackup is not currently waiting for WAL on a standby (but does on a primary) then that's pretty serious

Re: [HACKERS] pg_stop_backup(wait_for_archive := true) on standby server

2017-07-24 Thread David Steele
On 7/24/17 12:28 PM, Stephen Frost wrote: * David Steele (da...@pgmasters.net) wrote: While this patch brings pg_stop_backup() in line with the documentation, it also introduces a behavioral change compared to 9.6. Currently, the default 9.6 behavior on a standby is to return immediately

Re: [HACKERS] pg_stop_backup(wait_for_archive := true) on standby server

2017-07-24 Thread David Steele
On 7/23/17 11:48 PM, Masahiko Sawada wrote: On Sat, Jul 22, 2017 at 8:04 AM, Stephen Frost wrote: I started discussing this with David off-list and he'd like a chance to review it in a bit more detail (he's just returning from being gone for a few weeks). That review will

Re: [HACKERS] Should pg_current_wal_location() become pg_current_wal_lsn()

2017-05-09 Thread David Steele
On 5/9/17 10:00 AM, Stephen Frost wrote: * Tom Lane (t...@sss.pgh.pa.us) wrote: #2: Rename all these functions and columns to "lsn", as in this patch. +1 <...> #2 strikes me as the best option, though that's probably not much of a surprise to anyone whose been following my comments on

Re: [HACKERS] renaming "transaction log"

2017-05-03 Thread David Steele
On 5/2/17 9:09 PM, Peter Eisentraut wrote: > Most documentation and error messages still uses the term "transaction > log" to refer to the write-ahead log. Here is a patch to rename that, > which I think should be done, to match the xlog -> wal renaming in APIs. +1 for the idea. The

Re: [HACKERS] Should pg_current_wal_location() become pg_current_wal_lsn()

2017-04-18 Thread David Steele
On 4/15/17 12:56 PM, Robert Haas wrote: > On Fri, Apr 14, 2017 at 6:52 PM, Tom Lane wrote: >> Peter Eisentraut writes: >>> If we're talking about making things easier to understand, wouldn't a >>> random user rather know what a WAL "location"

Re: [HACKERS] snapbuild woes

2017-04-08 Thread David Steele
On 4/8/17 10:29 AM, Erik Rijkers wrote: > On 2017-04-08 15:56, Andres Freund wrote: >> On 2017-04-08 09:51:39 -0400, David Steele wrote: >>> On 3/2/17 7:54 PM, Petr Jelinek wrote: >>> > >>> > Yes the copy patch needs rebase as well. But these ones are f

[HACKERS] 2017-03 CF Closed

2017-04-08 Thread David Steele
The commitfest has officially ended. I have pushed the remaining patches to the next commitfest or returned then with feedback as appropriate: Features moved to next CF: - new plpgsql extra_checks - Better estimate merging for duplicate vars in clausesel.c - initdb configurable wal_segment_size -

Re: [HACKERS] postgres_fdw: support parameterized foreign joins

2017-04-08 Thread David Steele
On 4/7/17 9:54 AM, Arthur Zakirov wrote: > > Marked the patch as "Ready for Commiter". But the patch should be > commited only after the patch [1]. This submission has been moved to CF 2017-07. -- -David da...@pgmasters.net -- Sent via pgsql-hackers mailing list

Re: [HACKERS] Push down more UPDATEs/DELETEs in postgres_fdw

2017-04-08 Thread David Steele
On 3/22/17 6:20 AM, Etsuro Fujita wrote: > On 2017/02/22 19:57, Rushabh Lathia wrote: >> Marked this as Ready for Committer. > > I noticed that this item in the CF app was incorrectly marked as > Committed. This patch isn't committed, so I returned it to the previous > status. I also rebased

Re: [HACKERS] Bug in to_timestamp().

2017-04-08 Thread David Steele
On 2/27/17 4:19 AM, Artur Zakirov wrote: > On 15.02.2017 15:26, Amul Sul wrote: >> >> The new status of this patch is: Ready for Committer >> > > Thank you for your review! This submission has been moved to CF 2017-07. -- -David da...@pgmasters.net -- Sent via pgsql-hackers mailing list

Re: [HACKERS] Vacuum: allow usage of more than 1GB of work mem

2017-04-08 Thread David Steele
On 4/7/17 10:19 PM, Claudio Freire wrote: > > I rebased the early free patch (patch 3) to apply on top of the v9 > patch 2 (it needed some changes). I recognize the early free patch > didn't get nearly as much scrutiny, so I'm fine with commiting only 2 > if that one's ready to go but 3 isn't. >

Re: [HACKERS] increasing the default WAL segment size

2017-04-08 Thread David Steele
On 4/7/17 2:59 AM, Beena Emerson wrote: > I ran tests and following are the details: > > Machine details: > Architecture: ppc64le > Byte Order:Little Endian > CPU(s):192 > On-line CPU(s) list: 0-191 > Thread(s) per core:8 > Core(s) per socket:1 >

Re: [HACKERS] Crash on promotion when recovery.conf is renamed

2017-04-08 Thread David Steele
On 3/28/17 1:21 AM, Tsunakawa, Takayuki wrote: > From: Michael Paquier [mailto:michael.paqu...@gmail.com] >> Okay. I got the message, and I agree with what you say here. You are right >> by the way, the error messages just use "two-phase file" and not "two-phase >> STATE file", missed that

Re: [HACKERS] WIP: Covering + unique indexes.

2017-04-08 Thread David Steele
On 4/4/17 2:47 PM, Peter Geoghegan wrote: > > At the very least, you should change comments to note the issue. I > think it's highly unlikely that this could ever result in a failure to > find a split point, which there are many defenses against already, but > I think I would find that difficult

Re: [HACKERS] postgres_fdw bug in 9.6

2017-04-08 Thread David Steele
On 4/7/17 3:24 PM, Robert Haas wrote: > > There's probably more to think about here, but those are my question > on an initial read-through. This bug has been moved to CF 2017-07. -- -David da...@pgmasters.net -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make

Re: [HACKERS] snapbuild woes

2017-04-08 Thread David Steele
On 3/2/17 7:54 PM, Petr Jelinek wrote: > > Yes the copy patch needs rebase as well. But these ones are fine. This bug has been moved to CF 2017-07. -- -David da...@pgmasters.net -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription:

Re: [HACKERS] Making clausesel.c Smarter

2017-04-08 Thread David Steele
On 4/7/17 5:33 PM, Claudio Freire wrote: > > Also, can you add a test case? Cost values could make the test > fragile, so if that gives you trouble I'll understand, but it'd be > best to try and test this if possible. This submission has been moved to CF 2017-07. -- -David da...@pgmasters.net

Re: [HACKERS] plpgsql - additional extra checks

2017-04-08 Thread David Steele
> On 1/13/17 6:55 AM, Marko Tiikkaja wrote: >> On Fri, Jan 13, 2017 at 2:46 AM, Jim Nasby > > wrote: >> >> On 1/11/17 5:54 AM, Pavel Stehule wrote: >> >> +too_many_rows >> + >> + >> +

Re: [HACKERS] [PATCH] Warn users about duplicate configuration parameters

2017-04-07 Thread David Steele
On 4/7/17 11:22 AM, Tatsuo Ishii wrote: >>> Recently I've discovered that if there are multiple values of the same >>> parameter in postgresql.conf PostgreSQL will silently use the last one. >>> It looks like not the best approach to me. For instance, user can find >>> the first value in the

Re: [HACKERS] increasing the default WAL segment size

2017-04-06 Thread David Steele
On 4/6/17 6:52 PM, Tomas Vondra wrote: > On 04/06/2017 11:45 PM, David Steele wrote: >> >> How many people in the field are running custom builds of >> Postgres? And of those, how many have changed the WAL segment size? >> I've never encountered a non-standard segme

Re: [HACKERS] increasing the default WAL segment size

2017-04-06 Thread David Steele
On 4/6/17 5:05 PM, Tomas Vondra wrote: > On 04/06/2017 08:33 PM, David Steele wrote: >> On 4/5/17 7:29 AM, Simon Riggs wrote: >> >>> 2. It's not clear to me the advantage of being able to pick varying >>> filesizes. I see great disadvantage in having too many opt

Re: [HACKERS] increasing the default WAL segment size

2017-04-06 Thread David Steele
On 4/5/17 7:29 AM, Simon Riggs wrote: > On 5 April 2017 at 06:04, Beena Emerson wrote: >> >> The WALfilename - LSN mapping disruption for higher values you mean? Is >> there anything else I have missed? > > I see various issues raised but not properly addressed > > 1.

Re: [HACKERS] Re: PATCH: pageinspect / add page_checksum and bt_page_items(bytea)

2017-04-04 Thread David Steele
On 4/4/17 12:55 PM, Ashutosh Sharma wrote: > > As I am not seeing any response from Tomas for last 2-3 days and since > the commit-fest is coming towards end, I have planned to work on the > review comments that I had given few days back and submit the updated > patch. PFA new version of patch

Re: [HACKERS] PATCH: Make pg_stop_backup() archive wait optional

2017-04-04 Thread David Steele
On 4/4/17 11:42 AM, Stephen Frost wrote: > * David Steele (da...@pgmasters.net) wrote: >> On 3/22/17 4:42 PM, Peter Eisentraut wrote: >>> On 3/22/17 15:14, Stephen Frost wrote: >>>>> -SELECT * FROM pg_stop_backup(false); >>>>> +SELECT * FROM pg_s

Re: [HACKERS] tuplesort_gettuple_common() and *should_free argument

2017-04-04 Thread David Steele
Hi Anastasia, On 3/13/17 9:14 PM, Peter Geoghegan wrote: > On Wed, Jan 25, 2017 at 3:11 PM, Peter Geoghegan wrote: >> On Wed, Jan 25, 2017 at 3:11 PM, Tom Lane wrote: >>> Please. You might want to hit the existing ones with a separate patch, >>> but it

Re: [HACKERS] Re: PATCH: pageinspect / add page_checksum and bt_page_items(bytea)

2017-04-04 Thread David Steele
On 4/4/17 9:43 AM, Robert Haas wrote: > On Tue, Apr 4, 2017 at 9:32 AM, David Steele <da...@pgmasters.net> wrote: >> My goal is to help people focus on patches that have a chance. At this >> point I think that includes poking authors who are not being responsive >> us

Re: [HACKERS] Re: PATCH: pageinspect / add page_checksum and bt_page_items(bytea)

2017-04-04 Thread David Steele
On 4/4/17 9:11 AM, Simon Riggs wrote: > On 4 April 2017 at 09:05, David Steele <da...@pgmasters.net> wrote: >> Hi Tomas, >> >> On 4/1/17 5:40 AM, Ashutosh Sharma wrote: >>> >>> Apart from above comments, your patch looks good to me. I have also

Re: [HACKERS] New CORRESPONDING clause design

2017-04-04 Thread David Steele
On 4/1/17 1:54 PM, Tom Lane wrote: > > I'll set this back to Waiting on Author, but I think the chances of > getting to a committable patch before the end of the commitfest are > about nil. I think this is especially true now that another three days have passed. This submission has been marked

[HACKERS] Re: PATCH: pageinspect / add page_checksum and bt_page_items(bytea)

2017-04-04 Thread David Steele
Hi Tomas, On 4/1/17 5:40 AM, Ashutosh Sharma wrote: > > Apart from above comments, your patch looks good to me. I have also > marked this patch as 'Waiting for Author' in the commitfest. Thanks. The CF has been extended until April 7 but time is still growing short and this thread has been idle

Re: [HACKERS] Should we cacheline align PGXACT?

2017-04-04 Thread David Steele
On 4/4/17 8:55 AM, Alexander Korotkov wrote: > On Mon, Apr 3, 2017 at 9:58 PM, Andres Freund > I'm inclined to push this to the next CF, it seems we need a lot more > benchmarking here. > > > No objections. This submission has been moved to CF 2017-07. Thanks, --

Re: [HACKERS] Adding support for Default partition in partitioning

2017-04-04 Thread David Steele
On 3/31/17 10:45 AM, David Steele wrote: > On 3/29/17 8:13 AM, Rahila Syed wrote: > >> Thanks for reporting. I have identified the problem and have a fix. >> Currently working on allowing >> adding a partition after default partition if the default partition does >>

Re: [HACKERS] postgres_fdw: support parameterized foreign joins

2017-03-31 Thread David Steele
Hi Arthur. On 3/23/17 8:45 AM, Etsuro Fujita wrote: On 2017/03/21 18:38, Etsuro Fujita wrote: On 2017/03/16 22:23, Arthur Zakirov wrote: Can you rebase the patch? It is not applied now. Ok, will do. Thanks for the report! Done. Also, I added regression tests and revised code and

Re: [HACKERS] [PATCH] Transaction traceability - txid_status(bigint)

2017-03-31 Thread David Steele
On 3/31/17 10:46 AM, Craig Ringer wrote: Patches 1 and 2 were the key parts and thanks to Robert's helpful review, advice and edits they're committed now. Committed, done. Yay. Excellent. I have marked this a "Committed" by Robert. One down... -- -David da...@pgmasters.net -- Sent via

Re: [HACKERS] PATCH: Batch/pipelining support for libpq

2017-03-31 Thread David Steele
Hi, On 3/30/17 2:12 PM, Daniel Verite wrote: Vaishnavi Prabakaran wrote: Hmm, With batch mode, after sending COPY command to server(and server started processing the query and goes into COPY state) , client does not immediately read the result , but it keeps sending other queries to

[HACKERS] Re: PATCH: pageinspect / add page_checksum and bt_page_items(bytea)

2017-03-31 Thread David Steele
On 3/29/17 11:08 AM, Tomas Vondra wrote: The attached patch is the best I came up with - it essentially shares just the tuple-forming part, which is exactly the same in both cases. I have marked this submission "Needs Review". -- -David da...@pgmasters.net -- Sent via pgsql-hackers mailing

Re: [HACKERS] Adding support for Default partition in partitioning

2017-03-31 Thread David Steele
On 3/29/17 8:13 AM, Rahila Syed wrote: Thanks for reporting. I have identified the problem and have a fix. Currently working on allowing adding a partition after default partition if the default partition does not have any conflicting rows. Will update the patch with both of these. The CF has

Re: [HACKERS] BRIN cost estimate

2017-03-31 Thread David Steele
On 3/26/17 7:44 AM, Emre Hasegeli wrote: If we want to have a variable which stores the number of ranges, then I think numRanges is better than numBlocks. I can't imagine many people would disagree there. I renamed it with other two. At the very least please write a comment to explain this

Re: [HACKERS] [PATCH] Transaction traceability - txid_status(bigint)

2017-03-31 Thread David Steele
On 3/25/17 12:12 AM, Peter Eisentraut wrote: I'm wondering if this is a perl version/platform issue around $tx->pump until $stdout =~ /[[:digit:]]+[\r\n]$/; where we're not recognising the required output from psql when we get it. What's in src/test/recovery/tmp_check/log/regress_log_011*

Re: [HACKERS] GUC for cleanup indexes threshold.

2017-03-31 Thread David Steele
On 3/29/17 2:23 AM, Masahiko Sawada wrote: On Wed, Mar 29, 2017 at 12:23 AM, David Steele <da...@pgmasters.net> wrote: On 3/23/17 1:54 AM, Masahiko Sawada wrote: On Wed, Mar 15, 2017 at 7:51 AM, Peter Geoghegan <p...@bowt.ie> wrote: On Tue, Mar 14, 2017 at 3:10 PM, Peter

Re: [HACKERS] Supporting huge pages on Windows

2017-03-28 Thread David Steele
On 3/24/17 12:51 PM, Ashutosh Sharma wrote: Hi, On Fri, Mar 24, 2017 at 9:00 PM, David Steele <da...@pgmasters.net> wrote: Hi Ashutosh, On 3/22/17 8:52 AM, Amit Kapila wrote: On Wed, Mar 22, 2017 at 12:07 AM, David Steele <da...@pgmasters.net> wrote: Amit, Magnus, you a

Re: [HACKERS] WIP: [[Parallel] Shared] Hash

2017-03-28 Thread David Steele
Hi Thomas, On 3/28/17 1:41 AM, Rafia Sabih wrote: On Mon, Mar 27, 2017 at 12:20 PM, Thomas Munro I thought this last point about Windows might be fatal to my design, but it seems that Windows since at least version 2000 has support for Unixoid unlinkability via the special flag

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

2017-03-28 Thread David Steele
Hi Pavan, On 3/28/17 11:04 AM, Robert Haas wrote: On Mon, Mar 27, 2017 at 10:25 PM, Pavan Deolasee wrote: On Tue, Mar 28, 2017 at 1:59 AM, Robert Haas wrote: On Thu, Mar 23, 2017 at 2:47 PM, Pavan Deolasee wrote:

Re: [HACKERS] GUC for cleanup indexes threshold.

2017-03-28 Thread David Steele
On 3/23/17 1:54 AM, Masahiko Sawada wrote: On Wed, Mar 15, 2017 at 7:51 AM, Peter Geoghegan wrote: On Tue, Mar 14, 2017 at 3:10 PM, Peter Geoghegan wrote: We already have BTPageOpaqueData.btpo, a union whose contained type varies based on the page being dead. We

[HACKERS] Re: PATCH: pageinspect / add page_checksum and bt_page_items(bytea)

2017-03-28 Thread David Steele
On 3/23/17 11:27 PM, Peter Eisentraut wrote: On 3/17/17 18:35, Tomas Vondra wrote: On 03/17/2017 05:23 PM, Peter Eisentraut wrote: I'm struggling to find a good way to share code between bt_page_items(text, int4) and bt_page_items(bytea). If we do it via the SQL route, as I had suggested, it

Re: [HACKERS] [WIP] RE: DECLARE STATEMENT setting up a connection in ECPG

2017-03-28 Thread David Steele
Hi Takeshi, On 3/23/17 1:33 AM, Haribabu Kommi wrote: The test patch looks good to me. This thread has been idle for five days. Please respond with a new patch by 2017-03-30 00:00 AoE (UTC-12) or this submission will be marked "Returned with Feedback". -- -David da...@pgmasters.net --

Re: [HACKERS] [PATCH] Incremental sort

2017-03-28 Thread David Steele
Hi Alexander, On 3/20/17 10:19 AM, Heikki Linnakangas wrote: On 03/20/2017 11:33 AM, Alexander Korotkov wrote: Please, find rebased patch in the attachment. I had a quick look at this. <...> According to 'perf', 85% of the CPU time is spent in ExecCopySlot(). To alleviate that, it might

Re: [HACKERS] Index usage for elem-contained-by-const-range clauses

2017-03-28 Thread David Steele
On 3/24/17 10:50 AM, David Steele wrote: Hi Pritam, On 3/17/17 5:41 PM, Pritam Baral wrote: So sorry. I'm attaching the correct version of the original with this, in case you want to test the limited implementation, because I still have to go through Tom's list of suggestions. BTW, the patch

Re: [HACKERS] patch proposal

2017-03-27 Thread David Steele
On 3/26/17 7:34 PM, Venkata B Nagothi wrote: > Hi David, > > On Thu, Mar 23, 2017 at 4:21 AM, David Steele <da...@pgmasters.net > <mailto:da...@pgmasters.net>> wrote: > > On 3/21/17 8:45 PM, Venkata B Nagothi wrote: > > On Tue, Mar 21, 2017 at

Re: [HACKERS] increasing the default WAL segment size

2017-03-24 Thread David Steele
Hi Robert, On 3/24/17 3:00 PM, Robert Haas wrote: On Wed, Mar 22, 2017 at 6:05 PM, David Steele <da...@pgmasters.net> wrote: Wait, really? I thought you abandoned this approach because there's then no principled way to handle WAL segments of less than the default size. I did say that,

Re: [HACKERS] Supporting huge pages on Windows

2017-03-24 Thread David Steele
Hi Ashutosh, On 3/22/17 8:52 AM, Amit Kapila wrote: On Wed, Mar 22, 2017 at 12:07 AM, David Steele <da...@pgmasters.net> wrote: Amit, Magnus, you are signed up as reviewers for this patch. Do you know when you'll have a chance to take a look? I have provided my feedback and I

Re: [HACKERS] cast result of copyNode()

2017-03-24 Thread David Steele
On 3/21/17 6:52 PM, Mark Dilger wrote: On Mar 21, 2017, at 2:13 PM, David Steele <da...@pgmasters.net> wrote: Hi Mark, On 3/9/17 3:34 PM, Peter Eisentraut wrote: On 3/7/17 18:27, Mark Dilger wrote: You appear to be using a #define macro to wrap a function of the same name with th

Re: [HACKERS] PATCH: Batch/pipelining support for libpq

2017-03-24 Thread David Steele
Hi Vaishnavi, On 3/19/17 9:32 PM, Vaishnavi Prabakaran wrote: On Fri, Mar 17, 2017 at 12:37 AM, Daniel Verite I would also like to hear Craig's

Re: [HACKERS] LWLock optimization for multicore Power machines

2017-03-24 Thread David Steele
Hi Alexander, On 3/16/17 1:35 PM, David Steele wrote: On 2/21/17 9:54 AM, Bernd Helmle wrote: Am Dienstag, den 14.02.2017, 15:53 +0300 schrieb Alexander Korotkov: +1 And you could try to use pg_wait_sampling <https://github.com/postgrespro/pg_wait_sampling> to sampling of wait events.

Re: [HACKERS] pg_stat_wal_write statistics view

2017-03-24 Thread David Steele
On 3/16/17 1:54 AM, vinayak wrote: On 2017/03/16 14:46, Haribabu Kommi wrote: As the view name already contains WAL, I am not sure whether is it required to include WAL in every column? I am fine to change if others have the same opinion of adding WAL to column names. Ok. So what is the

Re: [HACKERS] Index usage for elem-contained-by-const-range clauses

2017-03-24 Thread David Steele
Hi Pritam, On 3/17/17 5:41 PM, Pritam Baral wrote: So sorry. I'm attaching the correct version of the original with this, in case you want to test the limited implementation, because I still have to go through Tom's list of suggestions. BTW, the patch is for applying on top of REL9_6_2, and

Re: [HACKERS] [PATCH] Generic type subscripting

2017-03-24 Thread David Steele
Hi Dmitry, On 3/21/17 4:42 PM, Dmitry Dolgov wrote: On 21 March 2017 at 18:16, David Steele <da...@pgmasters.net <mailto:da...@pgmasters.net>> wrote: This thread has been idle for over a week. Yes, sorry for the late reply. I'm still trying to find a better solut

Re: [HACKERS] PassDownLimitBound for ForeignScan/CustomScan [take-2]

2017-03-24 Thread David Steele
Hi Kaigai, On 3/21/17 1:11 PM, David Steele wrote: On 3/13/17 3:25 AM, Jeevan Chalke wrote: I have reviewed this patch further and here are my comments: This thread has been idle for over a week. Please respond and/or post a new patch by 2017-03-24 00:00 AoE (UTC-12) or this submission

Re: [HACKERS] make async slave to wait for lsn to be replayed

2017-03-24 Thread David Steele
Hi Ivan, On 3/21/17 1:06 PM, David Steele wrote: Hi Ivan, On 3/12/17 10:20 PM, Thomas Munro wrote: On Fri, Mar 10, 2017 at 1:49 AM, Ivan Kartyshov <i.kartys...@postgrespro.ru> wrote: Here I attached rebased patch waitlsn_10dev_v3 (core feature) I will leave the choice of implementation

Re: [HACKERS] [PATCH] kNN for SP-GiST

2017-03-24 Thread David Steele
On 3/21/17 11:45 AM, David Steele wrote: Hi Nikita, On 3/9/17 8:52 AM, Alexander Korotkov wrote: I take a look to this patchset. My first notes are following. This thread has been idle for quite a while. Please respond and/or post a new patch by 2017-03-24 00:00 AoE (UTC-12

Re: [HACKERS] Adding the optional clause 'AS' in CREATE TRIGGER

2017-03-24 Thread David Steele
Hi Naoki, On 3/17/17 11:58 AM, Surafel Temesgen wrote: I am sending the review of this patch This thread has been idle for a week. Please respond and/or post a new patch by 2017-03-28 00:00 AoE (UTC-12) or this submission will be marked "Returned with Feedback". -- -David

Re: [HACKERS] increasing the default WAL segment size

2017-03-24 Thread David Steele
On 3/23/17 4:45 PM, Peter Eisentraut wrote: On 3/22/17 17:33, David Steele wrote: I think if we don't change the default size it's very unlikely I would use alternate WAL segment sizes or recommend that anyone else does, at least in v10. I simply don't think it would get the level of testing

Re: [HACKERS] increasing the default WAL segment size

2017-03-24 Thread David Steele
On 3/24/17 12:27 AM, Peter Eisentraut wrote: On 3/23/17 16:58, Stephen Frost wrote: The backup tools need to also get the LSN from the pg_stop_backup and verify that they have the WAL file associated with that LSN. There is a function for that. They also need to make sure that they have all

Re: [HACKERS] increasing the default WAL segment size

2017-03-22 Thread David Steele
Hi Robert, On 3/22/17 3:45 PM, Robert Haas wrote: On Wed, Mar 22, 2017 at 3:24 PM, David Steele <da...@pgmasters.net> wrote: One of the reasons to go with the LSN is that we would actually be maintaining what happens when the WAL files are 16MB in size. David's initial expec

Re: [HACKERS] PATCH: Make pg_stop_backup() archive wait optional

2017-03-22 Thread David Steele
On 3/22/17 4:42 PM, Peter Eisentraut wrote: On 3/22/17 15:14, Stephen Frost wrote: -SELECT * FROM pg_stop_backup(false); +SELECT * FROM pg_stop_backup(false [, true ]); I think that it's better to get rid of "[" and "]" from the above because IMO this should be the command example that users

Re: [HACKERS] increasing the default WAL segment size

2017-03-22 Thread David Steele
On 3/22/17 3:39 PM, Peter Eisentraut wrote: On 3/22/17 15:37, Peter Eisentraut wrote: If changing WAL sizes catches on, I do think we should keep thinking about a new format for a future release, I think that means that I'm skeptical about changing the default size right now. I think if we

Re: [HACKERS] increasing the default WAL segment size

2017-03-22 Thread David Steele
On 3/22/17 3:09 PM, Stephen Frost wrote: * Robert Haas (robertmh...@gmail.com) wrote: On Wed, Mar 22, 2017 at 1:49 PM, Stephen Frost wrote: Then perhaps we do need to be thinking of moving this to PG11 instead of exposing an option that users will start to use which will

Re: [HACKERS] patch proposal

2017-03-22 Thread David Steele
On 3/21/17 8:45 PM, Venkata B Nagothi wrote: On Tue, Mar 21, 2017 at 8:46 AM, David Steele <da...@pgmasters.net Unfortunately, I don't think the first patch (recoveryStartPoint) will work as currently implemented. The problem I see is that the new function recoveryStartsH

Re: [HACKERS] PATCH: Make pg_stop_backup() archive wait optional

2017-03-21 Thread David Steele
On 3/21/17 2:34 PM, Fujii Masao wrote: On Tue, Mar 21, 2017 at 11:03 AM, Tsunakawa, Takayuki <tsunakawa.ta...@jp.fujitsu.com> wrote: From: David Steele [mailto:da...@pgmasters.net] Well, that's embarrassing. When I recreated the function to add defaults I messed up the AS clause a

[HACKERS] Re: proposal - psql: possibility to specify sort for describe commands, when size is printed

2017-03-21 Thread David Steele
Hi Alexander, On 3/11/17 7:06 AM, Pavel Stehule wrote: I am sending a updated version with separated sort direction in special variable There is a question. Has desc direction sense for columns like schema or table name? Using desc, asc for size is natural. But for tablename? Do you know

Re: [HACKERS] increasing the default WAL segment size

2017-03-21 Thread David Steele
On 3/21/17 3:22 PM, Robert Haas wrote: On Tue, Mar 21, 2017 at 9:04 AM, Stephen Frost wrote: In short, I'm also concerned about this change to make WAL file names no longer match up with LSNs and also about the odd stepping that you get as a result of this change when it

Re: [HACKERS] PATCH: recursive json_populate_record()

2017-03-21 Thread David Steele
On 3/21/17 2:31 PM, Andrew Dunstan wrote: On 03/21/2017 01:37 PM, David Steele wrote: >> This thread has been idle for months since Tom's review. The submission has been marked "Returned with Feedback". Please feel free to resubmit to a future commitfest. Please reviv

Re: [HACKERS] Should we cacheline align PGXACT?

2017-03-21 Thread David Steele
Hi Alexander On 3/10/17 8:08 AM, Alexander Korotkov wrote: Results look good for me. Idea of committing both of patches looks attractive. We have pretty much acceleration for read-only case and small acceleration for read-write case. I'll run benchmark on 72-cores machine as well. Have you

  1   2   3   4   5   6   >