Re: [HACKERS] Measuring replay lag

On 1 March 2017 at 10:47, Thomas Munro wrote: >>> I added a fourth case 'overwhelm.png' which you might find >>> interesting. It's essentially like one 'burst' followed by a 100% ide >>> primary. The primary stops sending new WAL around 50 seconds in and >>> then

Re: [HACKERS] Allow pg_dumpall to work without pg_authid

On 5 March 2017 at 18:00, Simon Riggs wrote: > I'm looking to commit the patch version I posted, so I would like your > comments that it does continue to solve the problems you raised. > ​Thanks Simon, for confirming. Yes, the updated patch does solve the problem.​ -

Re: [HACKERS] Measuring replay lag

On 1 March 2017 at 10:47, Thomas Munro wrote: > On Fri, Feb 24, 2017 at 9:05 AM, Simon Riggs wrote: >> On 21 February 2017 at 21:38, Thomas Munro >> wrote: >>> However, I think a call like

Re: [HACKERS] dropping partitioned tables without CASCADE

On 27 February 2017 at 02:38, Amit Langote wrote: > On 2017/02/26 5:30, Simon Riggs wrote: >> On 23 February 2017 at 16:33, Simon Riggs wrote: >> >>> I'll be happy to review >> >> Patch looks OK so far, but fails on a partition that has

Re: [HACKERS] Block level parallel vacuum WIP

On Sun, Mar 5, 2017 at 12:14 PM, David Steele wrote: > On 3/4/17 9:08 PM, Masahiko Sawada wrote: >> On Sat, Mar 4, 2017 at 5:47 PM, Robert Haas wrote: >>> On Fri, Mar 3, 2017 at 9:50 PM, Masahiko Sawada >>> wrote: Yes,

Re: [HACKERS] Allow pg_dumpall to work without pg_authid

On 28 February 2017 at 17:49, Simon Riggs wrote: > I've edited the stated reason for the patch on the CF app, so its > clearer as to why this might be acceptable. Robins, I'm looking to commit the patch version I posted, so I would like your comments that it does

Re: [HACKERS] Block level parallel vacuum WIP

On 3/4/17 9:08 PM, Masahiko Sawada wrote: > On Sat, Mar 4, 2017 at 5:47 PM, Robert Haas wrote: >> On Fri, Mar 3, 2017 at 9:50 PM, Masahiko Sawada >> wrote: >>> Yes, it's taking a time to update logic and measurement but it's >>> coming along. Also

Re: [HACKERS] DROP SUBSCRIPTION and ROLLBACK

On Sat, Mar 4, 2017 at 1:46 PM, Peter Eisentraut wrote: > On 3/3/17 13:58, Petr Jelinek wrote: >> On 23/02/17 08:24, Masahiko Sawada wrote: >>> Attached updated version patches. Please review these. >>> >> >> This version looks good to me, I'd only change the >>

Re: [HACKERS] Block level parallel vacuum WIP

On Sat, Mar 4, 2017 at 5:47 PM, Robert Haas wrote: > On Fri, Mar 3, 2017 at 9:50 PM, Masahiko Sawada wrote: >> Yes, it's taking a time to update logic and measurement but it's >> coming along. Also I'm working on changing deadlock detection. Will >>

Re: [HACKERS] mat views stats

On Wed, Mar 1, 2017 at 8:39 PM, Michael Paquier wrote: > On Thu, Mar 2, 2017 at 7:20 AM, Jim Mlodgenski wrote: > > > > > > On Sun, Feb 26, 2017 at 11:49 AM, Robert Haas > wrote: > >> > >> On Wed, Feb 22, 2017 at 11:13 AM, Jim

Re: [HACKERS] Cost model for parallel CREATE INDEX

On Sat, Mar 4, 2017 at 6:00 AM, Stephen Frost wrote: >> It is, but I was using that with index size, not table size. I can >> change it to be table size, based on what you said. But the workMem >> related cap, which probably won't end up being applied all that often >> in

Re: [HACKERS] Re: check failure with -DRELCACHE_FORCE_RELEASE -DCLOBBER_FREED_MEMORY

I wrote: > Andrew Dunstan writes: >> On 03/03/2017 02:24 PM, Andrew Dunstan wrote: >>> I have been setting up a buildfarm member with -DRELCACHE_FORCE_RELEASE >>> -DCLOBBER_FREED_MEMORY, settings which Alvaro suggested to me.I got core >>> dumps with these stack

Re: [HACKERS] Re: check failure with -DRELCACHE_FORCE_RELEASE -DCLOBBER_FREED_MEMORY

Andrew Dunstan writes: > On 03/03/2017 02:24 PM, Andrew Dunstan wrote: >> I have been setting up a buildfarm member with -DRELCACHE_FORCE_RELEASE >> -DCLOBBER_FREED_MEMORY, settings which Alvaro suggested to me.I got core >> dumps with these stack traces. The

Re: [HACKERS] [PATCH] Use $ parameters as replacement characters for pg_stat_statements

Hi, On 2017-03-04 11:02:14 -0500, Tom Lane wrote: > But speaking of ambiguity: isn't it possible for $n symbols to appear in > pg_stat_statements already? Indeed. > I think it is, both from extended-protocol > client queries and from SPI commands, which would mean that the proposal > as it

Re: [HACKERS] [PATCH] Use $ parameters as replacement characters for pg_stat_statements

On Sat, Mar 4, 2017 at 8:02 AM, Tom Lane wrote: >> Perhaps there could be a choice of behaviors. Even if we all agreed >> that parameter notation was better in theory, there's something to be >> said for maintaining backward compatibility, or having an option to do >> so. > >

Re: [HACKERS] PATCH: pageinspect / add page_checksum and bt_page_items(bytea)

On 3/4/17 12:39, Tomas Vondra wrote: >> Can we have a test case for page_checksum(), or is that too difficult to >> get running deterministicly? > > I'm not sure it can be made deterministic. Certainly not on heap pages, > because then it'd be susceptible to xmin/xmax changes, but we have other

Re: [HACKERS] Change in "policy" on dump ordering?

On 3/1/17 08:36, Peter Eisentraut wrote: > On 2/22/17 18:24, Jim Nasby wrote: >>> Yes, by that logic matview refresh should always be last. >> >> Patches for head attached. >> >> RLS was the first item added after DO_REFRESH_MATVIEW, which was added >> in 9.5. So if we want to treat this as a

Re: [HACKERS] PATCH: pageinspect / add page_checksum and bt_page_items(bytea)

On 03/04/2017 02:08 PM, Peter Eisentraut wrote: On 3/3/17 09:03, Tomas Vondra wrote: Damn. In my defense, the patch was originally created for an older PostgreSQL version (to investigate issue on a production system), which used that approach to building values. Should have notice it, though.

Re: [HACKERS] REINDEX CONCURRENTLY 2.0

On March 4, 2017 1:16:56 AM PST, Robert Haas wrote: >Maybe. But it looks to me like this patch is going to have >considerably more than its share of user-visible warts, and I'm not >very excited about that. I feel like what we ought to be doing is >keeping the index OID

[HACKERS] Parallel seq. plan is not coming against inheritance or partition table

Hi All, >From following git commit onwards, parallel seq scan is never getting selected for inheritance or partitioned tables. commit 51ee6f3160d2e1515ed6197594bda67eb99dc2cc Author: Robert Haas Date: Wed Feb 15 13:37:24 2017 -0500 Replace

Re: [HACKERS] GSoC 2017

Robert Haas writes: > On Thu, Mar 2, 2017 at 3:45 AM, Jim Nasby wrote: >> On 2/27/17 4:52 PM, Thomas Munro wrote: >>> By the way, that page claims that PostgreSQL runs on Irix and Tru64, >>> which hasn't been true for a few years. >> There could

Re: [HACKERS] [PATCH] Use $ parameters as replacement characters for pg_stat_statements

Robert Haas writes: > On Wed, Mar 1, 2017 at 8:21 PM, Peter Eisentraut > wrote: >> On 2/28/17 20:01, Lukas Fittl wrote: >>> I'd like to propose changing the replacement character from ? to instead >>> be a parameter (like $1). >> Hmm, I

Re: [HACKERS] Patch to implement pg_current_logfile() function

"Karl O. Pinc" writes: > On Sat, 4 Mar 2017 14:26:54 +0530 > Robert Haas wrote: >> On Fri, Mar 3, 2017 at 7:21 PM, Karl O. Pinc wrote: >>> But if the code does not exit the loop then >>> before calling elog(ERROR), shouldn't it >>> also call

Re: [HACKERS] Patch to implement pg_current_logfile() function

On Sat, 4 Mar 2017 14:26:54 +0530 Robert Haas wrote: > On Fri, Mar 3, 2017 at 7:21 PM, Karl O. Pinc wrote: > > But if the code does not exit the loop then > > before calling elog(ERROR), shouldn't it > > also call FreeFile(fd)? > > Hmm. Normally error

Re: [HACKERS] wait events for disk I/O

On Mon, Feb 20, 2017 at 4:04 PM, Rushabh Lathia wrote: > > My colleague Rahila reported compilation issue with > the patch. Issue was only coming with we do the clean > build on the branch. > > Fixed the same into latest version of patch. > Few assorted comments: 1. +

[HACKERS] Re: check failure with -DRELCACHE_FORCE_RELEASE -DCLOBBER_FREED_MEMORY

On 03/03/2017 02:24 PM, Andrew Dunstan wrote: > I have been setting up a buildfarm member with -DRELCACHE_FORCE_RELEASE > -DCLOBBER_FREED_MEMORY, settings which Alvaro suggested to me.I got core > dumps with these stack traces. The platform is Amazon Linux. > I have replicated this on a couple

Re: [HACKERS] PATCH: Make pg_stop_backup() archive wait optional

Hi Robert, On 3/4/17 1:58 AM, Robert Haas wrote: > On Wed, Mar 1, 2017 at 9:07 AM, David Steele wrote: >> On 2/28/17 10:22 PM, Robert Haas wrote: >>> On Tue, Feb 28, 2017 at 6:22 AM, David Steele wrote: >> I'm not sure that's the case. It seems

Re: [HACKERS] Cost model for parallel CREATE INDEX

Peter, * Peter Geoghegan (p...@bowt.ie) wrote: > On Sat, Mar 4, 2017 at 12:50 AM, Robert Haas wrote: > > If the result of > > compute_parallel_workers() based on min_parallel_table_scan_size is > > smaller, then use that value instead. I must be confused, because I > >

Re: [HACKERS] [FEATURE PATCH] pg_stat_statements with plans

Alright, for the next version of this patch I'll look into standard deviation (an implementation of Welfords' algorithm already exists in pg_stat_statements). On 3/4/17 14:18, Peter Eisentraut wrote: The other problem is that this measures execution time, which can vary for reasons other

Re: [HACKERS] [FEATURE PATCH] pg_stat_statements with plans

Hi Julian, On 3/4/17 8:41 AM, Julian Markwort wrote: >> Any improvements would be greatly welcome by the admin >> community, I'm sure. > That's good to hear - on the other hand, I really appreciate the opinion > of admins on this idea! >> However, this is a rather large change which could be

Re: [HACKERS] PATCH: pageinspect / add page_checksum and bt_page_items(bytea)

On 3/3/17 09:03, Tomas Vondra wrote: > Damn. In my defense, the patch was originally created for an older > PostgreSQL version (to investigate issue on a production system), which > used that approach to building values. Should have notice it, though. > > Attached is v2, fixing both issues.

Re: [HACKERS] [FEATURE PATCH] pg_stat_statements with plans

Any improvements would be greatly welcome by the admin community, I'm sure. That's good to hear - on the other hand, I really appreciate the opinion of admins on this idea! However, this is a rather large change which could be destabilizing to the many users of this extension. I'm fully aware

Re: [HACKERS] [WIP]Vertical Clustered Index (columnar store extension)

On 3/4/17 8:33 AM, Peter Eisentraut wrote: > On 3/3/17 16:16, David Steele wrote: >> While this looks like it could be a really significant performance >> improvement, I think the above demonstrates that it needs a lot of work. >> I know this is not new to the 2017-03 CF but it doesn't seem

Re: [HACKERS] [WIP]Vertical Clustered Index (columnar store extension)

On 3/3/17 16:16, David Steele wrote: > While this looks like it could be a really significant performance > improvement, I think the above demonstrates that it needs a lot of work. > I know this is not new to the 2017-03 CF but it doesn't seem enough > progress has been made since posting to

Re: [HACKERS] [FEATURE PATCH] pg_stat_statements with plans

On 1/25/17 12:43, Simon Riggs wrote: > On 25 January 2017 at 17:34, Julian Markwort > wrote: > >> Analogous to this, a bad_plan is saved, when the time has been exceeded by a >> factor greater than 1.1 . > ...and the plan differs? > > Probably best to use some

Re: [HACKERS] pg_dump, pg_dumpall and data durability

On Sat, Mar 4, 2017 at 3:08 PM, Robert Haas wrote: > On Thu, Mar 2, 2017 at 5:02 AM, Michael Paquier > wrote: >> On Thu, Mar 2, 2017 at 2:26 AM, David Steele wrote: >>> This patch is in need of a committer. Any takers? >>>

Re: [HACKERS] WAL Consistency checking for hash indexes

On Sat, Mar 4, 2017 at 2:29 PM, Robert Haas wrote: > On Fri, Mar 3, 2017 at 9:44 AM, Amit Kapila wrote: >> On Tue, Feb 28, 2017 at 11:06 AM, Kuntal Ghosh >> wrote: >>> Hello everyone, >>> >>> I've attached a patch which

Re: [HACKERS] GUC for cleanup indexes threshold.

On Sat, Mar 4, 2017 at 8:55 AM, Peter Geoghegan wrote: > On Fri, Mar 3, 2017 at 6:58 PM, Amit Kapila wrote: >> You are right that we don't want the number of unclaimed-by-FSM >> recyclable pages to grow forever, but I think that won't happen with >> this

Re: [HACKERS] Print correct startup cost for the group aggregate.

On Thu, Mar 2, 2017 at 6:48 PM, Ashutosh Bapat wrote: > On Thu, Mar 2, 2017 at 6:06 PM, Rushabh Lathia > wrote: >> While reading through the cost_agg() I found that startup cost for the >> group aggregate is not correctly assigned. Due

Re: [HACKERS] REINDEX CONCURRENTLY 2.0

On Thu, Mar 2, 2017 at 11:48 AM, Andres Freund wrote: > On 2017-03-01 19:25:23 -0600, Jim Nasby wrote: >> On 2/28/17 11:21 AM, Andreas Karlsson wrote: >> > The only downside I can see to this approach is that we no logner will >> > able to reindex catalog tables concurrently,

Re: [HACKERS] WAL Consistency checking for hash indexes

On Fri, Mar 3, 2017 at 9:44 AM, Amit Kapila wrote: > On Tue, Feb 28, 2017 at 11:06 AM, Kuntal Ghosh > wrote: >> Hello everyone, >> >> I've attached a patch which implements WAL consistency checking for >> hash indexes. This feature is going to

Re: [HACKERS] Cost model for parallel CREATE INDEX

On Sat, Mar 4, 2017 at 12:50 AM, Robert Haas wrote: > If you think parallelism isn't worthwhile unless the sort was going to > be external anyway, I don't -- that's just when it starts to look like a safe bet that parallelism is worthwhile. There are quite a few cases

Re: [HACKERS] Patch to implement pg_current_logfile() function

On Fri, Mar 3, 2017 at 7:21 PM, Karl O. Pinc wrote: > But if the code does not exit the loop then > before calling elog(ERROR), shouldn't it > also call FreeFile(fd)? Hmm. Normally error recovery cleans up opened files, but since logfile_open() directly calls fopen(), that's not

Re: [HACKERS] Patch to implement pg_current_logfile() function

On Fri, Mar 3, 2017 at 11:54 AM, Michael Paquier wrote: > On Fri, Mar 3, 2017 at 3:18 PM, Robert Haas wrote: >> Hopefully I haven't broken anything; please let me know if you >> encounter any issues. > > Reading what has just been committed... >

Re: [HACKERS] Cost model for parallel CREATE INDEX

On Sat, Mar 4, 2017 at 2:17 PM, Peter Geoghegan wrote: > On Sat, Mar 4, 2017 at 12:43 AM, Robert Haas wrote: >> Oh. But then I don't see why you need min_parallel_anything. That's >> just based on an estimate of the amount of data per worker vs. >>

Re: [HACKERS] Block level parallel vacuum WIP

On Fri, Mar 3, 2017 at 9:50 PM, Masahiko Sawada wrote: > Yes, it's taking a time to update logic and measurement but it's > coming along. Also I'm working on changing deadlock detection. Will > post new patch and measurement result. I think that we should push this patch

Re: [HACKERS] Cost model for parallel CREATE INDEX

On Sat, Mar 4, 2017 at 12:43 AM, Robert Haas wrote: > Oh. But then I don't see why you need min_parallel_anything. That's > just based on an estimate of the amount of data per worker vs. > maintenance_work_mem, isn't it? Yes -- and it's generally a pretty good estimate.

Re: [HACKERS] Cost model for parallel CREATE INDEX

On Sat, Mar 4, 2017 at 2:01 PM, Peter Geoghegan wrote: > On Sat, Mar 4, 2017 at 12:23 AM, Robert Haas wrote: >>> I guess that the workMem scaling threshold thing could be >>> min_parallel_index_scan_size, rather than min_parallel_relation_size >>> (which we

Re: [HACKERS] Cost model for parallel CREATE INDEX

On Sat, Mar 4, 2017 at 12:23 AM, Robert Haas wrote: >> I guess that the workMem scaling threshold thing could be >> min_parallel_index_scan_size, rather than min_parallel_relation_size >> (which we now call min_parallel_table_scan_size)? > > No, it should be based on

Re: [HACKERS] I propose killing PL/Tcl's "modules" infrastructure

On Thu, Mar 2, 2017 at 11:29 PM, Tom Lane wrote: > After thinking about that for awhile, it seemed like the most useful thing > to do is to set up an errcontext callback that will be active throughout > execution of the start_proc. That will cover both setup failures like >

Re: [HACKERS] Cost model for parallel CREATE INDEX

On Thu, Mar 2, 2017 at 10:38 PM, Peter Geoghegan wrote: > I'm glad. This justifies the lack of much of any "veto" on the > logarithmic scaling. The only thing that can do that is > max_parallel_workers_maintenance, the storage parameter > parallel_workers (maybe this isn't a storage