Re: [bug fix] Cascaded standby cannot start after a clean shutdown

On Mon, Feb 19, 2018 at 03:01:15AM +, Tsunakawa, Takayuki wrote: > From: Michael Paquier [mailto:mich...@paquier.xyz] Sorry for my late reply. I was looking at this problem for the last couple of days here and there, still thinking about it. >> It seems to me that the consolidation of the pa

Re: PATCH: pgbench - break out timing data for initialization phases

Hello Doug, Doing the "in progress" way suffers from everything before 'generating data' possibly scrolling off the screen/window. Yeah, that is a point. I tend to "| less" when I want to see a long output in details, so it is not an issue for me. Also, I like to have an information when

Re: SHA-2 functions

On Wed, Feb 21, 2018 at 03:45:17PM -0500, Peter Eisentraut wrote: > On 2/20/18 23:04, Michael Paquier wrote: >> I think that crypto_hash.c or hash_crypt.c would be adapted as well. >> crypt.c is too much generic, so including both concepts in the name is >> the way to go. The name given by Tom her

Re: ALTER TABLE ADD COLUMN fast default

On Wed, Feb 21, 2018 at 7:48 PM, Andres Freund wrote: > Hi, > [ Long and useful review] > This doesn't seem ready yet. > Thanks. I'm working through the issues you raised. Will reply in a few days. cheers andrew -- Andrew Dunstanhttps://www.2ndQuadrant.com PostgreSQL Devel

Re: [HACKERS] Constifying numeric.c's local vars

Mark Dilger writes: >> This means that the const variable 'const_zero' contains a pointer that is >> non-const, pointing at something that is static const, stored in read only >> memory. Yikes. > I still believe this is unsafe. I'm with Andres: I don't see the problem. It's true that we've cas

Re: [HACKERS] Partition-wise aggregation/grouping

On Thu, Feb 8, 2018 at 8:05 AM, Jeevan Chalke wrote: > 0003 - 0006 are refactoring patches as before. I have committed 0006 with some modifications. In particular, [1] I revised the comments and formatting; [2] I made cost_merge_append() add cpu_tuple_cost * APPEND_CPU_COST_MULTIPLIER in lieu of

Re: [bug fix] Produce a crash dump before main() on Windows

On 20 February 2018 at 23:43, Magnus Hagander wrote: > > I've seen a number of usecases where apps start it alongside the app > instead of as a service. I'm not sure how recent those apps are though, and > I'm not sure it's better than using a service in the first place (but it > does let you ins

Re: [HACKERS] Add support for tuple routing to foreign partitions

Fujita-san, On 2018/02/21 20:54, Etsuro Fujita wrote: > (2018/02/02 19:33), Etsuro Fujita wrote: >> (2018/01/25 23:33), Stephen Frost wrote: >>> I'm afraid a good bit of this patch is now failing to apply. I don't >>> have much else to say except to echo the performance concern that Amit >>> Lango

Re: Hash Joins vs. Bloom Filters / take 2

On 02/21/2018 08:17 AM, Thomas Munro wrote: > On Wed, Feb 21, 2018 at 10:23 AM, Tomas Vondra > wrote: >> In 2015/2016 I've been exploring if we could improve hash joins by >> leveraging bloom filters [1], and I was reminded about this idea in a >> thread about amcheck [2]. I also see that bloom f

Re: Hash Joins vs. Bloom Filters / take 2

On 02/21/2018 02:10 AM, Peter Geoghegan wrote: > On Tue, Feb 20, 2018 at 3:54 PM, Tomas Vondra > wrote: >>> I suspect that it could make sense to use a Bloom filter to >>> summarize the entire inner side of the join all at once, even >>> when there are multiple batches. I also suspect that this i

Re: Duplicate Item Pointers in Gin index

On Thu, Feb 22, 2018 at 8:28 AM, Peter Geoghegan wrote: > On Wed, Feb 21, 2018 at 3:02 PM, R, Siva wrote: >> Did you mean pin on the metapage buffer during ginInsertCleanup and not lock >> during addition of tuples to the accumulator? The exclusive lock on metapage >> buffer is released after rea

Re: file cloning in pg_upgrade and CREATE DATABASE

On 02/21/2018 04:00 AM, Peter Eisentraut wrote: > ... > > Some example measurements: > > 6 GB database, pg_upgrade unpatched 30 seconds, patched 3 seconds (XFS > and APFS) > > similar for a CREATE DATABASE from a large template > Nice improvement, of course. How does that affect performance on

Re: Duplicate Item Pointers in Gin index

On Wed, Feb 21, 2018 at 3:02 PM, R, Siva wrote: > Did you mean pin on the metapage buffer during ginInsertCleanup and not lock > during addition of tuples to the accumulator? The exclusive lock on metapage > buffer is released after reading/locking head of pending list and before we > process page

Re: Two small patches for the isolationtester lexer

Daniel Gustafsson writes: > On 21 Feb 2018, at 21:41, Tom Lane wrote: >> I can't think of one; but I wonder if it's worth working a bit harder and >> removing the fixed limit altogether, probably by using a PQExpBuffer. >> If you've hit 1024 today, somebody will bump up against 2048 tomorrow. >

Re: Two small patches for the isolationtester lexer

I wrote; > Daniel Gustafsson writes: >> I also (again) forgot about the # comments not being allowed inside setup and >> teardown blocks, so patch 0002 proposes adding support for these as the >> documentation implies. > Hmm, not sure this is a good idea. # is a valid SQL operator name, so > doi

ERROR: left and right pathkeys do not match in mergejoin

Hi hackers, I found a bug related to the planning of merge joins. The steps to reproduce are as follows: ``` CREATE TABLE J1_TBL (     i integer,     j integer,     t text ); CREATE TABLE J2_TBL (     i integer,     k integer ); set enable_hashjoin to off; explain select * from j1_tbl full join

Re: Two small patches for the isolationtester lexer

> On 21 Feb 2018, at 21:41, Tom Lane wrote: > > Daniel Gustafsson writes: >> When writing an isolation testcase recently I bumped into the 1024 line >> buffer >> size limit in the lexer for my setup block. Adding some stored procedures to >> the test makes it quite easy to break 1024 character

Online enabling of checksums

*Once more, here is an attempt to solve the problem of on-line enabling of checksums that me and Daniel have been hacking on for a bit. See for example https://www.postgresql.org/message-id/CABUevEx8KWhZE_XkZQpzEkZypZmBp3GbM9W90JLp%3D-7OJWBbcg%40mail.gmail.com

Re: SHA-2 functions

On 2/20/18 23:04, Michael Paquier wrote: > I think that crypto_hash.c or hash_crypt.c would be adapted as well. > crypt.c is too much generic, so including both concepts in the name is > the way to go. The name given by Tom here sounds actually nice. Updated patches -- Peter Eisentraut

Re: [doc fix] Correct calculation of vm.nr_hugepages

On Mon, Feb 19, 2018 at 9:43 PM, Tsunakawa, Takayuki wrote: > Thanks, I'd like to take this. Why are these values so large? The example in the documentation shows 6490428 kB, and in my test I got 8733888 kB. But 8733888 kB = 8.3 TB! 8.3 GB would make sense, but 8.3 TB does not. -- Robert Haa

Two small patches for the isolationtester lexer

When writing an isolation testcase recently I bumped into the 1024 line buffer size limit in the lexer for my setup block. Adding some stored procedures to the test makes it quite easy to break 1024 characters, and while these could be added as steps it, it’s not a good workaround since the permut

Re: Add PGDLLIMPORT to enable_hashagg

On Wed, Feb 21, 2018 at 10:14 AM, Andres Freund wrote: > Could you take the relevant commit, backport it to the > relevant branches, resolve conflicts, make possibly appropriate > adaptions, and post? > The original commit touched some new variables and therefore didn't apply cleanly. Attached a

Re: Two small patches for the isolationtester lexer

Daniel Gustafsson writes: > When writing an isolation testcase recently I bumped into the 1024 line buffer > size limit in the lexer for my setup block. Adding some stored procedures to > the test makes it quite easy to break 1024 characters, and while these could > be > added as steps it, it’s

Re: pgsql: Avoid valgrind complaint about write() of uninitalized bytes.

On Tue, Feb 20, 2018 at 9:34 PM, Andres Freund wrote: > Doesn't appear to have fixed the problem entirely: > https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=skink&dt=2018-02-20%2017%3A10%3A01 > > relevant excerpt: > ==12452== Syscall param write(buf) points to uninitialised byte(s) > ==124

Re: [HACKERS] pageinspect option to forgo buffer locking?

On Thu, Nov 9, 2017 at 2:21 PM, Andres Freund wrote: > On 2017-11-09 17:14:11 -0500, Tom Lane wrote: >> If we do this, I'd suggest exposing it as a separate SQL function >> get_raw_page_unlocked() rather than as an option to get_raw_page(). >> >> The reasoning is that if we ever allow these functi

Re: [doc fix] Correct calculation of vm.nr_hugepages

On Wed, Feb 21, 2018 at 03:14:57PM -0500, Robert Haas wrote: > On Mon, Feb 19, 2018 at 9:43 PM, Tsunakawa, Takayuki > wrote: > > Thanks, I'd like to take this. > > Why are these values so large? The example in the documentation shows > 6490428 kB, and in my test I got 8733888 kB. But 8733888 kB

Re: pgsql: Avoid valgrind complaint about write() of uninitalized bytes.

On Wed, Feb 21, 2018 at 10:55 AM, Peter Geoghegan wrote: > Sure, but it looks like it has the exact same underlying cause to the > LogicalTapeFreeze() issue. It shouldn't be very hard to write an > equivalent patch for LogicalTapeRewindForRead() -- I pointed out that > this could happen there inst

Typo in predicate.c comment

Hi, Here's a tiny patch to fix a typo. -- Thomas Munro http://www.enterprisedb.com typo.patch Description: Binary data

Re: [PATCH] pgbench - refactor some connection finish/null into common function

On 1/30/18, 03:41, "Fabien COELHO" wrote: Hello Doug, Hi Fabien, > This patch refactors all of the connection state PQfinish() and NULL’ing into a single function. > Excludes PQfinish() in doConnect(). My 0.02€: The argument could be "PGconn **" instead of

Re: Add PGDLLIMPORT to enable_hashagg

Hi, On 2018-02-16 12:57:40 -0800, Brian Cloutier wrote: > Thanks for committing this! We forgot to ask though, could you please > backport this patch to 10 and maybe even 9.6? As-is I don't think these > variables will be available until PG 11. I think these days our policy is to do that on reque

Weird failures on lorikeet

Hi Andrew, I noticed your animal lorikeet failed in the last two runs: https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=lorikeet&dt=2018-02-21%2009%3A47%3A17 TRAP: FailedAssertion("!(((PageHeader) (page))->pd_special >= (__builtin_offsetof (PageHeaderData, pd_linp)))", File: "/home/andrew

Re: PATCH: pgbench - break out timing data for initialization phases

On 1/29/18, 23:52, "Fabien COELHO" wrote: Hello Doug, Hi Fabien, > With patch and ‘-I dtgvpf’ options: > pgrun pgbench -i -s 2000 -F 90 -q -I dtgvpf > dropping old tables... > creating tables... > generating data... > … > 2 of 2 tuples (1

Re: Drop --disable-floatN-byval configure options?

Robert Haas writes: > On Wed, Feb 21, 2018 at 10:50 AM, Tom Lane wrote: >> I have a modest >> substitute proposal: let's just drop the --disable-float4-byval and >> --disable-float8-byval configure options as of v11. Those don't have any >> impact on on-disk storage. > USE_FLOAT4_BYVAL seems co

Re: [HACKERS] Constifying numeric.c's local vars

> This patch got committed as c1898c3e1e235ae35b4759d233253eff221b976a > on Sun Sep 10 16:20:41 2017 -0700, but I've only just gotten around to > reviewing it. > > I believe this is wrong and should be reverted, at least in part. > > The NumericVar struct has the field 'digits' as non-const: >

Re: [HACKERS] Constifying numeric.c's local vars

On February 21, 2018 8:49:51 AM PST, Mark Dilger wrote: > >The idea that set_var_from_var might be called on const_zero (or >const_one, >etc.) is not hypothetical. It is being done in numeric.c. > >If this is safe, somebody needs to be a lot clearer about why that is >so. There >are no commen

Re: missing toast table for pg_policy

On Sun, Feb 18, 2018 at 10:43 AM, Joe Conway wrote: > Is there really a compelling reason to not just create toast tables for > all system catalogs as in the attached? Then we could just check for 0 > rows in misc_sanity.sql. +1. I don't have a huge problem with excluding a few key catalogs for

Re: [HACKERS] Constifying numeric.c's local vars

> On Sep 11, 2017, at 5:10 AM, Tom Lane wrote: > > Andres Freund writes: >> One large user of unnecessary non-constant static variables is >> numeric.c. More out of curiosity - numeric is slow enough in itself to >> make inlining not a huge win - I converted it to use consts. > > LGTM. > >>

Re: Drop --disable-floatN-byval configure options?

On Wed, Feb 21, 2018 at 10:50 AM, Tom Lane wrote: > Per the discussion at > https://www.postgresql.org/message-id/flat/54dfd2022c205eda9aa35b88923f027a%40postgrespro.ru > it's become evident that use of --disable-float8-byval breaks at least one > current regression test case, because it results i

Re: file cloning in pg_upgrade and CREATE DATABASE

On Tue, Feb 20, 2018 at 10:00 PM, Peter Eisentraut wrote: > Some example measurements: > > 6 GB database, pg_upgrade unpatched 30 seconds, patched 3 seconds (XFS > and APFS) > > similar for a CREATE DATABASE from a large template > > Even if you don't have a file system with cloning support, the s

Re: master check fails on Windows Server 2008

On 21-02-2018 18:51, Tom Lane wrote: Marina Polyakova writes: On 20-02-2018 21:23, Tom Lane wrote: I continue to wonder if it's not better to just remove the option and thereby simplify our lives. What's the actual value of having it anymore? I agree with you, but I have too little experi

Re: master check fails on Windows Server 2008

Marina Polyakova writes: > On 20-02-2018 21:23, Tom Lane wrote: >> I continue to wonder if it's not better to just remove >> the option and thereby simplify our lives. What's the actual value of >> having it anymore? > I agree with you, but I have too little experience to vote for removing > th

Drop --disable-floatN-byval configure options?

Per the discussion at https://www.postgresql.org/message-id/flat/54dfd2022c205eda9aa35b88923f027a%40postgrespro.ru it's become evident that use of --disable-float8-byval breaks at least one current regression test case, because it results in a large change in hash table size for float8 hash keys.

Off-cycle back-branch releases next week

We will make a set of minor releases next week, ie wrap tarballs Monday 2/26 for announcement Thursday 3/1. The reason for this unusual scheduling is that an external security researcher found a problem and did not wish to wait for our next quarterly releases before publishing his results. So we

Re: CURRENT OF causes an error when IndexOnlyScan is used

20.02.2018 12:52, Aleksander Alekseev: Hi Anastasia, I'd like to propose the patch that fixes the issue. We already have a way to return heaptuple from IndexOnlyScan, but it was not applied to b-tree for some reason. Attached patch solves the reported bug. Moreover, it will come in handy for "

Re: [HACKERS] Runtime Partition Pruning

On 21 February 2018 at 22:45, Rajkumar Raghuwanshi wrote: > select count(*) from prt1 x where (x.a,x.b) in (select t1.a,t2.b from prt1 > t1,prt2 t2 where t1.a=t2.b) > and (x.c) in (select t3.c from plt1 t3,plt2 t4 where t3.c=t4.c); Thanks for the test case. It seems like the x.c Param for some r

Re: [HACKERS] Add support for tuple routing to foreign partitions

(2018/02/02 19:33), Etsuro Fujita wrote: (2018/01/25 23:33), Stephen Frost wrote: I'm afraid a good bit of this patch is now failing to apply. I don't have much else to say except to echo the performance concern that Amit Langote raised about expanding the inheritence tree twice. To address th

Re: NEXT VALUE FOR sequence

Ashutosh Bapat wrote: > On Tue, Feb 20, 2018 at 8:39 PM, Laurenz Albe > wrote: > > Tom Lane wrote: > > > Laurenz Albe writes: > > > > The SQL standard has the expression "NEXT VALUE FOR asequence" to do > > > > what we traditionally do with "nextval('asequence')". > > > > This is an attempt to i

Re: NEXT VALUE FOR sequence

On Tue, Feb 20, 2018 at 8:39 PM, Laurenz Albe wrote: > Tom Lane wrote: >> Laurenz Albe writes: >> > The SQL standard has the expression "NEXT VALUE FOR asequence" to do >> > what we traditionally do with "nextval('asequence')". >> > This is an attempt to implement this on top of the recently intr

Re: master check fails on Windows Server 2008

On 20-02-2018 21:23, Tom Lane wrote: Marina Polyakova writes: On 20-02-2018 3:37, Tom Lane wrote: 4. Try to tweak the stats_ext.sql test conditions in some more refined way to get the test to pass everywhere. This'd be a lot of work with no guarantee of success, so I'm not too excited about

Re: [HACKERS] path toward faster partition pruning

On 21 February 2018 at 14:53, Amit Langote wrote: > On 2018/02/21 10:19, David Rowley wrote: >> v30-0004-Faster-partition-pruning.patch contains: >> >> +create table coll_pruning_multi (a text) partition by range >> (substr(a, 1) collate "en_GB", substr(a, 1) collate "en_US"); >> >> This'll likely

Re: [HACKERS] Runtime Partition Pruning

On Wed, Feb 21, 2018 at 2:36 PM, David Rowley wrote: > I've attached v11 of the patch. > Hi, I have applied attached patch on head "6f1d723b6359507ef55a81617167507bc25e3e2b" over Amit's v30 patches. while testing further I got a server crash with below test case. Please take a look. CREATE TAB

Re: ALTER TABLE ADD COLUMN fast default

Hi, On 2018-02-20 17:03:07 +1030, Andrew Dunstan wrote: > + > + attmissingval > + bytea > + > + > + This column has a binary representation of the value used when the > + column is entirely missing from the row, as happens when the column is > + added a

Re: [HACKERS] asynchronous execution

At Thu, 11 Jan 2018 17:08:39 +0900 (Tokyo Standard Time), Kyotaro HORIGUCHI wrote in <20180111.170839.23674040.horiguchi.kyot...@lab.ntt.co.jp> > At Mon, 11 Dec 2017 20:07:53 +0900 (Tokyo Standard Time), Kyotaro HORIGUCHI > wrote in > <20171211.200753.191768178.horiguchi.kyot...@lab.ntt.co.jp

RE: Speed up the removal of WAL files

From: Michael Paquier [mailto:mich...@paquier.xyz] It seems to me that you would reintroduce partially the problems that > 1d4a0ab1 has fixed. In short, if a crash happens in the code paths calling > RemoveXlogFile with durable = false before fsync'ing pg_wal, then a rename > has no guarantee to b