Re: [HACKERS] path toward faster partition pruning

2018-04-10 Thread Amit Langote
Thanks for the comment. On 2018/04/10 21:11, Ashutosh Bapat wrote: > On Tue, Apr 10, 2018 at 5:32 PM, David Rowley > wrote: >> Apart from that confusion, looking at the patch: >> >> +CREATE OR REPLACE FUNCTION pp_hashint4_noop(int4, int8) RETURNS int8 AS >>

Re: Boolean partitions syntax

2018-04-10 Thread Amit Langote
On 2018/04/11 13:39, David Rowley wrote: > On 11 April 2018 at 05:22, Tom Lane wrote: >> David Rowley writes: >>> On 11 April 2018 at 03:34, Tom Lane wrote: Well, that just begs the question: why do these expressions

Re: vacuum_cost_limit doc description patch

2018-04-10 Thread David Rowley
On 11 April 2018 at 09:13, Martín Marqués wrote: > This is a patch to add some further description, plus the upper and > lower limits it has. Hi, + for vacuum_cost_delay. The parameter can take a value between 1 and 1. vacuum_cost_delay should be in tags.

RE: power() function in Windows: "value out of range: underflow"

2018-04-10 Thread Huong Dangminh
> > I updated the patch as David Rowley mentioned. > > Looks fine to me. Please add to the next commitfest. Thanks. Added. > -- > David Rowley http://www.2ndQuadrant.com/ > PostgreSQL Development, 24x7 Support, Training & Services > --- Thanks and best regards, Dang Minh

Re: Boolean partitions syntax

2018-04-10 Thread Amit Langote
Horiguchi-san, Thanks for working on this. On 2018/04/11 13:20, Kyotaro HORIGUCHI wrote: > At Wed, 11 Apr 2018 11:27:17 +0900, Amit Langote wrote: >> On 2018/04/11 10:44, Tom Lane wrote: >>> Kyotaro HORIGUCHI writes: At least partition bound *must* be a

Re: power() function in Windows: "value out of range: underflow"

2018-04-10 Thread David Rowley
On 11 April 2018 at 16:42, Huong Dangminh wrote: > I updated the patch as David Rowley mentioned. Looks fine to me. Please add to the next commitfest. -- David Rowley http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training &

RE: power() function in Windows: "value out of range: underflow"

2018-04-10 Thread Huong Dangminh
Thanks for confirming. > 2018-04-11 0:13 GMT-03:00 David Rowley : > > I can recreate this when building with MSVC 2012. I confirm that I see > > the same as you. Microsoft are setting errno to EDOM in the above 3 > > cases, where in Linux the result is still NaN,

Re: Boolean partitions syntax

2018-04-10 Thread David Rowley
On 11 April 2018 at 05:22, Tom Lane wrote: > David Rowley writes: >> On 11 April 2018 at 03:34, Tom Lane wrote: >>> Well, that just begs the question: why do these expressions need to >>> be immutable? What we really want, I

Re: Boolean partitions syntax

2018-04-10 Thread Kyotaro HORIGUCHI
At Wed, 11 Apr 2018 11:27:17 +0900, Amit Langote wrote in <1810b14f-3cd7-aff5-8358-c225c0231...@lab.ntt.co.jp> > On 2018/04/11 10:44, Tom Lane wrote: > > Kyotaro HORIGUCHI writes: > >> At least partition bound *must* be a

Re: power() function in Windows: "value out of range: underflow"

2018-04-10 Thread Euler Taveira
2018-04-11 0:13 GMT-03:00 David Rowley : > I can recreate this when building with MSVC 2012. I confirm that I see > the same as you. Microsoft are setting errno to EDOM in the above 3 > cases, where in Linux the result is still NaN, just the errno is not > set. >

Re: power() function in Windows: "value out of range: underflow"

2018-04-10 Thread David Rowley
On 10 April 2018 at 20:30, Huong Dangminh wrote: > Hi, > > There are some cases that power() function does not work > correctly with 'NaN' arguments in Windows environment. > Something like, > > postgres=# select power('NaN',11); > ERROR: value out of range: underflow

Re: Boolean partitions syntax

2018-04-10 Thread Amit Langote
On 2018/04/10 23:37, David Rowley wrote: > On 10 April 2018 at 23:13, Kyotaro HORIGUCHI > wrote: >> Note: This is not intended to be committed this time but just for >> information. >> >> At Tue, 10 Apr 2018 10:34:27 +0900 (Tokyo Standard Time), Kyotaro HORIGUCHI

Re: Boolean partitions syntax

2018-04-10 Thread Amit Langote
On 2018/04/11 10:44, Tom Lane wrote: > Kyotaro HORIGUCHI writes: >> At least partition bound *must* be a constant. Any expression >> that can be reduced to a constant at parse time ought to be >> accepted but must not be accepted if not. > > My point is that

Re: lazy detoasting

2018-04-10 Thread Jan Wieck
Maybe I'm missing something here, but let me put $.02 in anyway. TOAST reuses entries. If a toasted value is unchanged on UPDATE (i.e. the toast pointer didn't get replaced by a new value), the new tuple points to the same toast slices as the old. If it is changed, the current transaction DELETEs

Re: Boolean partitions syntax

2018-04-10 Thread Tom Lane
Kyotaro HORIGUCHI writes: > At least partition bound *must* be a constant. Any expression > that can be reduced to a constant at parse time ought to be > accepted but must not be accepted if not. My point is that *any* expression can be reduced to a constant, we

Re: Boolean partitions syntax

2018-04-10 Thread Kyotaro HORIGUCHI
At Wed, 11 Apr 2018 02:33:58 +1200, David Rowley wrote in > On 3 February 2018 at 12:04, Tom Lane wrote: > > Perhaps more useful to discuss: would that truly be the semantics

Re: User defined data types in Logical Replication

2018-04-10 Thread Masahiko Sawada
On Mon, Mar 19, 2018 at 7:57 PM, Masahiko Sawada wrote: > On Mon, Mar 19, 2018 at 12:50 PM, Masahiko Sawada > wrote: >> On Fri, Mar 16, 2018 at 10:24 AM, Alvaro Herrera >> wrote: >>> Masahiko Sawada wrote: On Thu, Mar

Re: [HACKERS] Moving relation extension locks out of heavyweight lock manager

2018-04-10 Thread Masahiko Sawada
On Wed, Apr 11, 2018 at 1:40 AM, Robert Haas wrote: > On Tue, Apr 10, 2018 at 5:40 AM, Masahiko Sawada > wrote: >> The probability of performance degradation can be reduced by >> increasing N_RELEXTLOCK_ENTS. But as Robert mentioned, while keeping

Re: [HACKERS] kqueue

2018-04-10 Thread Thomas Munro
On Wed, Dec 6, 2017 at 12:53 AM, Thomas Munro wrote: > On Thu, Jun 22, 2017 at 7:19 PM, Thomas Munro > wrote: >> I don't plan to resubmit this patch myself, but I was doing some >> spring cleaning and rebasing today and I figured it

Re: Partitioned tables and covering indexes

2018-04-10 Thread Amit Langote
Hi. On 2018/04/11 0:36, Teodor Sigaev wrote: >>     Does the attached fix look correct?  Haven't checked the fix with >> ATTACH >>     PARTITION though. >> >> >> Attached patch seems to fix the problem.  However, I would rather get >> rid of modifying stmt->indexParams.  That seems to be more

Re: Gotchas about pg_verify_checksums

2018-04-10 Thread Peter Geoghegan
On Tue, Apr 10, 2018 at 4:44 PM, Michael Paquier wrote: > Peter, the code does the right thing as it requires the instance's > control file state to be either DB_SHUTDOWNED_IN_RECOVERY or > DB_SHUTDOWNED. The documentation, on the contrary, implies that > the instance just

Re: [HACKERS] PATCH: Keep one postmaster monitoring pipe per process

2018-04-10 Thread Thomas Munro
On Wed, Apr 11, 2018 at 12:26 PM, Andres Freund wrote: > On 2018-04-11 12:17:14 +1200, Thomas Munro wrote: >> I arrived at this idea via the realisation that the closest thing to >> prctl(PR_SET_PDEATHSIG) on BSD-family systems today is >>

Re: WIP: Covering + unique indexes.

2018-04-10 Thread Peter Geoghegan
On Tue, Apr 10, 2018 at 1:37 PM, Peter Geoghegan wrote: > _bt_mark_page_halfdead() looked like it had a problem, but it now > looks like I was wrong. I did find another problem, though. Looks like the idea to explicitly represent the number of attributes directly has paid off

Re: [HACKERS] PATCH: Keep one postmaster monitoring pipe per process

2018-04-10 Thread Andres Freund
Hi, On 2018-04-11 12:17:14 +1200, Thomas Munro wrote: > I arrived at this idea via the realisation that the closest thing to > prctl(PR_SET_PDEATHSIG) on BSD-family systems today is > please-tell-my-kqueue-if-this-process-dies. It so happens that my > kqueue patch already uses that instead of

Re: Reopen logfile on SIGHUP

2018-04-10 Thread Grigory Smolkin
On 04/11/2018 12:00 AM, Tom Lane wrote: Alexander Kuzmenkov writes: Syslogger does already rotate logs properly on SIGHUP under some conditions, so we can just change this to unconditional rotation. Probably some people wouldn't want their logs to be rotated on

Re: [HACKERS] PATCH: Keep one postmaster monitoring pipe per process

2018-04-10 Thread Thomas Munro
On Wed, Apr 11, 2018 at 12:03 PM, Andres Freund wrote: > On 2018-04-11 11:57:20 +1200, Thomas Munro wrote: >> Then if pgarch_ArchiverCopyLoop() and HandleStartupProcInterrupts() >> (ie loops without waiting) adopt a prctl(PR_SET_PDEATHSIG)-based >> approach where available as

Re: pgsql: Validate page level checksums in base backups

2018-04-10 Thread Michael Paquier
On Tue, Apr 10, 2018 at 11:35:21PM +0200, Tomas Vondra wrote: > BTW pg_verify_checksums needs the same fix. Yes you are right here. Just for the record: what needs to be done is to check for PageIsNew() in scan_file() before fetching pg_checksum_page() and after reading an individual block,

Re: [HACKERS] PATCH: Keep one postmaster monitoring pipe per process

2018-04-10 Thread Andres Freund
Hi, On 2018-04-11 11:57:20 +1200, Thomas Munro wrote: > Rebased, but I don't actually like this patch any more. Over in > another thread[1] I proposed that we should just make exit(1) the > default behaviour built into latch.c for those cases that don't want > to do something special (eg

Re: 2018-03 CF Cleanup

2018-04-10 Thread Michael Paquier
On Tue, Apr 10, 2018 at 03:18:53PM -0400, Tom Lane wrote: > David Steele writes: >> OK, I did it that way and closed the CF. > > Yay! And many thanks for your hard work on this. Thanks for the cleanup, David! That's a hard task. -- Michael signature.asc Description: PGP

Re: [HACKERS] PATCH: Keep one postmaster monitoring pipe per process

2018-04-10 Thread Thomas Munro
On Tue, Sep 20, 2016 at 11:26 AM, Andres Freund wrote: > On 2016-09-20 11:07:03 +1200, Thomas Munro wrote: >> Yeah, I wondered why that was different than the pattern established >> elsewhere when I was hacking on replication code. There are actually >> several places where

Re: Gotchas about pg_verify_checksums

2018-04-10 Thread Michael Paquier
On Tue, Apr 10, 2018 at 10:27:19PM +0200, Daniel Gustafsson wrote: >> On 10 Apr 2018, at 06:21, Michael Paquier wrote: > Does it really imply that? Either way, the tool could potentially be useful > for debugging a broken cluster so I’m not sure that stating it requires a >

RE: power() function in Windows: "value out of range: underflow"

2018-04-10 Thread Huong Dangminh
Hi, Thanks for response # I will add this thread to current CF soon. > 2018-04-10 5:30 GMT-03:00 Huong Dangminh : > > There are some cases that power() function does not work correctly > > with 'NaN' arguments in Windows environment. > > Something like, > > > What is

Re: Gotchas about pg_verify_checksums

2018-04-10 Thread Michael Paquier
On Tue, Apr 10, 2018 at 02:40:58PM -0700, Peter Geoghegan wrote: > I agree with Michael -- shutting down the server using immediate mode > could lead to torn pages, that crash recovery will need to repair at a > later stage. I think that some strong caveats around this are required > in the

Re: lazy detoasting

2018-04-10 Thread Chapman Flack
On 04/10/2018 05:04 PM, Chapman Flack wrote: > ... I wonder if > there's at least a cheap way to check a particular snapshot > for suitability wrt a given toast pointer. Check a couple usual > suspects, find one most of the time, fall back to eager detoasting > otherwise? > > Guess I need to go

Re: pgsql: Validate page level checksums in base backups

2018-04-10 Thread David Steele
On 4/10/18 5:24 PM, Tomas Vondra wrote: I think there's a bug in sendFile(). We do check checksums on all pages that pass this LSN check: /* * Only check pages which have not been modified since the * start of the base backup. Otherwise, they might have been * written

Re: Gotchas about pg_verify_checksums

2018-04-10 Thread Peter Geoghegan
On Tue, Apr 10, 2018 at 1:27 PM, Daniel Gustafsson wrote: >> On 10 Apr 2018, at 06:21, Michael Paquier wrote: >> 1) The documentation states that the cluster needs to be offline. >> Doesn't this imply that the cluster can also be forcibly killed? It >>

Re: pgsql: Validate page level checksums in base backups

2018-04-10 Thread Tomas Vondra
On 04/10/2018 11:24 PM, Tomas Vondra wrote: > Hi, > > I think there's a bug in sendFile(). We do check checksums on all pages > that pass this LSN check: > > /* > * Only check pages which have not been modified since the > * start of the base backup. Otherwise, they might have

Re: [HACKERS] Runtime Partition Pruning

2018-04-10 Thread Alvaro Herrera
Robert Haas wrote: > On Mon, Apr 9, 2018 at 2:28 PM, Alvaro Herrera > wrote: > >> I don't get this. The executor surely had to (and did) open all of > >> the relations somewhere even before this patch. > > I was worried that this coding could be seen as breaking

Re: pgsql: Validate page level checksums in base backups

2018-04-10 Thread Tomas Vondra
Hi, I think there's a bug in sendFile(). We do check checksums on all pages that pass this LSN check: /* * Only check pages which have not been modified since the * start of the base backup. Otherwise, they might have been * written only halfway and the checksum would not be

Re: lazy detoasting

2018-04-10 Thread Chapman Flack
On 04/10/2018 05:04 PM, Chapman Flack wrote: > If I'm a function, and ... I found [the datum] myself, say by an > SPI query within the function, usually that's at a level of abstraction > somewhere above what-snapshot-was-used-in-the-scan. It looks like for that case (since the commit 08e261cbc

vacuum_cost_limit doc description patch

2018-04-10 Thread Martín Marqués
Hi, Today looking for information on hard limits for autovacuum_vacuum_cost_limit I found myself with a very short description in the docs. This is a patch to add some further description, plus the upper and lower limits it has. -- Martín Marqués select 'martin.marques' || '@' || 'gmail.com'

Re: lazy detoasting

2018-04-10 Thread Chapman Flack
On 04/10/2018 04:03 PM, Robert Haas wrote: > I suspect you want, or maybe need, to use the same snapshot as the > scan that retrieved the tuple containing the toasted datum. I'm sure it's worth more than that, but I don't know if it's implementable. If I'm a function, and the datum came to me

Re: Reopen logfile on SIGHUP

2018-04-10 Thread Tom Lane
Alexander Kuzmenkov writes: > Syslogger does already rotate logs properly on SIGHUP under some > conditions, so we can just change this to unconditional rotation. > Probably some people wouldn't want their logs to be rotated on SIGHUP, > so we could also add a GUC

Re: PostgreSQL's handling of fsync() errors is unsafe and risks data loss at least on XFS

2018-04-10 Thread Joshua D. Drake
On 04/10/2018 12:51 PM, Joshua D. Drake wrote: -hackers, The thread is picking up over on the ext4 list. They don't update their archives as often as we do, so I can't link to the discussion. What would be the preferred method of sharing the info? Thanks to Anthony for this link:

Re: Partitioned tables and covering indexes

2018-04-10 Thread Jaime Casanova
On 10 April 2018 at 10:36, Teodor Sigaev wrote: >> Does the attached fix look correct? Haven't checked the fix with >> ATTACH >> PARTITION though. >> >> >> Attached patch seems to fix the problem. However, I would rather get >> rid of modifying stmt->indexParams. That

Re: WIP: Covering + unique indexes.

2018-04-10 Thread Peter Geoghegan
On Tue, Apr 10, 2018 at 9:03 AM, Teodor Sigaev wrote: >> * Not sure that all calls to BTreeInnerTupleGetDownLink() are limited >> to inner tuples, which might be worth doing something about (perhaps >> just renaming the macro). > > What is suspicious place for you opinion?

Re: Reopen logfile on SIGHUP

2018-04-10 Thread Alexander Kuzmenkov
El 10/04/18 a las 22:40, Robert Haas escribió: Having said that, I'm not averse to providing a solution if it's robust, not too invasive and doesn't break other use-cases. So far we've not seen a patch that meets those conditions. Fair enough. Syslogger does already rotate logs properly

Re: Gotchas about pg_verify_checksums

2018-04-10 Thread Daniel Gustafsson
> On 10 Apr 2018, at 06:21, Michael Paquier wrote: > 1) The documentation states that the cluster needs to be offline. > Doesn't this imply that the cluster can also be forcibly killed? It > seems to me that the documentation ought to say that the cluster needs > to be shut

Re: submake-errcodes

2018-04-10 Thread Tom Lane
Alvaro Herrera writes: > Tom Lane wrote: >> $(OBJS): | submake-generated-headers >> but I took it out thinking it was no longer needed. >> >> The short-term solution seems to be to put that back, but that's sort >> of annoying because it means this isn't a bulletproof

Re: submake-errcodes

2018-04-10 Thread Alvaro Herrera
Tom Lane wrote: > $(OBJS): | submake-generated-headers > > but I took it out thinking it was no longer needed. > > The short-term solution seems to be to put that back, but that's sort > of annoying because it means this isn't a bulletproof solution. It > will only work for builds started in

Re: lazy detoasting

2018-04-10 Thread Robert Haas
On Tue, Apr 10, 2018 at 11:26 AM, Chapman Flack wrote: > Out of the six GetFooSnapshot()s, would I want to squirrel away > Active? Oldest? Transaction? I suspect you want, or maybe need, to use the same snapshot as the scan that retrieved the tuple containing the toasted

Re: PostgreSQL's handling of fsync() errors is unsafe and risks data loss at least on XFS

2018-04-10 Thread Joshua D. Drake
-hackers, The thread is picking up over on the ext4 list. They don't update their archives as often as we do, so I can't link to the discussion. What would be the preferred method of sharing the info? Thanks, JD -- Command Prompt, Inc. || http://the.postgres.company/ || @cmdpromptinc ***

Re: Reopen logfile on SIGHUP

2018-04-10 Thread Robert Haas
On Tue, Apr 10, 2018 at 3:17 PM, Tom Lane wrote: > We, as in the core project, are not shipping it. +1 for what JD said on that subject. > I'm also unclear > on why you want to exclude "fix the RPM packaging" as a reasonable > solution. Mostly because the complaint was

Re: Reopen logfile on SIGHUP

2018-04-10 Thread Joshua D. Drake
On 04/10/2018 12:17 PM, Tom Lane wrote: Robert Haas writes: On Tue, Feb 27, 2018 at 6:12 PM, Tom Lane wrote: IOW, I think a fair response to this is "if you're using logrotate with Postgres, you're doing it wrong". Well, the original post says

Re: 2018-03 CF Cleanup

2018-04-10 Thread Tom Lane
David Steele writes: > OK, I did it that way and closed the CF. Yay! And many thanks for your hard work on this. regards, tom lane

Re: Reopen logfile on SIGHUP

2018-04-10 Thread Tom Lane
Robert Haas writes: > On Tue, Feb 27, 2018 at 6:12 PM, Tom Lane wrote: >> IOW, I think a fair response to this is "if you're using logrotate with >> Postgres, you're doing it wrong". > Well, the original post says that this is how the PGDG RPMs are

Re: submake-errcodes

2018-04-10 Thread Tom Lane
I wrote: > Hm ... you're cd'ing into src/pl/plpython and issuing "make all"? > That works for me. > ... or, wait ... with -j it doesn't. That's strange, will look. So after a bit of digging, it seems that the locution all: submake-generated-headers doesn't result in ensuring that

Re: PostgreSQL's handling of fsync() errors is unsafe and risks data loss at least on XFS

2018-04-10 Thread Joshua D. Drake
-hackers, I reached out to the Linux ext4 devs, here is ty...@mit.edu response: """ Hi Joshua, This isn't actually an ext4 issue, but a long-standing VFS/MM issue. There are going to be multiple opinions about what the right thing to do. I'll try to give as unbiased a description as

Re: 2018-03 CF Cleanup

2018-04-10 Thread David Steele
On 4/10/18 11:24 AM, Alvaro Herrera wrote: > David Steele wrote: > >> For the moment I have left all bugs in the 2018-03 CF. I can can add >> them to the "Older Bugs" section of the PG 11 Open Items but I'm not >> convinced that is the best way to track them. If they are added to >> "Older

Re: Reopen logfile on SIGHUP

2018-04-10 Thread Robert Haas
On Tue, Feb 27, 2018 at 6:12 PM, Tom Lane wrote: > IOW, I think a fair response to this is "if you're using logrotate with > Postgres, you're doing it wrong". Well, the original post says that this is how the PGDG RPMs are doing it on Debian/Ubuntu. I wonder if that's due to

Re: [sqlsmith] Segfault in expand_tuple

2018-04-10 Thread Andrew Dunstan
On 04/07/2018 03:28 PM, Andreas Seltenreich wrote: > Hi, > > the following query triggers a segfault for me when run against the > regression database. Testing was done with master at 039eb6e92f. > Backtrace below. > [large query] > Core was generated by `postgres: smith regression [local]

Re: Boolean partitions syntax

2018-04-10 Thread Jonathan S. Katz
> On Apr 10, 2018, at 1:22 PM, Tom Lane wrote: > > David Rowley writes: >> On 11 April 2018 at 03:34, Tom Lane wrote: >>> Well, that just begs the question: why do these expressions need to >>> be immutable? What we really

Re: Function to track shmem reinit time

2018-04-10 Thread Robert Haas
On Tue, Apr 10, 2018 at 9:07 AM, David Steele wrote: > On 3/29/18 9:40 AM, Tomas Vondra wrote: >> On 03/28/2018 08:55 PM, David Steele wrote: >>> I'm setting this entry to Waiting on Author, but based on the discussion >>> I think it should be Returned with Feedback. >> >>

Re: submake-errcodes

2018-04-10 Thread Tom Lane
Devrim =?ISO-8859-1?Q?G=FCnd=FCz?= writes: > On Tue, 2018-04-10 at 10:01 -0400, Tom Lane wrote: >> You could replace it with submake-generated-headers, since that's more >> general, but in principle you shouldn't need anything because that >> target is invoked automatically as

Re: pgsql: Merge catalog/pg_foo_fn.h headers back into pg_foo.h headers.

2018-04-10 Thread Tom Lane
Julien Rouhaud writes: > On Tue, Apr 10, 2018 at 6:58 PM, Tom Lane wrote: >> none of the three if-guards in the temp-install rule itself should >> prevent this, so what is preventing it? I don't see it. > I just checked, and for the record the second

Re: [sqlsmith] Failed assertion in create_gather_path

2018-04-10 Thread Robert Haas
On Tue, Apr 10, 2018 at 2:59 AM, Jeevan Chalke wrote: > I actually wanted to have rel->consider_parallel in the condition (yes, for > additional safety) as we are adding a partial path into rel. But then > observed that it is same as that of

Re: submake-errcodes

2018-04-10 Thread Devrim Gündüz
Hi, On Tue, 2018-04-10 at 10:01 -0400, Tom Lane wrote: > You could replace it with submake-generated-headers, since that's more > general, but in principle you shouldn't need anything because that > target is invoked automatically as of yesterday. What's the larger > context here --- why do you

Re: pgsql: Merge catalog/pg_foo_fn.h headers back into pg_foo.h headers.

2018-04-10 Thread Julien Rouhaud
On Tue, Apr 10, 2018 at 6:58 PM, Tom Lane wrote: > > Well, the question that's bothering me is how come "make check" in > an external build doesn't try to execute the temp-install rule before > printing that error message. Experimentation shows that it doesn't, > but it sure

Re: Boolean partitions syntax

2018-04-10 Thread Tom Lane
David Rowley writes: > On 11 April 2018 at 03:34, Tom Lane wrote: >> Well, that just begs the question: why do these expressions need to >> be immutable? What we really want, I think, is to evaluate them >> and reduce them to constants. After

Re: pgsql: Merge catalog/pg_foo_fn.h headers back into pg_foo.h headers.

2018-04-10 Thread Tom Lane
Julien Rouhaud writes: > On Tue, Apr 10, 2018 at 3:46 PM, Tom Lane wrote: >> Hm. I wonder if we don't also want NO_TEMP_INSTALL, like the doc/src/sgml >> makefile. I don't know whether "make check" could be useful in a PGXS >> build, but certainly that

Re: PostgreSQL's handling of fsync() errors is unsafe and risks data loss at least on XFS

2018-04-10 Thread Greg Stark
On 10 April 2018 at 02:59, Craig Ringer wrote: > Nitpick: In most cases the kernel reserves disk space immediately, > before returning from write(). NFS seems to be the main exception > here. I'm kind of puzzled by this. Surely NFS servers store the data in the filesystem

Re: pg_dump should use current_database() instead of PQdb()

2018-04-10 Thread Tom Lane
Peter Eisentraut writes: > A report from a pgbouncer user revealed that running pg_dump -C/--create > does not work through a connection proxy if the virtual database name on > the proxy does not match the real database name on the database server. > That's

pg_dump should use current_database() instead of PQdb()

2018-04-10 Thread Peter Eisentraut
A report from a pgbouncer user revealed that running pg_dump -C/--create does not work through a connection proxy if the virtual database name on the proxy does not match the real database name on the database server. That's because pg_dump looks up the database to be dumped using the information

Re: [HACKERS] Moving relation extension locks out of heavyweight lock manager

2018-04-10 Thread Robert Haas
On Tue, Apr 10, 2018 at 5:40 AM, Masahiko Sawada wrote: > The probability of performance degradation can be reduced by > increasing N_RELEXTLOCK_ENTS. But as Robert mentioned, while keeping > fast and simple implementation like acquiring lock by a few atomic > operation

Re: Optimization of range queries

2018-04-10 Thread Konstantin Knizhnik
On 09.04.2018 20:05, Teodor Sigaev wrote: Hi! 12 years ago I proposed patch to which could "union" OR clauses into one range clause if it's possible. In that time pgsql could not use IS NULL as index clause, so patch doesn't support that

Re: pgsql: Support partition pruning at execution time

2018-04-10 Thread Robert Haas
On Tue, Apr 10, 2018 at 12:29 PM, Alvaro Herrera wrote: > In contrast, in an indexonly scan you have a single counter and it > doesn't really matter the distribution of fetches done by workers, so it > seems okay to aggregate them all in a single counter. And it being so

Re: [PATCH] pg_hba.conf : new auth option : clientcert=verify-full

2018-04-10 Thread Magnus Hagander
On Tue, Apr 10, 2018 at 2:10 PM, Julian Markwort < julian.markw...@uni-muenster.de> wrote: > On Fri, 2018-04-06 at 20:31 +0200, Magnus Hagander wrote: > > I've been through this one again. > > Thanks for taking the time! > > There is one big omission from it -- it fails to work with the view >

Re: Boolean partitions syntax

2018-04-10 Thread David Rowley
On 11 April 2018 at 03:34, Tom Lane wrote: > David Rowley writes: >> I imagined this would have had a check for volatile functions and some >> user-friendly error message to say partition bounds must be immutable, >> but instead, it does: > >>

Re: pgsql: Support partition pruning at execution time

2018-04-10 Thread Alvaro Herrera
Robert Haas wrote: > On Tue, Apr 10, 2018 at 11:14 AM, Alvaro Herrera > wrote: > > Questions: > > 1. Do we want to back-patch this to 10? I suppose (without checking) > > that EXPLAIN ANALYZE is already reporting bogus numbers for parallel > > index-only scans, so I

Re: Online enabling of checksums

2018-04-10 Thread Robert Haas
On Fri, Apr 6, 2018 at 8:59 PM, Andres Freund wrote: > This is PROPARALLEL_RESTRICTED. That doesn't strike me right, shouldn't > they be PROPARALLEL_UNSAFE? It might be fine, but I'd not want to rely > on it. Just a fine-grained note on this particular point: It's totally

Re: pgsql: Merge catalog/pg_foo_fn.h headers back into pg_foo.h headers.

2018-04-10 Thread Julien Rouhaud
On Tue, Apr 10, 2018 at 3:46 PM, Tom Lane wrote: > > Julien Rouhaud writes: >> I think the best fix if to define NO_GENERATED_HEADERS in pgxs.mk, >> patch attached. > > Hm. I wonder if we don't also want NO_TEMP_INSTALL, like the doc/src/sgml > makefile.

Re: WIP: Covering + unique indexes.

2018-04-10 Thread Teodor Sigaev
* There is no pfree() within _bt_buildadd() for truncated tuples, even though that's a context where it's clearly not okay. Agree * It might be a good idea to also pfree() the truncated tuple for most other _bt_buildadd() callers. Even though it's arguably okay in other cases, it seems worth

Re: pgsql: Support partition pruning at execution time

2018-04-10 Thread Robert Haas
On Tue, Apr 10, 2018 at 11:14 AM, Alvaro Herrera wrote: > Questions: > 1. Do we want to back-patch this to 10? I suppose (without checking) > that EXPLAIN ANALYZE is already reporting bogus numbers for parallel > index-only scans, so I think we should do that. I haven't

Re: pgsql: Support partition pruning at execution time

2018-04-10 Thread David Rowley
On 11 April 2018 at 03:42, Tom Lane wrote: > Alvaro Herrera writes: >> 2. Do we want to revert Andrew's test stabilization patch? If I >> understand correctly, the problem is the inverse of what was diagnosed: >> "any running transaction at the time

Re: pgsql: Support partition pruning at execution time

2018-04-10 Thread David Rowley
On 11 April 2018 at 03:14, Alvaro Herrera wrote: > 2. Do we want to revert Andrew's test stabilization patch? If I > understand correctly, the problem is the inverse of what was diagnosed: > "any running transaction at the time of the test could prevent pages > from

Re: pgsql: Support partition pruning at execution time

2018-04-10 Thread Tom Lane
Alvaro Herrera writes: > Questions: > 1. Do we want to back-patch this to 10? I suppose (without checking) > that EXPLAIN ANALYZE is already reporting bogus numbers for parallel > index-only scans, so I think we should do that. You can't back-patch a change in struct

Re: PostgreSQL's handling of fsync() errors is unsafe and risks data loss at least on XFS

2018-04-10 Thread Anthony Iliopoulos
Hi Robert, On Tue, Apr 10, 2018 at 11:15:46AM -0400, Robert Haas wrote: > On Mon, Apr 9, 2018 at 3:13 PM, Andres Freund wrote: > > Let's lower the pitchforks a bit here. Obviously a grand rewrite is > > absurd, as is some of the proposed ways this is all supposed to > >

Re: Partitioned tables and covering indexes

2018-04-10 Thread Teodor Sigaev
Does the attached fix look correct?  Haven't checked the fix with ATTACH PARTITION though. Attached patch seems to fix the problem.  However, I would rather get rid of modifying stmt->indexParams.  That seems to be more logical for me.  Also, it would be good to check some covering

Re: Boolean partitions syntax

2018-04-10 Thread Tom Lane
David Rowley writes: > I imagined this would have had a check for volatile functions and some > user-friendly error message to say partition bounds must be immutable, > but instead, it does: > postgres=# create table d_p1 partition of d for values in (Random()); >

Including SQL files in extension scripts

2018-04-10 Thread Jeremy Finzel
In writing extension update scripts, I find it to be really difficult to grok diffs for example in changed view or function definitions when a new extension script has to include the whole definition in a new file. I want to rather use separate files for these objects, then use something like

Re: PostgreSQL's handling of fsync() errors is unsafe and risks data loss at least on XFS

2018-04-10 Thread Robert Haas
On Tue, Apr 10, 2018 at 1:37 AM, Craig Ringer wrote: > ... but *only if they hit an I/O error* or they're on a FS that > doesn't reserve space and hit ENOSPC. > > It still does 99% of the job. It still flushes all buffers to > persistent storage and maintains write

Re: crash with sql language partition support function

2018-04-10 Thread Tom Lane
Alvaro Herrera writes: > Ashutosh Bapat wrote: >> On Tue, Apr 10, 2018 at 1:44 PM, Amit Langote >> wrote: >>> Attached fixes it. It teaches RelationBuildPartitionKey() to use >>> fmgr_info_cxt and pass rd_partkeycxt to it. >> The patch is

Re: [PATCH][PROPOSAL] Refuse setting toast.* reloptions when TOAST table does not exist

2018-04-10 Thread David Steele
On 4/10/18 9:17 AM, Alvaro Herrera wrote: > Nikolay Shaplov wrote: > >> But I need some confirmation, in order not to write patch in vain again :-) > > Don't worry, rest assured that you will still write *many* patches in > vain, not just this one. Despite the rather dubious pep talk, Álvaro is

Re: lazy detoasting

2018-04-10 Thread Chapman Flack
On 04/10/2018 10:06 AM, Tom Lane wrote: > Chapman Flack writes: >> Am I right in thinking that, for my original purpose of >> detoasting something later in a transaction, all that matters >> is that I registered a snapshot from the time at which I copied >> the toasted

Re: 2018-03 CF Cleanup

2018-04-10 Thread Alvaro Herrera
David Steele wrote: > Hackers, > > I have gone through all the remaining non-bug entries in CF 2018-03 and > pushed them or closed them as appropriate. The majority of the patches > were in Needs Review or Ready for Committer status and I did a brief > review of each to be sure the state was

2018-03 CF Cleanup

2018-04-10 Thread David Steele
Hackers, I have gone through all the remaining non-bug entries in CF 2018-03 and pushed them or closed them as appropriate. The majority of the patches were in Needs Review or Ready for Committer status and I did a brief review of each to be sure the state was reasonable. I moved a few Waiting

Re: PostgreSQL's handling of fsync() errors is unsafe and risks data loss at least on XFS

2018-04-10 Thread Robert Haas
On Mon, Apr 9, 2018 at 3:13 PM, Andres Freund wrote: > Let's lower the pitchforks a bit here. Obviously a grand rewrite is > absurd, as is some of the proposed ways this is all supposed to > work. But I think the case we're discussing is much closer to a near > irresolvable

Re: pgsql: Support partition pruning at execution time

2018-04-10 Thread Alvaro Herrera
Changed CC to pgsql-hackers. Tom Lane wrote: > Alvaro Herrera writes: > > Frankly, I don't like this. I would rather have an instrument->ntuples2 > > rather than these "divide this by nloops, sometimes" schizoid counters. > > This is already being misused by ON

Re: [HACKERS] [PATCH] Incremental sort

2018-04-10 Thread Alexander Korotkov
On Tue, Apr 10, 2018 at 4:15 PM, David Steele wrote: > On 4/9/18 11:56 AM, Alexander Korotkov wrote: > > > > Thank you very much for your efforts on reviewing this patch. > > I'm looking forward work with you on this feature for PG12. > > Since there's a new patch I have

Re: Partitioned tables and covering indexes

2018-04-10 Thread Alexander Korotkov
On Tue, Apr 10, 2018 at 12:47 PM, Amit Langote < langote_amit...@lab.ntt.co.jp> wrote: > On 2018/04/10 16:07, Jaime Casanova wrote: > > Hi, > > > > Trying covering indexes on partitioned tables i get this error > > """ > > postgres=# create index on t1_part (i) include (t); > > ERROR: cache

  1   2   >