Re: Boolean partitions syntax

2018-04-10 Thread Kyotaro HORIGUCHI
At Wed, 11 Apr 2018 14:22:29 +0900, Amit Langote wrote in <6e929961-4160-7338-3d26-ccf84f416...@lab.ntt.co.jp> > On 2018/04/11 13:39, David Rowley wrote: > > On 11 April 2018 at 05:22, Tom Lane wrote: > >> David Rowley writes: > >>> On 11 April 2018 at 03:34, Tom Lane wrote: > Well, that

Re: [PATCH] Add missing type conversion functions for PL/Python

2018-04-10 Thread Haozhou Wang
Hi David, Thanks for your email. Just wondering will I need to re-submit this patch? Thanks a lot! Regards, Haozhou On Tue, Apr 10, 2018 at 9:35 PM, David Steele wrote: > On 3/26/18 12:07 PM, Nikita Glukhov wrote: > > On 26.03.2018 17:19, David Steele wrote: > > > >> On 2/20/18 10:14 AM, Haoz

Re: [HACKERS] Runtime Partition Pruning

2018-04-10 Thread Amit Langote
On 2018/04/11 6:32, Alvaro Herrera wrote: > Robert Haas wrote: >> On Mon, Apr 9, 2018 at 2:28 PM, Alvaro Herrera >> wrote: > I don't get this. The executor surely had to (and did) open all of the relations somewhere even before this patch. > >>> I was worried that this coding could b

Re: [HACKERS] path toward faster partition pruning

2018-04-10 Thread Amit Langote
Thanks for the review. On 2018/04/10 21:02, David Rowley wrote: > On 10 April 2018 at 20:56, Amit Langote wrote: >> On 2018/04/10 13:27, Ashutosh Bapat wrote: >>> On Mon, Apr 9, 2018 at 8:56 PM, Robert Haas wrote: CREATE OR REPLACE FUNCTION hashint4_noop(int4, int8) RETURNS int8 AS $$S

Re: [HACKERS] path toward faster partition pruning

2018-04-10 Thread Amit Langote
Thanks for the comment. On 2018/04/10 21:11, Ashutosh Bapat wrote: > On Tue, Apr 10, 2018 at 5:32 PM, David Rowley > wrote: >> Apart from that confusion, looking at the patch: >> >> +CREATE OR REPLACE FUNCTION pp_hashint4_noop(int4, int8) RETURNS int8 AS >> +$$SELECT coalesce($1)::int8$$ LANGUAGE

Re: Boolean partitions syntax

2018-04-10 Thread Amit Langote
On 2018/04/11 13:39, David Rowley wrote: > On 11 April 2018 at 05:22, Tom Lane wrote: >> David Rowley writes: >>> On 11 April 2018 at 03:34, Tom Lane wrote: Well, that just begs the question: why do these expressions need to be immutable? What we really want, I think, is to evaluate t

Re: vacuum_cost_limit doc description patch

2018-04-10 Thread David Rowley
On 11 April 2018 at 09:13, Martín Marqués wrote: > This is a patch to add some further description, plus the upper and > lower limits it has. Hi, + for vacuum_cost_delay. The parameter can take a value between 1 and 1. vacuum_cost_delay should be in tags. +1 to mentioning that we

RE: power() function in Windows: "value out of range: underflow"

2018-04-10 Thread Huong Dangminh
> > I updated the patch as David Rowley mentioned. > > Looks fine to me. Please add to the next commitfest. Thanks. Added. > -- > David Rowley http://www.2ndQuadrant.com/ > PostgreSQL Development, 24x7 Support, Training & Services > --- Thanks and best regards, Dang Minh Hu

Re: Boolean partitions syntax

2018-04-10 Thread Amit Langote
Horiguchi-san, Thanks for working on this. On 2018/04/11 13:20, Kyotaro HORIGUCHI wrote: > At Wed, 11 Apr 2018 11:27:17 +0900, Amit Langote wrote: >> On 2018/04/11 10:44, Tom Lane wrote: >>> Kyotaro HORIGUCHI writes: At least partition bound *must* be a constant. Any expression that ca

Re: power() function in Windows: "value out of range: underflow"

2018-04-10 Thread David Rowley
On 11 April 2018 at 16:42, Huong Dangminh wrote: > I updated the patch as David Rowley mentioned. Looks fine to me. Please add to the next commitfest. -- David Rowley http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services

RE: power() function in Windows: "value out of range: underflow"

2018-04-10 Thread Huong Dangminh
Thanks for confirming. > 2018-04-11 0:13 GMT-03:00 David Rowley : > > I can recreate this when building with MSVC 2012. I confirm that I see > > the same as you. Microsoft are setting errno to EDOM in the above 3 > > cases, where in Linux the result is still NaN, just the errno is not > > set. >

Re: Boolean partitions syntax

2018-04-10 Thread David Rowley
On 11 April 2018 at 05:22, Tom Lane wrote: > David Rowley writes: >> On 11 April 2018 at 03:34, Tom Lane wrote: >>> Well, that just begs the question: why do these expressions need to >>> be immutable? What we really want, I think, is to evaluate them >>> and reduce them to constants. After th

Re: Boolean partitions syntax

2018-04-10 Thread Kyotaro HORIGUCHI
At Wed, 11 Apr 2018 11:27:17 +0900, Amit Langote wrote in <1810b14f-3cd7-aff5-8358-c225c0231...@lab.ntt.co.jp> > On 2018/04/11 10:44, Tom Lane wrote: > > Kyotaro HORIGUCHI writes: > >> At least partition bound *must* be a constant. Any expression > >> that can be reduced to a constant at parse

Re: power() function in Windows: "value out of range: underflow"

2018-04-10 Thread Euler Taveira
2018-04-11 0:13 GMT-03:00 David Rowley : > I can recreate this when building with MSVC 2012. I confirm that I see > the same as you. Microsoft are setting errno to EDOM in the above 3 > cases, where in Linux the result is still NaN, just the errno is not > set. > FWIW, I tested in MSVC 2017 (15.6.4

Re: power() function in Windows: "value out of range: underflow"

2018-04-10 Thread David Rowley
On 10 April 2018 at 20:30, Huong Dangminh wrote: > Hi, > > There are some cases that power() function does not work > correctly with 'NaN' arguments in Windows environment. > Something like, > > postgres=# select power('NaN',11); > ERROR: value out of range: underflow > postgres=# select power('N

Re: Boolean partitions syntax

2018-04-10 Thread Amit Langote
On 2018/04/10 23:37, David Rowley wrote: > On 10 April 2018 at 23:13, Kyotaro HORIGUCHI > wrote: >> Note: This is not intended to be committed this time but just for >> information. >> >> At Tue, 10 Apr 2018 10:34:27 +0900 (Tokyo Standard Time), Kyotaro HORIGUCHI >> wrote: >>> Just adding negatio

Re: Boolean partitions syntax

2018-04-10 Thread Amit Langote
On 2018/04/11 10:44, Tom Lane wrote: > Kyotaro HORIGUCHI writes: >> At least partition bound *must* be a constant. Any expression >> that can be reduced to a constant at parse time ought to be >> accepted but must not be accepted if not. > > My point is that *any* expression can be reduced to a c

Re: lazy detoasting

2018-04-10 Thread Jan Wieck
Maybe I'm missing something here, but let me put $.02 in anyway. TOAST reuses entries. If a toasted value is unchanged on UPDATE (i.e. the toast pointer didn't get replaced by a new value), the new tuple points to the same toast slices as the old. If it is changed, the current transaction DELETEs

Re: Boolean partitions syntax

2018-04-10 Thread Tom Lane
Kyotaro HORIGUCHI writes: > At least partition bound *must* be a constant. Any expression > that can be reduced to a constant at parse time ought to be > accepted but must not be accepted if not. My point is that *any* expression can be reduced to a constant, we just have to do so. > Maybe we co

Re: Boolean partitions syntax

2018-04-10 Thread Kyotaro HORIGUCHI
At Wed, 11 Apr 2018 02:33:58 +1200, David Rowley wrote in > On 3 February 2018 at 12:04, Tom Lane wrote: > > Perhaps more useful to discuss: would that truly be the semantics we want, > > or should we just evaluate the expression and have done? It's certainly > > arguable that "IN (random())"

Re: User defined data types in Logical Replication

2018-04-10 Thread Masahiko Sawada
On Mon, Mar 19, 2018 at 7:57 PM, Masahiko Sawada wrote: > On Mon, Mar 19, 2018 at 12:50 PM, Masahiko Sawada > wrote: >> On Fri, Mar 16, 2018 at 10:24 AM, Alvaro Herrera >> wrote: >>> Masahiko Sawada wrote: On Thu, Mar 15, 2018 at 9:41 AM, Alvaro Herrera wrote: >>> > I think thi

Re: [HACKERS] Moving relation extension locks out of heavyweight lock manager

2018-04-10 Thread Masahiko Sawada
On Wed, Apr 11, 2018 at 1:40 AM, Robert Haas wrote: > On Tue, Apr 10, 2018 at 5:40 AM, Masahiko Sawada > wrote: >> The probability of performance degradation can be reduced by >> increasing N_RELEXTLOCK_ENTS. But as Robert mentioned, while keeping >> fast and simple implementation like acquiring

Re: [HACKERS] kqueue

2018-04-10 Thread Thomas Munro
On Wed, Dec 6, 2017 at 12:53 AM, Thomas Munro wrote: > On Thu, Jun 22, 2017 at 7:19 PM, Thomas Munro > wrote: >> I don't plan to resubmit this patch myself, but I was doing some >> spring cleaning and rebasing today and I figured it might be worth >> quietly leaving a working patch here just in c

Re: Partitioned tables and covering indexes

2018-04-10 Thread Amit Langote
Hi. On 2018/04/11 0:36, Teodor Sigaev wrote: >>     Does the attached fix look correct?  Haven't checked the fix with >> ATTACH >>     PARTITION though. >> >> >> Attached patch seems to fix the problem.  However, I would rather get >> rid of modifying stmt->indexParams.  That seems to be more logi

Re: Gotchas about pg_verify_checksums

2018-04-10 Thread Peter Geoghegan
On Tue, Apr 10, 2018 at 4:44 PM, Michael Paquier wrote: > Peter, the code does the right thing as it requires the instance's > control file state to be either DB_SHUTDOWNED_IN_RECOVERY or > DB_SHUTDOWNED. The documentation, on the contrary, implies that > the instance just needs to be offline, wh

Re: [HACKERS] PATCH: Keep one postmaster monitoring pipe per process

2018-04-10 Thread Thomas Munro
On Wed, Apr 11, 2018 at 12:26 PM, Andres Freund wrote: > On 2018-04-11 12:17:14 +1200, Thomas Munro wrote: >> I arrived at this idea via the realisation that the closest thing to >> prctl(PR_SET_PDEATHSIG) on BSD-family systems today is >> please-tell-my-kqueue-if-this-process-dies. It so happens

Re: WIP: Covering + unique indexes.

2018-04-10 Thread Peter Geoghegan
On Tue, Apr 10, 2018 at 1:37 PM, Peter Geoghegan wrote: > _bt_mark_page_halfdead() looked like it had a problem, but it now > looks like I was wrong. I did find another problem, though. Looks like the idea to explicitly represent the number of attributes directly has paid off already: pg@~[3711]

Re: [HACKERS] PATCH: Keep one postmaster monitoring pipe per process

2018-04-10 Thread Andres Freund
Hi, On 2018-04-11 12:17:14 +1200, Thomas Munro wrote: > I arrived at this idea via the realisation that the closest thing to > prctl(PR_SET_PDEATHSIG) on BSD-family systems today is > please-tell-my-kqueue-if-this-process-dies. It so happens that my > kqueue patch already uses that instead of the

Re: Reopen logfile on SIGHUP

2018-04-10 Thread Grigory Smolkin
On 04/11/2018 12:00 AM, Tom Lane wrote: Alexander Kuzmenkov writes: Syslogger does already rotate logs properly on SIGHUP under some conditions, so we can just change this to unconditional rotation. Probably some people wouldn't want their logs to be rotated on SIGHUP, so we could also add a G

Re: [HACKERS] PATCH: Keep one postmaster monitoring pipe per process

2018-04-10 Thread Thomas Munro
On Wed, Apr 11, 2018 at 12:03 PM, Andres Freund wrote: > On 2018-04-11 11:57:20 +1200, Thomas Munro wrote: >> Then if pgarch_ArchiverCopyLoop() and HandleStartupProcInterrupts() >> (ie loops without waiting) adopt a prctl(PR_SET_PDEATHSIG)-based >> approach where available as suggested by Andres[2

Re: pgsql: Validate page level checksums in base backups

2018-04-10 Thread Michael Paquier
On Tue, Apr 10, 2018 at 11:35:21PM +0200, Tomas Vondra wrote: > BTW pg_verify_checksums needs the same fix. Yes you are right here. Just for the record: what needs to be done is to check for PageIsNew() in scan_file() before fetching pg_checksum_page() and after reading an individual block, which

Re: [HACKERS] PATCH: Keep one postmaster monitoring pipe per process

2018-04-10 Thread Andres Freund
Hi, On 2018-04-11 11:57:20 +1200, Thomas Munro wrote: > Rebased, but I don't actually like this patch any more. Over in > another thread[1] I proposed that we should just make exit(1) the > default behaviour built into latch.c for those cases that don't want > to do something special (eg SyncRepW

Re: 2018-03 CF Cleanup

2018-04-10 Thread Michael Paquier
On Tue, Apr 10, 2018 at 03:18:53PM -0400, Tom Lane wrote: > David Steele writes: >> OK, I did it that way and closed the CF. > > Yay! And many thanks for your hard work on this. Thanks for the cleanup, David! That's a hard task. -- Michael signature.asc Description: PGP signature

Re: [HACKERS] PATCH: Keep one postmaster monitoring pipe per process

2018-04-10 Thread Thomas Munro
On Tue, Sep 20, 2016 at 11:26 AM, Andres Freund wrote: > On 2016-09-20 11:07:03 +1200, Thomas Munro wrote: >> Yeah, I wondered why that was different than the pattern established >> elsewhere when I was hacking on replication code. There are actually >> several places where we call PostmasterIsAl

Re: Gotchas about pg_verify_checksums

2018-04-10 Thread Michael Paquier
On Tue, Apr 10, 2018 at 10:27:19PM +0200, Daniel Gustafsson wrote: >> On 10 Apr 2018, at 06:21, Michael Paquier wrote: > Does it really imply that? Either way, the tool could potentially be useful > for debugging a broken cluster so I’m not sure that stating it requires a > cleanly shut down serv

RE: power() function in Windows: "value out of range: underflow"

2018-04-10 Thread Huong Dangminh
Hi, Thanks for response # I will add this thread to current CF soon. > 2018-04-10 5:30 GMT-03:00 Huong Dangminh : > > There are some cases that power() function does not work correctly > > with 'NaN' arguments in Windows environment. > > Something like, > > > What is your exact OS version? What i

Re: Gotchas about pg_verify_checksums

2018-04-10 Thread Michael Paquier
On Tue, Apr 10, 2018 at 02:40:58PM -0700, Peter Geoghegan wrote: > I agree with Michael -- shutting down the server using immediate mode > could lead to torn pages, that crash recovery will need to repair at a > later stage. I think that some strong caveats around this are required > in the pg_veri

Re: lazy detoasting

2018-04-10 Thread Chapman Flack
On 04/10/2018 05:04 PM, Chapman Flack wrote: > ... I wonder if > there's at least a cheap way to check a particular snapshot > for suitability wrt a given toast pointer. Check a couple usual > suspects, find one most of the time, fall back to eager detoasting > otherwise? > > Guess I need to go ba

Re: pgsql: Validate page level checksums in base backups

2018-04-10 Thread David Steele
On 4/10/18 5:24 PM, Tomas Vondra wrote: I think there's a bug in sendFile(). We do check checksums on all pages that pass this LSN check: /* * Only check pages which have not been modified since the * start of the base backup. Otherwise, they might have been * written onl

Re: Gotchas about pg_verify_checksums

2018-04-10 Thread Peter Geoghegan
On Tue, Apr 10, 2018 at 1:27 PM, Daniel Gustafsson wrote: >> On 10 Apr 2018, at 06:21, Michael Paquier wrote: >> 1) The documentation states that the cluster needs to be offline. >> Doesn't this imply that the cluster can also be forcibly killed? It >> seems to me that the documentation ought to

Re: pgsql: Validate page level checksums in base backups

2018-04-10 Thread Tomas Vondra
On 04/10/2018 11:24 PM, Tomas Vondra wrote: > Hi, > > I think there's a bug in sendFile(). We do check checksums on all pages > that pass this LSN check: > > /* > * Only check pages which have not been modified since the > * start of the base backup. Otherwise, they might have bee

Re: [HACKERS] Runtime Partition Pruning

2018-04-10 Thread Alvaro Herrera
Robert Haas wrote: > On Mon, Apr 9, 2018 at 2:28 PM, Alvaro Herrera > wrote: > >> I don't get this. The executor surely had to (and did) open all of > >> the relations somewhere even before this patch. > > I was worried that this coding could be seen as breaking modularity, or > > trying to do

Re: pgsql: Validate page level checksums in base backups

2018-04-10 Thread Tomas Vondra
Hi, I think there's a bug in sendFile(). We do check checksums on all pages that pass this LSN check: /* * Only check pages which have not been modified since the * start of the base backup. Otherwise, they might have been * written only halfway and the checksum would not be va

Re: lazy detoasting

2018-04-10 Thread Chapman Flack
On 04/10/2018 05:04 PM, Chapman Flack wrote: > If I'm a function, and ... I found [the datum] myself, say by an > SPI query within the function, usually that's at a level of abstraction > somewhere above what-snapshot-was-used-in-the-scan. It looks like for that case (since the commit 08e261cbc t

vacuum_cost_limit doc description patch

2018-04-10 Thread Martín Marqués
Hi, Today looking for information on hard limits for autovacuum_vacuum_cost_limit I found myself with a very short description in the docs. This is a patch to add some further description, plus the upper and lower limits it has. -- Martín Marqués select 'martin.marques' || '@' || 'gmail.com' DB

Re: lazy detoasting

2018-04-10 Thread Chapman Flack
On 04/10/2018 04:03 PM, Robert Haas wrote: > I suspect you want, or maybe need, to use the same snapshot as the > scan that retrieved the tuple containing the toasted datum. I'm sure it's worth more than that, but I don't know if it's implementable. If I'm a function, and the datum came to me as

Re: Reopen logfile on SIGHUP

2018-04-10 Thread Tom Lane
Alexander Kuzmenkov writes: > Syslogger does already rotate logs properly on SIGHUP under some > conditions, so we can just change this to unconditional rotation. > Probably some people wouldn't want their logs to be rotated on SIGHUP, > so we could also add a GUC to control this. Please see th

Re: PostgreSQL's handling of fsync() errors is unsafe and risks data loss at least on XFS

2018-04-10 Thread Joshua D. Drake
On 04/10/2018 12:51 PM, Joshua D. Drake wrote: -hackers, The thread is picking up over on the ext4 list. They don't update their archives as often as we do, so I can't link to the discussion. What would be the preferred method of sharing the info? Thanks to Anthony for this link: http://lis

Re: Partitioned tables and covering indexes

2018-04-10 Thread Jaime Casanova
On 10 April 2018 at 10:36, Teodor Sigaev wrote: >> Does the attached fix look correct? Haven't checked the fix with >> ATTACH >> PARTITION though. >> >> >> Attached patch seems to fix the problem. However, I would rather get >> rid of modifying stmt->indexParams. That seems to be more l

Re: WIP: Covering + unique indexes.

2018-04-10 Thread Peter Geoghegan
On Tue, Apr 10, 2018 at 9:03 AM, Teodor Sigaev wrote: >> * Not sure that all calls to BTreeInnerTupleGetDownLink() are limited >> to inner tuples, which might be worth doing something about (perhaps >> just renaming the macro). > > What is suspicious place for you opinion? _bt_mark_page_halfdead(

Re: Reopen logfile on SIGHUP

2018-04-10 Thread Alexander Kuzmenkov
El 10/04/18 a las 22:40, Robert Haas escribió: Having said that, I'm not averse to providing a solution if it's robust, not too invasive and doesn't break other use-cases. So far we've not seen a patch that meets those conditions. Fair enough. Syslogger does already rotate logs properly on

Re: Gotchas about pg_verify_checksums

2018-04-10 Thread Daniel Gustafsson
> On 10 Apr 2018, at 06:21, Michael Paquier wrote: > 1) The documentation states that the cluster needs to be offline. > Doesn't this imply that the cluster can also be forcibly killed? It > seems to me that the documentation ought to say that the cluster needs > to be shut down cleanly instead.

Re: submake-errcodes

2018-04-10 Thread Tom Lane
Alvaro Herrera writes: > Tom Lane wrote: >> $(OBJS): | submake-generated-headers >> but I took it out thinking it was no longer needed. >> >> The short-term solution seems to be to put that back, but that's sort >> of annoying because it means this isn't a bulletproof solution. > Isn't it possib

Re: submake-errcodes

2018-04-10 Thread Alvaro Herrera
Tom Lane wrote: > $(OBJS): | submake-generated-headers > > but I took it out thinking it was no longer needed. > > The short-term solution seems to be to put that back, but that's sort > of annoying because it means this isn't a bulletproof solution. It > will only work for builds started in on

Re: lazy detoasting

2018-04-10 Thread Robert Haas
On Tue, Apr 10, 2018 at 11:26 AM, Chapman Flack wrote: > Out of the six GetFooSnapshot()s, would I want to squirrel away > Active? Oldest? Transaction? I suspect you want, or maybe need, to use the same snapshot as the scan that retrieved the tuple containing the toasted datum. (This advice may

Re: PostgreSQL's handling of fsync() errors is unsafe and risks data loss at least on XFS

2018-04-10 Thread Joshua D. Drake
-hackers, The thread is picking up over on the ext4 list. They don't update their archives as often as we do, so I can't link to the discussion. What would be the preferred method of sharing the info? Thanks, JD -- Command Prompt, Inc. || http://the.postgres.company/ || @cmdpromptinc ***

Re: Reopen logfile on SIGHUP

2018-04-10 Thread Robert Haas
On Tue, Apr 10, 2018 at 3:17 PM, Tom Lane wrote: > We, as in the core project, are not shipping it. +1 for what JD said on that subject. > I'm also unclear > on why you want to exclude "fix the RPM packaging" as a reasonable > solution. Mostly because the complaint was about the *Debian* packag

Re: Reopen logfile on SIGHUP

2018-04-10 Thread Joshua D. Drake
On 04/10/2018 12:17 PM, Tom Lane wrote: Robert Haas writes: On Tue, Feb 27, 2018 at 6:12 PM, Tom Lane wrote: IOW, I think a fair response to this is "if you're using logrotate with Postgres, you're doing it wrong". Well, the original post says that this is how the PGDG RPMs are doing it on

Re: 2018-03 CF Cleanup

2018-04-10 Thread Tom Lane
David Steele writes: > OK, I did it that way and closed the CF. Yay! And many thanks for your hard work on this. regards, tom lane

Re: Reopen logfile on SIGHUP

2018-04-10 Thread Tom Lane
Robert Haas writes: > On Tue, Feb 27, 2018 at 6:12 PM, Tom Lane wrote: >> IOW, I think a fair response to this is "if you're using logrotate with >> Postgres, you're doing it wrong". > Well, the original post says that this is how the PGDG RPMs are doing > it on Debian/Ubuntu. I wonder if that'

Re: submake-errcodes

2018-04-10 Thread Tom Lane
I wrote: > Hm ... you're cd'ing into src/pl/plpython and issuing "make all"? > That works for me. > ... or, wait ... with -j it doesn't. That's strange, will look. So after a bit of digging, it seems that the locution all: submake-generated-headers doesn't result in ensuring that submake-genera

Re: PostgreSQL's handling of fsync() errors is unsafe and risks data loss at least on XFS

2018-04-10 Thread Joshua D. Drake
-hackers, I reached out to the Linux ext4 devs, here is ty...@mit.edu response: """ Hi Joshua, This isn't actually an ext4 issue, but a long-standing VFS/MM issue. There are going to be multiple opinions about what the right thing to do. I'll try to give as unbiased a description as possible,

Re: 2018-03 CF Cleanup

2018-04-10 Thread David Steele
On 4/10/18 11:24 AM, Alvaro Herrera wrote: > David Steele wrote: > >> For the moment I have left all bugs in the 2018-03 CF. I can can add >> them to the "Older Bugs" section of the PG 11 Open Items but I'm not >> convinced that is the best way to track them. If they are added to >> "Older Bugs"

Re: pgsql: Support partition pruning at execution time

2018-04-10 Thread Alvaro Herrera
Robert Haas wrote: > > I don't have a strong opinion on that. Since we know how many tuples > were processed by each worker, knowing how many heap fetches we have > on a per-worker basis seems like a good thing to have, too. On the > other hand, maybe EXPLAIN (ANALYZE, VERBOSE) would give us tha

Re: Reopen logfile on SIGHUP

2018-04-10 Thread Robert Haas
On Tue, Feb 27, 2018 at 6:12 PM, Tom Lane wrote: > IOW, I think a fair response to this is "if you're using logrotate with > Postgres, you're doing it wrong". Well, the original post says that this is how the PGDG RPMs are doing it on Debian/Ubuntu. I wonder if that's due to some Debian/Ubuntu p

Re: [sqlsmith] Segfault in expand_tuple

2018-04-10 Thread Andrew Dunstan
On 04/07/2018 03:28 PM, Andreas Seltenreich wrote: > Hi, > > the following query triggers a segfault for me when run against the > regression database. Testing was done with master at 039eb6e92f. > Backtrace below. > [large query] > Core was generated by `postgres: smith regression [local] SELE

Re: Boolean partitions syntax

2018-04-10 Thread Jonathan S. Katz
> On Apr 10, 2018, at 1:22 PM, Tom Lane wrote: > > David Rowley writes: >> On 11 April 2018 at 03:34, Tom Lane wrote: >>> Well, that just begs the question: why do these expressions need to >>> be immutable? What we really want, I think, is to evaluate them >>> and reduce them to constants.

Re: Function to track shmem reinit time

2018-04-10 Thread Robert Haas
On Tue, Apr 10, 2018 at 9:07 AM, David Steele wrote: > On 3/29/18 9:40 AM, Tomas Vondra wrote: >> On 03/28/2018 08:55 PM, David Steele wrote: >>> I'm setting this entry to Waiting on Author, but based on the discussion >>> I think it should be Returned with Feedback. >> >> Fine with me. > > This e

Re: submake-errcodes

2018-04-10 Thread Tom Lane
Devrim =?ISO-8859-1?Q?G=FCnd=FCz?= writes: > On Tue, 2018-04-10 at 10:01 -0400, Tom Lane wrote: >> You could replace it with submake-generated-headers, since that's more >> general, but in principle you shouldn't need anything because that >> target is invoked automatically as of yesterday. What'

Re: pgsql: Merge catalog/pg_foo_fn.h headers back into pg_foo.h headers.

2018-04-10 Thread Tom Lane
Julien Rouhaud writes: > On Tue, Apr 10, 2018 at 6:58 PM, Tom Lane wrote: >> none of the three if-guards in the temp-install rule itself should >> prevent this, so what is preventing it? I don't see it. > I just checked, and for the record the second rule (ifneq > ($(abs_top_builddir),) is actu

Re: [sqlsmith] Failed assertion in create_gather_path

2018-04-10 Thread Robert Haas
On Tue, Apr 10, 2018 at 2:59 AM, Jeevan Chalke wrote: > I actually wanted to have rel->consider_parallel in the condition (yes, for > additional safety) as we are adding a partial path into rel. But then > observed that it is same as that of final_rel->consider_parallel and thus > used it along wi

Re: submake-errcodes

2018-04-10 Thread Devrim Gündüz
Hi, On Tue, 2018-04-10 at 10:01 -0400, Tom Lane wrote: > You could replace it with submake-generated-headers, since that's more > general, but in principle you shouldn't need anything because that > target is invoked automatically as of yesterday. What's the larger > context here --- why do you

Re: pgsql: Merge catalog/pg_foo_fn.h headers back into pg_foo.h headers.

2018-04-10 Thread Julien Rouhaud
On Tue, Apr 10, 2018 at 6:58 PM, Tom Lane wrote: > > Well, the question that's bothering me is how come "make check" in > an external build doesn't try to execute the temp-install rule before > printing that error message. Experimentation shows that it doesn't, > but it sure looks to me like it s

Re: Boolean partitions syntax

2018-04-10 Thread Tom Lane
David Rowley writes: > On 11 April 2018 at 03:34, Tom Lane wrote: >> Well, that just begs the question: why do these expressions need to >> be immutable? What we really want, I think, is to evaluate them >> and reduce them to constants. After that, it hardly matters whether >> the original expr

Re: pgsql: Merge catalog/pg_foo_fn.h headers back into pg_foo.h headers.

2018-04-10 Thread Tom Lane
Julien Rouhaud writes: > On Tue, Apr 10, 2018 at 3:46 PM, Tom Lane wrote: >> Hm. I wonder if we don't also want NO_TEMP_INSTALL, like the doc/src/sgml >> makefile. I don't know whether "make check" could be useful in a PGXS >> build, but certainly that recipe for making a temp install isn't gon

Re: PostgreSQL's handling of fsync() errors is unsafe and risks data loss at least on XFS

2018-04-10 Thread Greg Stark
On 10 April 2018 at 02:59, Craig Ringer wrote: > Nitpick: In most cases the kernel reserves disk space immediately, > before returning from write(). NFS seems to be the main exception > here. I'm kind of puzzled by this. Surely NFS servers store the data in the filesystem using write(2) or the i

Re: pg_dump should use current_database() instead of PQdb()

2018-04-10 Thread Tom Lane
Peter Eisentraut writes: > A report from a pgbouncer user revealed that running pg_dump -C/--create > does not work through a connection proxy if the virtual database name on > the proxy does not match the real database name on the database server. > That's because pg_dump looks up the database to

pg_dump should use current_database() instead of PQdb()

2018-04-10 Thread Peter Eisentraut
A report from a pgbouncer user revealed that running pg_dump -C/--create does not work through a connection proxy if the virtual database name on the proxy does not match the real database name on the database server. That's because pg_dump looks up the database to be dumped using the information f

Re: [HACKERS] Moving relation extension locks out of heavyweight lock manager

2018-04-10 Thread Robert Haas
On Tue, Apr 10, 2018 at 5:40 AM, Masahiko Sawada wrote: > The probability of performance degradation can be reduced by > increasing N_RELEXTLOCK_ENTS. But as Robert mentioned, while keeping > fast and simple implementation like acquiring lock by a few atomic > operation it's hard to improve or at

Re: PostgreSQL's handling of fsync() errors is unsafe and risks data loss at least on XFS

2018-04-10 Thread Greg Stark
On 9 April 2018 at 11:50, Anthony Iliopoulos wrote: > On Mon, Apr 09, 2018 at 09:45:40AM +0100, Greg Stark wrote: >> On 8 April 2018 at 22:47, Anthony Iliopoulos wrote: > To make things a bit simpler, let us focus on EIO for the moment. > The contract between the block layer and the filesystem l

Re: Optimization of range queries

2018-04-10 Thread Konstantin Knizhnik
On 09.04.2018 20:05, Teodor Sigaev wrote: Hi! 12 years ago I proposed patch to which could "union" OR clauses into one range clause if it's possible. In that time pgsql could not use IS NULL as index clause, so patch doesn't support that https://www.postgresql.org/message-id/flat/45742C51.

Re: pgsql: Support partition pruning at execution time

2018-04-10 Thread Robert Haas
On Tue, Apr 10, 2018 at 12:29 PM, Alvaro Herrera wrote: > In contrast, in an indexonly scan you have a single counter and it > doesn't really matter the distribution of fetches done by workers, so it > seems okay to aggregate them all in a single counter. And it being so > simple, it seems reason

Re: [PATCH] pg_hba.conf : new auth option : clientcert=verify-full

2018-04-10 Thread Magnus Hagander
On Tue, Apr 10, 2018 at 2:10 PM, Julian Markwort < julian.markw...@uni-muenster.de> wrote: > On Fri, 2018-04-06 at 20:31 +0200, Magnus Hagander wrote: > > I've been through this one again. > > Thanks for taking the time! > > There is one big omission from it -- it fails to work with the view > pg_

Re: Boolean partitions syntax

2018-04-10 Thread David Rowley
On 11 April 2018 at 03:34, Tom Lane wrote: > David Rowley writes: >> I imagined this would have had a check for volatile functions and some >> user-friendly error message to say partition bounds must be immutable, >> but instead, it does: > >> postgres=# create table d_p1 partition of d for value

Re: pgsql: Support partition pruning at execution time

2018-04-10 Thread Alvaro Herrera
Robert Haas wrote: > On Tue, Apr 10, 2018 at 11:14 AM, Alvaro Herrera > wrote: > > Questions: > > 1. Do we want to back-patch this to 10? I suppose (without checking) > > that EXPLAIN ANALYZE is already reporting bogus numbers for parallel > > index-only scans, so I think we should do that. > >

Re: Online enabling of checksums

2018-04-10 Thread Robert Haas
On Fri, Apr 6, 2018 at 8:59 PM, Andres Freund wrote: > This is PROPARALLEL_RESTRICTED. That doesn't strike me right, shouldn't > they be PROPARALLEL_UNSAFE? It might be fine, but I'd not want to rely > on it. Just a fine-grained note on this particular point: It's totally fine for parallel-restr

Re: pgsql: Merge catalog/pg_foo_fn.h headers back into pg_foo.h headers.

2018-04-10 Thread Julien Rouhaud
On Tue, Apr 10, 2018 at 3:46 PM, Tom Lane wrote: > > Julien Rouhaud writes: >> I think the best fix if to define NO_GENERATED_HEADERS in pgxs.mk, >> patch attached. > > Hm. I wonder if we don't also want NO_TEMP_INSTALL, like the doc/src/sgml > makefile. I don't know whether "make check" could

Re: WIP: Covering + unique indexes.

2018-04-10 Thread Teodor Sigaev
* There is no pfree() within _bt_buildadd() for truncated tuples, even though that's a context where it's clearly not okay. Agree * It might be a good idea to also pfree() the truncated tuple for most other _bt_buildadd() callers. Even though it's arguably okay in other cases, it seems worth b

Re: pgsql: Support partition pruning at execution time

2018-04-10 Thread Robert Haas
On Tue, Apr 10, 2018 at 11:14 AM, Alvaro Herrera wrote: > Questions: > 1. Do we want to back-patch this to 10? I suppose (without checking) > that EXPLAIN ANALYZE is already reporting bogus numbers for parallel > index-only scans, so I think we should do that. I haven't looked at this closely, b

Re: pgsql: Support partition pruning at execution time

2018-04-10 Thread David Rowley
On 11 April 2018 at 03:42, Tom Lane wrote: > Alvaro Herrera writes: >> 2. Do we want to revert Andrew's test stabilization patch? If I >> understand correctly, the problem is the inverse of what was diagnosed: >> "any running transaction at the time of the test could prevent pages >> from being

Re: pgsql: Support partition pruning at execution time

2018-04-10 Thread David Rowley
On 11 April 2018 at 03:14, Alvaro Herrera wrote: > 2. Do we want to revert Andrew's test stabilization patch? If I > understand correctly, the problem is the inverse of what was diagnosed: > "any running transaction at the time of the test could prevent pages > from being set as all-visible". Th

Re: pgsql: Support partition pruning at execution time

2018-04-10 Thread Tom Lane
Alvaro Herrera writes: > Questions: > 1. Do we want to back-patch this to 10? I suppose (without checking) > that EXPLAIN ANALYZE is already reporting bogus numbers for parallel > index-only scans, so I think we should do that. You can't back-patch a change in struct Instrumentation; that'd be a

Re: PostgreSQL's handling of fsync() errors is unsafe and risks data loss at least on XFS

2018-04-10 Thread Anthony Iliopoulos
Hi Robert, On Tue, Apr 10, 2018 at 11:15:46AM -0400, Robert Haas wrote: > On Mon, Apr 9, 2018 at 3:13 PM, Andres Freund wrote: > > Let's lower the pitchforks a bit here. Obviously a grand rewrite is > > absurd, as is some of the proposed ways this is all supposed to > > work. But I think the cas

Re: Partitioned tables and covering indexes

2018-04-10 Thread Teodor Sigaev
Does the attached fix look correct?  Haven't checked the fix with ATTACH PARTITION though. Attached patch seems to fix the problem.  However, I would rather get rid of modifying stmt->indexParams.  That seems to be more logical for me.  Also, it would be good to check some covering index

Re: Boolean partitions syntax

2018-04-10 Thread Tom Lane
David Rowley writes: > I imagined this would have had a check for volatile functions and some > user-friendly error message to say partition bounds must be immutable, > but instead, it does: > postgres=# create table d_p1 partition of d for values in (Random()); > ERROR: specified value cannot b

Including SQL files in extension scripts

2018-04-10 Thread Jeremy Finzel
In writing extension update scripts, I find it to be really difficult to grok diffs for example in changed view or function definitions when a new extension script has to include the whole definition in a new file. I want to rather use separate files for these objects, then use something like psql

Re: PostgreSQL's handling of fsync() errors is unsafe and risks data loss at least on XFS

2018-04-10 Thread Robert Haas
On Tue, Apr 10, 2018 at 1:37 AM, Craig Ringer wrote: > ... but *only if they hit an I/O error* or they're on a FS that > doesn't reserve space and hit ENOSPC. > > It still does 99% of the job. It still flushes all buffers to > persistent storage and maintains write ordering. It may not detect and

Re: crash with sql language partition support function

2018-04-10 Thread Tom Lane
Alvaro Herrera writes: > Ashutosh Bapat wrote: >> On Tue, Apr 10, 2018 at 1:44 PM, Amit Langote >> wrote: >>> Attached fixes it. It teaches RelationBuildPartitionKey() to use >>> fmgr_info_cxt and pass rd_partkeycxt to it. >> The patch is using partkeycxt and not rd_partkeycxt. Probably a typo

Re: [PATCH][PROPOSAL] Refuse setting toast.* reloptions when TOAST table does not exist

2018-04-10 Thread David Steele
On 4/10/18 9:17 AM, Alvaro Herrera wrote: > Nikolay Shaplov wrote: > >> But I need some confirmation, in order not to write patch in vain again :-) > > Don't worry, rest assured that you will still write *many* patches in > vain, not just this one. Despite the rather dubious pep talk, Álvaro is

Re: lazy detoasting

2018-04-10 Thread Chapman Flack
On 04/10/2018 10:06 AM, Tom Lane wrote: > Chapman Flack writes: >> Am I right in thinking that, for my original purpose of >> detoasting something later in a transaction, all that matters >> is that I registered a snapshot from the time at which I copied >> the toasted datum, and the resource own

  1   2   >