Re: Local partitioned indexes and pageinspect

2018-04-29 Thread Michael Paquier
On Sun, Apr 29, 2018 at 06:20:02PM -0700, Peter Geoghegan wrote: > What about amcheck? I did change the example query in the docs to > account for this, so anyone that generalizes from that won't have a > problem, but it would be nice if it had a friendlier message. I didn't > change amcheck to acc

Re: Postgres, fsync, and OSs (specifically linux)

2018-04-29 Thread Craig Ringer
Hrm, something else that just came up. On 9.6+ we use sync_file_range. It's surely going to eat errors: rc = sync_file_range(fd, offset, nbytes, SYNC_FILE_RANGE_WRITE); /* don't error out, this is just a performance optimization */ if (rc != 0)

Re: PostgreSQL's handling of fsync() errors is unsafe and risks data loss at least on XFS

2018-04-29 Thread Craig Ringer
For archive readers, this thread is continued as https://www.postgresql.org/message-id/20180427222842.in2e4mibx45zd...@alap3.anarazel.de and there's a follow-up lwn article at https://lwn.net/Articles/752613/ too.

Intermittent ECPG test failures on Windows buildfarm machines

2018-04-29 Thread Tom Lane
Observe the following buildfarm failures: https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=whelk&dt=2018-03-29%2013%3A41%3A13 https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=woodlouse&dt=2018-04-18%2016%3A42%3A03 https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=dory&dt=2018-0

Re: Accounting of zero-filled buffers in EXPLAIN (BUFFERS)

2018-04-29 Thread Thomas Munro
On Mon, Apr 30, 2018 at 3:13 PM, Andres Freund wrote: > On 2018-04-30 14:59:31 +1200, Thomas Munro wrote: >> In EXPLAIN (BUFFERS), there are two kinds of cache misses that show up >> as "reads" when in fact they are not reads at all: >> >> 1. Relation extension, which in fact writes a zero-filled

Re: Accounting of zero-filled buffers in EXPLAIN (BUFFERS)

2018-04-29 Thread Andres Freund
On 2018-04-30 14:59:31 +1200, Thomas Munro wrote: > Hi, > > In EXPLAIN (BUFFERS), there are two kinds of cache misses that show up > as "reads" when in fact they are not reads at all: > > 1. Relation extension, which in fact writes a zero-filled block. > 2. The RBM_ZERO_* modes, which provoke n

Accounting of zero-filled buffers in EXPLAIN (BUFFERS)

2018-04-29 Thread Thomas Munro
Hi, In EXPLAIN (BUFFERS), there are two kinds of cache misses that show up as "reads" when in fact they are not reads at all: 1. Relation extension, which in fact writes a zero-filled block. 2. The RBM_ZERO_* modes, which provoke neither read nor write. Here's a suggested fix. I noticed this

Re: Postgres, fsync, and OSs (specifically linux)

2018-04-29 Thread Craig Ringer
> Not quite sure what you're getting at with "a file we don't fsync" - if > we don't, we don't care about durability anyway, no? Or do you mean > where we fsync in a different process? Right. > Either way, the answer is mostly no: On NFS et al where close() implies > an fsync you'll get the error

Re: Postgres, fsync, and OSs (specifically linux)

2018-04-29 Thread Andres Freund
On 2018-04-30 10:14:23 +0800, Craig Ringer wrote: > Meanwhile, do we know if, on Linux 4.13+, if we get a buffered write > error due to dirty writeback before we close() a file we don't > fsync(), we'll get the error on close()? Not quite sure what you're getting at with "a file we don't fsync" -

Re: Postgres, fsync, and OSs (specifically linux)

2018-04-29 Thread Craig Ringer
On 30 April 2018 at 09:09, Thomas Munro wrote: > Considering the variety in interpretation and liberties taken, I > wonder if fsync() is underspecified and someone should file an issue > over at http://www.opengroup.org/austin/ about that. All it's going to achieve is adding an "is implementatio

Re: Is a modern build system acceptable for older platforms

2018-04-29 Thread Yuriy Zhuravlev
> > this is not about having a working build environment, it is about having > a fully working and correct source tarball before distributing it as a > new release. Sorry, I did not understand correctly it before. I suppose it's not big problem especial if you have CI and tests farm. And anyway i

Re: Is a modern build system acceptable for older platforms

2018-04-29 Thread Yuriy Zhuravlev
> > Which is nice > OK Again, nice > Yeah, that's nice I already use CMake I'd do it, personally > I suppose we have no one silver bullet reason to change autoconf+make to CMake but it's cumulative impression. Also, I suppose this thread started to resolve at least one small question.

Re: Local partitioned indexes and pageinspect

2018-04-29 Thread Peter Geoghegan
On Mon, Apr 23, 2018 at 9:58 PM, Michael Paquier wrote: > Hm, the docs about taking backups with the low-level APIs don't care > much about relkind now: > https://www.postgresql.org/docs/devel/static/continuous-archiving.html#BACKUP-LOWLEVEL-BASE-BACKUP > Or do you have another section in the docs

Re: Postgres, fsync, and OSs (specifically linux)

2018-04-29 Thread Thomas Munro
On Sun, Apr 29, 2018 at 1:58 PM, Craig Ringer wrote: > On 28 April 2018 at 23:25, Simon Riggs wrote: >> On 27 April 2018 at 15:28, Andres Freund wrote: >>> While I'm a bit concerned adding user-code before a checkpoint, if >>> we'd do it as a shell command it seems pretty reasonable. And use

Re: Postgres, fsync, and OSs (specifically linux)

2018-04-29 Thread Thomas Munro
On Mon, Apr 30, 2018 at 11:02 AM, Thomas Munro wrote: > MySQL: The default is still buffered Someone pulled me up on this off-list: the default is buffered (fsync) on Unix, but it's unbuffered on Windows. That's quite interesting. https://dev.mysql.com/doc/refman/8.0/en/innodb-parameters.html#s

Re: power() function in Windows: "value out of range: underflow"

2018-04-29 Thread David Rowley
On 30 April 2018 at 10:22, Tom Lane wrote: > David Rowley writes: >> Wouldn't this machine have returned 1 before this patch though? > > No, don't think so, because it doesn't set EDOM for the case. > > Basically what we're doing here is making sure that we get results > conforming to current POS

Re: Cold welcoming message when installing anything because of LLVM bitcode stuff

2018-04-29 Thread Michael Paquier
On Sun, Apr 29, 2018 at 03:49:53PM -0700, Andres Freund wrote: > Isn't that what bc19b7836215b1a847524041a1bd138d7bb5cbef did? Oops, sorry. Missed this one. -- Michael signature.asc Description: PGP signature

Re: Postgres, fsync, and OSs (specifically linux)

2018-04-29 Thread Thomas Munro
On Sun, Apr 29, 2018 at 10:42 PM, Simon Riggs wrote: > On 28 April 2018 at 09:15, Andres Freund wrote: >> On 2018-04-28 08:25:53 -0700, Simon Riggs wrote: >>> The people I've spoken to so far have encouraged us to continue >>> working with the filesystem layer, offering encouragement of our >>> d

Re: Cold welcoming message when installing anything because of LLVM bitcode stuff

2018-04-29 Thread Andres Freund
On 2018-04-30 07:43:46 +0900, Michael Paquier wrote: > On Sat, Apr 28, 2018 at 02:32:06PM -0400, Tom Lane wrote: > > Ugh. I'd tried the core code but not contrib. Will look. > > The thread has stalled a bit. Huh? It's a weekend. The last message was yesterday afternoon. > What are the other i

Re: Cold welcoming message when installing anything because of LLVM bitcode stuff

2018-04-29 Thread Michael Paquier
On Sat, Apr 28, 2018 at 02:32:06PM -0400, Tom Lane wrote: > Ugh. I'd tried the core code but not contrib. Will look. The thread has stalled a bit. What are the other issues you were seeing? Are those related to --with-llvm? On my side, if I just apply something like the attached I am able to

Re: power() function in Windows: "value out of range: underflow"

2018-04-29 Thread Tom Lane
David Rowley writes: > Wouldn't this machine have returned 1 before this patch though? No, don't think so, because it doesn't set EDOM for the case. Basically what we're doing here is making sure that we get results conforming to current POSIX even on machines that predate that standard. There

Re: power() function in Windows: "value out of range: underflow"

2018-04-29 Thread David Rowley
On 30 April 2018 at 09:19, Tom Lane wrote: > http://netbsd.gw.com/cgi-bin/man-cgi?pow+3+NetBSD-5.2.3 > > which goes to great lengths to justify NaN^0 = 1 while saying nothing > to suggest that 1^NaN might not yield NaN. > > I'm not sure if we should add more special-case code for that, or just > r

Re: power() function in Windows: "value out of range: underflow"

2018-04-29 Thread Tom Lane
David Rowley writes: > I wonder if it's better just to hard code these two cases before even > calling the pow() function. Yeah, see my followup --- I also found out that SUSv2 (POSIX 1997) doesn't require either of these special cases, which helps explain why the inconsistency on older platforms

Re: power() function in Windows: "value out of range: underflow"

2018-04-29 Thread David Rowley
On 30 April 2018 at 07:24, Tom Lane wrote: > Pushed. I'd mainly note that you need to update all the variant float8 > expected-files, not just the primary one. (Sometimes this requires a > bit of guesswork, but here we're expecting all platforms to produce > the same result. The buildfarm shoul

Re: power() function in Windows: "value out of range: underflow"

2018-04-29 Thread Tom Lane
David Rowley writes: > You patch fixes the problem for me, and importantly the two following > cases still work: > postgres=# select power(1,'NaN'); > power > --- > 1 > (1 row) > postgres=# select power('NaN', 0); > power > --- > 1 > (1 row) > There's no mention in the SQL s

Re: [HACKERS] Clock with Adaptive Replacement

2018-04-29 Thread Andres Freund
On 2018-04-25 11:31:12 +0500, Andrey Borodin wrote: > > > > 24 апр. 2018 г., в 23:14, Andres Freund написал(а): > > > > On 2018-04-24 17:16:47 +0500, Andrey Borodin wrote: > >> But, I think that cost of development of real page eviction strategy > >> itself is neglectable small compared to infr

Re: [HACKERS] Clock with Adaptive Replacement

2018-04-29 Thread Юрий Соколов
вс, 29 апр. 2018 г., 2:17 Peter Geoghegan : > On Sat, Apr 28, 2018 at 7:39 AM, Stephen Frost wrote: > > Perhaps I'm misunderstanding the case here, but with the ring buffer we > > use for sequential access, aren't we already discouraging keeping heap > > pages around, particularly when they're co

Re: Implementing SQL ASSERTION

2018-04-29 Thread David Fetter
On Sun, Apr 29, 2018 at 07:18:00PM +0100, Joe Wildish wrote: > On 28 Mar 2018, at 16:13, David Fetter wrote: > > > > Sorry to bother you again, but this now doesn't compile atop master. > > Attached is a rebased patch for the prototype. Thanks! This is great timing for the 12 cycle :) Best, D

Re: power() function in Windows: "value out of range: underflow"

2018-04-29 Thread Tom Lane
Huong Dangminh writes: >> 2018-04-11 0:13 GMT-03:00 David Rowley : >>> I can recreate this when building with MSVC 2012. I confirm that I see >>> the same as you. Microsoft are setting errno to EDOM in the above 3 >>> cases, where in Linux the result is still NaN, just the errno is not >>> set. >

Re: Transform for pl/perl

2018-04-29 Thread Tom Lane
Peter Eisentraut writes: > On 4/24/18 14:31, Andrew Dunstan wrote: >> There is the routine IsValidJsonNumber that helps - see among others >> hstore_io.c for an example use. > I would need something like that taking a double/float8 input. But I > think there is no such shortcut available, so I j

Re: Explain buffers wrong counter with parallel plans

2018-04-29 Thread Amit Kapila
On Fri, Apr 20, 2018 at 6:36 PM, Adrien Nayrat wrote: > Hello, > > I tried to understand this issue and it seems Gather node only take account of > this own buffer usage : > > > create unlogged table t1 (c1 int); > insert into t1 select generate_series(1,100); > vacuum t1; > > explain (analyze

Re: Explain buffers wrong counter with parallel plans

2018-04-29 Thread Amit Kapila
On Wed, Mar 28, 2018 at 12:07 AM, Adrien Nayrat wrote: > Hello, > > I notice Explain report wrong counters with parallel plans : > > create unlogged table t1 (c1 int); > insert into t1 select generate_series(1,1); > vacuum t1; > > (stop PG, drop caches,start) > > set track_io_timing to on;

Re: Toast issues with OldestXmin going backwards

2018-04-29 Thread Amit Kapila
On Sun, Apr 29, 2018 at 11:56 AM, Andrew Gierth wrote: >> "Amit" == Amit Kapila writes: > > >>> (Or do we need to track it across restarts? maybe we do, to deal with > >>> replication slaves without slots, or changes in parameters) > > >> Yeah, I'm worried that it might need to be persiste

Re: Verbosity of genbki.pl

2018-04-29 Thread Michael Paquier
On Sat, Apr 28, 2018 at 01:02:10PM -0400, Stephen Frost wrote: > +1 for making them not output anything if all is well. +1. -- Michael signature.asc Description: PGP signature

Re: Postgres, fsync, and OSs (specifically linux)

2018-04-29 Thread Simon Riggs
On 28 April 2018 at 09:15, Andres Freund wrote: > Hi, > > On 2018-04-28 08:25:53 -0700, Simon Riggs wrote: >> > - Use direct IO. Due to architectural performance issues in PG and the >> > fact that it'd not be applicable for all installations I don't think >> > this is a reasonable fix for the

Re: Postgres, fsync, and OSs (specifically linux)

2018-04-29 Thread Simon Riggs
On 28 April 2018 at 08:25, Simon Riggs wrote: > On 27 April 2018 at 15:28, Andres Freund wrote: > >> - Add a pre-checkpoint hook that checks for filesystem errors *after* >> fsyncing all the files, but *before* logging the checkpoint completion >> record. Operating systems, filesystems, etc.