Re: Bug in reindexdb's error reporting

On Fri, May 10, 2019 at 09:25:58PM -0400, Tom Lane wrote: > Michael Paquier writes: > > The refactoring bits are fine for HEAD. For back-branches I would > > suggest using the simplest patch of upthread. > > Makes sense to me too. The refactoring is mostly to make future > additions easier, so

Re: pg12 release notes

On Fri, May 10, 2019 at 7:11 PM Bruce Momjian wrote: > > Whether or not you include more details is not what I care about here > > -- I *agree* that this is insignificant. > Well, we can move the entire item up to the incompatibility section, but > that seems unbalanced since the

Re: pg12 release notes

On Fri, May 10, 2019 at 06:53:55PM -0700, Peter Geoghegan wrote: > On Fri, May 10, 2019 at 6:02 PM Bruce Momjian wrote: > > Have new btree indexes sort duplicate index entries in heap-storage > > order (Peter Geoghegan) > > > > This slightly reduces the maximum-allowed

Re: pg12 release notes

> "Bruce" == Bruce Momjian writes: >> I noticed that the compatibility note for Andrew Gierth's RYU >> floating point patch seems to simply say why the feature is useful. >> Shouldn't it be listed separately, and its impact on users upgrading >> be listed here instead? Bruce> The text

Re: pg12 release notes

On Fri, May 10, 2019 at 6:02 PM Bruce Momjian wrote: > Have new btree indexes sort duplicate index entries in heap-storage > order (Peter Geoghegan) > > This slightly reduces the maximum-allowed length of indexed values. >

Re: Bug in reindexdb's error reporting

Michael Paquier writes: > The refactoring bits are fine for HEAD. For back-branches I would > suggest using the simplest patch of upthread. Makes sense to me too. The refactoring is mostly to make future additions easier, so there's not much point for back branches. > That's perhaps too much

Re: pg12 release notes

On Fri, May 10, 2019 at 03:34:15PM +1200, David Rowley wrote: > On Fri, 10 May 2019 at 13:03, Bruce Momjian wrote: > > > > On Thu, May 9, 2019 at 07:13:35PM -0500, Justin Pryzby wrote: > > > On Thu, May 09, 2019 at 07:35:18PM -0400, Bruce Momjian wrote: > > > > I have made your other changes,

Re: pg12 release notes

On Thu, May 9, 2019 at 07:10:43PM -0700, Peter Geoghegan wrote: > On Thu, May 9, 2019 at 6:03 PM Bruce Momjian wrote: > > These were all very helpful. I adjusted your changes to create the > > attached applied patch. URL updated: > > I noticed that the compatibility note for Andrew Gierth's

Re: Bug in reindexdb's error reporting

On Fri, May 10, 2019 at 05:58:03PM +0200, Julien Rouhaud wrote: > On Fri, May 10, 2019 at 5:33 PM Alvaro Herrera > wrote: >> I did have the same thought. It seem clear now that we should do it :-) >> ISTM that the way to fix that problem is to use the proposed enum >> everywhere and turn it

Re: pg12 release notes

On Thu, May 9, 2019 at 08:34:49PM -0500, Justin Pryzby wrote: > On Thu, May 09, 2019 at 09:02:51PM -0400, Bruce Momjian wrote: > > These were all very helpful. I adjusted your changes to create the > > attached applied patch. URL updated: > > Thanks. > > > -Allow more comparisons with

Re: Adding a test for speculative insert abort case

Hi, On 2019-05-01 11:41:48 -0700, Andres Freund wrote: > I'm imagining something like > > if (pg_try_advisory_xact_lock(1)) > pg_advisory_xact_lock(2); > else > pg_advisory_xact_lock(1); > > in t1_lock_func. If you then make the session something roughly like > > s1:

Re: Why is infinite_recurse test suddenly failing?

On 5/10/19 3:35 PM, Tom Lane wrote: > Andres Freund writes: >> On 2019-05-10 11:38:57 -0400, Tom Lane wrote: >>> I am wondering if, somehow, the stack depth limit seen by the postmaster >>> sometimes doesn't apply to its children. That would be pretty wacko >>> kernel behavior, especially if

Re: Problems with pg_upgrade and extensions referencing catalog tables/views

> On May 9, 2019, at 7:14 PM, Bruce Momjian wrote: > > On Wed, May 8, 2019 at 10:07:23PM +, Nasby, Jim wrote: >> I don’t recall why pg_upgrade wants to control OIDs… don’t we >> re-create all catalog entries for user objects from scratch? > > The C comment at top of pg_upgrade.c explains

Re: Unexpected "shared memory block is still in use"

I wrote: > Will go fix/backpatch in a minute. Done now, but while thinking more about the issue, I had an idea: why is it that we base the shmem key on the postmaster's port number, and not on the data directory's inode number? Using the port number not only increases the risk of collisions

Re: REINDEX INDEX results in a crash for an index of pg_class since 9.6

Andres Freund writes: > On 2019-05-07 09:17:11 -0700, Andres Freund wrote: >> Well, it rejiggers the way table locks are acquired for all REINDEX >> INDEX commands, not just in the CONCURRENTLY. But yea, it's probably >> easy to catch issues there on user tables. > Pushed now. OK. I marked the

Re: Inconsistency between table am callback and table function names

Alvaro Herrera writes: > On 2019-May-10, Andres Freund wrote: >> My personal opinion is that this is more churn than I think is useful to >> tackle after feature freeze, with not sufficient benefits. If others >> chime in, voting to do this, I'm OK with doing that, but otherwise I >> think

Re: Inconsistency between table am callback and table function names

Hi, On 2019-05-10 16:18:32 -0400, Alvaro Herrera wrote: > On 2019-May-10, Andres Freund wrote: > > > My personal opinion is that this is more churn than I think is useful to > > tackle after feature freeze, with not sufficient benefits. If others > > chime in, voting to do this, I'm OK with

Re: Inconsistency between table am callback and table function names

On 2019-May-10, Andres Freund wrote: > My personal opinion is that this is more churn than I think is useful to > tackle after feature freeze, with not sufficient benefits. If others > chime in, voting to do this, I'm OK with doing that, but otherwise I > think there's more important stuff to

Re: REINDEX INDEX results in a crash for an index of pg_class since 9.6

Hi, On 2019-05-07 09:17:11 -0700, Andres Freund wrote: > Hi, > > On 2019-05-07 12:14:43 -0400, Tom Lane wrote: > > Andres Freund writes: > > > On 2019-05-07 12:07:37 -0400, Tom Lane wrote: > > >> The number of deadlock failures is kind of annoying, so I'd rather remove > > >> the tests from

Re: What's the point of allow_system_table_mods?

Hi, On 2019-05-10 15:48:49 -0400, Tom Lane wrote: > Andres Freund writes: > > On 2019-05-10 15:00:18 -0400, Tom Lane wrote: > >> What exactly is the motivation for changing this now, after 20 years? > > > That I've seen enough corruption and other hard to investigate issues > > related to

Re: Inconsistency between table am callback and table function names

Hi, On 2019-05-10 12:43:06 -0700, Ashwin Agrawal wrote: > On Fri, May 10, 2019 at 10:51 AM Andres Freund wrote: > > > > Hi, > > > > On 2019-05-10 10:43:44 -0700, Ashwin Agrawal wrote: > > > On Thu, May 9, 2019 at 8:52 AM Andres Freund wrote: > > > > The changes necessary for tableam were

Re: What's the point of allow_system_table_mods?

Andres Freund writes: > On 2019-05-10 15:00:18 -0400, Tom Lane wrote: >> What exactly is the motivation for changing this now, after 20 years? > That I've seen enough corruption and other hard to investigate issues > related to manual catalog modifications to make me complain. Note that > other

Re: Inconsistency between table am callback and table function names

On Fri, May 10, 2019 at 10:51 AM Andres Freund wrote: > > Hi, > > On 2019-05-10 10:43:44 -0700, Ashwin Agrawal wrote: > > On Thu, May 9, 2019 at 8:52 AM Andres Freund wrote: > > > The changes necessary for tableam were already huge. Changing naming > > > schemes for functions that are used all

Re: Why is infinite_recurse test suddenly failing?

Andres Freund writes: > On 2019-05-10 11:38:57 -0400, Tom Lane wrote: >> I am wondering if, somehow, the stack depth limit seen by the postmaster >> sometimes doesn't apply to its children. That would be pretty wacko >> kernel behavior, especially if it's only intermittently true. >> But we're

Re: What's the point of allow_system_table_mods?

Hi, On 2019-05-10 15:00:18 -0400, Tom Lane wrote: > What exactly is the motivation for changing this now, after 20 years? That I've seen enough corruption and other hard to investigate issues related to manual catalog modifications to make me complain. Note that other have complained about this

Re: What's the point of allow_system_table_mods?

Hi, On 2019-05-10 19:51:10 +0100, Andrew Gierth wrote: > > "Andres" == Andres Freund writes: > > Andres> Why is it so much more dangerous? I've seen plenty of corrupted > Andres> clusters due to people doing DML against the catalogs. I'm OK > Andres> with adding separate GUCs for both,

Re: What's the point of allow_system_table_mods?

Andrew Gierth writes: > "Andres" == Andres Freund writes: > Andres> Why is it so much more dangerous? I've seen plenty of corrupted > Andres> clusters due to people doing DML against the catalogs. I'm OK > Andres> with adding separate GUCs for both, if we want to do that, but > Andres> I do

Re: What's the point of allow_system_table_mods?

> "Andres" == Andres Freund writes: Andres> Why is it so much more dangerous? I've seen plenty of corrupted Andres> clusters due to people doing DML against the catalogs. I'm OK Andres> with adding separate GUCs for both, if we want to do that, but Andres> I do think we shouldn't allow

Re: Why is infinite_recurse test suddenly failing?

Hi, On 2019-05-10 11:38:57 -0400, Tom Lane wrote: > Core was generated by `postgres: debian regression [local] SELECT > '. > Program terminated with signal SIGSEGV, Segmentation fault. > #0 sysmalloc (nb=8208, av=0x3fff916e0d28 ) at malloc.c:2748 > 2748

Re: What's the point of allow_system_table_mods?

Hi, On 2019-05-09 12:22:39 -0400, Tom Lane wrote: > Andres Freund writes: > > I'm not quite clear what the goal of allow_system_table_mods > > is. Obviously, it's extremely dangerous to target catalogs with DDL. But > > at the same time we allow DML to catalog tables without any sort of > >

Re: Inconsistency between table am callback and table function names

Hi, On 2019-05-10 10:43:44 -0700, Ashwin Agrawal wrote: > On Thu, May 9, 2019 at 8:52 AM Andres Freund wrote: > > The changes necessary for tableam were already huge. Changing naming > > schemes for functions that are used all over the backend (e.g. ~80 calls > > to table_beginscan), and where

Re: Inconsistency between table am callback and table function names

On Thu, May 9, 2019 at 8:52 AM Andres Freund wrote: > The changes necessary for tableam were already huge. Changing naming > schemes for functions that are used all over the backend (e.g. ~80 calls > to table_beginscan), and where there's other wrapper functions that also > widely used (237 calls

Re: Bug in reindexdb's error reporting

On Fri, May 10, 2019 at 5:33 PM Alvaro Herrera wrote: > > On 2019-May-10, Julien Rouhaud wrote: > > > On Fri, May 10, 2019 at 4:43 PM Tom Lane wrote: > > > > Patch is good as far as it goes, but I wonder if it'd be smarter to > > > convert the function's "type" argument from a string to an enum,

Re: Why is infinite_recurse test suddenly failing?

We just got another one of these, on yet another member of Mark's ppc64 armada: https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=shoveler=2019-05-10%2014%3A04%3A34 Now we have a stack trace (thanks Mark!), but it is pretty unsurprising: Core was generated by `postgres: debian regression

Re: Bug in reindexdb's error reporting

On 2019-May-10, Julien Rouhaud wrote: > On Fri, May 10, 2019 at 4:43 PM Tom Lane wrote: > > Patch is good as far as it goes, but I wonder if it'd be smarter to > > convert the function's "type" argument from a string to an enum, > > and then replace the if/else chains with switches? > > I've

Re: Bug in reindexdb's error reporting

Alvaro Herrera writes: > On 2019-May-10, Julien Rouhaud wrote: >> I just noticed that reindexdb could report an extraneous message >> saying an error happened while reindexing a database if it failed >> reindexing a table or an index. > Kudos, good find -- that's a 14 years old bug, introduced

Re: att_isnull

Alvaro Herrera writes: > Yeah, I've noticed this inconsistency too. I doubt we want to change > the macro definition or its name, but +1 for expanding the comment. > Your proposed wording seems sufficient. +1 >> There is some kind of broader confusion here, I think, because we >> refer in many

Re: att_isnull

On 2019-May-10, Robert Haas wrote: > Obviously, this macro does not do what it claims to do: > > /* > * check to see if the ATT'th bit of an array of 8-bit bytes is set. > */ > #define att_isnull(ATT, BITS) (!((BITS)[(ATT) >> 3] & (1 << ((ATT) & 0x07 > > OK, I lied. It's not at all

att_isnull

Obviously, this macro does not do what it claims to do: /* * check to see if the ATT'th bit of an array of 8-bit bytes is set. */ #define att_isnull(ATT, BITS) (!((BITS)[(ATT) >> 3] & (1 << ((ATT) & 0x07 OK, I lied. It's not at all obvious, or at least it wasn't to me. The macro actually

Re: Bug in reindexdb's error reporting

On 2019-May-10, Julien Rouhaud wrote: > I just noticed that reindexdb could report an extraneous message > saying an error happened while reindexing a database if it failed > reindexing a table or an index. Kudos, good find -- that's a 14 years old bug, introduced in this commit: Author: Bruce

Re: pg12 release notes

Alvaro Herrera writes: > Still, it's an inconsistency in pg11. I vote -0 to getting it > backpatched, mostly because it seems like more work than is warranted. > (I think the work consists not only of testing that the backpatched > commit works correctly, but also documenting for 11.4 release

Re: pg12 release notes

On 2019-May-10, Amit Langote wrote: > On 2019/05/10 12:18, David Rowley wrote: > > On Fri, 10 May 2019 at 12:08, Bruce Momjian wrote: > >>> 17f206f Set pg_class.relhassubclass for partitioned indexes > >> > >> I need help with this one. I know the system column existed in previous > >>

Re: vacuumdb and new VACUUM options

On Thu, May 9, 2019 at 1:39 PM Kyotaro HORIGUCHI wrote: > > At Thu, 9 May 2019 20:14:51 +0900, Masahiko Sawada > wrote in > > On Thu, May 9, 2019 at 10:01 AM Michael Paquier wrote: > > > > > > On Wed, May 08, 2019 at 06:21:09PM -0300, Euler Taveira wrote: > > > > Em qua, 8 de mai de 2019 às

Re: Fwd: Add tablespace tap test to pg_rewind

On Thu, May 09, 2019 at 02:36:48PM +0900, Michael Paquier wrote: > Yes, I can see that. The issue is that even if we do a backup with > --tablespace-mapping then we still need a tweak to allow the copy of > symlinks. I am not sure that this is completely what we are looking > for either, as it

Re: POC: Cleaning up orphaned files using undo logs

Hello Thomas, In pg_buffercache contrib module, the file pg_buffercache--1.3--1.4.sql is missing. AFAICS, this file should be added as part of the following commit: Add SmgrId to smgropen() and BufferTag

Re: Bug in reindexdb's error reporting

> On 10 May 2019, at 12:24, Michael Paquier wrote: > > On Fri, May 10, 2019 at 11:02:52AM +0200, Julien Rouhaud wrote: >> I just noticed that reindexdb could report an extraneous message >> saying an error happened while reindexing a database if it failed >> reindexing a table or an index. >>

Re: Bug in reindexdb's error reporting

On Fri, May 10, 2019 at 11:02:52AM +0200, Julien Rouhaud wrote: > I just noticed that reindexdb could report an extraneous message > saying an error happened while reindexing a database if it failed > reindexing a table or an index. > > Trivial fix attached. Oops. That's true, nice catch. This

Multivariate MCV stats can leak data to unprivileged users

While working on 1aebfbea83c, I noticed that the new multivariate MCV stats feature suffers from the same problem, and also the original problems that were fixed in e2d4ef8de8 and earlier --- namely that a user can see values in the MCV lists that they shouldn't see (values from tables that they

Bug in reindexdb's error reporting

Hi, I just noticed that reindexdb could report an extraneous message saying an error happened while reindexing a database if it failed reindexing a table or an index. Trivial fix attached. diff --git a/src/bin/scripts/reindexdb.c b/src/bin/scripts/reindexdb.c index d6f3efd313..897ad9a71a 100644

Re: [HACKERS] Weaker shmem interlock w/o postmaster.pid

On Wed, May 08, 2019 at 10:10:35PM +1200, Thomas Munro wrote: > On Mon, May 6, 2019 at 3:30 PM Thomas Munro wrote: > > I put the second sentence back and tweaked it thus: s/fails/might > > fail/. Maybe I'm being too pedantic here, but binding to 127.0.0.2 > > works on other OSes too, as long as

Re: Unexpected "shared memory block is still in use"

On Thu, May 09, 2019 at 06:47:58PM -0400, Tom Lane wrote: > I wrote: > > However, I have a new theory, after noticing that c09850992 moved the > > check for shm_nattch == 0. Previously, if a shmem segment had zero attach > > count, it was unconditionally considered not-a-threat. Now, we'll try >