Re: found xmin from before relfrozenxid on pg_catalog.pg_authid

2018-08-09 Thread Bruce Momjian
On Wed, Aug 8, 2018 at 10:27:59AM -0700, Peter Geoghegan wrote: > On Wed, Aug 8, 2018 at 10:23 AM, Kyle Samson wrote: > > We found the exact same issue on a production database at TripAdvisor. We > > have no new information to add for debugging. Bumping this to up the > > priority on a patch ve

Re: found xmin from before relfrozenxid on pg_catalog.pg_authid

2018-08-08 Thread Peter Geoghegan
On Wed, Aug 8, 2018 at 10:23 AM, Kyle Samson wrote: > We found the exact same issue on a production database at TripAdvisor. We > have no new information to add for debugging. Bumping this to up the priority > on a patch version release getting out. There is a batch of point releases that were

Re: found xmin from before relfrozenxid on pg_catalog.pg_authid

2018-08-08 Thread Kyle Samson
Hello, We found the exact same issue on a production database at TripAdvisor. We have no new information to add for debugging. Bumping this to up the priority on a patch version release getting out. Thanks, -Kyle Samson

Re: found xmin from before relfrozenxid on pg_catalog.pg_authid

2018-06-26 Thread Jerry Sievers
Hackers, felt like reporting this relevant tidbit in case of interest... My site is among the few who've hit this bug. We observed recently a case of pg_database having joined pg_authid and pg_auth_members in the bad xmin, unable to vacuum shcatalog group, however... pg_database then started wor

Re: found xmin from before relfrozenxid on pg_catalog.pg_authid

2018-06-22 Thread Matheus de Oliveira
Hello again... On Tue, Jun 19, 2018 at 12:53 PM, Andres Freund wrote: > ... > > After that, yes, deleting the > global/pg_internal.init file is the way to go, and I can't think of a > case where it's problematic, even without stopping the server. > > Just to let you know. I applied the work arou

Re: found xmin from before relfrozenxid on pg_catalog.pg_authid

2018-06-20 Thread Andres Freund
Hi, On 2018-06-20 15:05:59 +0300, Sergey Burladyan wrote: > Andres Freund writes: > > > You should first make sure it's actually this problem - which tables are > > holding back the xmin horizon? After that, yes, deleting the > > global/pg_internal.init file is the way to go, and I can't think

Re: found xmin from before relfrozenxid on pg_catalog.pg_authid

2018-06-19 Thread Andres Freund
Hi, On 2018-06-19 17:05:48 -0300, Matheus de Oliveira wrote: > > You should first make sure it's actually this problem - which tables are > > holding back the xmin horizon? > > > How can I be sure? The tables are `pg_authid` and `pg_auth_members`, the > following message is logged every minute (

Re: found xmin from before relfrozenxid on pg_catalog.pg_authid

2018-06-19 Thread Matheus de Oliveira
On Tue, Jun 19, 2018 at 12:53 PM, Andres Freund wrote: > Hi, > > On 2018-06-19 10:26:26 -0300, Matheus de Oliveira wrote: > > Hello friends. > > > > On Tue, Jun 12, 2018 at 3:31 PM, Andres Freund > wrote: > > > > > > > > On 2018-06-11 17:39:14 -0700, Andres Freund wrote: > > > > I plan to go ove

Re: found xmin from before relfrozenxid on pg_catalog.pg_authid

2018-06-19 Thread Andres Freund
Hi, On 2018-06-19 08:25:33 -0500, Jeremy Finzel wrote: > I have a question related to this - and specifically, preventing the error > until we have a patch :). The patch has been committed to postgres, although no release has been made including it yet. > We are encountering this error every fe

Re: found xmin from before relfrozenxid on pg_catalog.pg_authid

2018-06-19 Thread Andres Freund
Hi, On 2018-06-19 10:26:26 -0300, Matheus de Oliveira wrote: > Hello friends. > > On Tue, Jun 12, 2018 at 3:31 PM, Andres Freund wrote: > > > > > On 2018-06-11 17:39:14 -0700, Andres Freund wrote: > > > I plan to go over the change again tomorrow, and then push. Unless > > > somebody has commen

Re: found xmin from before relfrozenxid on pg_catalog.pg_authid

2018-06-19 Thread Jeremy Finzel
On Tue, Jun 19, 2018 at 8:26 AM Matheus de Oliveira < matioli.math...@gmail.com> wrote: > Hello friends. > > On Tue, Jun 12, 2018 at 3:31 PM, Andres Freund wrote: > >> >> On 2018-06-11 17:39:14 -0700, Andres Freund wrote: >> > I plan to go over the change again tomorrow, and then push. Unless >>

Re: found xmin from before relfrozenxid on pg_catalog.pg_authid

2018-06-19 Thread Matheus de Oliveira
Hello friends. On Tue, Jun 12, 2018 at 3:31 PM, Andres Freund wrote: > > On 2018-06-11 17:39:14 -0700, Andres Freund wrote: > > I plan to go over the change again tomorrow, and then push. Unless > > somebody has comments before then, obviously. > > Done. > > Sorry to bother about this, but do yo

Re: found xmin from before relfrozenxid on pg_catalog.pg_authid

2018-06-19 Thread Jeremy Finzel
On Fri, May 25, 2018 at 3:37 PM, Andres Freund wrote: > Hi, > > Moving discussion to -hackers. Tom, I think you worked most with this > code, your input would be appreciated. > > Original discussion is around: > http://archives.postgresql.org/message-id/20180524211311. > tnswfnjwnii54htx%40alvhe

Re: found xmin from before relfrozenxid on pg_catalog.pg_authid

2018-06-12 Thread Andres Freund
Hi, On 2018-06-11 17:39:14 -0700, Andres Freund wrote: > I plan to go over the change again tomorrow, and then push. Unless > somebody has comments before then, obviously. Done. - Andres

Re: found xmin from before relfrozenxid on pg_catalog.pg_authid

2018-06-11 Thread Andres Freund
Hi, On 2018-05-28 12:52:06 -0700, Andres Freund wrote: > Hi, > > On 2018-05-27 13:00:06 -0700, Andres Freund wrote: > > I've a patch that seems to work, that mostly needs some comment > > polishing. > > Attached is what I currently have. Still needs some more work, but I > think it's more than g

Re: found xmin from before relfrozenxid on pg_catalog.pg_authid

2018-06-11 Thread Andres Freund
Hi, On 2018-05-29 19:14:51 -0400, Alvaro Herrera wrote: > I added an Assert(DatabasePath != NULL) to > RelationCacheInitFilePreInvalidate() and didn't see a single crash when > running the tests. I thought that adding a "VACUUM FREEZE pg_class" > in the recovery tests (where there is a standby) o

Re: found xmin from before relfrozenxid on pg_catalog.pg_authid

2018-05-29 Thread Alvaro Herrera
On 2018-May-29, Alvaro Herrera wrote: > I added an Assert(DatabasePath != NULL) to > RelationCacheInitFilePreInvalidate() and didn't see a single crash when > running the tests. I thought that adding a "VACUUM FREEZE pg_class" > in the recovery tests (where there is a standby) ought to do it, but

Re: found xmin from before relfrozenxid on pg_catalog.pg_authid

2018-05-29 Thread Alvaro Herrera
I added an Assert(DatabasePath != NULL) to RelationCacheInitFilePreInvalidate() and didn't see a single crash when running the tests. I thought that adding a "VACUUM FREEZE pg_class" in the recovery tests (where there is a standby) ought to do it, but it doesn't. What's the deal there? Here are

Re: found xmin from before relfrozenxid on pg_catalog.pg_authid

2018-05-29 Thread Andres Freund
On 2018-05-29 18:06:12 +, Nishant, Fnu wrote: > Hi, > > > To achieve this we can allocate Form_pg_class structures (for shared > > relations… a small number) on shared memory. > > But why would this be necessary / a good idea? Even if we decided it > were, it seems like it

Re: found xmin from before relfrozenxid on pg_catalog.pg_authid

2018-05-29 Thread Nishant, Fnu
Hi, > To achieve this we can allocate Form_pg_class structures (for shared > relations… a small number) on shared memory. But why would this be necessary / a good idea? Even if we decided it were, it seems like it'd end up being quite invasive. But I doubt it's a good pla

Re: found xmin from before relfrozenxid on pg_catalog.pg_authid

2018-05-28 Thread Andres Freund
Hi, On 2018-05-27 13:00:06 -0700, Andres Freund wrote: > I've a patch that seems to work, that mostly needs some comment > polishing. Attached is what I currently have. Still needs some more work, but I think it's more than good enough to review the approach. Basically the approach consists out

Re: found xmin from before relfrozenxid on pg_catalog.pg_authid

2018-05-28 Thread Andres Freund
Hi, (please don't top post) On 2018-05-28 15:07:52 +, Nishant, Fnu wrote: > We were working on this issue and thinking if we could actually make > pg_class(rd_rel) part of recache entry upgradable. Right, that's necessary. See the patch I just sent. > To achieve this we can allocate Form_p

Re: found xmin from before relfrozenxid on pg_catalog.pg_authid

2018-05-28 Thread Nishant, Fnu
Hi, We were working on this issue and thinking if we could actually make pg_class(rd_rel) part of recache entry upgradable. To achieve this we can allocate Form_pg_class structures (for shared relations… a small number) on shared memory. We do not need global pg_internal_init file as new backend

Re: found xmin from before relfrozenxid on pg_catalog.pg_authid

2018-05-27 Thread Andres Freund
Hi, On 2018-05-27 13:22:21 -0400, Tom Lane wrote: > But I don't think there's any need for special magic here: we just > have to accept the fact that there's a need to flush that cache > sometimes. In normal use it shouldn't happen often enough to be a > performance problem. Yea, it's not that p

Re: found xmin from before relfrozenxid on pg_catalog.pg_authid

2018-05-27 Thread Tom Lane
Andres Freund writes: > On May 27, 2018 9:39:49 AM PDT, "Nasby, Jim" wrote: >> How about only keeping the critical info for being able to find >> relations in the .init files, and then fully populate the cache by >> doing a normal lookup? > Then the cache wouldn't have any benefits, no? It's bee

Re: found xmin from before relfrozenxid on pg_catalog.pg_authid

2018-05-27 Thread Andres Freund
On May 27, 2018 9:39:49 AM PDT, "Nasby, Jim" wrote: >On May 26, 2018, at 1:45 PM, Andres Freund wrote: >> Does anybody see a way to not have to remove the .init file? > >How about only keeping the critical info for being able to find >relations in the .init files, and then fully populate the ca

Re: found xmin from before relfrozenxid on pg_catalog.pg_authid

2018-05-27 Thread Nasby, Jim
On May 26, 2018, at 1:45 PM, Andres Freund wrote: > Does anybody see a way to not have to remove the .init file? How about only keeping the critical info for being able to find relations in the .init files, and then fully populate the cache by doing a normal lookup? Since there’s multiple entri

Re: found xmin from before relfrozenxid on pg_catalog.pg_authid

2018-05-26 Thread Andres Freund
On 2018-05-26 13:45:06 -0700, Andres Freund wrote: > On 2018-05-25 15:05:31 -0700, Andres Freund wrote: > > On 2018-05-25 17:47:37 -0400, Tom Lane wrote: > > > For nailed indexes, we allow updating of some additional fields, and I > > > guess what has to happen here is that we teach the code to upd

Re: found xmin from before relfrozenxid on pg_catalog.pg_authid

2018-05-26 Thread Andres Freund
On 2018-05-25 15:05:31 -0700, Andres Freund wrote: > On 2018-05-25 17:47:37 -0400, Tom Lane wrote: > > For nailed indexes, we allow updating of some additional fields, and I > > guess what has to happen here is that we teach the code to update some > > additional fields for nailed tables too. > >

Re: found xmin from before relfrozenxid on pg_catalog.pg_authid

2018-05-25 Thread Andres Freund
On 2018-05-25 17:47:37 -0400, Tom Lane wrote: > Andres Freund writes: > > Moving discussion to -hackers. Tom, I think you worked most with this > > code, your input would be appreciated. > > Yeah, the assumption in the relcache is that the only part of a nailed > catalog's relcache entry that re

Re: found xmin from before relfrozenxid on pg_catalog.pg_authid

2018-05-25 Thread Tom Lane
Andres Freund writes: > Moving discussion to -hackers. Tom, I think you worked most with this > code, your input would be appreciated. Yeah, the assumption in the relcache is that the only part of a nailed catalog's relcache entry that really needs to be updated intrasession is the relfilenode m

Re: found xmin from before relfrozenxid on pg_catalog.pg_authid

2018-05-25 Thread Andres Freund
Hi, Moving discussion to -hackers. Tom, I think you worked most with this code, your input would be appreciated. Original discussion is around: http://archives.postgresql.org/message-id/20180524211311.tnswfnjwnii54htx%40alvherre.pgsql On 2018-05-24 17:13:11 -0400, Alvaro Herrera wrote: > On 201