RE: integrate Postgres Users Authentication with our own LDAP Server

2019-05-09 Thread M Tarkeshwar Rao
We want to setup ldap authentication in pg_hba.conf, for Postgresql users(other than postgres super user). We are getting issue with special characters by following steps given in postgres documentation. It is not accepting any special characters as special characters are mandatory in our us

Re: Fuzzy thinking in is_publishable_class

2019-05-09 Thread Peter Eisentraut
On 2019-05-09 04:37, Tom Lane wrote: > I tried removing the FirstNormalObjectId check, and found that the > reason for it seems to be "the subscription/t/004_sync.pl test > falls over without it". That's because that test supposes that > the *only* entry in pg_subscription_rel will be for the test

Re: We're leaking predicate locks in HEAD

2019-05-09 Thread Thomas Munro
On Wed, May 8, 2019 at 4:50 PM Thomas Munro wrote: > On Wed, May 8, 2019 at 3:53 PM Tom Lane wrote: > > Thomas Munro writes: > > > Reproduced here. Once the system reaches a state where it's leaking > > > (which happens only occasionally for me during installcheck-parallel), > > > it keeps leak

Re: [Proposal] Table-level Transparent Data Encryption (TDE) and Key Management Service (KMS)

2019-05-09 Thread Masahiko Sawada
On Wed, May 8, 2019 at 10:32 PM Robert Haas wrote: > > On Tue, May 7, 2019 at 2:10 AM Masahiko Sawada wrote: > > > > That better not be true. If you have a design where reading the WAL > > > > lets you get *any* encryption key, you have a bad design, I think. > > > > How does the startup process

Re: pgsql: Add strict_multi_assignment and too_many_rows plpgsql checks

2019-05-09 Thread Pavel Stehule
Hi čt 9. 5. 2019 v 8:52 odesílatel Peter Eisentraut < peter.eisentr...@2ndquadrant.com> napsal: > On 2018-07-25 01:47, Tomas Vondra wrote: > > Add strict_multi_assignment and too_many_rows plpgsql checks > > Could you clarify this error message: > > /* translator: %s represents a name of an e

Re: postgres_fdw: another oddity in costing aggregate pushdown paths

2019-05-09 Thread Etsuro Fujita
On Wed, May 8, 2019 at 12:45 PM Etsuro Fujita wrote: > This doesn't get applied cleanly after commit 1d33858406. Here is a > rebased version of the patch. I also modified the comments a little > bit. If there are no objections from Antonin or anyone else, I'll > commit the patch. Pushed. Than

Re: Fixing order of resowner cleanup in 12, for Windows

2019-05-09 Thread Thomas Munro
On Tue, May 7, 2019 at 6:08 AM Robert Haas wrote: > On Mon, May 6, 2019 at 1:58 PM Tom Lane wrote: > > Robert Haas writes: > > > Right. That's why I favor applying the change to move DSM cleanup to > > > the end for now, and seeing how that goes. It could be that we'll > > > eventually discove

Re: Missing FDW documentation about GetForeignUpperPaths

2019-05-09 Thread Etsuro Fujita
On Wed, May 8, 2019 at 3:51 PM Etsuro Fujita wrote: > In commit d50d172e51, which adds support for FINAL relation pushdown > in postgres_fdw, I forgot to update the FDW documentation about > GetForeignUpperPaths to mention that the extra parameter of that > function points to a FinalPathExtraData

Re: vacuumdb and new VACUUM options

2019-05-09 Thread Masahiko Sawada
On Thu, May 9, 2019 at 10:01 AM Michael Paquier wrote: > > On Wed, May 08, 2019 at 06:21:09PM -0300, Euler Taveira wrote: > > Em qua, 8 de mai de 2019 às 14:19, Fujii Masao > > escreveu: > >> The question is; we should support vacuumdb option for (1), i.e.,, > >> something like --index-cleanup o

Re: vacuumdb and new VACUUM options

2019-05-09 Thread Kyotaro HORIGUCHI
At Thu, 9 May 2019 20:14:51 +0900, Masahiko Sawada wrote in > On Thu, May 9, 2019 at 10:01 AM Michael Paquier wrote: > > > > On Wed, May 08, 2019 at 06:21:09PM -0300, Euler Taveira wrote: > > > Em qua, 8 de mai de 2019 às 14:19, Fujii Masao > > > escreveu: > > >> The question is; we should su

Re: Fuzzy thinking in is_publishable_class

2019-05-09 Thread Petr Jelinek
On 09/05/2019 04:37, Tom Lane wrote: > I wrote: >> is_publishable_class has a test "relid >= FirstNormalObjectId", >> which I think we should drop, for two reasons: >> ... >> So what is the motivation for this test? If there's an important >> reason for it, we need to find a less fragile way to

Re: Patch to document base64 encoding

2019-05-09 Thread Karl O. Pinc
On Thu, 9 May 2019 06:50:12 +0200 (CEST) Fabien COELHO wrote: > You might consider reviewing other people patches, that is expected > to make the overall process work. There are several documentation or > comment patches in the queue. Understood. I thought I had built up some reviewing credit,

Re: any suggestions to detect memory corruption

2019-05-09 Thread Tom Lane
Alex writes: > Someone add some code during backend init which used palloc. but at that > time, the CurrentMemoryContext is PostmasterContext. at the end of > backend initialization, the PostmasterContext is deleted, then the error > happens. the reason why it happens randomly is before the pa

Re: Fuzzy thinking in is_publishable_class

2019-05-09 Thread Tom Lane
Peter Eisentraut writes: > On 2019-05-09 04:37, Tom Lane wrote: >> But, if what we want is the definition that "information_schema is >> excluded from publishable tables", I'm not satisfied with this >> implementation of that rule. > ... It would also make sense to integrate both of these concept

Re: Unexpected "shared memory block is still in use"

2019-05-09 Thread Tom Lane
Noah Misch writes: > On Wed, May 08, 2019 at 02:32:46PM -0400, Tom Lane wrote: >> Just now, while running a parallel check-world on HEAD according to the >> same script I've been using for quite some time, one of the TAP tests >> died during initdb: >> performing post-bootstrap initialization ...

Re: Fuzzy thinking in is_publishable_class

2019-05-09 Thread Tom Lane
Petr Jelinek writes: > I think we need a better way of identifying what's part of system and > what's user created in general. The FirstNormalObjectId seems somewhat > okay approximation, but then we have plenty of other ways for checking, > maybe it's time to consolidate it into some extra column

What's the point of allow_system_table_mods?

2019-05-09 Thread Andres Freund
Hi, I'm not quite clear what the goal of allow_system_table_mods is. Obviously, it's extremely dangerous to target catalogs with DDL. But at the same time we allow DML to catalog tables without any sort of restriction. I also don't understand what's achieved by having allow_system_table_mods be P

Re: Inconsistency between table am callback and table function names

2019-05-09 Thread Andres Freund
Hi, On 2019-05-08 17:05:07 -0700, Ashwin Agrawal wrote: > On Wed, May 8, 2019 at 2:51 PM Andres Freund wrote: > > On 2019-05-08 00:32:22 -0700, Ashwin Agrawal wrote: > > > The general theme for table function names seem to be > > > "table_". For example table_scan_getnextslot() and its > > > corr

Re: Fuzzy thinking in is_publishable_class

2019-05-09 Thread Andres Freund
Hi, On 2019-05-09 09:30:50 +0200, Peter Eisentraut wrote: > On 2019-05-09 04:37, Tom Lane wrote: > > I'm of two minds what to do about that. One approach is to just > > define a "FOR ALL TABLES" publication as including the information_schema > > tables, > > certainly not Yea, that strikes me a

Re: What's the point of allow_system_table_mods?

2019-05-09 Thread Tom Lane
Andres Freund writes: > I'm not quite clear what the goal of allow_system_table_mods > is. Obviously, it's extremely dangerous to target catalogs with DDL. But > at the same time we allow DML to catalog tables without any sort of > restriction. The last is not true, see pg_class_aclmask(). > I a

Re: [Proposal] Table-level Transparent Data Encryption (TDE) and Key Management Service (KMS)

2019-05-09 Thread Bruce Momjian
On Wed, May 8, 2019 at 09:32:08AM -0400, Robert Haas wrote: > On Tue, May 7, 2019 at 2:10 AM Masahiko Sawada wrote: > > > > That better not be true. If you have a design where reading the WAL > > > > lets you get *any* encryption key, you have a bad design, I think. > > > > How does the startup

Re: PG12, PGXS and linking pgfeutils

2019-05-09 Thread Tom Lane
I wrote: > Ian Barwick writes: >> Commit cc8d4151 [*] introduced a dependency between some functions in >> libpgcommon and libpgfeutils, > This seems rather seriously broken. I do not think the answer is to > create a global dependency on libpgfeutils. Or, to be clearer: fe_utils has had depend

Re: PG12, PGXS and linking pgfeutils

2019-05-09 Thread Alvaro Herrera
On 2019-May-09, Tom Lane wrote: > I'm not especially on board with the idea of moving FE-specific error > handling code into libpgcommon, as that breaks the concept that > src/common/ is broadly for code that can work in either frontend or > backend contexts. However, we already have a few violat

Re: [Proposal] Table-level Transparent Data Encryption (TDE) and Key Management Service (KMS)

2019-05-09 Thread Bruce Momjian
On Thu, May 9, 2019 at 05:49:12PM +0900, Masahiko Sawada wrote: > In terms of keys, one advantage could be that we have less keys with > per-tablespace keys. > > Let me briefly explain the current design I'm thinking. The design > employees 2-tier key architecture. That is, a database cluster has

Re: PG12, PGXS and linking pgfeutils

2019-05-09 Thread Tom Lane
Alvaro Herrera writes: > On 2019-May-09, Tom Lane wrote: >> I'm not especially on board with the idea of moving FE-specific error >> handling code into libpgcommon, as that breaks the concept that >> src/common/ is broadly for code that can work in either frontend or >> backend contexts. However,

Re: Inconsistent error message wording for REINDEX CONCURRENTLY

2019-05-09 Thread Tom Lane
Michael Paquier writes: > On Wed, May 08, 2019 at 11:28:35PM -0400, Tom Lane wrote: >> Pushed my stuff, have at it. > Thanks. Attached is what I get to after scanning the error messages > in indexcmds.c and index.c. Perhaps you have more comments about it? LGTM, thanks.

Re: Pluggable Storage - Andres's take

2019-05-09 Thread Ashwin Agrawal
On Wed, May 8, 2019 at 2:46 PM Andres Freund wrote: > > Hi, > > On 2019-05-07 23:18:39 -0700, Ashwin Agrawal wrote: > > On Mon, May 6, 2019 at 1:39 PM Ashwin Agrawal wrote: > > > Also wish to point out, while working on Zedstore, we realized that > > > TupleDesc from Relation object can be truste

Re: reloption to prevent VACUUM from truncating empty pages at the end of relation

2019-05-09 Thread Bruce Momjian
On Wed, May 8, 2019 at 02:14:04AM +0900, Fujii Masao wrote: > On Tue, May 7, 2019 at 5:33 PM Masahiko Sawada wrote: > > Sorry for the late. I've reviewed the patch and it looks good to me. > > Thanks for the review! I committed the patch. Great. I noticed from the release notes that it was odd

Re: pg12 release notes

2019-05-09 Thread Thomas Munro
On Thu, May 9, 2019 at 8:32 AM Justin Pryzby wrote: > I noticed you added release notes at bdf595adbca195fa54a909c74a5233ebc30641a1, > thanks for writing them. +1 > I reviewed notes; find proposed changes attached+included. +1 to all the corrections shown. One more: "Allow parallel query when

Re: Unexpected "shared memory block is still in use"

2019-05-09 Thread Tom Lane
I wrote: > Noah Misch writes: >> The odds are very high that you would not have gotten that error before that >> commit. But if the cause matches your guess, it's not something wrong with >> the commit ... > Fair point. After more study and testing, I no longer believe my original thought about

Re: _bt_split(), and the risk of OOM before its critical section

2019-05-09 Thread Peter Geoghegan
On Wed, May 8, 2019 at 3:37 PM Peter Geoghegan wrote: > It makes perfect sense for _bt_split() to call _bt_findsplitloc() > directly, since _bt_findsplitloc() is already aware of almost every > _bt_split() implementation detail, whereas those same details are not > of interest anywhere else. I di

Re: Unexpected "shared memory block is still in use"

2019-05-09 Thread Tom Lane
I wrote: > However, I have a new theory, after noticing that c09850992 moved the > check for shm_nattch == 0. Previously, if a shmem segment had zero attach > count, it was unconditionally considered not-a-threat. Now, we'll try > shmat() anyway, and if that fails for any reason other than EACCES

Re: Inconsistent error message wording for REINDEX CONCURRENTLY

2019-05-09 Thread Michael Paquier
On Thu, May 09, 2019 at 02:08:39PM -0400, Tom Lane wrote: > LGTM, thanks. Thanks for double-checking, committed. I am closing the open item. -- Michael signature.asc Description: PGP signature

Re: pg12 release notes

2019-05-09 Thread Bruce Momjian
On Wed, May 8, 2019 at 03:32:04PM -0500, Justin Pryzby wrote: > I noticed you added release notes at bdf595adbca195fa54a909c74a5233ebc30641a1, > thanks for writing them. > -This improve optimization for columns with non-uniform distributions that > often appear in WHERE clauses. > +This improves

Re: pg12 release notes

2019-05-09 Thread Bruce Momjian
On Wed, May 8, 2019 at 03:32:04PM -0500, Justin Pryzby wrote: > I noticed you added release notes at bdf595adbca195fa54a909c74a5233ebc30641a1, > thanks for writing them. > > I reviewed notes; find proposed changes attached+included. > > I think these should also be mentioned? > > f7cb284 Add pl

Re: pg12 release notes

2019-05-09 Thread Justin Pryzby
On Thu, May 09, 2019 at 08:08:53PM -0400, Bruce Momjian wrote: > On Wed, May 8, 2019 at 03:32:04PM -0500, Justin Pryzby wrote: > > I think these should also be mentioned? > > > > f7cb284 Add plan_cache_mode setting > > 387a5cf Add pg_dump --on-conflict-do-nothing option. > > Applied patch attach

Re: pg12 release notes

2019-05-09 Thread Justin Pryzby
On Thu, May 09, 2019 at 07:35:18PM -0400, Bruce Momjian wrote: > I have made your other changes, with adjustment, patch attached. > > The results are here: > > http://momjian.us/pgsql_docs/release-12.html Thanks > -Many system tables now have an 'oid' column that will be expanded with > -

Re: Problems with pg_upgrade and extensions referencing catalog tables/views

2019-05-09 Thread Bruce Momjian
On Wed, May 8, 2019 at 10:07:23PM +, Nasby, Jim wrote: > I don’t recall why pg_upgrade wants to control OIDs… don’t we > re-create all catalog entries for user objects from scratch? The C comment at top of pg_upgrade.c explains why some oids must be preserved: https://git.postgresql

Re: pg12 release notes

2019-05-09 Thread Bruce Momjian
On Thu, May 9, 2019 at 07:19:59PM -0500, Justin Pryzby wrote: > > +Author: Thomas Munro > > +2018-07-13 [387a5cfb9] Add pg_dump - -on-conflict-do-nothing option. > > +--> > > + > > + > > +Allow restore of INSERT statements to skip rows which would cause > > conflicts (Surafel Temesgen) > > + >

Re: copy-past-o comment in lock.h

2019-05-09 Thread Michael Paquier
On Wed, May 08, 2019 at 05:03:31PM +0900, Michael Paquier wrote: > Good idea to move the comments so what you proposes looks fine to me. > Are there any objections? Okay, committed. -- Michael signature.asc Description: PGP signature

Re: any suggestions to detect memory corruption

2019-05-09 Thread Alex
On Thu, May 9, 2019 at 9:30 PM Tom Lane wrote: > Alex writes: > > Someone add some code during backend init which used palloc. but at that > > time, the CurrentMemoryContext is PostmasterContext. at the end of > > backend initialization, the PostmasterContext is deleted, then the error > > ha

Re: pg12 release notes

2019-05-09 Thread Bruce Momjian
On Thu, May 9, 2019 at 07:13:35PM -0500, Justin Pryzby wrote: > On Thu, May 09, 2019 at 07:35:18PM -0400, Bruce Momjian wrote: > > I have made your other changes, with adjustment, patch attached. > > > > The results are here: > > > > http://momjian.us/pgsql_docs/release-12.html > > Thanks

Re: pg12 release notes

2019-05-09 Thread Justin Pryzby
On Thu, May 09, 2019 at 09:02:51PM -0400, Bruce Momjian wrote: > These were all very helpful. I adjusted your changes to create the > attached applied patch. URL updated: Thanks. > -Allow more comparisons with information_schema text columns to use indexes > (Tom Lane) > +Allow more use of ind

Re: pg12 release notes

2019-05-09 Thread Peter Geoghegan
On Thu, May 9, 2019 at 6:03 PM Bruce Momjian wrote: > These were all very helpful. I adjusted your changes to create the > attached applied patch. URL updated: I noticed that the compatibility note for Andrew Gierth's RYU floating point patch seems to simply say why the feature is useful. Shoul

Re: pg12 release notes

2019-05-09 Thread David Rowley
On Fri, 10 May 2019 at 12:08, Bruce Momjian wrote: > > 17f206f Set pg_class.relhassubclass for partitioned indexes > > I need help with this one. I know the system column existed in previous > releases, so how is it different now? Do we document system table > changes that are implementation-beh

Re: pg12 release notes

2019-05-09 Thread David Rowley
On Fri, 10 May 2019 at 13:03, Bruce Momjian wrote: > > On Thu, May 9, 2019 at 07:13:35PM -0500, Justin Pryzby wrote: > > On Thu, May 09, 2019 at 07:35:18PM -0400, Bruce Momjian wrote: > > > I have made your other changes, with adjustment, patch attached. > > > > > > The results are here: > > > >

Re: pg12 release notes

2019-05-09 Thread Tom Lane
David Rowley writes: > Just a question about the item: "Allow IN comparisons with arrays to > use IS NOT NULL partial indexes more frequently (Tom Lane)" > From what I can tell this must refer to 65ce07e0202f. You can tell for sure by looking into the SGML comments in release-12.sgml: Allow

Re: pg12 release notes

2019-05-09 Thread Amit Langote
On 2019/05/10 12:18, David Rowley wrote: > On Fri, 10 May 2019 at 12:08, Bruce Momjian wrote: >>> 17f206f Set pg_class.relhassubclass for partitioned indexes >> >> I need help with this one. I know the system column existed in previous >> releases, so how is it different now? Do we document syst

Re: pg12 release notes

2019-05-09 Thread David Rowley
On Fri, 10 May 2019 at 15:45, Tom Lane wrote: > > > > Allow IS NOT NULL with mis-matching types to use partial indexes more > frequently (Tom Lane) > > > > If so, I think James Coleman should be the author. > > ... and yeah, James should get the credit. But there's more wrong with > the summa

Re: pg12 release notes

2019-05-09 Thread Tom Lane
David Rowley writes: > I think you might be mixing up two items. I'm talking about: > > Allow IN comparisons with arrays to use IS NOT NULL > partial indexes more frequently (Tom Lane) > > to which the sgml references 65ce07e02. Wups, sorry, I was talking about 65ce07e02 also, but I managed t

Re: pg12 release notes

2019-05-09 Thread Justin Pryzby
On Thu, May 09, 2019 at 08:08:53PM -0400, Bruce Momjian wrote: > On Wed, May 8, 2019 at 03:32:04PM -0500, Justin Pryzby wrote: > > I noticed you added release notes at > > bdf595adbca195fa54a909c74a5233ebc30641a1, > > thanks for writing them. > > > > I reviewed notes; find proposed changes attac

Re: pg12 release notes

2019-05-09 Thread Justin Pryzby
On Fri, May 10, 2019 at 12:45:54PM +0900, Amit Langote wrote: > On 2019/05/10 12:18, David Rowley wrote: > > On Fri, 10 May 2019 at 12:08, Bruce Momjian wrote: > >>> 17f206f Set pg_class.relhassubclass for partitioned indexes > >> > >> I need help with this one. I know the system column existed i

Re: pg12 release notes

2019-05-09 Thread David Rowley
On Fri, 10 May 2019 at 16:52, Justin Pryzby wrote: > I'm rechecking my list from last month. What about these ? > > > c076f3d Remove pg_constraint.conincluding > > bd09503 Increase the default vacuum_cost_limit from 200 to 2000 bd09503 was reverted in 52985e4fea7 and replaced by cbccac371, which

Re: pg12 release notes

2019-05-09 Thread David Rowley
On Fri, 10 May 2019 at 16:57, Justin Pryzby wrote: > > 8edd0e7 Suppress Append and MergeAppend plan nodes that have a single child. You could say that I'm biased, but I think this should get a mention. It's not just a matter of tidying up the plan by getting rid of nodes that are not requires, it

Re: pg12 release notes

2019-05-09 Thread Amit Langote
Hi David, On 2019/05/10 14:51, David Rowley wrote: > On Fri, 10 May 2019 at 16:57, Justin Pryzby wrote: >>> 959d00e Use Append rather than MergeAppend for scanning ordered partitions. > > I also think this is worth a mention. The speedup can be quite large > when the query involves a LIMIT clau

RE: integrate Postgres Users Authentication with our own LDAP Server

2019-05-09 Thread M Tarkeshwar Rao
We want to setup ldap authentication in pg_hba.conf, for Postgresql users(other than postgres super user). We are getting issue with special characters by following steps given in postgres documentation. It is not accepting any special characters as special characters are mandatory in our us

Re: Fuzzy thinking in is_publishable_class

2019-05-09 Thread Peter Eisentraut
On 2019-05-09 04:37, Tom Lane wrote: > I tried removing the FirstNormalObjectId check, and found that the > reason for it seems to be "the subscription/t/004_sync.pl test > falls over without it". That's because that test supposes that > the *only* entry in pg_subscription_rel will be for the test

Re: We're leaking predicate locks in HEAD

2019-05-09 Thread Thomas Munro
On Wed, May 8, 2019 at 4:50 PM Thomas Munro wrote: > On Wed, May 8, 2019 at 3:53 PM Tom Lane wrote: > > Thomas Munro writes: > > > Reproduced here. Once the system reaches a state where it's leaking > > > (which happens only occasionally for me during installcheck-parallel), > > > it keeps leak

Re: [Proposal] Table-level Transparent Data Encryption (TDE) and Key Management Service (KMS)

2019-05-09 Thread Masahiko Sawada
On Wed, May 8, 2019 at 10:32 PM Robert Haas wrote: > > On Tue, May 7, 2019 at 2:10 AM Masahiko Sawada wrote: > > > > That better not be true. If you have a design where reading the WAL > > > > lets you get *any* encryption key, you have a bad design, I think. > > > > How does the startup process

Re: pgsql: Add strict_multi_assignment and too_many_rows plpgsql checks

2019-05-09 Thread Pavel Stehule
Hi čt 9. 5. 2019 v 8:52 odesílatel Peter Eisentraut < peter.eisentr...@2ndquadrant.com> napsal: > On 2018-07-25 01:47, Tomas Vondra wrote: > > Add strict_multi_assignment and too_many_rows plpgsql checks > > Could you clarify this error message: > > /* translator: %s represents a name of an e

Re: postgres_fdw: another oddity in costing aggregate pushdown paths

2019-05-09 Thread Etsuro Fujita
On Wed, May 8, 2019 at 12:45 PM Etsuro Fujita wrote: > This doesn't get applied cleanly after commit 1d33858406. Here is a > rebased version of the patch. I also modified the comments a little > bit. If there are no objections from Antonin or anyone else, I'll > commit the patch. Pushed. Than

Re: Fixing order of resowner cleanup in 12, for Windows

2019-05-09 Thread Thomas Munro
On Tue, May 7, 2019 at 6:08 AM Robert Haas wrote: > On Mon, May 6, 2019 at 1:58 PM Tom Lane wrote: > > Robert Haas writes: > > > Right. That's why I favor applying the change to move DSM cleanup to > > > the end for now, and seeing how that goes. It could be that we'll > > > eventually discove

Re: Missing FDW documentation about GetForeignUpperPaths

2019-05-09 Thread Etsuro Fujita
On Wed, May 8, 2019 at 3:51 PM Etsuro Fujita wrote: > In commit d50d172e51, which adds support for FINAL relation pushdown > in postgres_fdw, I forgot to update the FDW documentation about > GetForeignUpperPaths to mention that the extra parameter of that > function points to a FinalPathExtraData

Re: vacuumdb and new VACUUM options

2019-05-09 Thread Masahiko Sawada
On Thu, May 9, 2019 at 10:01 AM Michael Paquier wrote: > > On Wed, May 08, 2019 at 06:21:09PM -0300, Euler Taveira wrote: > > Em qua, 8 de mai de 2019 às 14:19, Fujii Masao > > escreveu: > >> The question is; we should support vacuumdb option for (1), i.e.,, > >> something like --index-cleanup o

Re: vacuumdb and new VACUUM options

2019-05-09 Thread Kyotaro HORIGUCHI
At Thu, 9 May 2019 20:14:51 +0900, Masahiko Sawada wrote in > On Thu, May 9, 2019 at 10:01 AM Michael Paquier wrote: > > > > On Wed, May 08, 2019 at 06:21:09PM -0300, Euler Taveira wrote: > > > Em qua, 8 de mai de 2019 às 14:19, Fujii Masao > > > escreveu: > > >> The question is; we should su

Re: Fuzzy thinking in is_publishable_class

2019-05-09 Thread Petr Jelinek
On 09/05/2019 04:37, Tom Lane wrote: > I wrote: >> is_publishable_class has a test "relid >= FirstNormalObjectId", >> which I think we should drop, for two reasons: >> ... >> So what is the motivation for this test? If there's an important >> reason for it, we need to find a less fragile way to

Re: Patch to document base64 encoding

2019-05-09 Thread Karl O. Pinc
On Thu, 9 May 2019 06:50:12 +0200 (CEST) Fabien COELHO wrote: > You might consider reviewing other people patches, that is expected > to make the overall process work. There are several documentation or > comment patches in the queue. Understood. I thought I had built up some reviewing credit,

Re: any suggestions to detect memory corruption

2019-05-09 Thread Tom Lane
Alex writes: > Someone add some code during backend init which used palloc. but at that > time, the CurrentMemoryContext is PostmasterContext. at the end of > backend initialization, the PostmasterContext is deleted, then the error > happens. the reason why it happens randomly is before the pa

Re: Fuzzy thinking in is_publishable_class

2019-05-09 Thread Tom Lane
Peter Eisentraut writes: > On 2019-05-09 04:37, Tom Lane wrote: >> But, if what we want is the definition that "information_schema is >> excluded from publishable tables", I'm not satisfied with this >> implementation of that rule. > ... It would also make sense to integrate both of these concept

Re: Unexpected "shared memory block is still in use"

2019-05-09 Thread Tom Lane
Noah Misch writes: > On Wed, May 08, 2019 at 02:32:46PM -0400, Tom Lane wrote: >> Just now, while running a parallel check-world on HEAD according to the >> same script I've been using for quite some time, one of the TAP tests >> died during initdb: >> performing post-bootstrap initialization ...

Re: Fuzzy thinking in is_publishable_class

2019-05-09 Thread Tom Lane
Petr Jelinek writes: > I think we need a better way of identifying what's part of system and > what's user created in general. The FirstNormalObjectId seems somewhat > okay approximation, but then we have plenty of other ways for checking, > maybe it's time to consolidate it into some extra column

What's the point of allow_system_table_mods?

2019-05-09 Thread Andres Freund
Hi, I'm not quite clear what the goal of allow_system_table_mods is. Obviously, it's extremely dangerous to target catalogs with DDL. But at the same time we allow DML to catalog tables without any sort of restriction. I also don't understand what's achieved by having allow_system_table_mods be P

Re: Inconsistency between table am callback and table function names

2019-05-09 Thread Andres Freund
Hi, On 2019-05-08 17:05:07 -0700, Ashwin Agrawal wrote: > On Wed, May 8, 2019 at 2:51 PM Andres Freund wrote: > > On 2019-05-08 00:32:22 -0700, Ashwin Agrawal wrote: > > > The general theme for table function names seem to be > > > "table_". For example table_scan_getnextslot() and its > > > corr

Re: Fuzzy thinking in is_publishable_class

2019-05-09 Thread Andres Freund
Hi, On 2019-05-09 09:30:50 +0200, Peter Eisentraut wrote: > On 2019-05-09 04:37, Tom Lane wrote: > > I'm of two minds what to do about that. One approach is to just > > define a "FOR ALL TABLES" publication as including the information_schema > > tables, > > certainly not Yea, that strikes me a

Re: What's the point of allow_system_table_mods?

2019-05-09 Thread Tom Lane
Andres Freund writes: > I'm not quite clear what the goal of allow_system_table_mods > is. Obviously, it's extremely dangerous to target catalogs with DDL. But > at the same time we allow DML to catalog tables without any sort of > restriction. The last is not true, see pg_class_aclmask(). > I a

Re: [Proposal] Table-level Transparent Data Encryption (TDE) and Key Management Service (KMS)

2019-05-09 Thread Bruce Momjian
On Wed, May 8, 2019 at 09:32:08AM -0400, Robert Haas wrote: > On Tue, May 7, 2019 at 2:10 AM Masahiko Sawada wrote: > > > > That better not be true. If you have a design where reading the WAL > > > > lets you get *any* encryption key, you have a bad design, I think. > > > > How does the startup

Re: PG12, PGXS and linking pgfeutils

2019-05-09 Thread Tom Lane
I wrote: > Ian Barwick writes: >> Commit cc8d4151 [*] introduced a dependency between some functions in >> libpgcommon and libpgfeutils, > This seems rather seriously broken. I do not think the answer is to > create a global dependency on libpgfeutils. Or, to be clearer: fe_utils has had depend

Re: PG12, PGXS and linking pgfeutils

2019-05-09 Thread Alvaro Herrera
On 2019-May-09, Tom Lane wrote: > I'm not especially on board with the idea of moving FE-specific error > handling code into libpgcommon, as that breaks the concept that > src/common/ is broadly for code that can work in either frontend or > backend contexts. However, we already have a few violat

Re: [Proposal] Table-level Transparent Data Encryption (TDE) and Key Management Service (KMS)

2019-05-09 Thread Bruce Momjian
On Thu, May 9, 2019 at 05:49:12PM +0900, Masahiko Sawada wrote: > In terms of keys, one advantage could be that we have less keys with > per-tablespace keys. > > Let me briefly explain the current design I'm thinking. The design > employees 2-tier key architecture. That is, a database cluster has

Re: PG12, PGXS and linking pgfeutils

2019-05-09 Thread Tom Lane
Alvaro Herrera writes: > On 2019-May-09, Tom Lane wrote: >> I'm not especially on board with the idea of moving FE-specific error >> handling code into libpgcommon, as that breaks the concept that >> src/common/ is broadly for code that can work in either frontend or >> backend contexts. However,

Re: Inconsistent error message wording for REINDEX CONCURRENTLY

2019-05-09 Thread Tom Lane
Michael Paquier writes: > On Wed, May 08, 2019 at 11:28:35PM -0400, Tom Lane wrote: >> Pushed my stuff, have at it. > Thanks. Attached is what I get to after scanning the error messages > in indexcmds.c and index.c. Perhaps you have more comments about it? LGTM, thanks.

Re: Pluggable Storage - Andres's take

2019-05-09 Thread Ashwin Agrawal
On Wed, May 8, 2019 at 2:46 PM Andres Freund wrote: > > Hi, > > On 2019-05-07 23:18:39 -0700, Ashwin Agrawal wrote: > > On Mon, May 6, 2019 at 1:39 PM Ashwin Agrawal wrote: > > > Also wish to point out, while working on Zedstore, we realized that > > > TupleDesc from Relation object can be truste

Re: reloption to prevent VACUUM from truncating empty pages at the end of relation

2019-05-09 Thread Bruce Momjian
On Wed, May 8, 2019 at 02:14:04AM +0900, Fujii Masao wrote: > On Tue, May 7, 2019 at 5:33 PM Masahiko Sawada wrote: > > Sorry for the late. I've reviewed the patch and it looks good to me. > > Thanks for the review! I committed the patch. Great. I noticed from the release notes that it was odd

Re: pg12 release notes

2019-05-09 Thread Thomas Munro
On Thu, May 9, 2019 at 8:32 AM Justin Pryzby wrote: > I noticed you added release notes at bdf595adbca195fa54a909c74a5233ebc30641a1, > thanks for writing them. +1 > I reviewed notes; find proposed changes attached+included. +1 to all the corrections shown. One more: "Allow parallel query when

Re: Unexpected "shared memory block is still in use"

2019-05-09 Thread Tom Lane
I wrote: > Noah Misch writes: >> The odds are very high that you would not have gotten that error before that >> commit. But if the cause matches your guess, it's not something wrong with >> the commit ... > Fair point. After more study and testing, I no longer believe my original thought about

Re: _bt_split(), and the risk of OOM before its critical section

2019-05-09 Thread Peter Geoghegan
On Wed, May 8, 2019 at 3:37 PM Peter Geoghegan wrote: > It makes perfect sense for _bt_split() to call _bt_findsplitloc() > directly, since _bt_findsplitloc() is already aware of almost every > _bt_split() implementation detail, whereas those same details are not > of interest anywhere else. I di

Re: Unexpected "shared memory block is still in use"

2019-05-09 Thread Tom Lane
I wrote: > However, I have a new theory, after noticing that c09850992 moved the > check for shm_nattch == 0. Previously, if a shmem segment had zero attach > count, it was unconditionally considered not-a-threat. Now, we'll try > shmat() anyway, and if that fails for any reason other than EACCES

Re: Inconsistent error message wording for REINDEX CONCURRENTLY

2019-05-09 Thread Michael Paquier
On Thu, May 09, 2019 at 02:08:39PM -0400, Tom Lane wrote: > LGTM, thanks. Thanks for double-checking, committed. I am closing the open item. -- Michael signature.asc Description: PGP signature

Re: pg12 release notes

2019-05-09 Thread Bruce Momjian
On Wed, May 8, 2019 at 03:32:04PM -0500, Justin Pryzby wrote: > I noticed you added release notes at bdf595adbca195fa54a909c74a5233ebc30641a1, > thanks for writing them. > -This improve optimization for columns with non-uniform distributions that > often appear in WHERE clauses. > +This improves

Re: pg12 release notes

2019-05-09 Thread Bruce Momjian
On Wed, May 8, 2019 at 03:32:04PM -0500, Justin Pryzby wrote: > I noticed you added release notes at bdf595adbca195fa54a909c74a5233ebc30641a1, > thanks for writing them. > > I reviewed notes; find proposed changes attached+included. > > I think these should also be mentioned? > > f7cb284 Add pl

Re: pg12 release notes

2019-05-09 Thread Justin Pryzby
On Thu, May 09, 2019 at 08:08:53PM -0400, Bruce Momjian wrote: > On Wed, May 8, 2019 at 03:32:04PM -0500, Justin Pryzby wrote: > > I think these should also be mentioned? > > > > f7cb284 Add plan_cache_mode setting > > 387a5cf Add pg_dump --on-conflict-do-nothing option. > > Applied patch attach

Re: pg12 release notes

2019-05-09 Thread Justin Pryzby
On Thu, May 09, 2019 at 07:35:18PM -0400, Bruce Momjian wrote: > I have made your other changes, with adjustment, patch attached. > > The results are here: > > http://momjian.us/pgsql_docs/release-12.html Thanks > -Many system tables now have an 'oid' column that will be expanded with > -

Re: Problems with pg_upgrade and extensions referencing catalog tables/views

2019-05-09 Thread Bruce Momjian
On Wed, May 8, 2019 at 10:07:23PM +, Nasby, Jim wrote: > I don’t recall why pg_upgrade wants to control OIDs… don’t we > re-create all catalog entries for user objects from scratch? The C comment at top of pg_upgrade.c explains why some oids must be preserved: https://git.postgresql

Re: pg12 release notes

2019-05-09 Thread Bruce Momjian
On Thu, May 9, 2019 at 07:19:59PM -0500, Justin Pryzby wrote: > > +Author: Thomas Munro > > +2018-07-13 [387a5cfb9] Add pg_dump - -on-conflict-do-nothing option. > > +--> > > + > > + > > +Allow restore of INSERT statements to skip rows which would cause > > conflicts (Surafel Temesgen) > > + >

Re: copy-past-o comment in lock.h

2019-05-09 Thread Michael Paquier
On Wed, May 08, 2019 at 05:03:31PM +0900, Michael Paquier wrote: > Good idea to move the comments so what you proposes looks fine to me. > Are there any objections? Okay, committed. -- Michael signature.asc Description: PGP signature

Re: any suggestions to detect memory corruption

2019-05-09 Thread Alex
On Thu, May 9, 2019 at 9:30 PM Tom Lane wrote: > Alex writes: > > Someone add some code during backend init which used palloc. but at that > > time, the CurrentMemoryContext is PostmasterContext. at the end of > > backend initialization, the PostmasterContext is deleted, then the error > > ha

Re: pg12 release notes

2019-05-09 Thread Bruce Momjian
On Thu, May 9, 2019 at 07:13:35PM -0500, Justin Pryzby wrote: > On Thu, May 09, 2019 at 07:35:18PM -0400, Bruce Momjian wrote: > > I have made your other changes, with adjustment, patch attached. > > > > The results are here: > > > > http://momjian.us/pgsql_docs/release-12.html > > Thanks

Re: pg12 release notes

2019-05-09 Thread Justin Pryzby
On Thu, May 09, 2019 at 09:02:51PM -0400, Bruce Momjian wrote: > These were all very helpful. I adjusted your changes to create the > attached applied patch. URL updated: Thanks. > -Allow more comparisons with information_schema text columns to use indexes > (Tom Lane) > +Allow more use of ind

Re: pg12 release notes

2019-05-09 Thread Peter Geoghegan
On Thu, May 9, 2019 at 6:03 PM Bruce Momjian wrote: > These were all very helpful. I adjusted your changes to create the > attached applied patch. URL updated: I noticed that the compatibility note for Andrew Gierth's RYU floating point patch seems to simply say why the feature is useful. Shoul

Re: pg12 release notes

2019-05-09 Thread David Rowley
On Fri, 10 May 2019 at 12:08, Bruce Momjian wrote: > > 17f206f Set pg_class.relhassubclass for partitioned indexes > > I need help with this one. I know the system column existed in previous > releases, so how is it different now? Do we document system table > changes that are implementation-beh

  1   2   >