Re: pg_restore V12 fails consistently against piped pg_dumps

2020-05-06 Thread Tom Lane
Mohamed Wael Khobalatte writes: > When doing a parallel pg_restore (v12) against a dump created through a > pipe using an earlier version (11 all the way to 9.6), it fails with the > known error of not finding data offsets. I understand the reasons for it > (potential inability to seek), which is

pg_restore V12 fails consistently against piped pg_dumps

2020-05-06 Thread Mohamed Wael Khobalatte
Hi all, When doing a parallel pg_restore (v12) against a dump created through a pipe using an earlier version (11 all the way to 9.6), it fails with the known error of not finding data offsets. I understand the reasons for it (potential inability to seek), which is documented in pg_restore. What

Re: New Role drop with Grant/Revokes stop working after subsequent runs

2020-05-06 Thread David G. Johnston
On Wed, May 6, 2020 at 5:05 PM AC Gomez wrote: > We have developed some code that creates a new role to be used as the main > role for DB usage. This code will be called on a predetermined frequency to > act a role/pwd rotation mechanism. > > Each time the code is run we feed it the prior role t

New Role drop with Grant/Revokes stop working after subsequent runs

2020-05-06 Thread AC Gomez
Hi, On PostgreSQL 9.6. We have developed some code that creates a new role to be used as the main role for DB usage. This code will be called on a predetermined frequency to act a role/pwd rotation mechanism. Each time the code is run we feed it the prior role that was created (the Db owner bein

Re: Lock Postgres account after X number of failed logins?

2020-05-06 Thread Geoff Winkless
On Wed, 6 May 2020, 14:28 Stephen Frost, wrote: > Greetings, > > * Geoff Winkless (pgsqlad...@geoff.dj) wrote: > > On Wed, 6 May 2020 at 00:05, Tim Cross wrote: > > > Where Tom's solution fails is with smaller companies that cannot afford > > > this level of infrastructure. > > > > Is there an o

previous replication slot and new initdb

2020-05-06 Thread Support
Hi Folks, is there a way to refresh/reset/reinit the internal replication slot identifier for a freshly new hot standby initdb and pg_dumpall? For now the master complain that the identifier (a bigint) of the node does not match with the replication slot identifier. So if there is a way to res

Re: White space affecting parsing of range values

2020-05-06 Thread Thom Brown
On Wed, 6 May 2020 at 17:33, Tom Lane wrote: > > Thom Brown writes: > > I guess I should read the docs more carefully. Shouldn't this be > > insignificant for a numeric value? > > That would require the range code to know whether the subtype considers > whitespace significant (or perhaps more us

Re: White space affecting parsing of range values

2020-05-06 Thread Thom Brown
On Wed, 6 May 2020 at 17:30, Adrian Klaver wrote: > > On 5/6/20 9:19 AM, Thom Brown wrote: > > On Wed, 6 May 2020 at 17:13, Adrian Klaver > > wrote: > >> > >> On 5/6/20 9:00 AM, Thom Brown wrote: > >>> Hi, > >>> > >>> I noticed I'm getting an error when adding white space to a numeric > >>> rang

Re: White space affecting parsing of range values

2020-05-06 Thread Tom Lane
Thom Brown writes: > I guess I should read the docs more carefully. Shouldn't this be > insignificant for a numeric value? That would require the range code to know whether the subtype considers whitespace significant (or perhaps more usefully, whether an all-spaces input is valid). We've staye

Re: White space affecting parsing of range values

2020-05-06 Thread Adrian Klaver
On 5/6/20 9:19 AM, Thom Brown wrote: On Wed, 6 May 2020 at 17:13, Adrian Klaver wrote: On 5/6/20 9:00 AM, Thom Brown wrote: Hi, I noticed I'm getting an error when adding white space to a numeric range. I can run this: postgres=# SELECT 5::numeric <@ '(,10]'::numrange; ?column?

Re: pg_basebackup inconsistent performance

2020-05-06 Thread Stephen Frost
Greetings, * Jasen Lentz (jle...@sescollc.com) wrote: > Where are the machines you are backing up from/to relative to each on the > network? > Direct ethernet connection between 10G network interfaces Is the backup server shared among other systems..? > Is there increased activity on the databa

Re: White space affecting parsing of range values

2020-05-06 Thread Thom Brown
On Wed, 6 May 2020 at 17:13, Adrian Klaver wrote: > > On 5/6/20 9:00 AM, Thom Brown wrote: > > Hi, > > > > I noticed I'm getting an error when adding white space to a numeric > > range. I can run this: > > > > postgres=# SELECT 5::numeric <@ '(,10]'::numrange; > > ?column? > > -- > >

Re: White space affecting parsing of range values

2020-05-06 Thread Adrian Klaver
On 5/6/20 9:00 AM, Thom Brown wrote: Hi, I noticed I'm getting an error when adding white space to a numeric range. I can run this: postgres=# SELECT 5::numeric <@ '(,10]'::numrange; ?column? -- t (1 row) But I can't run this: postgres=# SELECT 5::numeric <@ '( ,10]'::numrange;

RE: pg_basebackup inconsistent performance

2020-05-06 Thread Jasen Lentz
I'm guessing the above happens from one run to another correct? Yes Where are the machines you are backing up from/to relative to each on the network? Direct ethernet connection between 10G network interfaces Is there increased activity on the database servers e.g. inserts, updates, etc during

White space affecting parsing of range values

2020-05-06 Thread Thom Brown
Hi, I noticed I'm getting an error when adding white space to a numeric range. I can run this: postgres=# SELECT 5::numeric <@ '(,10]'::numrange; ?column? -- t (1 row) But I can't run this: postgres=# SELECT 5::numeric <@ '( ,10]'::numrange; ERROR: invalid input syntax for type num

Re: Lock Postgres account after X number of failed logins?

2020-05-06 Thread Stephen Frost
Greetings, * Peter J. Holzer (hjp-pg...@hjp.at) wrote: > On 2020-05-06 09:28:28 -0400, Stephen Frost wrote: > > LDAP-based authentication in PG involves passing the user's password to > > the database server in the clear (or tunneled through SSL, but that > > doesn't help if the DB is compromised)

Re: Abnormal Growth of Index Size - Index Size 3x large than table size.

2020-05-06 Thread Ravi Krishna
> On May 6, 2020, at 10:52 AM, Ashish Chugh > wrote: > > Hello Ravi, > > > Total number of indexes are 10 and size is 65 GB. Shall we consider this as a > normal scenario or we need to look into the growth of the indexes as this is > increasing day by day and table data is not increasing

Re: Lock Postgres account after X number of failed logins?

2020-05-06 Thread Magnus Hagander
On Wed, May 6, 2020 at 5:26 PM Peter J. Holzer wrote: > On 2020-05-06 09:28:28 -0400, Stephen Frost wrote: > > LDAP-based authentication in PG involves passing the user's password to > > the database server in the clear (or tunneled through SSL, but that > > doesn't help if the DB is compromised)

Re: Lock Postgres account after X number of failed logins?

2020-05-06 Thread Peter J. Holzer
On 2020-05-06 09:28:28 -0400, Stephen Frost wrote: > LDAP-based authentication in PG involves passing the user's password to > the database server in the clear (or tunneled through SSL, but that > doesn't help if the DB is compromised), so it's really not a good > solution. Still a lot better than

Re: Abnormal Growth of Index Size - Index Size 3x large than table size.

2020-05-06 Thread Ravi Krishna
> > Hello Ravi, > > Total number of index is 10 and 65GB is the sum total of index size of all > indexes for table “tstock_movement” > I am attaching the screen shot for your reference. > In that case 65GB is not surprising.

Re: pg_basebackup inconsistent performance

2020-05-06 Thread Adrian Klaver
On 5/6/20 5:44 AM, Jasen Lentz wrote: We have pg_basebackup running on two of our DB servers that are replicated.  We are running postgres 11, and it seems we are getting inconsistent performance from the backups and unsure as of why.  We start out at 5-6 hours over a dedicated 10G port for 7TB

Re: pg_basebackup inconsistent performance

2020-05-06 Thread Adrian Klaver
On 5/6/20 5:44 AM, Jasen Lentz wrote: We have pg_basebackup running on two of our DB servers that are replicated.  We are running postgres 11, and it seems we are getting inconsistent performance from the backups and unsure as of why.  We start out at 5-6 hours over a dedicated 10G port for 7TB

RE: Abnormal Growth of Index Size - Index Size 3x large than table size.

2020-05-06 Thread Ram Pratap Maurya
Hello Ravi, Total number of index is 10 and 65GB is the sum total of index size of all indexes for table "tstock_movement" I am attaching the screen shot for your reference. [cid:image001.png@01D623D7.65173990] Regards, Ram Pratap. Lava International Limited. Tel+ 91-120-4637148 [cid:image001

Re: EXTERNAL: Re: Lock Postgres account after X number of failed logins?

2020-05-06 Thread Stephen Frost
Greetings, * Wolff, Ken L (ken.l.wo...@lmco.com) wrote: > Thanks again, everyone, for all the responses and ideas. I'm still getting > caught up on the various responses but with respect to LDAP/AD, I truly wish > it were an option. I agree with the various sentiments that AD > authentication

Re: Abnormal Growth of Index Size - Index Size 3x large than table size.

2020-05-06 Thread Michael Lewis
Indexes larger than the table may be expected if there are many. It may be prudent to check if they are being used in pg_stat_all_indexes. If there are just a few indexes that are becoming bloated quickly, you'd want to ensure your autovacuum settings are tuned more aggressively, and consider lowe

RE: EXTERNAL: Re: Lock Postgres account after X number of failed logins?

2020-05-06 Thread Wolff, Ken L
Thanks again, everyone, for all the responses and ideas. I'm still getting caught up on the various responses but with respect to LDAP/AD, I truly wish it were an option. I agree with the various sentiments that AD authentication is more manageable, scalable, secure, etc. However, if there we

Re: Encoding conversion: Use replacement character instead of failing query with "ERROR: character with byte sequence 0xd0 0xad in encoding "UTF8" has no equivalent in encoding LATIN1" ?

2020-05-06 Thread Christian Ramseyer
On 06.05.20 02:00, Tom Lane wrote: > Christian Ramseyer writes: >> Can I somehow influence the client:UTF8->server:LATIN1 character set >> conversion so that instead of failing, it inserts an invalid codepoint >> character, the utf8 hex bytes as string, drops the character or >> something like

Re: Lock Postgres account after X number of failed logins?

2020-05-06 Thread Stephen Frost
Greetings, * Geoff Winkless (pgsqlad...@geoff.dj) wrote: > On Wed, 6 May 2020 at 00:05, Tim Cross wrote: > > Where Tom's solution fails is with smaller companies that cannot afford > > this level of infrastructure. > > Is there an objection to openldap? It's lightweight (so could > reasonably be

pg_basebackup inconsistent performance

2020-05-06 Thread Jasen Lentz
We have pg_basebackup running on two of our DB servers that are replicated. We are running postgres 11, and it seems we are getting inconsistent performance from the backups and unsure as of why. We start out at 5-6 hours over a dedicated 10G port for 7TB. It creeps up to 8-9 hours then all o

Re: Abnormal Growth of Index Size - Index Size 3x large than table size.

2020-05-06 Thread Christian Ramseyer
Hi On 06.05.20 11:48, Ram Pratap Maurya wrote: > We are facing a problem in our PostgreSQL production database related to > abnormal growth of index size. Some of the indexes are having abnormal > growth and index size is larger than table data size. > > One table is having 75 G.B of index thoug

Re: Lock Postgres account after X number of failed logins?

2020-05-06 Thread Tim Cross
Geoff Winkless writes: > On Wed, 6 May 2020 at 00:05, Tim Cross wrote: >> Where Tom's solution fails is with smaller companies that cannot afford >> this level of infrastructure. > > Is there an objection to openldap? It's lightweight (so could > reasonably be run on the same hardware without

Re: Lock Postgres account after X number of failed logins?

2020-05-06 Thread Guillaume Lelarge
Le mer. 6 mai 2020 à 04:18, Christian Ramseyer a écrit : > > > On 05.05.20 16:13, Wolff, Ken L wrote: > > Hi, everyone. Wondering if there’s a way in PostgreSQL to automatically > > lock accounts after a number of failed logins (a security requirement > > for my organization). > > > > Locking ac

Re: Abnormal Growth of Index Size - Index Size 3x large than table size.

2020-05-06 Thread Ravi Krishna
> On May 6, 2020, at 5:48 AM, Ram Pratap Maurya > wrote: > > Hi Team, > > We are facing a problem in our PostgreSQL production database related to > abnormal growth of index size. Some of the indexes are having abnormal growth > and index size is larger than table data size. > One table is

Abnormal Growth of Index Size - Index Size 3x large than table size.

2020-05-06 Thread Ram Pratap Maurya
Hi Team, We are facing a problem in our PostgreSQL production database related to abnormal growth of index size. Some of the indexes are having abnormal growth and index size is larger than table data size. One table is having 75 G.B of index though table size is only 25 G.B. On monthly basis w

Re: Lock Postgres account after X number of failed logins?

2020-05-06 Thread Paul Förster
Hi Geoff, > On 06. May, 2020, at 10:33, Geoff Winkless wrote: > > Is there an objection to openldap? It's lightweight (so could > reasonably be run on the same hardware without significant impact), > BSD-ish and mature, and (with the password policy overlay) should > provide exactly the function

Re: Lock Postgres account after X number of failed logins?

2020-05-06 Thread Geoff Winkless
On Wed, 6 May 2020 at 00:05, Tim Cross wrote: > Where Tom's solution fails is with smaller companies that cannot afford > this level of infrastructure. Is there an objection to openldap? It's lightweight (so could reasonably be run on the same hardware without significant impact), BSD-ish and mat