Re: meaning of (to prevent wraparound) ..??

2020-11-25 Thread Michael Paquier
On Wed, Nov 25, 2020 at 11:10:50PM -0700, Jessica Sharp wrote: > On Wed, Nov 25, 2020 at 23:09 Atul Kumar wrote: >> Thanks Jessica. Could help me out by sharing documents that can help me >> understand “to prevent wraparound “ in simplest way, postgres doc is little >> bit harder for a newbee lik

Re: meaning of (to prevent wraparound) ..??

2020-11-25 Thread Jessica Sharp
Absolutely. This may be helpful: https://youtu.be/bA1SgWn6Gv4 On Wed, Nov 25, 2020 at 23:09 Atul Kumar wrote: > Thanks Jessica. Could help me out by sharing documents that can help me > understand “to prevent wraparound “ in simplest way, postgres doc is little > bit harder for a newbee like me.

Re: meaning of (to prevent wraparound) ..??

2020-11-25 Thread Atul Kumar
Thanks Jessica. Could help me out by sharing documents that can help me understand “to prevent wraparound “ in simplest way, postgres doc is little bit harder for a newbee like me. Regards Atul On Thursday, November 26, 2020, Jessica Sharp wrote: > Hi Atul, > > This means autovacuum is do

Re: meaning of (to prevent wraparound) ..??

2020-11-25 Thread Jessica Sharp
Hi Atul, This means autovacuum is doing it’s job — most likely no need to be alarmed here based on this alone. The autovacuum is helping avoid a txid limit. This is part of PostgreSQL MVCC. You may check that section of the documentation for more information on this. On Wed, Nov 25, 2020 at 22:54

meaning of (to prevent wraparound) ..??

2020-11-25 Thread Atul Kumar
Hi, I have autovacuum running in background from last 30-40 minutes but I noticed that it is with "(to prevent wraparound)" say for example autovacuum: VACUUM trk.move (to prevent wraparound). So what is the meaning of "(to prevent wraparound)" here ? Will it impact the autovacuum anyhow ? Plea

Re: archive file "00000001000000000000006F" has wrong size: 67118648 instead of 16777216

2020-11-25 Thread 江川潔
Hi I have corrected it on "restore_command = 'copy "D:\\BKUP\\pg_archivedir\\PostgreSQL_DEV\\%f" "%p"'". Thanks, Kiyoshi 2020年11月25日(水) 19:35 Magnus Hagander : > On Wed, Nov 25, 2020 at 3:02 AM 江川潔 wrote: > > > > Hi, > > > > WAL log recovery was failed on wrong log record size. Could you please

Re: Potential BRIN Index Corruption

2020-11-25 Thread Alvaro Herrera
On 2020-Nov-26, Huan Ruan wrote: > Hi All > > We cannot work out a reproducible case but we have a copy of the offending > database. I was hoping to know I think the way to debug this would be to see what WAL records have been emitted for the index, using pageinspect to find the problem index tu

Potential BRIN Index Corruption

2020-11-25 Thread Huan Ruan
Hi All We have a table with 623 million records. It appears a BRIN index of this table on a timestamp column is missing some records, as illustrated below in a cut-down version with additional columns and indices omitted. We cannot work out a reproducible case but we have a copy of the offending

Re: limit of data type character varying

2020-11-25 Thread David G. Johnston
On Wed, Nov 25, 2020 at 1:43 PM Mark Phillips wrote: > The actual maximum length supported by postgresql 12 is 10,485,760. We dug > into the postgres code and found a limit in the config with a comment > dating it to the pg 8.5 era. Being the simple folk that we are, we changed > the setting to t

limit of data type character varying

2020-11-25 Thread Mark Phillips
Ran into a problem with SymmetricDS when attempting to sync two postgres 12 databases. The problem occurs when Symmetric builds the XML for a table that has the character varying datatype with an undefined length. SymmetricDS creates in the xml character field that is 2,147,483,647 length. Sub

Re: Number of parallel workers chosen by the optimizer for parallel append

2020-11-25 Thread Michael Lewis
What have you tried? Changing the relevant cost parameters I assume? Nothing else going on that may be taking up those workers, right? https://www.postgresql.org/docs/current/runtime-config-query.html#GUC-PARALLEL-SETUP-COST

Re: postgres_fdw insert extremely slow

2020-11-25 Thread Tom Lane
Mats Julian Olsen writes: > I've got some more numbers here: > ... > To me this does indicate some sort of networking issue, but I'm > wondering if INSERTs are treated differently than SELECTs in > postgres_fdw? The only feasibly explanation I have is that postgres_fdw > does many more network

Re: postgres_fdw insert extremely slow

2020-11-25 Thread Mats Julian Olsen
On 25.11.2020 18:12, Tom Lane wrote: Mats Julian Olsen writes: On 25.11.2020 17:58, Tom Lane wrote: ... Have you tried looking into pg_locks on the remote server while this query is running? Thanks Tom, I'll try to spin up a regular Postgres instance on both rds and ec2 and see if that help

Re: INSERT Trigger to check for existing records : Does this do what I hope it is doing?

2020-11-25 Thread Adrian Klaver
On 11/25/20 9:13 AM, Adrian Klaver wrote: On 11/25/20 8:43 AM, ha...@datasundae.com wrote: Adrian, Until the previous questions are addressed the above is not doable. 3. Not positive the IF is doing what I want,  but if I copy a sfdc.dealid into the same_test() parameter field the FUNTI

RE: INSERT Trigger to check for existing records : Does this do what I hope it is doing?

2020-11-25 Thread hagen
[Hagen] Answers inline -Original Message- From: Adrian Klaver Sent: Wednesday, November 25, 2020 10:13 AM To: ha...@datasundae.com; pgsql-general@lists.postgresql.org Subject: Re: INSERT Trigger to check for existing records : Does this do what I hope it is doing? On 11/25/20 8:43 AM

Re: INSERT Trigger to check for existing records : Does this do what I hope it is doing?

2020-11-25 Thread Adrian Klaver
On 11/25/20 8:43 AM, ha...@datasundae.com wrote: Adrian, Thanks for your detailed response. That's very kind and much appreciated. 1. OK that's just me groping for a RETURN statement that doesn't throw a rod. Things can still work, sort of. I once cranked up and ran(for a short time) a JD 4

Re: postgres_fdw insert extremely slow

2020-11-25 Thread Tom Lane
Mats Julian Olsen writes: > On 25.11.2020 17:58, Tom Lane wrote: >> ... Have you tried looking into pg_locks on the >> remote server while this query is running? > Thanks Tom, I'll try to spin up a regular Postgres instance on both rds > and ec2 and see if that helps. As for the locks, I can not

Re: Performance hit if I create multiple DBs on same instance

2020-11-25 Thread Bruce Momjian
On Wed, Nov 25, 2020 at 03:59:06PM +0100, Laurenz Albe wrote: > On Tue, 2020-11-24 at 11:34 -0600, Ron wrote: > > And if you're afraid of autovacuum and autoanalyze stealing resources, then > > disable them (at the table level). > > Ugh, bad advice. > > Better would be to VACUUM (FREEZE) these s

Re: postgres_fdw insert extremely slow

2020-11-25 Thread Mats Julian Olsen
On 25.11.2020 17:58, Tom Lane wrote: Mats Julian Olsen writes: Postgres version(s)? x: Aurora PostgreSQL 11.7 on x86_64-pc-linux-gnu, compiled by gcc (GCC) 4.9.3, 64-bit (RDS) y: PostgreSQL 12.2 (Ubuntu 12.2-2.pgdg19.10+1) on x86_64-pc-linux-gnu, compiled by gcc (Ubuntu 9.2.1-9ubuntu2) 9.2.1

Re: postgres_fdw insert extremely slow

2020-11-25 Thread Mats Julian Olsen
On 25.11.2020 17:57, Adrian Klaver wrote: On 11/25/20 8:48 AM, Mats Julian Olsen wrote: Apologies for the sloppiness! Postgres version(s)? x: Aurora PostgreSQL 11.7 on x86_64-pc-linux-gnu, compiled by gcc (GCC) 4.9.3, 64-bit (RDS) y: PostgreSQL 12.2 (Ubuntu 12.2-2.pgdg19.10+1) on x86_64

Re: postgres_fdw insert extremely slow

2020-11-25 Thread Tom Lane
Mats Julian Olsen writes: >> Postgres version(s)? > x: Aurora PostgreSQL 11.7 on x86_64-pc-linux-gnu, compiled by gcc (GCC) > 4.9.3, 64-bit (RDS) > y: PostgreSQL 12.2 (Ubuntu 12.2-2.pgdg19.10+1) on x86_64-pc-linux-gnu, > compiled by gcc (Ubuntu 9.2.1-9ubuntu2) 9.2.1 20191008, 64-bit (GCP) Hmm,

Re: postgres_fdw insert extremely slow

2020-11-25 Thread Adrian Klaver
On 11/25/20 8:48 AM, Mats Julian Olsen wrote: Apologies for the sloppiness! Postgres version(s)? x: Aurora PostgreSQL 11.7 on x86_64-pc-linux-gnu, compiled by gcc (GCC) 4.9.3, 64-bit (RDS) y: PostgreSQL 12.2 (Ubuntu 12.2-2.pgdg19.10+1) on x86_64-pc-linux-gnu, compiled by gcc (Ubuntu 9.2.1-

Re: postgres_fdw insert extremely slow

2020-11-25 Thread Mats Julian Olsen
Apologies for the sloppiness! Postgres version(s)? x: Aurora PostgreSQL 11.7 on x86_64-pc-linux-gnu, compiled by gcc (GCC) 4.9.3, 64-bit (RDS) y: PostgreSQL 12.2 (Ubuntu 12.2-2.pgdg19.10+1) on x86_64-pc-linux-gnu, compiled by gcc (Ubuntu 9.2.1-9ubuntu2) 9.2.1 20191008, 64-bit (GCP) With

RE: INSERT Trigger to check for existing records : Does this do what I hope it is doing?

2020-11-25 Thread hagen
Adrian, Thanks for your detailed response. That's very kind and much appreciated. 1. OK that's just me groping for a RETURN statement that doesn't throw a rod. I don't actually need to return anything as the goal of the FUNCTION (for the moment) is to perform updates to a table. It might be ni

Re: postgres_fdw insert extremely slow

2020-11-25 Thread Adrian Klaver
On 11/25/20 8:37 AM, Mats Julian Olsen wrote: Helle pgsql-general, We have just set up postgres_fdw between two postgres databases, x and y, with the plan to periodically insert data from x into y. We've successfully set up the connection with a few options: `use_remote_estimate 'true'` and

postgres_fdw insert extremely slow

2020-11-25 Thread Mats Julian Olsen
Helle pgsql-general, We have just set up postgres_fdw between two postgres databases, x and y, with the plan to periodically insert data from x into y. We've successfully set up the connection with a few options: `use_remote_estimate 'true'` and `fetch_size '5'` (the latter we've played aroun

Number of parallel workers chosen by the optimizer for parallel append

2020-11-25 Thread Laurenz Albe
I have a partitioned table, each partition has "parallel_workers = 10" set. SET max_parallel_workers_per_gather = 8; SET enable_partitionwise_aggregate = on; EXPLAIN (COSTS OFF) SELECT applicant_name, count(ipc_4) FROM laurenz.z_flat GROUP BY applicant_name; QU

Re: INSERT Trigger to check for existing records : Does this do what I hope it is doing?

2020-11-25 Thread Adrian Klaver
On 11/25/20 7:41 AM, Hagen Finley wrote: Folks, Just a quick question. *Using this FUNCTION:* CREATE OR REPLACE FUNCTION same_test(did numeric) RETURNS numeric AS $$ BEGIN   IF $1 IN       (SELECT dealid from hygiene_112)   THEN     UPDATE hygiene_119 SET paid =

Re: Performance hit if I create multiple DBs on same instance

2020-11-25 Thread Ron
On 11/25/20 8:59 AM, Laurenz Albe wrote: On Tue, 2020-11-24 at 11:34 -0600, Ron wrote: And if you're afraid of autovacuum and autoanalyze stealing resources, then disable them (at the table level). Ugh, bad advice. Better would be to VACUUM (FREEZE) these static table once, then autovacuum wo

Re: INSERT Trigger to check for existing records : Does this do what I hope it is doing?

2020-11-25 Thread Hagen Finley
Folks, Just a quick question. *Using this FUNCTION:* CREATE OR REPLACE FUNCTION same_test(did numeric) RETURNS numeric AS $$ BEGIN   IF $1 IN       (SELECT dealid from hygiene_112)   THEN     UPDATE hygiene_119 SET paid = 'SAME';   ELSE       UPDATE hygiene_119 S

Re: Performance hit if I create multiple DBs on same instance

2020-11-25 Thread Laurenz Albe
On Tue, 2020-11-24 at 11:34 -0600, Ron wrote: > And if you're afraid of autovacuum and autoanalyze stealing resources, then > disable them (at the table level). Ugh, bad advice. Better would be to VACUUM (FREEZE) these static table once, then autovacuum won't ever perform resource consuming acti

Re: archive file "00000001000000000000006F" has wrong size: 67118648 instead of 16777216

2020-11-25 Thread Magnus Hagander
On Wed, Nov 25, 2020 at 3:02 AM 江川潔 wrote: > > Hi, > > WAL log recovery was failed on wrong log record size. Could you please advise > me what is wrong in the setting ? Any suggestions will be highly appreciated. > > Thanks, > Kiyoshi > > postgres.conf: > wal_level = replica > archive_mode = on >