postgres materialized view refresh in parallel

2020-08-18 Thread Ayub M
Created materialized view in parallel by setting max_parallel_workers_per_gather to 4. I could see parallel sessions kicking off and creating the mview much faster than serial execution (10mins - parallel vs over an hour - serial). Now I want the refresh of the mview to happen taking around same

Re: "Go" (lang) standard driver

2020-08-18 Thread David Gallagher
I can recommend sqlx: https://github.com/jmoiron/sqlx On Tue, Aug 18, 2020 at 4:56 PM Scottix wrote: > I have been using pgx and seems to be working well. > > I recommend if your just starting to use the v4 version. > > On Tue, Aug 18, 2020 at 5:53 AM Stephen Frost wrote: > >> Greetings, >>

Re: Index tuple deduplication limitations in pg13

2020-08-18 Thread Peter Geoghegan
On Tue, Aug 18, 2020 at 1:31 PM Matthias van de Meent wrote: > Would this extra ordering not effectively be an extra tiebreaker in > the ordering, applied before the TID? I do not know the full > implications of that, but I believe that would not result in the > limitations that you are

Re: Index tuple deduplication limitations in pg13

2020-08-18 Thread Matthias van de Meent
On Tue, 18 Aug 2020 at 22:00, Peter Geoghegan wrote: > > On Tue, Aug 18, 2020 at 11:52 AM Matthias van de Meent > wrote: > > Given that the above could work, the current btree tuple ordering is > > not optimized for opclass-equal but datum image-distinct values: > > ordering of opclass-equal

Re: "Go" (lang) standard driver

2020-08-18 Thread Scottix
I have been using pgx and seems to be working well. I recommend if your just starting to use the v4 version. On Tue, Aug 18, 2020 at 5:53 AM Stephen Frost wrote: > Greetings, > > * Olivier Gautherot (ogauthe...@gautherot.net) wrote: > > Le mar. 18 août 2020 à 09:36, Tony Shelver a écrit > : >

Re: Index tuple deduplication limitations in pg13

2020-08-18 Thread Matthias van de Meent
On Tue, 18 Aug 2020 at 18:44, Peter Geoghegan wrote: > > On Mon, Aug 17, 2020 at 11:44 PM Matthias van de Meent > wrote: > > But, if the ordering of operator-class equal tuples is already > > system-defined, could the physical ordering of index tuples in a btree > > (with deduplication enabled

Re: Orphaned relations after crash/sigkill during CREATE TABLE

2020-08-18 Thread Adrian Klaver
On 8/18/20 1:19 PM, Jason Myers wrote: > On Tue, Aug 18, 2020 at 3:49 PM Adrian Klaver mailto:adrian.kla...@aklaver.com>> wrote: > Have you tried with: > BEGIN; > CREATE TABLE some_table SELECT some_data FROM other_table LIMIT 1 WITH > NO DATA; > COMMIT; > > The above gets you the

Re: Orphaned relations after crash/sigkill during CREATE TABLE

2020-08-18 Thread Jason Myers
> On Tue, Aug 18, 2020 at 3:49 PM Adrian Klaver wrote: > > Have you tried with: > BEGIN; > CREATE TABLE some_table SELECT some_data FROM other_table LIMIT 1 WITH > NO DATA; > COMMIT; > > The above gets you the table structure, but no data. > > BEGIN; > INSERT into some_table SELECT * FROM

Re: Index tuple deduplication limitations in pg13

2020-08-18 Thread Peter Geoghegan
On Tue, Aug 18, 2020 at 11:52 AM Matthias van de Meent wrote: > Given that the above could work, the current btree tuple ordering is > not optimized for opclass-equal but datum image-distinct values: > ordering of opclass-equal values is currently determined only by tid, > with as an example

Re: Orphaned relations after crash/sigkill during CREATE TABLE

2020-08-18 Thread Adrian Klaver
On 8/18/20 12:35 PM, Jason Myers wrote: Postgres 12.4 I was directed in slack to mention here that we're being impacted by Postgres leaving orphaned pages in /base/ after a crash while a CREATE TABLE is being run in transaction. The issue is the same as the reproduction steps listed here

Orphaned relations after crash/sigkill during CREATE TABLE

2020-08-18 Thread Jason Myers
Postgres 12.4 I was directed in slack to mention here that we're being impacted by Postgres leaving orphaned pages in /base/ after a crash while a CREATE TABLE is being run in transaction. The issue is the same as the reproduction steps listed here [1], that is: - Start a CREATE TABLE

Re: Index tuple deduplication limitations in pg13

2020-08-18 Thread Peter Geoghegan
On Tue, Aug 18, 2020 at 11:52 AM Matthias van de Meent wrote: > Deduplication does not need to destroy semantic differences? 'equal' > can (in my book) mean: > - 'opclass-equal', that is the opclass returns true for an equality check > - 'binary equal' or 'datum-equal' (? maybe incorrect term),

Re: Index tuple deduplication limitations in pg13

2020-08-18 Thread Peter Geoghegan
On Tue, Aug 18, 2020 at 9:44 AM Peter Geoghegan wrote: > If we wanted to fix this for numeric, we'd have to invent a new > numeric datatype (called numeric2, say). That probably isn't as hard > as it sounds, since it could be part of the same B-Tree operator > family as numeric. It could also be

Re: Index tuple deduplication limitations in pg13

2020-08-18 Thread Peter Geoghegan
On Mon, Aug 17, 2020 at 11:44 PM Matthias van de Meent wrote: > But, if the ordering of operator-class equal tuples is already > system-defined, could the physical ordering of index tuples in a btree > (with deduplication enabled for "unsafe" opclasses) be updated from > [index_columns, tid] to

Re: import XML

2020-08-18 Thread David G. Johnston
On Tue, Aug 18, 2020 at 8:27 AM PASCAL CROZET < pascal.cro...@qualis-consulting.com> wrote: > I want to import XML file into PG database table. > I've find functions to get the XML content of a cell after imported an XML > file with the pg_get_file function. > But, I want to explode the XML

import XML

2020-08-18 Thread PASCAL CROZET
Hi all, I want to import XML file into PG database table. I've find functions to get the XML content of a cell after imported an XML file with the pg_get_file function. But, I want to explode the XML content to colums. How can I do this ? PG 10 under Ubuntu 18 _

Re: Database logins taking longer and longer, showing up as "authentication" in ps(1)

2020-08-18 Thread Adam Sjøgren
Stephen writes: > * Adam Sjøgren (a...@koldfront.dk) wrote: >> >> We have 60 processes (workers) running on different machines accessing >> the database, that all grab jobs from a queue and update rows in a table >> after doing some calculations (which vary in time from <1s to perhaps a >>

Re: Database logins taking longer and longer, showing up as "authentication" in ps(1)

2020-08-18 Thread Tom Lane
Stephen Frost writes: > * Adam Sjøgren (a...@koldfront.dk) wrote: >> Sometimes new database logins slow down, from usually taking <0.05s to >> taking minutes. This is for psql as a normal user using Kerberos, for >> psql as the postgres superuser, for the web-application logging into the >>

Re: Database logins taking longer and longer, showing up as "authentication" in ps(1)

2020-08-18 Thread Srinivasa T N
On Tue, Aug 18, 2020, 8:04 PM Stephen Frost wrote: > Greetings, > > * Adam Sjøgren (a...@koldfront.dk) wrote: > > We have a PostgreSQL 11.3¹ running on an Ubuntu 16.04 server, which > > sometimes exhibits a behaviour I can't quite understand: simply logging > > into the database starts to take

Re: Database logins taking longer and longer, showing up as "authentication" in ps(1)

2020-08-18 Thread Adam Sjøgren
Laurenz writes: > On Tue, 2020-08-18 at 14:53 +0200, Adam Sjøgren wrote: >> >> We have 60 processes (workers) running on different machines accessing >> the database, that all grab jobs from a queue and update rows in a table >> after doing some calculations (which vary in time from <1s to

Re: Database logins taking longer and longer, showing up as "authentication" in ps(1)

2020-08-18 Thread Stephen Frost
Greetings, * Adam Sjøgren (a...@koldfront.dk) wrote: > We have a PostgreSQL 11.3¹ running on an Ubuntu 16.04 server, which > sometimes exhibits a behaviour I can't quite understand: simply logging > into the database starts to take minutes to complete. > > We have 60 processes (workers) running

Re: Database logins taking longer and longer, showing up as "authentication" in ps(1)

2020-08-18 Thread Laurenz Albe
On Tue, 2020-08-18 at 14:53 +0200, Adam Sjøgren wrote: > We have a PostgreSQL 11.3¹ running on an Ubuntu 16.04 server, which > sometimes exhibits a behaviour I can't quite understand: simply logging > into the database starts to take minutes to complete. > > We have 60 processes (workers) running

Database logins taking longer and longer, showing up as "authentication" in ps(1)

2020-08-18 Thread Adam Sjøgren
Hi, We have a PostgreSQL 11.3¹ running on an Ubuntu 16.04 server, which sometimes exhibits a behaviour I can't quite understand: simply logging into the database starts to take minutes to complete. We have 60 processes (workers) running on different machines accessing the database, that all

Re: Point in time recovery

2020-08-18 Thread Stephen Frost
Greetings, Please don't top-post. * Ron (ronljohnso...@gmail.com) wrote: > Search the log file for errors? Sadly, this doesn't actually work if you're really doing PITR- you need to look for the specific message saying "recovery stopping before commit" or similar, eg: 2020-08-18 12:55:31.240

Re: "Go" (lang) standard driver

2020-08-18 Thread Stephen Frost
Greetings, * Olivier Gautherot (ogauthe...@gautherot.net) wrote: > Le mar. 18 août 2020 à 09:36, Tony Shelver a écrit : > > -- Forwarded message - > > From: Tony Shelver > > Date: Tue, 18 Aug 2020 at 09:33 > > Subject: Re: "Go" (lang) standard driver > > To: Edson Richter > > >

Re: Point in time recovery

2020-08-18 Thread Ron
Search the log file for errors? Query the database(s) to verify that the latest data s there? On 8/18/20 5:10 AM, Daulat Ram wrote: Hello Team, I want to know the best way to ensure/verify that the Point in time recovery has done successfully after the crash and the restore. Thanks,

Point in time recovery

2020-08-18 Thread Daulat Ram
Hello Team, I want to know the best way to ensure/verify that the Point in time recovery has done successfully after the crash and the restore. Thanks,

BUG? Slave don't reconnect to the master

2020-08-18 Thread Олег Самойлов
Hi all. I found some strange behaviour of postgres, which I recognise as a bug. First of all, let me explain situation. I created a "test bed" (not sure how to call it right), to test high availability clusters based on Pacemaker and PostgreSQL. The test bed consist of 12 virtual machines (on

Re: passing linux user to PG server as a variable ?

2020-08-18 Thread Daniel Verite
David Gauthier wrote: > I can avoid the error by just throwing a namespace in there... > atletx7-reg036:/home/dgauthie[ 214 ] --> setenv PGOPTIONS "-c > 'os.user=$USER' " > But once in, "show os.user" is undefined. It's documented to work [1], but you need to remove these single quotes.

Re: "Go" (lang) standard driver

2020-08-18 Thread Olivier Gautherot
Hi Edson, Le mar. 18 août 2020 à 09:36, Tony Shelver a écrit : > -- Forwarded message - > From: Tony Shelver > Date: Tue, 18 Aug 2020 at 09:33 > Subject: Re: "Go" (lang) standard driver > To: Edson Richter > > > A quick Google search found https://github.com/lib/pq. Has 6.1K

Fwd: "Go" (lang) standard driver

2020-08-18 Thread Tony Shelver
-- Forwarded message - From: Tony Shelver Date: Tue, 18 Aug 2020 at 09:33 Subject: Re: "Go" (lang) standard driver To: Edson Richter A quick Google search found https://github.com/lib/pq. Has 6.1K stars so I would guess fairly well used. On Tue, 18 Aug 2020 at 05:56, Edson

Index tuple deduplication limitations in pg13

2020-08-18 Thread Matthias van de Meent
Hi, I was reading through the new features of PG13 (beta), and noticed that deduplication is disabled for float(4, 8) and numeric (and jsonb, ...) due to that the datums of those types could be not binary equal, but equal according for the opclass used. But, if the ordering of operator-class