HASH partitioning not working properly

2020-06-18 Thread Srinivasa T N
Hi, After seeing the below, I feel partitioning is not working properly or it maybe case that my understanding is wrong. Can somebody explain me what is happening? postgres=# create table busbar_version (objectid int, ver_id int) partition by hash(ver_id); CREATE TABLE postgres=# CREATE TABLE

Re: Netapp SnapCenter

2020-06-18 Thread Paul Förster
Hi Ron, > On 18. Jun, 2020, at 21:30, Ron wrote: > But it's trivial to make pg_xlog a separate mount point. technically yes, but that would mean it would go to a different volume and hence destroy atomicity of the Netapp snapshots. Also, you'd have a huge administrative barrier against such

Re: Netapp SnapCenter

2020-06-18 Thread Paul Förster
Hi Stephen, > On 18. Jun, 2020, at 21:26, Stephen Frost wrote: > If the entire database, all tablespaces, and pg_wal, are on the same > volume and the snapshot of the volume is atomic, then you don't actually > need to go through the start/stop backup- a snapshot being restored will > look just

Re: Importing a Large .ndjson file

2020-06-18 Thread Sankar P
> Sankar P writes: > > I spoke too soon. While this worked fine when there were no indexes > > and finished within 10 minutes, with GIN index on the jsonb column, it > > is taking hours and still not completing. > > There's the generic advice that building an index after-the-fact > is often

Re: Parallel safety of contrib extensions

2020-06-18 Thread Michael Paquier
On Thu, Jun 18, 2020 at 02:26:04PM +, Winfield, Steven wrote: > Many thanks for the pointers - I've submitted a patch. Thanks Steve, I have noticed the patch. For the sake of the archives, it is here:

Re: create batch script to import into postgres tables

2020-06-18 Thread cgerard999
Remove the quotes around echo echo select count(*) from web_20200619; | "C:\Program Files\postgresql\11\bin\psql" -d *** -h *** -U *** or, store your query into a text file and use psql -f query.sql Sent from my mobile phone > Le 19 juin 2020 à 02:00, Adrian Klaver a écrit : > > On 6/18/20

Re: create batch script to import into postgres tables

2020-06-18 Thread Adrian Klaver
On 6/18/20 4:37 PM, Pepe TD Vo wrote: thank you for the link.  I did try it and it's still error echo 'SELECT count(*) FROM tableA;' | C:\Program Files\PostgreSQL\11\bin\psql -U PSmasteruser -d PSCIDR -h hostname.amazonaws.com -p 5432 This is getting old. The error is? I can run 'psql -u

Re: create batch script to import into postgres tables

2020-06-18 Thread Pepe TD Vo
thank you for the link.  I did try it and it's still errorecho 'SELECT count(*) FROM tableA;' | C:\Program Files\PostgreSQL\11\bin\psql -U PSmasteruser -d PSCIDR -h hostname.amazonaws.com -p 5432 I can run 'psql -u postgres -d PSCIDR -p 5432' (on Linux server, still learning how to run it from

Re: create batch script to import into postgres tables

2020-06-18 Thread Adrian Klaver
On 6/18/20 12:54 PM, Pepe TD Vo wrote: psql -d production  -U postgres -c 'select count(*) from cell_per'; Null display is "NULL".   count ---     68 (1 row) you can do this once you are in psql. But if you are running from shell script, it will be an error >>What error messages do

Re: Conflict with recovery on PG version 11.6

2020-06-18 Thread Toomas Kristin
Hi, > There can be other reasons: > > - replicated ACCESS EXCLUSIVE locks that conflict with queries > - replicated ACCESS EXCLUSIVE locks that cause deadlocks > - buffer pins that are needed for replication but held by a query > - dropped tablespaces that hold temporary files on the standby

Re: create batch script to import into postgres tables

2020-06-18 Thread Pepe TD Vo
>>psql -d production  -U postgres -c 'select count(*) from cell_per'; Null display is "NULL".   count ---     68 (1 row) you can do this once you are in psql. But if you are running from shell script, it will be an error >>What error messages do you get? my shell script is:@echo off 

Re: Netapp SnapCenter

2020-06-18 Thread Ron
On 6/18/20 11:49 AM, Paul Förster wrote: Hi Magnus, On 18. Jun, 2020, at 16:19, Magnus Hagander wrote: I don't know specifically about SnapCenter, but for snapshots in general, it does require backup mode *unless* all your data is on the same disk and you have an atomic snapshot across that

Re: Netapp SnapCenter

2020-06-18 Thread Stephen Frost
Greetings, * Paul Förster (paul.foers...@gmail.com) wrote: > > On 18. Jun, 2020, at 16:19, Magnus Hagander wrote: > > I don't know specifically about SnapCenter, but for snapshots in general, > > it does require backup mode *unless* all your data is on the same disk and > > you have an atomic

Re: create batch script to import into postgres tables

2020-06-18 Thread Adrian Klaver
On 6/18/20 9:40 AM, Pepe TD Vo wrote: I get this part that separates SQL script for import each table, (import.sql) begin; \copy table_1 (c1, c2, c3) from '/path/tabledata1.csv' csv header; commit; but when open the psql sql shell script it prompts line by line for localhost, port, db, user,

Re: Importing a Large .ndjson file

2020-06-18 Thread Tom Lane
Sankar P writes: > I spoke too soon. While this worked fine when there were no indexes > and finished within 10 minutes, with GIN index on the jsonb column, it > is taking hours and still not completing. There's the generic advice that building an index after-the-fact is often cheaper than

Re: Importing a Large .ndjson file

2020-06-18 Thread Michael Lewis
> > I spoke too soon. While this worked fine when there were no indexes > and finished within 10 minutes, with GIN index on the jsonb column, it > is taking hours and still not completing. > It is always recommended to create indexes AFTER loading data. Sometimes it can be faster to drop all

Re: Importing a Large .ndjson file

2020-06-18 Thread Sankar P
> > It looks like plain old COPY would do this just fine, along the lines > > of (in psql) > > > > \copy myTable(content) from 'myfile.ndjson' I spoke too soon. While this worked fine when there were no indexes and finished within 10 minutes, with GIN index on the jsonb column, it is taking hours

Re: Hiding a GUC from SQL

2020-06-18 Thread Michel Pelletier
On Thu, Jun 18, 2020 at 7:47 AM Laurenz Albe wrote: > On Wed, 2020-06-17 at 13:23 -0700, Michel Pelletier wrote: > > > > Any thoughts on weaknesses to this approach would be welcome. Thanks! > > A superuser can access files and start programs on the server machine. > > A dedicated superuser

Re: Netapp SnapCenter

2020-06-18 Thread Paul Förster
Hi Magnus, > On 18. Jun, 2020, at 16:19, Magnus Hagander wrote: > I don't know specifically about SnapCenter, but for snapshots in general, it > does require backup mode *unless* all your data is on the same disk and you > have an atomic snapshot across that disk (in theory it can be on

Re: create batch script to import into postgres tables

2020-06-18 Thread Pepe TD Vo
I get this part that separates SQL script for import each table,  (import.sql)begin; \copy table_1 (c1, c2, c3) from '/path/tabledata1.csv' csv header; commit; but when open the psql sql shell script it prompts line by line for localhost, port, db, user, and password.  If I set up a script and

Re: Hiding a GUC from SQL

2020-06-18 Thread Michel Pelletier
On Wed, Jun 17, 2020 at 3:55 PM Tom Lane wrote: > Michel Pelletier writes: > > In my extension pgsodium I'm defining a custom variable at startup to > store > > a key: > > > https://github.com/michelp/pgsodium/blob/master/src/pgsodium.c#L1107 > > > I'm using the flags GUC_NO_SHOW_ALL |

Re: create batch script to import into postgres tables

2020-06-18 Thread Adrian Klaver
On 6/18/20 8:20 AM, Pepe TD Vo wrote: Please don't top post. The preferred style on this list is inline or bottom posting(https://en.wikipedia.org/wiki/Posting_style). I have a Postgresql client installed and connected.  how can i create a batch script running from the client window?

Re: ESQL/C no indicator variables ./. error -213

2020-06-18 Thread Tom Lane
I wrote: > Hmm. Trying this here, I get > ... Oh, scratch that, I'd failed to duplicate the test case. (I had an empty string, not NULL, as the value of the tstchar25 column.) I concur with Laurenz's diagnosis that you need to use an indicator variable if you want to retrieve a NULL.

Re: create batch script to import into postgres tables

2020-06-18 Thread Pepe TD Vo
I have a Postgresql client installed and connected.  how can i create a batch script running from the client window? Bach-Nga No one in this world is pure and perfect.  If you avoid people for their mistakes you will be alone. So judge less, love, and forgive more.To call him a dog hardly

Re: ESQL/C no indicator variables ./. error -213

2020-06-18 Thread Tom Lane
Matthias Apitz writes: > El día Mittwoch, Juni 17, 2020 a las 01:39:53 -0400, Tom Lane escribió: >> Could you provide a self-contained test case for this? It's hard to >> guess at what the problem might be. > attached is a simple ESQL/C code; > ... > [6485] [18.06.2020 08:26:38:436]:

Re: autovacuum failing on pg_largeobject and disk usage of the pg_largeobject growing unchecked

2020-06-18 Thread Laurenz Albe
On Wed, 2020-06-17 at 14:26 -0400, Jim Hurne wrote: > On one of the instances that is exhibiting the "disk leak" behavior, the > VACUUM ANALYZE VERBOSE command doesn't generate any output or complete > before I loose the connection to the database (presumably because I hit a > connection read

Re: Hiding a GUC from SQL

2020-06-18 Thread Laurenz Albe
On Wed, 2020-06-17 at 13:23 -0700, Michel Pelletier wrote: > In my extension pgsodium I'm defining a custom variable at startup to store a > key: > > https://github.com/michelp/pgsodium/blob/master/src/pgsodium.c#L1107 > > I'm using the flags GUC_NO_SHOW_ALL | GUC_NO_RESET_ALL |

Re: Conflict with recovery on PG version 11.6

2020-06-18 Thread Laurenz Albe
On Thu, 2020-06-18 at 10:09 +0300, Toomas Kristin wrote: > What is difference between following error messages? > > ERROR: canceling statement due to conflict with recovery > FATAL: terminating connection due to conflict with recovery One kills your database session, the other doesn't. > I

Re: Conflict with recovery on PG version 11.6

2020-06-18 Thread Laurenz Albe
Toomas Kristin wrote: > > > Basically after upgrade to version 11.5 from 10.6 I experience error > > > messages on streaming > > > replica host “FATAL: terminating connection due to conflict with > > > recovery” and > > > “ERROR: canceling statement due to conflict with recovery”. There is no

Re: Parallel safety of contrib extensions

2020-06-18 Thread Winfield, Steven
> This would not be a complicated change as it requires creating a new > version script for those modules. You can look at commit 20eb273 as > an example, and the work is even simpler now that we just need update > scripts when bumping a module's version (those named foo--1.0--1.1.sql > and not

Re: ESQL/C no indicator variables ./. error -213

2020-06-18 Thread Laurenz Albe
On Wed, 2020-06-17 at 19:02 +0200, Matthias Apitz wrote: > We encountered that if our ESQL/C written servers see on SELECT or FETCH > in a row a NULL value, it will raise correctly the error -213 as written > and explained in >

Re: Netapp SnapCenter

2020-06-18 Thread Magnus Hagander
On Thu, Jun 18, 2020 at 4:07 PM Paul Förster wrote: > Hi Ken, > > > On 18. Jun, 2020, at 15:56, Wolff, Ken L wrote: > > PostgreSQL doesn’t need to be in a special mode for backups to work. > > this is curious. Doesn't the PostgreSQL cluster need to be set to backup > mode to use SnapCenter? >

Re: Netapp SnapCenter

2020-06-18 Thread Paul Förster
Hi Ken, > On 18. Jun, 2020, at 15:56, Wolff, Ken L wrote: > PostgreSQL doesn’t need to be in a special mode for backups to work. this is curious. Doesn't the PostgreSQL cluster need to be set to backup mode to use SnapCenter? The problem is, one can't test that and get a reliable answer,

RE: Netapp SnapCenter

2020-06-18 Thread Wolff, Ken L
We got the following from one of our NetApp contacts regarding a similar question. I'd be interested to hear if anyone feels the steps mentioned are not accurate: There is presently no NetApp supported plugin for PostgreSQL for SnapCenter. There is a community support plugin for

rejecting "interactive" connections

2020-06-18 Thread Luca Ferrari
Hi all, I'm just wondering if there is a way to discriminate a connection opened from an application server from one opened by an interactive client (e.g., psql) using the same credentials: the aim is to reject connections established from an interactive client rather than an application server. I

Re: i am getting issue as FATAL: password authentication failed for user ' postgres' tried

2020-06-18 Thread o1bigtenor
On Wed, Jun 17, 2020 at 9:14 PM prachi surangalikar wrote: > > hello team, > i have tried every thing but still i could not find the solution to this > problem. > i made changes in the pg_hba.conf file also , please help me to solve this > problem. As a noob to programming and under the

Netapp SnapCenter

2020-06-18 Thread Paul Förster
Hi, first of all, it wasn't my glorious idea and I know that volume snapshots are no backups. :-) Leaving that aside: Our company has decided to move away from Netapp SnapCreator to its SnapCenter. I was told there is some community plugin for PostgreSQL to use with SnapCenter but can't find

[HELP] General query - performance metrics

2020-06-18 Thread Praveen Kumar K S
Hello, I have set up pgpool and there are 2 postgres servers behind it. I had run pgbench with and without pgpool and got the results. I would like to showcase the results for a presentation. Just thought of asking you people if there are any tools to generate nice results with charts and graphs

Using postgresql and JSONB on very large datasets

2020-06-18 Thread Sankar P
Hi I have a table t, with just two columns, an `id SERIAL` and a `record JSONB`. I have about 3 million records where I save some JSON data in each record. Each `record` column jsonb has about 40 json key-values, of which about 10 are nested fields. Now, if I run a query, like: ``` select

Re: Importing a Large .ndjson file

2020-06-18 Thread Sankar P
> It looks like plain old COPY would do this just fine, along the lines > of (in psql) > > \copy myTable(content) from 'myfile.ndjson' > Indeed. Thanks. -- Sankar P http://psankar.blogspot.com

Re: ESQL/C no indicator variables ./. error -213

2020-06-18 Thread Matthias Apitz
El día Mittwoch, Juni 17, 2020 a las 01:39:53 -0400, Tom Lane escribió: > Matthias Apitz writes: > > We encountered that if our ESQL/C written servers see on SELECT or FETCH > > in a row a NULL value, it will raise correctly the error -213 as written > > and explained in > >

Re: Conflict with recovery on PG version 11.6

2020-06-18 Thread Toomas Kristin
Hi! What is difference between following error messages? ERROR: canceling statement due to conflict with recovery FATAL: terminating connection due to conflict with recovery I tried to correlate process termination on standby and vacuuming on master. Here is a sample timeline 1) 2020-06-17

Re: Table partitioning with sequence field in postgresql12

2020-06-18 Thread Tim Cross
Srinivasa T N writes: > Hi, >I have a parent table with one of the field as ""gid" int4 DEFAULT > nextval('"ami_smart_new".aoi_boundary_gid_seq'::regclass)". > >I create child tables which inherit parent and use hash partition. When > I directly insert into child tables, will there be

Re: Table partitioning with sequence field in postgresql12

2020-06-18 Thread Srinivasa T N
I am using declarative partitioning on some other field of the same table in which gid also exists. Regards, Seenu. On Thu, Jun 18, 2020 at 12:31 PM amul sul wrote: > On Thu, Jun 18, 2020 at 12:22 PM Srinivasa T N wrote: > > > > Hi, > >I have a parent table with one of the field as

Re: Table partitioning with sequence field in postgresql12

2020-06-18 Thread amul sul
On Thu, Jun 18, 2020 at 12:22 PM Srinivasa T N wrote: > > Hi, >I have a parent table with one of the field as ""gid" int4 DEFAULT > nextval('"ami_smart_new".aoi_boundary_gid_seq'::regclass)". > >I create child tables which inherit parent and use hash partition. When I > directly insert

Re: Table partitioning with sequence field in postgresql12

2020-06-18 Thread Tim Cross
Srinivasa T N writes: > Hi, >Partitioning of a table with sequence id as one of its fields is > supported in postgresql12? > > Regards, > Seenu. A sequence is really just an 'atomic' number generator, you get the next value, which is guaranteed to be larger than the last 'nextval' (up

Re: Table partitioning with sequence field in postgresql12

2020-06-18 Thread Srinivasa T N
Hi, I have a parent table with one of the field as ""gid" int4 DEFAULT nextval('"ami_smart_new".aoi_boundary_gid_seq'::regclass)". I create child tables which inherit parent and use hash partition. When I directly insert into child tables, will there be any race condition causing two child

Re: Table partitioning with sequence field in postgresql12

2020-06-18 Thread amul sul
On Thu, Jun 18, 2020 at 12:00 PM Srinivasa T N wrote: > > Hi, >Partitioning of a table with sequence id as one of its fields is supported > in postgresql12? > Could you please elaborate on your case a bit more? Regards, Amul

Table partitioning with sequence field in postgresql12

2020-06-18 Thread Srinivasa T N
Hi, Partitioning of a table with sequence id as one of its fields is supported in postgresql12? Regards, Seenu.