Possible improvement

2020-06-04 Thread Paul van der Linden
Hi, Don't know if this already came up earlier but I have an idea for improvement. If I have a query like: SELECT * FROM ( SELECT CASE WHEN field='value1' THEN 1 WHEN field='value2' THEN 2 END AS category FROM table1 ) AS foo WHERE category=1 doesn't use the index on field,

Multitenent architecture

2020-06-04 Thread Vasu Madhineni
Hi All, We are planning a POC on multitenant architecture in Postgres, Could you please help us with steps for multitenant using schema for each application model. Thank you so much all. Regards, Vasu

Re: Binary encoding of TIMESTAMP WITH TIME ZONE

2020-06-04 Thread Tom Lane
Joe Abbate writes: > However, when using the same query using the Rust adapter the transition > to a new row started showing up after midgnight GMT. I opened an issue > on Github (https://github.com/sfackler/rust-postgres/issues/608 ) and > the maintainer claimed the Rust adapter *had* to

Binary encoding of TIMESTAMP WITH TIME ZONE

2020-06-04 Thread Joe Abbate
I'm dealing with an issue where a query uses 'today'::date to select one of a number of rows depending on the day modulo the number of rows. The intent is that different information will be shown starting after midnight local time. The query runs as expected in psql and using psycopg2.

Re: Move configuration files with pg_upgrade

2020-06-04 Thread Adrian Klaver
On 6/4/20 12:52 PM, Tanja Savic wrote: Hello, I did Postgresql upgrade from v10 to v12 using pg_upgrade (on Linux Ubuntu server). Of course i wanted settings inpg_hba.conf and postgresql.conf to keep, but after the upgrade there were new configuration files and I moved it manually. That is

Re: Move configuration files with pg_upgrade

2020-06-04 Thread Peter J. Holzer
On 2020-06-04 19:52:19 +, Tanja Savic wrote: > I did Postgresql upgrade from v10 to v12 using pg_upgrade (on Linux Ubuntu > server). Did you use the packages from the PGDG server or from Ubuntu? > Of course i wanted settings in pg_hba.conf and postgresql.conf to keep, but > after the

Re: Monitoring for long running transactions

2020-06-04 Thread Thomas Kellerer
Samuel Smith schrieb am 04.06.2020 um 21:59: Sorry, I should have clarified that I was aware of the pg_stat_activity table. That is how we found the problem in the first place. And yes I could just write a bash script and run it in cron. I just didn't know if there was a more "official" way to

Re: Monitoring for long running transactions

2020-06-04 Thread Christoph Moench-Tegeder
## Samuel Smith (pg...@net153.net): > Sorry, I should have clarified that I was aware of the pg_stat_activity > table. That is how we found the problem in the first place. And yes I > could just write a bash script and run it in cron. I just didn't know if > there was a more "official" way to

Re: Monitoring for long running transactions

2020-06-04 Thread Adrian Klaver
On 6/4/20 12:59 PM, Samuel Smith wrote: On 6/4/20 2:29 PM, Adrian Klaver wrote: Sorry, I should have clarified that I was aware of the pg_stat_activity table. That is how we found the problem in the first place. And yes I could just write a bash script and run it in cron. I just didn't

Re: Shared memory error

2020-06-04 Thread Thomas Munro
On Fri, Jun 5, 2020 at 1:00 AM Sonam Sharma wrote: > The dynamic_shared_memory_type was set to POSIX . Because of this it was > using tmpfs /dev/shm. When the query was running I saw the file system was > filling. So I extended the file system and luckily the query worked for that > time Oh,

Re: Monitoring for long running transactions

2020-06-04 Thread Samuel Smith
On 6/4/20 2:29 PM, Adrian Klaver wrote: On 6/4/20 10:00 AM, Samuel Smith wrote: We had a customer complaining of random data loss for the last 6 months or so. We eventually tracked it down to a combination of bad coding and a couple of bugs with the ORM. Basically, 'BEGIN' was being emitted

Move configuration files with pg_upgrade

2020-06-04 Thread Tanja Savic
Hello, I did Postgresql upgrade from v10 to v12 using pg_upgrade (on Linux Ubuntu server). Of course i wanted settings in pg_hba.conf and postgresql.conf to keep, but after the upgrade there were new configuration files and I moved it manually. Is it possible somehow to migrate these old

Re: Monitoring for long running transactions

2020-06-04 Thread Adrian Klaver
On 6/4/20 10:00 AM, Samuel Smith wrote: We had a customer complaining of random data loss for the last 6 months or so. We eventually tracked it down to a combination of bad coding and a couple of bugs with the ORM. Basically, 'BEGIN' was being emitted by the web app when viewing a certain page

Re: PostgreSQL 11 with SSL on Linux

2020-06-04 Thread Susan Joseph
OK thanks, I figured I had something messed up in my thought process.  I will work through these steps and let you know how it goes. Susan -Original Message- From: Mohammed Bhatti To: Susan Joseph Cc: ahodg...@lists.simkin.ca ; pgsql-general@lists.postgresql.org Sent: Thu, Jun 4,

Re: PostgreSQL 11 with SSL on Linux

2020-06-04 Thread Mohammed Bhatti
On Thu, Jun 4, 2020 at 1:50 PM Susan Joseph wrote: > OK, when I went to PostgreSQL to get the rpm file to install I was given: > > yum install > https://download.postgresql.org/pub/repos/yum/reporpms/EL-7-x86_64/pgdg-redhat-repo-latest.noarch.rpm > > I put the file

Re: PostgreSQL 11 with SSL on Linux

2020-06-04 Thread Tom Lane
Susan Joseph writes: > OK, when I went to PostgreSQL to get the rpm file to install I was given: > yum install > https://download.postgresql.org/pub/repos/yum/reporpms/EL-7-x86_64/pgdg-redhat-repo-latest.noarch.rpm > I put the file pgdg-redhat-repo-latest.noarch.rpm in my /tmp directory and >

Re: PostgreSQL 11 with SSL on Linux

2020-06-04 Thread Susan Joseph
OK, when I went to PostgreSQL to get the rpm file to install I was given: yum install https://download.postgresql.org/pub/repos/yum/reporpms/EL-7-x86_64/pgdg-redhat-repo-latest.noarch.rpm I put the file pgdg-redhat-repo-latest.noarch.rpm in my /tmp directory and ran rpm -i on the file. If I

Re: PostgreSQL 11 with SSL on Linux

2020-06-04 Thread Tom Lane
Susan Joseph writes: > So when I run rpm -i it says that the package is installed, but I can't find > a postgresql directory with all the files and executables.  So what am I > missing? Looking in the right place, perhaps ;-). If the package is installed then "rpm -ql" will list all the files

Re: PostgreSQL 11 with SSL on Linux

2020-06-04 Thread Alan Hodgson
On Thu, 2020-06-04 at 17:32 +, Susan Joseph wrote: > So when I run rpm -i it says that the package is installed, but I > can't find a postgresql directory with all the files and executables. > So what am I missing? > > > > > > The server stuff is in postgresql11-serverif you're using

Re: PostgreSQL 11 with SSL on Linux

2020-06-04 Thread Susan Joseph
So when I run rpm -i it says that the package is installed, but I can't find a postgresql directory with all the files and executables.  So what am I missing? -Original Message- From: Tom Lane To: Susan Joseph Cc: mag...@hagander.net ; pgsql-general@lists.postgresql.org Sent:

Re: PostgreSQL 11 with SSL on Linux

2020-06-04 Thread Tom Lane
Susan Joseph writes: > So the other issue I have is that I am running this on a server that is not > connected to the Internet.  So I have downloaded the RPM file but I can't > figure out how to install it without it trying to access files on the > Internet.  Are there other libraries I need

Re: Index no longer being used, destroying and recreating it restores use.

2020-06-04 Thread Tom Lane
Adrian Klaver writes: > On 6/4/20 9:43 AM, Tom Lane wrote: >> It's possible that the index had bloated to the point where the planner >> thought it was cheaper to use a seqscan. Did you make a note of the >> cost estimates for the different plans? > I missed the part where the OP pointed to a

Monitoring for long running transactions

2020-06-04 Thread Samuel Smith
We had a customer complaining of random data loss for the last 6 months or so. We eventually tracked it down to a combination of bad coding and a couple of bugs with the ORM. Basically, 'BEGIN' was being emitted by the web app when viewing a certain page and 'COMMIT' was never emitted after

Re: PostgreSQL 11 with SSL on Linux

2020-06-04 Thread Susan Joseph
So the other issue I have is that I am running this on a server that is not connected to the Internet.  So I have downloaded the RPM file but I can't figure out how to install it without it trying to access files on the Internet.  Are there other libraries I need to include in my download for

Re: Index no longer being used, destroying and recreating it restores use.

2020-06-04 Thread Adrian Klaver
On 6/4/20 9:43 AM, Tom Lane wrote: Koen De Groote writes: I've got a table with about 30 million rows and a particular index that up until recently was actively being used. And then it stopped being used and the query that the index was made for, is now doing sequential scans. Deleting the

Re: Index no longer being used, destroying and recreating it restores use.

2020-06-04 Thread Tom Lane
Koen De Groote writes: > I've got a table with about 30 million rows and a particular index that up > until recently was actively being used. > And then it stopped being used and the query that the index was made for, > is now doing sequential scans. > Deleting the index and creating it again,

Re: Index no longer being used, destroying and recreating it restores use.

2020-06-04 Thread Adrian Klaver
On 6/4/20 8:37 AM, Koen De Groote wrote: Greetings, The following is using version 11.2 of PostgreSQL. I've got a table with about 30 million rows and a particular index that up until recently was actively being used. And then it stopped being used and the query that the index was made

Index no longer being used, destroying and recreating it restores use.

2020-06-04 Thread Koen De Groote
Greetings, The following is using version 11.2 of PostgreSQL. I've got a table with about 30 million rows and a particular index that up until recently was actively being used. And then it stopped being used and the query that the index was made for, is now doing sequential scans. Deleting the

Re: PostgreSQL 11 with SSL on Linux

2020-06-04 Thread Magnus Hagander
On Thu, Jun 4, 2020 at 5:01 PM Susan Joseph wrote: > Can someone tell me if there is a yum version of PostgreSQL 11 that can be > installed on Linux that has SSL enabled? Currently the only way I have > gotten SSL turned on in PostgreSQL is by doing the following commands: > > >- *tar xzvf

Re: Can we get SQL Server-like cross database queries

2020-06-04 Thread David G. Johnston
On Wednesday, June 3, 2020, Laurenz Albe wrote: > On Thu, 2020-06-04 at 16:41 +1200, Thomas Munro wrote: > > There's no doubt it's useful, and it's also part of the SQL spec, > > which says you can do catalog.schema.table. I would guess that we > > might get that as a byproduct of any project

PostgreSQL 11 with SSL on Linux

2020-06-04 Thread Susan Joseph
Can someone tell me if there is a yum version of PostgreSQL 11 that can be installed on Linux that has SSL enabled?  Currently the only way I have gotten SSL turned on in PostgreSQL is by doing the following commands: - tar xzvf /tmp/postgresql-11.2.tar.gz -C /data - mv

Re: Fine grained permissions on User Mapping

2020-06-04 Thread Paul Bonaud
Hi again Tom, I re-read your point 2. “You don't want to grant USAGE on the foreign server to the localuser, either.” to find out this was exactly the solution I was looking for. That is: it's fine to not let the basic user create the foreign tables. Wow, it was as easy as moving the foreign

Re: When to use PARTITION BY HASH?

2020-06-04 Thread Jeff Janes
On Wed, Jun 3, 2020 at 7:55 AM Oleksandr Shulgin < oleksandr.shul...@zalando.de> wrote: With hash partitioning you are not expected, in general, to end up with a > small number of partitions being accessed more heavily than the rest. So > your indexes will also not fit into memory. > > I have

Re: Can we get SQL Server-like cross database queries

2020-06-04 Thread Matt Zagrabelny
On Thu, Jun 4, 2020 at 12:56 AM Laurenz Albe wrote: > On Thu, 2020-06-04 at 16:41 +1200, Thomas Munro wrote: > > There's no doubt it's useful, and it's also part of the SQL spec, > > which says you can do catalog.schema.table. I would guess that we > > might get that as a byproduct of any

Re: Shared memory error

2020-06-04 Thread Sonam Sharma
The dynamic_shared_memory_type was set to POSIX . Because of this it was using tmpfs /dev/shm. When the query was running I saw the file system was filling. So I extended the file system and luckily the query worked for that time On Thu, Jun 4, 2020, 5:12 PM Thomas Munro wrote: > >> Do you see

Re: Shared memory error

2020-06-04 Thread Thomas Munro
>> Do you see any other errors around this one, in the PostgreSQL logs? > No , only this is the error from db and jdbc end .. and queries are failing If you need a workaround right now you could always set max_parallel_workers_per_gather=0 so that it doesn't try to use parallel query. That could

Re: Calculate hardware requirements

2020-06-04 Thread Praveen Kumar K S
Thank you for the detailed reply. I understand my question is generic. But just thought if I can get some good place to start. I will look into the suggestions you made. On Thu, Jun 4, 2020 at 4:02 PM Fabio Pardi wrote: > Hi, > > in my opinion your question is too generic to get an accurate

Re: Calculate hardware requirements

2020-06-04 Thread Fabio Pardi
Hi, in my opinion your question is too generic to get an accurate answer. To educate yourself reading Postgres docs or some good books would be in my opinion the best way to give an answer yourself to your own question. Then you can still post to the ML on some specific setting (postgres

Calculate hardware requirements

2020-06-04 Thread Praveen Kumar K S
Hello All, I'm looking for hardware configurations to set up 1 master and 2 hot-standby slaves using 9.6 in one DC. Also, I need to have DR with the same setup with cross-site replication enabled. I went through a lot of docs/blogs suggesting 4cores and at least 4/8GB RAM. But I'm looking for

Re: select count(id) on RDS replica causing high CPU load on RDS master

2020-06-04 Thread Azul
Thanks Jeremy, Plan is on the cards to upgrade to 10 this or next week, and then later on towards v12. hot_standby_feedback is enabled, yes. So the process list only shows high cpu usage for the postgres process itself, and all other ones are < 1.5%. it's a bit baffling as I don't have anything

Re: Shared memory error

2020-06-04 Thread Thomas Munro
On Thu, Jun 4, 2020 at 6:18 AM Sonam Sharma wrote: >>> 1) Postgres version : 11.2 FYI This is missing over a year's worth of bugfixes. That said, I don't know of anything fixed that has this symptom. >>> 4) Is this only with one query and if so what is it doing? : No , few >>> queries work,

Re: Shared memory error

2020-06-04 Thread Naresh g
Hi, If the dynamic shared memory segment type is POSIX then *dynamic shared memory control segment in your case it is **PostgreSQL.1511679208* is stored in */dev/shm* The file you mentioned will be created at the time of PostgreSQL startup and cleared automatically on successful shutdown.

Re: Shared memory error

2020-06-04 Thread Sonam Sharma
Can someone please help on this On Wed, Jun 3, 2020, 11:27 PM Sonam Sharma wrote: > I have set the dynamic_shared_memory_type to none which was set as POSIX. > For now we are not seeing the error... Can someone please explain what is > the impact of setting it as none . > > On Wed, Jun 3, 2020,

Potential optimisation for the creation of a partial index condition over a newly created nullable, non-default-valued column?

2020-06-04 Thread Tim Dawborn
Hi all, I was playing around with ways to make a schema change recently to a ~30M record table. I wanted to add a new nullable, non-default-valued column to this existing table, and then add a new partial to that table, where the partial index condition refers to a value in that newly added