Statistics information.

2024-03-22 Thread arun chirappurath
Dear All, Apologies the way i am asking question as i am more a SQL Server person and a new postgre man.. I have used a query store in SQL server. it provides me option to load statistics data to temp table and get below important information. 1. Last run duration 2. Average time for

Re: Seq scan vs index scan

2024-03-22 Thread arun chirappurath
Thanks Tom,David and Chris for detailed opinions Regards, Arun On Sat, 23 Mar 2024 at 09:25, arun chirappurath wrote: > Hi All, > > I have a table named users with index on user name. > > CREATE TABLE users ( > user_id SERIAL PRIMARY KEY, > username VARCHAR(50) NOT NULL, > email

Re: Seq scan vs index scan

2024-03-22 Thread Tom Lane
arun chirappurath writes: > I have a table named users with index on user name. > ... > When I try to do below select query it's taking seq scan and query returns > in 5ms. 5ms is an okay runtime, I would think. Is the table empty? > I am trying to force query to use indexes using query

Re: Seq scan vs index scan

2024-03-22 Thread David G. Johnston
On Fri, Mar 22, 2024 at 8:55 PM arun chirappurath wrote: > > I am trying to force query to use indexes using query hints. > > Set enable indexscan to ON, > Same for bitmap and index only scan > Everything is on by default in the planner. You need to think in terms of what you don't want to

Re: Seq scan vs index scan

2024-03-22 Thread Christophe Pettus
> On Mar 22, 2024, at 20:55, arun chirappurath wrote: > I am trying to force query to use indexes using query hints. PostgreSQL does not have query hints. Enabling index scans using parameters doesn't *disable* other types of query nodes. You can disable sequential scans using:

Seq scan vs index scan

2024-03-22 Thread arun chirappurath
Hi All, I have a table named users with index on user name. CREATE TABLE users ( user_id SERIAL PRIMARY KEY, username VARCHAR(50) NOT NULL, email VARCHAR(100) UNIQUE NOT NULL, age INT ); CREATE INDEX idx_username ON users (username); When I try to do below select query it's

Re: PostgreSQL as advanced job queuing system

2024-03-22 Thread Merlin Moncure
On Fri, Mar 22, 2024 at 6:58 AM ushi wrote: > Hello List, > > i am playing with the idea to implement a job queuing system using > PostgreSQL. To meet requirements the system needs to offer some advanced > features compared to "classic" queuing systems: > > - users can create new queues at any

Re: PostgreSQL as advanced job queuing system

2024-03-22 Thread Shaheed Haque
Generally, I'd suggest you think carefully about the nature of the jobs, and draw up a list of must-have properties (performance of course, but also things like whether jobs have to survive planned or unplanned outages, be visible across a WAN, numbers of readers and writers, delivery guarantees,

Re: Timing out A Blocker Based on Time or Count of Waiters

2024-03-22 Thread Adrian Klaver
On 3/22/24 12:41, Fred Habash wrote: Lock tree: All PID's waiting on a lock held by/blocked by single blocker PID. Similar to what you see in the output of this script: https://github.com/dataegret/pg-utils/blob/master/sql/locktree.sql

Re: Timing out A Blocker Based on Time or Count of Waiters

2024-03-22 Thread Fred Habash
Lock tree: All PID's waiting on a lock held by/blocked by single blocker PID. Similar to what you see in the output of this script: https://github.com/dataegret/pg-utils/blob/master/sql/locktree.sql . It uses the dot connotation to draw a tree. Waiters: The PID (first column) returned by this

Re: soft lockup - CPU#16 stuck for 3124s! [postmaster:2273]

2024-03-22 Thread Ron Johnson
On Fri, Mar 22, 2024 at 1:27 PM Tom Lane wrote: > Matthias Apitz writes: > > We have a PostgreSQL 15.1 server in production at a customer for some > > weeks (migrated from an older version) on SuSE SLES 15. > > > The customer is facing machine locks and before the Linux server does > > not

Re: soft lockup - CPU#16 stuck for 3124s! [postmaster:2273]

2024-03-22 Thread Tom Lane
Matthias Apitz writes: > We have a PostgreSQL 15.1 server in production at a customer for some > weeks (migrated from an older version) on SuSE SLES 15. > The customer is facing machine locks and before the Linux server does > not respond any more (not even on SSH, only power-cycle reset helps

soft lockup - CPU#16 stuck for 3124s! [postmaster:2273]

2024-03-22 Thread Matthias Apitz
We have a PostgreSQL 15.1 server in production at a customer for some weeks (migrated from an older version) on SuSE SLES 15. The customer is facing machine locks and before the Linux server does not respond any more (not even on SSH, only power-cycle reset helps to get it back), short before

Re: Timing out A Blocker Based on Time or Count of Waiters

2024-03-22 Thread Christophe Pettus
> On Mar 22, 2024, at 09:25, Fred Habash wrote: > > Facing an issue where sometimes humans login to a database and run DDL > statements causing a long locking tree of over 1000 waiters. As a workaround, > we asked developers to always start their DDL sessions with 'SET lock_timeout > =

Re: Timing out A Blocker Based on Time or Count of Waiters

2024-03-22 Thread Adrian Klaver
On 3/22/24 09:25, Fred Habash wrote: Facing an issue where sometimes humans login to a database and run DDL statements causing a long locking tree of over 1000 waiters. As a The above needs more explanation: 1) Define locking tree. 2) Define waiters. 3) Provide examples of the DDL.

Timing out A Blocker Based on Time or Count of Waiters

2024-03-22 Thread Fred Habash
Facing an issue where sometimes humans login to a database and run DDL statements causing a long locking tree of over 1000 waiters. As a workaround, we asked developers to always start their DDL sessions with 'SET lock_timeout = 'Xs'. I reviewed the native lock timeout parameter in Postgres and

Re: PostgreSQL as advanced job queuing system

2024-03-22 Thread Thiemo Kellner
Am 22.03.2024 um 14:15 schrieb Fred Habash: We developed a home-grown queue system using Postgres, but its performance was largely hindered by que tables bloating and the need to continuously vacuum them. It did not scale whatsoever. With some workarounds, we ended up designing three sets

Re: could not open file "global/pg_filenode.map": Operation not permitted

2024-03-22 Thread Alban Hertroys
On Fri, 22 Mar 2024 at 15:01, Nick Renders wrote: > > We now have a second machine with this issue: it is an Intel Mac mini > running macOS Sonoma (14.4) and PostgreSQL 16.2. > This one only has a single Data directory, so there are no multiple > instances running. > I don't think that having a

Re: could not open file "global/pg_filenode.map": Operation not permitted

2024-03-22 Thread Adrian Klaver
On 3/22/24 07:01, Nick Renders wrote: On 13 Mar 2024, at 12:35, Stephen Frost wrote: We now have a second machine with this issue: it is an Intel Mac mini running macOS Sonoma (14.4) and PostgreSQL 16.2. This one only has a single Data directory, so there are no multiple instances running.

Re: pg_locks-exclusivelock for select queries

2024-03-22 Thread Tom Lane
arun chirappurath writes: > I am running below query on a database. why is it creating a exclusive lock > on a virtualxid? I am running some SELECT queries and its creating an > ExclusiveLock in virtualxid? is this normal? Yes. That lock has nothing to do with any table, only with the

Re: could not open file "global/pg_filenode.map": Operation not permitted

2024-03-22 Thread Nick Renders
On 13 Mar 2024, at 12:35, Stephen Frost wrote: > Greetings, > > * Nick Renders (postg...@arcict.com) wrote: >>> ...run them under different users on the system. >> >> Are you referring to the "postgres" user / role? Does that also mean setting >> up 2 postgres installation directories? > > Yes,

Re: PostgreSQL as advanced job queuing system

2024-03-22 Thread Fred Habash
We developed a home-grown queue system using Postgres, but its performance was largely hindered by que tables bloating and the need to continuously vacuum them. It did not scale whatsoever. With some workarounds, we ended up designing three sets of queue tables, switching between them based on

Re: PostgreSQL as advanced job queuing system

2024-03-22 Thread Peter J. Holzer
On 2024-03-22 12:43:40 +0100, ushi wrote: > i am playing with the idea to implement a job queuing system using > PostgreSQL. To meet requirements the system needs to offer some advanced > features compared to "classic" queuing systems: > > - users can create new queues at any time > - queues have

Re: uncommitted xmin 3100586 from before xid cutoff 10339367 needs to be frozen

2024-03-22 Thread Laurenz Albe
On Fri, 2024-03-22 at 16:07 +0530, Vijaykumar Jain wrote: > On Fri, 22 Mar 2024 at 15:39, Laurenz Albe wrote: > > On Fri, 2024-03-22 at 13:41 +0530, Daulat wrote: > > > We are unable to take the backup of our database. While taking backup we > > > are getting the same error.  > > > > > > psql:

Re: PostgreSQL as advanced job queuing system

2024-03-22 Thread Robert Treat
On Fri, Mar 22, 2024 at 8:05 AM Dominique Devienne wrote: > > On Fri, Mar 22, 2024 at 12:58 PM ushi wrote: >> >> i am playing with the idea to implement a job queuing system using >> PostgreSQL. > > > FYI, two bookmarks I have on this subject, that I plan to revisit eventually: > *

Re: PostgreSQL as advanced job queuing system

2024-03-22 Thread Dominique Devienne
On Fri, Mar 22, 2024 at 12:58 PM ushi wrote: > i am playing with the idea to implement a job queuing system using > PostgreSQL. FYI, two bookmarks I have on this subject, that I plan to revisit eventually: * https://news.ycombinator.com/item?id=20020501 *

PostgreSQL as advanced job queuing system

2024-03-22 Thread ushi
Hello List, i am playing with the idea to implement a job queuing system using PostgreSQL. To meet requirements the system needs to offer some advanced features compared to "classic" queuing systems: - users can create new queues at any time - queues have priorities - priorities of queues can

Re: uncommitted xmin 3100586 from before xid cutoff 10339367 needs to be frozen

2024-03-22 Thread Vijaykumar Jain
On Fri, 22 Mar 2024 at 15:39, Laurenz Albe wrote: > On Fri, 2024-03-22 at 13:41 +0530, Daulat wrote: > > We are unable to take the backup of our database. While taking backup we > are getting the same error. > > > > psql: error: connection to server at "localhost" (::1), port 5014 > failed:

Re: pg_locks-exclusivelock for select queries

2024-03-22 Thread Laurenz Albe
On Fri, 2024-03-22 at 12:43 +0530, arun chirappurath wrote: > I am running below query on a database. why is it creating a exclusive lock > on a virtualxid? > I am running some SELECT queries and its creating an ExclusiveLock in > virtualxid? is this normal? > > SELECT datname, pid, state,

Re: uncommitted xmin 3100586 from before xid cutoff 10339367 needs to be frozen

2024-03-22 Thread Laurenz Albe
On Fri, 2024-03-22 at 13:41 +0530, Daulat wrote: > We are unable to take the backup of our database. While taking backup we are > getting the same error.  > > psql: error: connection to server at "localhost" (::1), port 5014 failed: > FATAL:  pg_attribute catalog is missing 1 attribute(s) for

Re: uncommitted xmin 3100586 from before xid cutoff 10339367 needs to be frozen

2024-03-22 Thread Daulat
We are unable to take the backup of our database. While taking backup we are getting the same error. psql: error: connection to server at "localhost" (::1), port 5014 failed: FATAL: pg_attribute catalog is missing 1 attribute(s) for relation OID 2662 Thanks. On Fri, Mar 22, 2024 at 12:35 PM

pg_locks-exclusivelock for select queries

2024-03-22 Thread arun chirappurath
Dear all, I am running below query on a database. why is it creating a exclusive lock on a virtualxid? I am running some SELECT queries and its creating an ExclusiveLock in virtualxid? is this normal? SELECT datname, pid, state, query, age(clock_timestamp(), query_start) AS age FROM

Re: uncommitted xmin 3100586 from before xid cutoff 10339367 needs to be frozen

2024-03-22 Thread Laurenz Albe
On Fri, 2024-03-22 at 10:56 +0530, Daulat wrote: > We recently started seeing an error “ERROR:  uncommitted xmin 3100586 > from before xid cutoff 10339367 needs to be frozen” on our user tables. > I’m unable to do ‘vacuum’, ‘vacuum freeze’ or ‘vacuum full’ on Postgres 14.4 > running on a windows