Re: Unnecessary buffer usage with multicolumn index, row comparison, and equility constraint

2024-05-10 Thread Ron Johnson
On Fri, May 10, 2024 at 11:28 PM WU Yan <4wu...@gmail.com> wrote: > Hi everyone, first time here. Please kindly let me know if this is not the > right place to ask. > > I notice a simple query can read a lot of buffer blocks in a meaningless > way, when > 1. there is an index scan on a

Forcing INTERVAL days display, even if the interval is less than one day

2024-05-07 Thread Ron Johnson
PG 9.6.24, if relevant. (Hopefully we're migrating next month.) Displaying how long ago a date was is easy, but interval casts "helpfully" suppress "X days ago" if the interval is less than one day ago. How do I make it display "days ago", even when days ago is zero? Explicitly casting "day to

Re: Ora2pg Delta Migration: Oracle to PostgreSQL

2024-05-02 Thread Ron Johnson
On Thu, May 2, 2024 at 8:28 PM Amit Sharma wrote: > Hello, > > Has anyone tried delta/incremental data migration for Oracle to PostgreSQL > using Ora2pg? Or what are the best options to run delta migration for > Oracle to PostgreSQL? > What do the ora2pg docs say about whether or not that

Re: Prevent users from executing pg_dump against tables

2024-05-02 Thread Ron Johnson
On Thu, May 2, 2024 at 1:47 AM RAJAMOHAN wrote: > Hello all, > > In our production db infrastructure, we have one read_only role which has > read privileges against all tables in schema A. > > We are planning to grant this role to some developers for viewing the > data, but also I want to limit

Re: Linked directory or explicit reference

2024-05-02 Thread Ron Johnson
On Thu, May 2, 2024 at 12:50 AM Senor Cervesa wrote: [snip] > I'm not sure what would trigger "directory not empty". The lost+found directory.

Re: Posgresql 14 and CarbonBlack on RHEL8?

2024-05-01 Thread Ron Johnson
On Tue, Apr 30, 2024 at 10:07 PM Tom Lane wrote: > Ron Johnson writes: > > When running stress tests on the systems (in prod, during the maintenance > > window), 171K events/second are generated on the RHEL8 servers, and CB > > needs (according to top(1)) 325% of CPU to h

Posgresql 14 and CarbonBlack on RHEL8?

2024-04-30 Thread Ron Johnson
(CarbonBlack is cross-platform AV software sold by VMware.) Currently we're running PG 9.6.24 on RHEL 6.10 with CB (version unknown to me) in production, and testing PG 14.11 on RHEL 8.9 with CB 2.15.2 (hopefully going into production next month). Both old and new VMs are 32 CPU with 128GB RAM.

Re: Linked directory or explicit reference

2024-04-30 Thread Ron Johnson
On Tue, Apr 30, 2024 at 7:00 PM Senor Cervesa wrote: > Hi All; > > When doing an initial install of PostgreSQL on RHEL 7 or 8 derived OS via > rpm, what are pros, cons and recommendations of these 2 procedures for > utilizing a second disk? > > Secondary SSD or RAID mounted at /disk2. > > Option

Re: best migration solution

2024-04-25 Thread Ron Johnson
On Thu, Apr 25, 2024 at 3:55 AM Zwettler Markus (OIZ) < markus.zwett...@zuerich.ch> wrote: > we have to migrate from hosted PG12 to containerized PG16 on private > cloud. > > > > some of the installed PG12 extensions are not offered on the containerized > PG16, eg. PostGIS related extensions like

Re: Backup_Long Running

2024-04-24 Thread Ron Johnson
PgBackRest is in the PGDG repositories (RHEL & Debian). The documentation is thorough, and discoverable via Google, On Wed, Apr 24, 2024 at 8:30 AM jaya kumar wrote: > Thanks for your update. Can you have any link or document to configure L0 > & L1 backup using pgbackrest tool. Also share the

Re: CLUSTER vs. VACUUM FULL

2024-04-22 Thread Ron Johnson
On Mon, Apr 22, 2024 at 5:56 PM Adrian Klaver wrote: > On 4/22/24 14:35, Ron Johnson wrote: > > > > > > > On Stack Exchange, I've got a question on how to determine when > > to run > > > CLUSTER. It ties in strongly with this threa

Re: issue with reading hostname

2024-04-22 Thread Ron Johnson
; Can we edit the socket to change the hostname in it ? > > Regards. > > On Tue, Apr 23, 2024 at 2:41 AM Ron Johnson > wrote: > >> On Mon, Apr 22, 2024 at 4:14 PM Atul Kumar wrote: >> >>> Hi, >>> >>> I have postgresql version 15 runnin

Re: CLUSTER vs. VACUUM FULL

2024-04-22 Thread Ron Johnson
On Mon, Apr 22, 2024 at 5:03 PM Adrian Klaver wrote: > On 4/22/24 13:59, Ron Johnson wrote: > > On Mon, Apr 22, 2024 at 4:21 PM Adrian Klaver > <mailto:adrian.kla...@aklaver.com>> wrote: > > [snip] > > > > Which gets us back to your comment upst

Re: issue with reading hostname

2024-04-22 Thread Ron Johnson
On Mon, Apr 22, 2024 at 4:14 PM Atul Kumar wrote: > Hi, > > I have postgresql version 15 running on centos7. > > I have below query that reads hostname from /tmp directory: > > psql postgres -A -t -p 5432 -h /tmp/ -c 'SELECT pg_is_in_recovery();' > If you installed from the PGDG repository

Re: CLUSTER vs. VACUUM FULL

2024-04-22 Thread Ron Johnson
On Mon, Apr 22, 2024 at 4:21 PM Adrian Klaver wrote: [snip] > Which gets us back to your comment upstream: > > "What the VACUUM docs "should" do, it seems, is suggest CLUSTER on the > PK, if the PK is a sequence (whether that be an actual sequence, or a > timestamp or something else that grows

Re: CLUSTER vs. VACUUM FULL

2024-04-22 Thread Ron Johnson
On Mon, Apr 22, 2024 at 3:14 PM Adrian Klaver wrote: > > > On 4/22/24 11:45 AM, Ron Johnson wrote: > > On Mon, Apr 22, 2024 at 12:29 PM David G. Johnston > > mailto:david.g.johns...@gmail.com>> wrote: > > > > > > > > On Mon, Apr 22

Re: CLUSTER vs. VACUUM FULL

2024-04-22 Thread Ron Johnson
On Mon, Apr 22, 2024 at 12:29 PM David G. Johnston < david.g.johns...@gmail.com> wrote: > > > On Mon, Apr 22, 2024, 08:37 Ron Johnson wrote: > >> On Mon, Apr 22, 2024 at 10:25 AM Tom Lane wrote: >> >>> Marcos Pegoraro writes: >>> > But w

Re: CLUSTER vs. VACUUM FULL

2024-04-22 Thread Ron Johnson
On Mon, Apr 22, 2024 at 10:25 AM Tom Lane wrote: > Marcos Pegoraro writes: > > But wouldn't it be good that VACUUM FULL uses that index defined by > > Cluster, if it exists ? > > No ... what would be the difference then? > What the VACUUM docs "should" do, it seems, is suggest CLUSTER on the

Re: CLUSTER vs. VACUUM FULL

2024-04-21 Thread Ron Johnson
On Sun, Apr 21, 2024 at 9:35 PM David Rowley wrote: > On Mon, 22 Apr 2024 at 12:16, Ron Johnson wrote: > > > > On Sun, Apr 21, 2024 at 6:45 PM Tom Lane wrote: > >> > >> Ron Johnson writes: > >> > Why is VACUUM FULL recommended for compressing a tab

Re: CLUSTER vs. VACUUM FULL

2024-04-21 Thread Ron Johnson
On Sun, Apr 21, 2024 at 6:45 PM Tom Lane wrote: > Ron Johnson writes: > > Why is VACUUM FULL recommended for compressing a table, when CLUSTER does > > the same thing (similarly doubling disk space), and apparently runs just > as > > fast? > > CLUSTER makes

Re: CLUSTER vs. VACUUM FULL

2024-04-21 Thread Ron Johnson
On Sun, Apr 21, 2024 at 6:45 PM Tom Lane wrote: > Ron Johnson writes: > > Why is VACUUM FULL recommended for compressing a table, when CLUSTER does > > the same thing (similarly doubling disk space), and apparently runs just > as > > fast? > > CLUSTER makes

CLUSTER vs. VACUUM FULL

2024-04-21 Thread Ron Johnson
PG 14.11 on RHEL8 Why is VACUUM FULL recommended for compressing a table, when CLUSTER does the same thing (similarly doubling disk space), and apparently runs just as fast? My tests: Table: CDSLBXW.public.log Time 1 Time 2 Time 3 secssecssecs VACUUM FULL 44.2

Re: error in trigger creation

2024-04-21 Thread Ron Johnson
On Sun, Apr 21, 2024 at 2:58 PM yudhi s wrote: > the partition drop from parent is taking longer as it scans all the > partitions of the child table > Does the relevant supporting index exist on the child table?

Re: [help] Error in database import

2024-04-19 Thread Ron Johnson
On Fri, Apr 19, 2024 at 11:58 PM Tu Ho wrote: > Hi, > > I am currently having a project where I need to combine 2 large database. > I was not however able to import an excel file .csv into the database. The > error was ERROR: syntax error at or near "OIDS" LINE 1: ...ing Site" , > "International

Re: efficiency random values / sequential ID values in indexes

2024-04-15 Thread Ron Johnson
On Mon, Apr 15, 2024 at 6:05 AM Sanjay Minni wrote: > Hi > > Is there any appreciable difference in using random values or sequential > values in indexes > > in a multi tenanted application there is a choice that the single field > ID's value is totally random / UUID or the numbers are created

Re: Failing streaming replication on PostgreSQL 14

2024-04-15 Thread Ron Johnson
On Mon, Apr 15, 2024 at 2:53 AM Nicolas Seinlet wrote: > Hello everyone, > > Since I moved some clusters from PostgreSQL 12 to 14, I noticed random > failures in streaming replication. I say "random" mostly because I haven't > got the source of the issue. > > I'm using the Ubuntu/cyphered

Recursively trace all Foreign Key "referenced by" tables?

2024-04-13 Thread Ron Johnson
The attached PG function dba.get_fk_referenced_by() has been tested on PG 9.6 and 14. A recursive bash function (also attached) calls the PG function, and displays the whole tree of tables that the table in question depends on. Output also attached. Is there a better way to do this? (I'm

PL/pgSQL techniques better than bash for dynamic DO?

2024-04-09 Thread Ron Johnson
PG 9.6.11, if relevant, migrating to PG 14 Real Soon Now. I must purge the oldest X period of records from 70 tables, every Sunday. The field name, interval (X days or months) and date (CURRENT_DATE or CURRENT_TIMESTAMP) varies for each table. Thus, I put all the relevant data in a tab-separated

Re: What is referential_action?

2024-04-08 Thread Ron Johnson
On Mon, Apr 8, 2024 at 9:41 AM David G. Johnston wrote: > On Monday, April 8, 2024, Ron Johnson wrote: > >> Four times, the word "referential_action" is used on this page, but it's >> never mentioned what the possible referential actions are. >> >

What is referential_action?

2024-04-08 Thread Ron Johnson
Four times, the word "referential_action" is used on this page, but it's never mentioned what the possible referential actions are. Am I missing something? https://www.postgresql.org/docs/14/sql-altertable.html [ CONSTRAINT constraint_name ] [snip] PRIMARY KEY index_parameters | REFERENCES

Re: Multiple COPY statements for one table vs one for ~half a billion records

2024-04-04 Thread Ron Johnson
On Thu, Apr 4, 2024 at 2:04 PM Carl L wrote: > Hi there, > > I have around half a billion records that are being generated from a back > end that are split into 80 threads (one per core) and I'm performing a copy > from memory ( from stdin binary) into Postgres from each of these threads - >

Re: how to check if the license is expired.

2024-03-30 Thread Ron Johnson
On Sat, Mar 30, 2024 at 9:15 AM 黄宁 wrote: > I want to develop a postgresql paid extension, then there is a local > license file, how do I check if the license file is expired, check it once > at each api execution, will that affect the performance of the api, is > there any other way? > What

Re: Cron not running

2024-03-28 Thread Ron Johnson
On Thu, Mar 28, 2024 at 5:28 PM Lok P wrote: > Hello All, > In RDS postgres(version 15.4), we have scheduled partition maintenance > through pg_partman and it's scheduled through pg_cron as below. The same > script has been executed in dev and test environments, and we are seeing > the cron job

Re: Table level restore in postgres

2024-03-28 Thread Ron Johnson
On Thu, Mar 28, 2024 at 2:27 PM arun chirappurath wrote: > Dear all, > > I am a new bie in postgres world > > Suppose I have accidently deleted a table or deleted few rows ,is it safe > to drop this table and restore just this table from custom backup to same > database? > By "custom

Re: Is this a buggy behavior?

2024-03-25 Thread Ron Johnson
On Mon, Mar 25, 2024 at 9:49 AM Christophe Pettus wrote: > > > > On Mar 25, 2024, at 02:50, Thiemo Kellner > wrote: > > My bad. I was under the impression that the create table statement was > an atomic process/transaction with all its bells and whistles for > constraints and keys, instead of a

Re: Statistics information.

2024-03-23 Thread Ron Johnson
On Sat, Mar 23, 2024 at 12:33 AM arun chirappurath wrote: > Dear All, > > Apologies the way i am asking question as i am more a SQL Server person > and a new postgre man.. > > I have used a query store in SQL server. it provides me option to load > statistics data to temp table and get below

Re: soft lockup - CPU#16 stuck for 3124s! [postmaster:2273]

2024-03-22 Thread Ron Johnson
On Fri, Mar 22, 2024 at 1:27 PM Tom Lane wrote: > Matthias Apitz writes: > > We have a PostgreSQL 15.1 server in production at a customer for some > > weeks (migrated from an older version) on SuSE SLES 15. > > > The customer is facing machine locks and before the Linux server does > > not

Re: How to store in hours:minutes:seconds where hours may be bigger than 24

2024-03-19 Thread Ron Johnson
On Tue, Mar 19, 2024 at 10:05 PM Celia McInnis wrote: > Hi: > > I want to store times in a database as hours:minutes:seconds where hours > can be greater than 24. How do I do this? I will want to be able to add > such times. > Try the INTERVAL data type.

Re: Postgres database encryption

2024-03-19 Thread Ron Johnson
On Tue, Mar 19, 2024 at 9:22 AM Johnathan Tiamoh wrote: > Hello, > > > Good morning. Please I wish to find out the best way of encrypting a > Postgres database. > > This could either be encrypting a whole database ( I don’t if this is > possible), a specific table and a specific tablespace. >

Re: Simple way to simulate a bug in logical replication

2024-03-12 Thread Ron Johnson
On Tue, Mar 12, 2024 at 11:16 AM Avi Weinberg wrote: > Hi All, > > > > I think I hit a bug in logical replication in version 15.2. > But 15.6 is the latest version. Maybe it's been fixed since then.

Re: Seeing high query planning time on Azure Postgres Single Server version 11.

2024-03-10 Thread Ron Johnson
On Sun, Mar 10, 2024 at 1:34 PM Greg Sabino Mullane wrote: > > On Sat, Mar 9, 2024 at 1:57 PM hassan rafi > wrote: > >> Would upgrading to the latest version of Postgres potentially solve the >> issue? >> > > Potentially, yes, but the only one who can answer that for sure is you. > Upgrade to

Re: Seeing high query planning time on Azure Postgres Single Server version 11.

2024-03-09 Thread Ron Johnson
On Sat, Mar 9, 2024 at 7:18 AM hassan rafi wrote: > Hi team, > > We are seeing unusually high query planning times on our Postgres server. > I am attaching a few query plans. > Postgresql version number? Rows in the tables? System load?

Re: creating a subset DB efficiently ?

2024-03-08 Thread Ron Johnson
On Fri, Mar 8, 2024 at 11:22 AM David Gauthier wrote: > Here's the situation > > - The DB contains data for several projects. > - The tables of the DB contain data for all projects (data is not > partitioned on project name or anything like that) > - The "project" identifier (table column)

Re: Windows service randomly stops with no indication why

2024-03-08 Thread Ron Johnson
On Fri, Mar 8, 2024 at 9:17 AM Jay Madren wrote: > Running PostgreSQL 15.6 on Windows Server 2022. The database service > randomly just stops and the Windows Service auto-restart options don't kick > in. The stop is unexpected (not a controlled shut down) because after > restarting the service

Re: update to 16.2

2024-03-08 Thread Ron Johnson
On Fri, Mar 8, 2024 at 5:01 AM Matthias Apitz wrote: > El día viernes, marzo 08, 2024 a las 12:56:16 -0800, Christophe Pettus > escribió: > > > > > > > > On Mar 8, 2024, at 00:53, Matthias Apitz wrote: > > > It does not say definitely that for all other versions a dump/restore > is > > >

Re: v11.5- v15.3 upgrade (linux)

2024-03-07 Thread Ron Johnson
would only make sense > if it's a significant win and not just options/features that we don't use. > > On Thu, Mar 7, 2024 at 12:53 AM Ron Johnson > wrote: > >> On Wed, Mar 6, 2024 at 7:19 PM David Gauthier >> wrote: >> >>> Hi: >>> I'm a PG

Re: v11.5- v15.3 upgrade (linux)

2024-03-06 Thread Ron Johnson
On Wed, Mar 6, 2024 at 7:19 PM David Gauthier wrote: > Hi: > I'm a PG user in a big corp with an IT dept that administers a PG > server/instance that I use. It's an old install, v11.5, and we need to > upgrade to v15.3. They want to bring the upgraded DB up on a new linux vm > which has OS

Re: How to add columns faster

2024-03-03 Thread Ron Johnson
On Sun, Mar 3, 2024 at 2:06 PM yudhi s wrote: > Hello, > We have an application in which the tables will have approx ~200 columns > in some of the big transaction tables when we will be serving all the > business use cases. Currently it has ~100 columns to serve current business > use cases to

Re: Non-Stored Generated Columns

2024-02-28 Thread Ron Johnson
On Wed, Feb 28, 2024 at 2:11 PM Tom Lane wrote: > Dominique Devienne writes: > > Views can have foreign-keys? > > Surely you'd put the FK on the underlying table. > > > Generated view columns be indexed? > > You want an index on a virtual column? Sure, just build an expression > index (on the

Re: PostgreSQL Guard

2024-02-28 Thread Ron Johnson
re the steps the same for other Linux distributions like Debian? > > On Wed, Feb 28, 2024 at 9:29 AM, Ron Johnson > wrote: > As before, I encourage you to read > https://www.tecmint.com/configure-postgresql-streaming-replication-in-centos-8/ > . > > On Tue, Feb 27, 2024 at 3

Re: PostgreSQL Guard

2024-02-27 Thread Ron Johnson
ant to transfer the database of a website like Amazon.com to > a new server and delete the old one. Many users are buying and selling on > this website at the same time and it is not possible to turn off the > server. What do you do to move a database to another server? > > On Tue, Feb 27

Re: PostgreSQL Guard

2024-02-27 Thread Ron Johnson
I used this web page to implement hot standby via physical streaming. This command sets up everything for you: pg_basebackup \ --pgdata=$PGDATA \ --dbname=service=basebackup \ --verbose --progress \ --checkpoint=fast \

Re: PostgreSQL Guard

2024-02-27 Thread Ron Johnson
On Tue, Feb 27, 2024 at 3:43 AM Jason Long wrote: > Hello, > I have some questions about the PostgreSQL database: > > 1- If I want to distribute the PostgreSQL database on several servers at > the same time. what should I do? Something similar to high availability. > HA replication is native in

Re: Encryption Options

2024-02-16 Thread Ron Johnson
www.varonis.com/blog/pci-dss-requirements > > Agreed. The on-premise vs aurora will take a different approach for > catering to above needs. We are currently evaluating , what would be the > possible options in each of these cases? and if this would be a factor in > choosing the on-premise p

Re: Encryption Options

2024-02-16 Thread Ron Johnson
On Fri, Feb 16, 2024 at 1:53 AM sud wrote: > Hello Friends, > > We are newly moving to postgres database (yet to decide if it would be an > on premise one or AWS aurora postgres). However , we want to understand > what encryption / decryption techniques are available in the postgres >

Re: Encryption Options

2024-02-16 Thread Ron Johnson
The phrases "personal information" and "data at rest encryption" strongly indicate PCI, or something similar. On Fri, Feb 16, 2024 at 12:20 PM Greg Sabino Mullane wrote: > You need to clearly define your threat model. What exactly are you > defending against? What scenario do you want to avoid?

Re: How to do faster DML

2024-02-14 Thread Ron Johnson
On Wed, Feb 14, 2024 at 11:58 PM veem v wrote: > > > On Thu, 15 Feb 2024 at 00:43, Adrian Klaver > wrote: > >> It depends: >> >> https://www.postgresql.org/docs/current/sql-altertable.html >> >> "Adding a column with a volatile DEFAULT or changing the type of an >> existing column will require

Re: PITR

2024-02-14 Thread Ron Johnson
On Wed, Feb 14, 2024 at 1:39 PM Yongye Serkfem wrote: > Hi, > I hope you are all doing well. I am trying to do PITR on Postgresql v12. > Now that the recovery.conf file is not available in this version, where > should I set the recovery_target_time? I checked the Postgresql.conf file > and can't

Re: Postgres pg_cron extension

2024-02-14 Thread Ron Johnson
On Wed, Feb 14, 2024 at 10:23 AM Greg Sabino Mullane wrote: > Yes, definitely the wrong forum. RDS is not Postgres, and "parameter > groups" is an Amazon thing. A quick web search would seem to indicate that > the default group is replaced by the custom one, rather than enhancing it. > But

Re: How to do faster DML

2024-02-13 Thread Ron Johnson
On Tue, Feb 13, 2024 at 4:17 PM veem v wrote: [sni[] > One question here, if we have defined one column as a fixed length data > type "integer" and slowly we noticed the length of data keeps increasing > (in case of a sequence generated PK column which will keep increasing), and > we want to

Re: Compressing large column by moving it to a unique table

2024-02-13 Thread Ron Johnson
('2024-01-01 00:00:00.00+00'::timestamptz + interval > '1 day') > AND > start_timestamp < ('2024-01-01 00:00:00.00+00'::timestamptz + interval > '2 day') > ); > -- > https://app.pgmustard.com/#/explore/a75b20bd-07d5-4402-a0d8-22419682307a > > explain (analyze, buffers,

Re: MAT. VIEW security problems and PG 10-11 versions?

2024-02-13 Thread Ron Johnson
On Tue, Feb 13, 2024 at 3:44 AM Daniel Gustafsson wrote: > > On 13 Feb 2024, at 08:56, Durumdara wrote: > > > But maybe that's because PG 10 and 11 are no longer supported - and not > because they aren't affected by the issues. > > EOL versions do not recieve security updates and are not

Re: Compressing large column by moving it to a unique table

2024-02-13 Thread Ron Johnson
practical aspects, in particular that case of > selecting a subset of columns from the view that I know doesn’t need the > join but the query planner thinks does. > > On Tue, Feb 13, 2024 at 3:16 AM Ron Johnson > wrote: > >> On Mon, Feb 12, 2024 at 10:12 PM Adrian Garcia Badaracco &l

Re: Compressing large column by moving it to a unique table

2024-02-12 Thread Ron Johnson
On Mon, Feb 12, 2024 at 10:12 PM Adrian Garcia Badaracco < adr...@adriangb.com> wrote: > I am using Timescale so I'll be mentioning some timestamp stuff but I > think this is a general postgres question for the most part. > > I have a table with some fixed, small columns (id, timestamp, etc) and

Re: How to do faster DML

2024-02-12 Thread Ron Johnson
On Mon, Feb 12, 2024 at 3:23 PM veem v wrote: [snip] > So it looks like the fixed length data type(like integer, float) should be > the first choice while choosing the data type of the attributes > wherever possible, as these are native types. > Correct. > (Like choosing "Integer/float" over

Re: Safest pgupgrade jump distance

2024-02-12 Thread Ron Johnson
On Mon, Feb 12, 2024 at 5:08 AM Dávid Suchan wrote: > Hi, I was wondering what is the safest pg_upgrade version upgrade distance > going from 9.6 version. Do I need to go version by version or I can go from > 9.6 to 15? We have a very huge database(TBs) with one replication server, > so we will

Re: Query hangs (and then timeout) after using COPY to import data

2024-02-11 Thread Ron Johnson
Since the query works in PgAdmin, but not in npgsql, the problem has to be somewhere in Npgsql. https://www.npgsql.org/doc/diagnostics/overview.html Maybe increasing the log level will lead to a solution. On Sun, Feb 11, 2024 at 6:13 PM wrote: > Thanks, Adrian, for the suggestion, but same

Re: Query hangs (and then timeout) after using COPY to import data

2024-02-11 Thread Ron Johnson
On Sun, Feb 11, 2024 at 4:41 PM Adrian Klaver wrote: > On 2/11/24 13:37, ste...@gmail.com wrote: > [snip] > > > > The same query, executed from pgAdmin, returns the result in less than a > > second (even if it’s executed while the query from my app is running). > > > > (actually the result are

Re: How to do faster DML

2024-02-11 Thread Ron Johnson
On Sun, Feb 11, 2024 at 11:54 AM veem v wrote: [snip] > When you said *"you would normally prefer those over numeric " *I was > thinking the opposite. As you mentioned integer is a fixed length data type > and will occupy 4 bytes whether you store 15 or .But in case of > variable length

Re: Question on partitioning

2024-02-06 Thread Ron Johnson
On Tue, Feb 6, 2024 at 2:40 PM veem v wrote: > Thank you Laurenz. Got it. > > So basically , you mean to say any DDL on a table won't allow the table to > be read by other processes. I was under the assumption that it should allow > the read queries to move ahead at least. I must be wrong here.

Exclude certain application pgaudit logging?

2024-02-06 Thread Ron Johnson
Currently, we use Object audit logging to capture all READ access to columns FOO_1, FOO_2 and FOO_3 in table BAR.SCRABBLE. (They are the three columns have PII data.) The problem is that the application legitimately reads these columns thousands of times per day. Thus, the log fills up with

Re: Unused indexes

2024-02-06 Thread Ron Johnson
On Mon, Feb 5, 2024 at 9:54 PM Greg Sabino Mullane wrote: > The pg_stat_*_tables tables idx_* columns has accumulated usage since the >> last time you started the postmaster. >> > > Actually, those persist at restart - you can use > > select datname, stats_reset from pg_stat_database; > > to

Re: Improving pg_dump performance when handling large numbers of LOBs

2024-02-05 Thread Ron Johnson
that use of LOBs for this purpose was > not necessary. > > Wyatt > > On Mon, Feb 5, 2024 at 11:36 AM Ron Johnson > wrote: > >> On Mon, Feb 5, 2024 at 2:01 PM Wyatt Tellis >> wrote: >> >>> Hi, >>> >>> We've inherited a series of leg

Re: Improving pg_dump performance when handling large numbers of LOBs

2024-02-05 Thread Ron Johnson
On Mon, Feb 5, 2024 at 2:01 PM Wyatt Tellis wrote: > Hi, > > We've inherited a series of legacy PG 12 clusters that each contain a > database that we need to migrate to a PG 15 cluster. Each database contains > about 150 million large objects totaling about 250GB. > 250*10^9 / (150*10^6) = 1667

Re: Unused indexes

2024-02-05 Thread Ron Johnson
On Mon, Feb 5, 2024 at 9:02 AM arun chirappurath wrote: > Hi All, > > Do we have a script to get unused indexes for 30 days and once identified > do we have an option to disable and enable when required? > The pg_stat_*_tables tables idx_* columns has accumulated usage since the last time you

Re: Why doesn't Postgres apply limit on groups when retrieving N results per group using WHERE IN + ORDER BY

2024-02-05 Thread Ron Johnson
On Mon, Feb 5, 2024 at 7:23 AM Sean v wrote: > This is related to a question I asked on dbs.stackexchange.com: > https://dba.stackexchange.com/questions/335501/why-doesnt-postgres-apply-limit-on-groups-when-retrieving-n-results-per-group > > But to reiterate - I have a query like this: > >

Re: Question on partitioning

2024-02-04 Thread Ron Johnson
On Sun, Feb 4, 2024 at 4:40 PM veem v wrote: > Hello All, > In postgresql, Is it possible to partition an existing nonpartitioned > table having data already residing in it and indexes and constraints > defined in it, without the need of manually moving the data around, to make > it faster?

Re: How to do faster DML

2024-02-04 Thread Ron Johnson
On Sat, Feb 3, 2024 at 11:09 AM Lok P wrote: [snip] > show maintenance_work_mem; - 4155MB > show work_mem; - 8MB > show shared_buffers ; -22029684 > Those are pretty small values. What are your server specs?

Re: How to do faster DML

2024-02-04 Thread Ron Johnson
On Sun, Feb 4, 2024 at 10:30 AM Lok P wrote: > On Sun, Feb 4, 2024 at 8:14 PM Dennis White > wrote: > >> I'm surprised no one has mentioned perhaps it's a good idea to partition >> this table while adding the pk. By your own statements the table is >> difficult to work with as is. Without

Re: How to do faster DML

2024-02-03 Thread Ron Johnson
On Sat, Feb 3, 2024 at 8:55 AM Lok P wrote: > Apology. One correction, the query is like below. I. E filter will be on > on ctid which I believe is equivalent of rowid in oracle and we will not > need the index on Id column then. > > But, it still runs long, so thinking any other way to make

Re: Query running longer

2024-02-01 Thread Ron Johnson
On Thu, Feb 1, 2024 at 4:13 PM Laurenz Albe wrote: > On Fri, 2024-02-02 at 02:27 +0530, veem v wrote: > > We have the below query which is running for ~45 seconds on postgres > aurora reader instance. > > I have captured the explain analyze. Want to understand, where exactly > the resources are

Re: Query performance in 9.6.24 vs 14.10

2024-01-29 Thread Ron Johnson
According to my tests, sometimes JIT is a little faster, and sometimes it's a little slower. Mostly within the realm of statistical noise (especially with each query having a sample size of only 13, on a VM that lives on a probably-busy host). On Mon, Jan 29, 2024 at 9:18 AM Ron Johnson wrote

Re: Query performance in 9.6.24 vs 14.10

2024-01-29 Thread Ron Johnson
Yes, jit=on. I'll test them with jit=off, to see the difference. (The application is 3rd party, so will change it at the system level.) On Mon, Jan 29, 2024 at 7:09 AM Bob Jolliffe wrote: > Out of curiosity, is the pg14 running with the default jit=on setting? > > This is obviously entirely

Re: Scriptable way to validate a pg_dump restore ?

2024-01-29 Thread Ron Johnson
On Mon, Jan 29, 2024 at 3:12 AM Laura Smith < n5d9xq3ti233xiyif...@protonmail.ch> wrote: > Hi > > Let's say I've got a scenario where I'm doing a pg_dump replication rather > than online streaming, e.g. due to air-gap or whatever. > > Is there a scriptable way to validate the restore ? e.g.

Re: Query performance in 9.6.24 vs 14.10

2024-01-28 Thread Ron Johnson
On Sun, Jan 28, 2024 at 10:44 PM David Rowley wrote: > On Mon, 29 Jan 2024 at 07:37, Ron Johnson wrote: > >> 08 9.6.24 1,142.164 1,160.801 1,103.716 1,249.852 1,191.081 >> 14.10 159.354 155.111 155.111 162.797 158.157 86.72% >> > > Your speedup per cent calcul

Query performance in 9.6.24 vs 14.10

2024-01-28 Thread Ron Johnson
(I don't know how this will look in text mode. Hopefully it will be comprehensible in the archives.) This is the summary of EXPLAIN (ANALYZE) on eight frequently-run complex queries from our application, extracted from the Postgresql log because either the BIND or SELECT takes longer than 3000

Re: permission denied on socket

2024-01-25 Thread Ron Johnson
On Thu, Jan 25, 2024 at 3:32 PM Adrian Klaver wrote: [snip] > Best guess is you are using a version of psql that is expecting the > socket to be somewhere else then where it actually is. > Is "permission denied" really the error you get when the socket does not exist? Trying "psql

Re: permission denied on socket

2024-01-25 Thread Ron Johnson
On Thu, Jan 25, 2024 at 3:23 PM Atul Kumar wrote: > Hi, > > I have postgres 13 running on centos 7. > > I am facing an issue while trying to connect the cluster using the below > command. > > -bash-4.2$ psql postgres > > psql: error: connection to server on socket "/tmp/.s.PGSQL.5432" failed: >

Re:

2024-01-25 Thread Ron Johnson
On Thu, Jan 25, 2024 at 12:29 PM Tom Lane wrote: > Ron Johnson writes: > > EXPLAIN SELECT works inside a FOR loop, but only the first line of the > > EXPLAIN output is stored. What's the magic sauce for seeing the whole > > EXPLAIN output? > > The usual way is to ru

[no subject]

2024-01-25 Thread Ron Johnson
PG 14.10 (and 9.6.24, which we're migrating off of). EXPLAIN SELECT works inside a FOR loop, but only the first line of the EXPLAIN output is stored. What's the magic sauce for seeing the whole EXPLAIN output? (The purpose is to generate many queries and see how the BIND and SELECT times change

Re: Backup certain months old data

2024-01-22 Thread Ron Johnson
On Mon, Jan 22, 2024 at 10:12 PM Siraj G wrote: > Hello! > > I would like to know how we can backup certain months old data from PgSQL > Use the COPY command. > and then delete it. > That would depend on how many records, how big the records are, and if there's index support on the "date"

Re: Mimic ALIAS in Postgresql?

2024-01-22 Thread Ron Johnson
On Mon, Jan 22, 2024 at 6:40 PM Rob Sargent wrote: > On 1/17/24 16:25, Jim Nasby wrote: > > On 1/16/24 6:41 PM, Rob Sargent wrote: > > On 1/16/24 17:39, Jim Nasby wrote: > > On 1/16/24 4:57 PM, Rob Sargent wrote: > > Or perhaps you have to beef the sed up to use word boundaries just > in

Re: unbale to list schema

2024-01-22 Thread Ron Johnson
On Wed, Jan 17, 2024 at 1:46 PM Atul Kumar wrote: > Hi, > > I am not able to find any solution to list all schemas in all databases at > once, to check the structure of the whole cluster. > > As I need to give a few privileges to a user to all databases, their > schemas and schemas' objects

Re: B-tree index balance?

2024-01-19 Thread Ron Johnson
On Fri, Jan 19, 2024 at 11:37 AM Tom Lane wrote: > Ron Johnson writes: > > On an RDMS which I used in the 1990s and 2000s, b-tree indices of > sequences > > would get unbalanced, since every new leaf was added to the far right > > corner of the tree. > >

B-tree index balance?

2024-01-19 Thread Ron Johnson
On an RDMS which I used in the 1990s and 2000s, b-tree indices of sequences would get unbalanced, since every new leaf was added to the far right corner of the tree. Sure, they would auto-balance *to a degree* during node splits, but all those "far-right corner" inserts still left them pretty

Re: pg_basebackup Restore problem

2024-01-17 Thread Ron Johnson
Perfectly understandable, but tar *did* fail. Time to start debugging your shell script. On Wed, Jan 17, 2024 at 4:26 PM Johnathan Tiamoh wrote: > Ok. > > I'm a little confused because has always work > > On Wed, Jan 17, 2024 at 4:11 PM Ron Johnson > wrote: > >

Re: pg_basebackup Restore problem

2024-01-17 Thread Ron Johnson
t;${PG_DUMP_OPTS[@]}" >>> >>> /bin/mv "$baseback_dir"/* "/enf/backup/current/" >>> /bin/rm -r "$baseback_dir" >>> >>> >>> 3. Why aren't you letting pg_basebackup maintain the WAL files it needs?

Re: pg_basebackup Restore problem

2024-01-17 Thread Ron Johnson
Why aren't you letting pg_basebackup maintain the WAL files it needs? > > > On Wed, Jan 17, 2024 at 11:46 AM Ron Johnson > wrote: > >> 1. What's in $PATH_FOLDER? >> 2. What pg_basebackup command did you use? >> 3. Why aren't you letting pg_basebackup maintain the

Re: pg_basebackup Restore problem

2024-01-17 Thread Ron Johnson
t? > /tnt/backup/current > > This is where the backup files are. > > > > > > On Wed, Jan 17, 2024 at 11:15 AM Ron Johnson > wrote: > >> On Wed, Jan 17, 2024 at 11:11 AM Johnathan Tiamoh < >> johnathantia...@gmail.com> wrote: >> >>> Hell

Re: Initiate backup from routine?

2024-01-17 Thread Ron Johnson
On Wed, Jan 17, 2024 at 9:41 AM Troels Arvin wrote: > Hello, > > I would like to allow a co-worker to perform a backup of a database, such > that the backup is saved to the database server itself. One use case is > that (s)he would like an extra backup of a database, just before an > application

  1   2   3   4   5   6   7   8   9   10   >