Re: Is there any limit on the number of rows to import using copy command

2025-07-24 Thread Adrian Klaver
On 7/24/25 16:59, sivapostg...@yahoo.com wrote: 1.  Testcase.  Created a new database, modified the triggers (split into three), populated required master data, lookup tables.  Then transferred 86420 records. Checked whether all the 86420 records inserted in table1 and also whether the trigger

Re: Is there any limit on the number of rows to import using copy command

2025-07-24 Thread Ron Johnson
On Thu, Jul 24, 2025 at 8:00 PM sivapostg...@yahoo.com < sivapostg...@yahoo.com> wrote: [snip] > 3. Regarding moving the logic to procedure. Won't the trigger work? > Will it be a burden for 86420 records? It's working, if we insert few > thousand records. After split of trigger function, it's

Re: Is there any limit on the number of rows to import using copy command

2025-07-24 Thread sivapostg...@yahoo.com
1.  Testcase.  Created a new database, modified the triggers (split into three), populated required master data, lookup tables.  Then transferred 86420 records. Checked whether all the 86420 records inserted in table1 and also whether the trigger created the required records in table2.   Yes, i

Re: Is there any limit on the number of rows to import using copy command

2025-07-24 Thread sivapostg...@yahoo.com
Those two columns have null values when the color turns into Red. And the status column shows 'active' only.   All those three column values are same (null, null, active) even after 2 hours! It works when I split the trigger function into three, one each for insert, delete and update call from t

Re: PostgreSQL on S3-backed Block Storage with Near-Local Performance

2025-07-24 Thread Pierre Barre
> This then begs the obvious question of how fast is this with > synchronous_commit = on? Probably not awful, especially with commit_delay. I'll try that and report back. Best, Pierre On Fri, Jul 25, 2025, at 00:03, Jeff Ross wrote: > On 7/24/25 13:50, Pierre Barre wrote: > >> It’s not “safe”

Re: PostgreSQL on S3-backed Block Storage with Near-Local Performance

2025-07-24 Thread Pierre Barre
Hi Marco, Thanks for the kind words! > and potentially elaborate on other projects for an active/active cluster! I > applaud you. I wrote an argument there: https://github.com/Barre/ZeroFS?tab=readme-ov-file#cap-theorem I definitely want to write a proof of concept when I get some time. Best

Re: PostgreSQL on S3-backed Block Storage with Near-Local Performance

2025-07-24 Thread Marco Torres
My humble take on this project: well done! You are opening the doors to work on a much-needed endeavor, decouple compute from storage, and potentially elaborate on other projects for an active/active cluster! I applaud you. On Thu, Jul 17, 2025, 4:59 PM Pierre Barre wrote: > Hi everyone, > > I w

Re: PostgreSQL on S3-backed Block Storage with Near-Local Performance

2025-07-24 Thread Jeff Ross
On 7/24/25 13:50, Pierre Barre wrote: It’s not “safe” or “unsafe”, there’s mountains of valid workloads which don’t require synchronous_commit. Synchronous_commit don’t make your system automatically safe either, and if that’s a requirement, there’s many workarounds, as you suggested, it cert

Re: PostgreSQL on S3-backed Block Storage with Near-Local Performance

2025-07-24 Thread Pierre Barre
It’s not “safe” or “unsafe”, there’s mountains of valid workloads which don’t require synchronous_commit. Synchronous_commit don’t make your system automatically safe either, and if that’s a requirement, there’s many workarounds, as you suggested, it certainly doesn’t make the setup useless. Be

Re: PostgreSQL on S3-backed Block Storage with Near-Local Performance

2025-07-24 Thread Nico Williams
On Fri, Jul 18, 2025 at 12:57:39PM +0200, Pierre Barre wrote: > - Postgres configured accordingly memory-wise as well as with > synchronous_commit = off, wal_init_zero = off and wal_recycle = off. Bingo. That's why it's fast (synchronous_commit = off). It's also why it's not safe _unless_ you

Re: PostgreSQL on S3-backed Block Storage with Near-Local Performance

2025-07-24 Thread Nico Williams
On Fri, Jul 18, 2025 at 06:40:58AM +0200, Laurenz Albe wrote: > On Fri, 2025-07-18 at 00:57 +0200, Pierre Barre wrote: > > Looking forward to your feedback and questions! > > I think the biggest hurdle you will have to overcome is to > convince notoriously paranoid DBAs that this tall stack > prov

Re: PostgreSQL on S3-backed Block Storage with Near-Local Performance

2025-07-24 Thread Pierre Barre
> "NFS" is a key word that does not inspire confidence in PostgreSQL circles... Coming back to this, I just implemented 9P, which should translates to proper semantics for FSYNC. mount -t 9p -o trans=tcp,port=5564,version=9p2000.L,msize=65536,access=user 127.0.0.1 /mnt/9p Best, Pierre On Fri,

Re: Is there any limit on the number of rows to import using copy command

2025-07-24 Thread David Barbour
You might also consider Backup/Restore. It appears you're not concerned with data being inserted into the source table after the backup is complete? If so, you can then easily sync the two post restore. On Wed, Jul 23, 2025 at 6:55 AM sivapostg...@yahoo.com < sivapostg...@yahoo.com> wrote: > Th

Re: Is there any limit on the number of rows to import using copy command

2025-07-24 Thread Adrian Klaver
On 7/24/25 05:18, sivapostg...@yahoo.com wrote: Thanks Merlin, adrain, Laurenz As a testcase, I split the trigger function into three, one each for insert, update, delete, each called from a separate trigger. IT WORKS!. It worked before, it just slowed down as your cases got bigger. You nee

Re: Is there any limit on the number of rows to import using copy command

2025-07-24 Thread sivapostg...@yahoo.com
Thanks Merlin, adrain, Laurenz As a testcase, I split the trigger function into three, one each for insert, update, delete, each called from a separate trigger.   IT WORKS!. Shouldn't we have one trigger function for all the three trigger events?  Is it prohibited for bulk insert like this? I tr