Re: index prefetching

2024-01-16 Thread Jim Nasby
for a very long time (it also *only* reads blocks that aren't marked all visble/frozen; none of the "only skip if skipping at least 32 blocks" logic is used). 1: https://d1.awsstatic.com/events/reinvent/2019/REPEAT_1_Deep_dive_on_Amazon_Aurora_with_PostgreSQL_compatibility_DAT328-R1.p

Re: Emit fewer vacuum records by reaping removable tuples during pruning

2024-01-16 Thread Jim Nasby
so crucial. -- Jim Nasby, Data Architect, Austin TX

Re: Make NUM_XLOGINSERT_LOCKS configurable

2024-01-12 Thread Jim Nasby
In-progress Inserts to Finish Time (wal_wait_for_insert_to_finish_time). Unfortunately this leaves the question of how frequently is WaitXLogInsertionsToFinish() being called and by whom. One possibility here is that wal_buffers is too small so backends are constantly having to write WAL data to free up buffers. -- Jim Nasby, Data Architect, Austin TX

Re: Confine vacuum skip logic to lazy_scan_skip

2024-01-12 Thread Jim Nasby
On 1/11/24 5:50 PM, Melanie Plageman wrote: On Fri, Jan 5, 2024 at 5:51 AM Nazir Bilal Yavuz wrote: On Fri, 5 Jan 2024 at 02:25, Jim Nasby wrote: On 1/4/24 2:23 PM, Andres Freund wrote: On 2024-01-02 12:36:18 -0500, Melanie Plageman wrote: Subject: [PATCH v2 1/6] lazy_scan_skip remove

Re: Add BF member koel-like indentation checks to SanityCheck CI

2024-01-09 Thread Jim Nasby
[2] (which also supports line-by-line formatting if the format tool allows it); presumably any modern editor has similar support. 1: Literally 3rd item at https://go.dev/doc/effective_go 2: https://github.com/vim-autoformat/vim-autoformat -- Jim Nasby, Data Architect, Austin TX

Re: Emit fewer vacuum records by reaping removable tuples during pruning

2024-01-09 Thread Jim Nasby
aking for indexes on integers, for example. But we'd still need to be cautious. -- Jim Nasby, Data Architect, Austin TX

Re: Random pg_upgrade test failure on drongo

2024-01-08 Thread Jim Nasby
I realize that synthetic test workloads like pg_upgrade in a loop aren't themselves real-world scenarios, but what about other cases? Even if we're certain it's not possible for these issues to wedge a server, it's still not a good experience for users to get random, unexplained IO-related

Re: Confine vacuum skip logic to lazy_scan_skip

2024-01-04 Thread Jim Nasby
ges, but I have to wonder if the concept of skipping should go away in the context of vector IO? Instead of thinking about "we can skip this range of blocks", why not maintain a list of "here's the next X number of blocks that we need to vacuum"?

Re: the s_lock_stuck on perform_spin_delay

2024-01-04 Thread Jim Nasby
Another possibility might be using the CPUs timestamp counter. 1: https://valgrind.org/docs/manual/lk-manual.html -- Jim Nasby, Data Architect, Austin TX

Re: doing also VM cache snapshot and restore with pg_prewarm, having more information of the VM inside PostgreSQL

2024-01-04 Thread Jim Nasby
raining, R -- Jim Nasby, Data Architect, Austin TX

Re: SET ROLE x NO RESET

2024-01-04 Thread Jim Nasby
their role back to themselves, but RESET ROLE removes the risk of typos. -- Jim Nasby, Data Architect, Austin TX

Re: add function argument names to regex* functions.

2024-01-03 Thread Jim Nasby
esumably misspelling wouldn't be a big issue. But I think "instance" is OK as well. -- Jim Nasby, Data Architect, Austin TX

Re: Extension Enhancement: Buffer Invalidation in pg_buffercache

2024-01-03 Thread Jim Nasby
L(1); I think you also need to test PG_ARGISNULL with force parameter. Actually, that's true for the first argument as well. Or, just mark the function as STRICT. -- Jim Nasby, Data Architect, Austin TX

Re: Password leakage avoidance

2024-01-03 Thread Jim Nasby
of setting passwords. -- Jim Nasby, Data Architect, Austin TX

Re: add function argument names to regex* functions.

2024-01-03 Thread Jim Nasby
ot as precise but would avoid the conceptual overloading of ordinary indices. I'm not a fan of "index" since that leaves the question of whether it's 0 or 1 based. "Position" is a bit better, but I think Jian's suggestion of "occurance" is best. -- Jim Nasby, Data Architect, Austin TX

Re: Things I don't like about \du's "Attributes" column

2024-01-02 Thread Jim Nasby
fence when it comes to SQL syntax vs what we have now. What we currenly have is more readable, but off-hand I think the other places we list attributes we do it in SQL syntax. It might be worth changing just for consistency sake. -- Jim Nasby, Data Architect, Austin TX

Re: Next step towards 64bit XIDs: Switch to FullTransactionId for PGPROC->xid and XLogRecord->xl_xid

2024-01-02 Thread Jim Nasby
On 1/2/24 1:58 PM, Robert Haas wrote: Maybe this analysis I've just given isn't quite right, but my point is that we should try to think hard about where in the system 32-bit XIDs suck and for what reason, and use that as a guide to what to change first. Very

Re: SET ROLE x NO RESET

2024-01-02 Thread Jim Nasby
On 12/31/23 1:19 PM, Joe Conway wrote: On 12/30/23 17:19, Michał Kłeczek wrote: On 30 Dec 2023, at 17:16, Eric Hanson wrote: What do you think of adding a NO RESET option to the SET ROLE

Re: Should vacuum process config file reload more often

2023-03-08 Thread Jim Nasby
On 3/2/23 1:36 AM, Masahiko Sawada wrote: For example, I guess we will need to take care of changes of maintenance_work_mem. Currently we initialize the dead tuple space at the beginning of lazy vacuum, but perhaps we would need to enlarge/shrink it based on the new value? Doesn't the dead

Re: refactoring relation extension and BufferAlloc(), faster COPY

2023-02-21 Thread Jim Nasby
On 2/21/23 3:12 PM, Andres Freund wrote: CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you can confirm the sender and know the content is safe. Hi, On 2023-02-21 15:00:15 -0600, Jim Nasby wrote: Some food for thought: I

Re: refactoring relation extension and BufferAlloc(), faster COPY

2023-02-21 Thread Jim Nasby
On 10/28/22 9:54 PM, Andres Freund wrote: b) I found that is quite beneficial to bulk-extend the relation with smgrextend() even without concurrency. The reason for that is the primarily the aforementioned dirty buffers that our current extension method causes. One bit that stumped

Re: Direct I/O

2022-11-04 Thread Jim Nasby
On 11/1/22 2:36 AM, Thomas Munro wrote: Hi, Here is a patch to allow PostgreSQL to use $SUBJECT. It is from the This is exciting to see! There's two other items to add to the TODO list before this would be ready for production: 1) work_mem. This is a significant impediment to scaling

Re: autovacuum: change priority of the vacuumed tables

2018-03-03 Thread Jim Nasby
in the foot in that regard already. We can always have safeguards in place if we get too close to wrap-around, just like we currently do. -- Jim Nasby, Chief Data Architect, Austin TX OpenSCG http://OpenSCG.com

Re: autovacuum: change priority of the vacuumed tables

2018-03-03 Thread Jim Nasby
les a list contains. A more fine-grained approach would be to have workers make a new selection after every vacuum they complete. That would provide the ultimate in control, since you'd be able to see exactly what all the other workers are doing. -- Jim Nasby, Chief Data Architect, Austin