pgsql: Improve read_stream.c advice for dense streams.
Improve read_stream.c advice for dense streams. read_stream.c tries not to issue read-ahead advice when it thinks the kernel's own read-ahead should be active, ie when using buffered I/O and reading sequential blocks. It previously gave up too easily, and issued advice only for the first read of up to io_combine_limit blocks in a larger range of sequential blocks after random jump. The following read could suffer an avoidable I/O stall. Fix, by continuing to issue advice until the corresponding preadv() calls catch up with the start of the region we're currently issuing advice for, if ever. That's when the kernel actually sees the sequential pattern. Advice is now disabled only when the stream is entirely sequential as far as we can see in the look-ahead window, or in other words, when a sequential region is larger than we can cover with the current io_concurrency and io_combine_limit settings. While refactoring the advice control logic, also get rid of the "suppress_advice" argument that was passed around between functions to skip useless posix_fadvise() calls immediately followed by preadv(). read_stream_start_pending_read() can figure that out, so let's concentrate knowledge of advice heuristics in fewer places (our goal being to make advice-based I/O concurrency a legacy mode soon). The problem cases were revealed by Tomas Vondra's extensive regression testing with many different disk access patterns using Melanie Plageman's streaming Bitmap Heap Scan patch, in a battle against the venerable always-issue-advice-and-always-one-block-at-a-time code. Reviewed-by: Andres Freund (earlier version) Reported-by: Melanie Plageman Reported-by: Tomas Vondra Reported-by: Andres Freund Tested-by: Melanie Plageman Discussion: https://postgr.es/m/CA%2BhUKGK_%3D4CVmMHvsHjOVrK6t4F%3DLBpFzsrr3R%2BaJYN8kcTfWg%40mail.gmail.com Discussion: https://postgr.es/m/CA%2BhUKGJ3HSWciQCz8ekP1Zn7N213RfA4nbuotQawfpq23%2Bw-5Q%40mail.gmail.com Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/7ea8cd15661e3b0da4b57be2f25fdd512951576f Modified Files -- src/backend/storage/aio/read_stream.c | 65 --- 1 file changed, 45 insertions(+), 20 deletions(-)
pgsql: doc: Explain more thoroughly when a table rewrite is needed
doc: Explain more thoroughly when a table rewrite is needed Author: Masahiro Ikeda Reviewed-by: Robert Treat Discussion: https://postgr.es/m/00e6eb5f5c793b8ef722252c7a519...@oss.nttdata.com Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/11bd8318602fc2282a6201f714c15461dc2009c6 Modified Files -- doc/src/sgml/ddl.sgml | 8 +++ doc/src/sgml/ref/alter_table.sgml | 46 ++- 2 files changed, 29 insertions(+), 25 deletions(-)
pgsql: Remove direct handling of reloptions for toast tables
Remove direct handling of reloptions for toast tables It doesn't actually work, even with allow_system_table_mods turned on: the ALTER TABLE operation is rejected by ATSimplePermissions(), so even the error message we're adding in this commit is unreachable. Add a test case for it. Author: Nikolay Shaplov Discussion: https://postgr.es/m/1913854.tdWV9SEqCh@thinkpad-pgpro Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/1548c3a30436dd825cfbf57923c6766b2fddd355 Modified Files -- src/backend/commands/tablecmds.c | 3 ++- src/test/modules/unsafe_tests/expected/alter_system_table.out | 7 +++ src/test/modules/unsafe_tests/sql/alter_system_table.sql | 4 3 files changed, 13 insertions(+), 1 deletion(-)
pgsql: Activate Python "Limited API" in PL/Python
Activate Python "Limited API" in PL/Python This allows building PL/Python against any Python 3.x version and using another Python 3.x version at run time. This is useful for installers that want to run against a separately downloaded Python, so that they don't have to bundle it themselves. This builds on the earlier patch to only use APIs supported by the Limited API. At the moment, this is not activated on MSVC because that leads to build failures that no one could explain or cared enough to address. This could be done later. Reviewed-by: Jakob Egger Discussion: https://www.postgresql.org/message-id/flat/ee410de1-1e0b-4770-b125-eeefd4726...@eisentraut.org Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/0793ab810038999c0659b3aad9525aa7ef4c8c26 Modified Files -- src/pl/plpython/plpython.h | 9 + 1 file changed, 9 insertions(+)
pgsql: Respect changing pin limits in read_stream.c.
Respect changing pin limits in read_stream.c. To avoid pinning too much of the buffer pool at once, read_stream.c previously used LimitAdditionalPins(). The coding was naive, and only considered the available buffers at stream construction time. This commit checks before each StartReadBuffers() call with GetAdditionalPinLimit(). The result might change over time due to pins acquired outside this stream by the same backend. No extra CPU cycles are added to the all-buffered fast-path code, but the I/O-starting path now considers the up-to-date remaining buffer limit. In practice it was quite difficult to exceed limits and cause any real problems in v17, so no back-patch for now, but proposed changes will make it easier. Per code review from Andres, in the course of testing his AIO patches. Reviewed-by: Andres Freund (earlier versions) Discussion: https://postgr.es/m/CA%2BhUKGK_%3D4CVmMHvsHjOVrK6t4F%3DLBpFzsrr3R%2BaJYN8kcTfWg%40mail.gmail.com Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/92fc6856cb4c598f99c58b862bc34aebc6f2ec25 Modified Files -- src/backend/storage/aio/read_stream.c | 106 +- 1 file changed, 90 insertions(+), 16 deletions(-)
pgsql: Optimize iteration over PGPROC for fast-path lock searches.
Optimize iteration over PGPROC for fast-path lock searches. This commit improves efficiency in FastPathTransferRelationLocks() and GetLockConflicts(), which iterate over PGPROCs to search for fast-path locks. Previously, these functions recalculated the fast-path group during every loop iteration, even though it remained constant. This update optimizes the process by calculating the group once and reusing it throughout the loop. The functions also now skip empty fast-path groups, avoiding unnecessary scans of their slots. Additionally, groups belonging to inactive backends (with pid=0) are always empty, so checking the group is sufficient to bypass these backends, further enhancing performance. Author: Fujii Masao Reviewed-by: Heikki Linnakangas Reviewed-by: Ashutosh Bapat Discussion: https://postgr.es/m/07d5fd6a-71f1-4ce8-8602-4cc6883f4...@oss.nttdata.com Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/e80171d57c25caf4362a7de17e96195f905386ea Modified Files -- src/backend/storage/lmgr/lock.c | 28 1 file changed, 16 insertions(+), 12 deletions(-)
pgsql: Add GUC option to log lock acquisition failures.
Add GUC option to log lock acquisition failures. This commit introduces a new GUC, log_lock_failure, which controls whether a detailed log message is produced when a lock acquisition fails. Currently, it only supports logging lock failures caused by SELECT ... NOWAIT. The log message includes information about all processes holding or waiting for the lock that couldn't be acquired, helping users analyze and diagnose the causes of lock failures. Currently, this option does not log failures from SELECT ... SKIP LOCKED, as that could generate excessive log messages if many locks are skipped, causing unnecessary noise. This mechanism can be extended in the future to support for logging lock failures from other commands, such as LOCK TABLE ... NOWAIT. Author: Yuki Seino Co-authored-by: Fujii Masao Reviewed-by: Jelte Fennema-Nio Discussion: https://postgr.es/m/411280a186cc26ef7034e0f2dfe54...@oss.nttdata.com Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/6d376c3b0d1e79c318d2a1c04097025784e28377 Modified Files -- doc/src/sgml/config.sgml | 20 src/backend/access/heap/heapam.c | 33 --- src/backend/access/heap/heapam_handler.c | 4 +- src/backend/storage/lmgr/lmgr.c | 33 --- src/backend/storage/lmgr/lock.c | 55 ++- src/backend/storage/lmgr/proc.c | 129 -- src/backend/utils/misc/guc_tables.c | 9 ++ src/backend/utils/misc/postgresql.conf.sample | 1 + src/include/storage/lmgr.h| 5 +- src/include/storage/lock.h| 4 +- src/include/storage/proc.h| 4 + 11 files changed, 211 insertions(+), 86 deletions(-)
