pgsql: Optimize visibilitymap_count() with AVX-512 instructions.

Optimize visibilitymap_count() with AVX-512 instructions. Commit 792752af4e added infrastructure for using AVX-512 intrinsic functions, and this commit uses that infrastructure to optimize visibilitymap_count(). Specificially, a new pg_popcount_masked() function is introduced that applies a bitma

pgsql: Optimize pg_popcount() with AVX-512 instructions.

Optimize pg_popcount() with AVX-512 instructions. Presently, pg_popcount() processes data in 32-bit or 64-bit chunks when possible. Newer hardware that supports AVX-512 instructions can use 512-bit chunks, which provides a nice speedup, especially for larger buffers. This commit introduces the i

pgsql: Fix if/while thinko in read_stream.c edge case.

Fix if/while thinko in read_stream.c edge case. When we determine that a wanted block can't be combined with the current pending read, it's time to start that read to get it out of the way. An "if" in that code path should have been a "while", because it might take more than one go in case of par

pgsql: Disable parallel query in psql error-with-FETCH_COUNT test.

Disable parallel query in psql error-with-FETCH_COUNT test. The buildfarm members using debug_parallel_query = regress are mostly unhappy with this test. I guess what is happening is that rows generated by a parallel worker are buffered, and might or might not get to the leader before the expecte

pgsql: Support retrieval of results in chunks with libpq.

Support retrieval of results in chunks with libpq. This patch generalizes libpq's existing single-row mode to allow individual partial-result PGresults to contain up to N rows, rather than always one row. This reduces malloc overhead compared to plain single-row mode, and it is very useful for ps

pgsql: Re-implement psql's FETCH_COUNT feature atop libpq's chunked mod

Re-implement psql's FETCH_COUNT feature atop libpq's chunked mode. Formerly this was done with a cursor, which is problematic since not all result-set-returning query types can be put into a cursor. The new implementation is better integrated into other psql features, too. Daniel Vérité, reviewed

pgsql: Change BitmapAdjustPrefetchIterator to accept BlockNumber

Change BitmapAdjustPrefetchIterator to accept BlockNumber BitmapAdjustPrefetchIterator() only used the blockno member of the passed in TBMIterateResult to ensure that the prefetch iterator and regular iterator stay in sync. Pass it the BlockNumber only, so that we can move away from using the TBMI

pgsql: BitmapHeapScan: Use correct recheck flag for skip_fetch

BitmapHeapScan: Use correct recheck flag for skip_fetch As of 7c70996ebf0949b142a9, BitmapPrefetch() used the recheck flag for the current block to determine whether or not it should skip prefetching the proposed prefetch block. As explained in the comment, this assumed the index AM will report th

pgsql: BitmapHeapScan: Push skip_fetch optimization into table AM

BitmapHeapScan: Push skip_fetch optimization into table AM Commit 7c70996ebf0949b142 introduced an optimization to allow bitmap scans to operate like index-only scans by not fetching a block from the heap if none of the underlying data is needed and the block is marked all visible in the visibilit

pgsql: Implement ALTER TABLE ... MERGE PARTITIONS ... command

Implement ALTER TABLE ... MERGE PARTITIONS ... command This new DDL command merges several partitions into the one partition of the target table. The target partition is created using new createPartitionTable() function with parent partition as the template. This commit comprises quite naive imp

pgsql: Implement ALTER TABLE ... SPLIT PARTITION ... command

Implement ALTER TABLE ... SPLIT PARTITION ... command This new DDL command splits a single partition into several parititions. Just like ALTER TABLE ... MERGE PARTITIONS ... command, new patitions are created using createPartitionTable() function with parent partition as the template. This commit

pgsql: BitmapHeapScan: postpone setting can_skip_fetch

BitmapHeapScan: postpone setting can_skip_fetch Set BitmapHeapScanState->can_skip_fetch in BitmapHeapNext() instead of in ExecInitBitmapHeapScan(). This is a preliminary step to pushing the skip fetch optimization into heap AM code. Author: Melanie Plageman Reviewed-by: Tomas Vondra, Andres Freun

pgsql: Use an LWLock instead of a spinlock in waitlsn.c

Use an LWLock instead of a spinlock in waitlsn.c This should prevent busy-waiting when number of waiting processes is high. Discussion: https://postgr.es/m/202404030658.hhj3vfxeyhft%40alvherre.pgsql Author: Alvaro Herrera Branch -- master Details --- https://git.postgresql.org/pg/commit

pgsql: Call WaitLSNCleanup() in AbortTransaction()

Call WaitLSNCleanup() in AbortTransaction() Even though waiting for replay LSN happens without explicit transaction, AbortTransaction() is responsible for the cleanup of the shared memory if the error is thrown in a stored procedure. So, we need to do WaitLSNCleanup() there to clean up after some

pgsql: Clarify what is protected by WaitLSNLock

Clarify what is protected by WaitLSNLock Not just WaitLSNState.waitersHeap, but also WaitLSNState.procInfos and updating of WaitLSNState.minWaitedLSN is protected by WaitLSNLock. There is one now documented exclusion on fast-path checking of WaitLSNProcInfo.inHeap flag. Discussion: https://postg

pgsql: BitmapHeapScan: begin scan after bitmap creation

BitmapHeapScan: begin scan after bitmap creation Change the order so that the table scan is initialized only after initializing the index scan and building the bitmap. This is mostly a cosmetic change for now, but later commits will need to pass parameters to table_beginscan_bm() that are unavail

pgsql: Backport IPC::Run optimization to src/test/perl.

Backport IPC::Run optimization to src/test/perl. This one-liner makes the TAP portion of "make check-world" 7% faster on a non-Windows machine. Discussion: https://postgr.es/m/20240331050310...@rfd.leadboat.com Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/06558f

pgsql: Enhance nbtree ScalarArrayOp execution.

Enhance nbtree ScalarArrayOp execution. Commit 9e8da0f7 taught nbtree to handle ScalarArrayOpExpr quals natively. This works by pushing down the full context (the array keys) to the nbtree index AM, enabling it to execute multiple primitive index scans that the planner treats as one continuous in

pgsql: Remove obsolete comment in CopyReadLineText().

Remove obsolete comment in CopyReadLineText(). When this bit of commentary was written, it was alluding to the fact that we looked for newlines and EOD markers in the raw (not yet encoding-converted) input data. We don't do that anymore, preferring to batch the conversion of larger chunks of inpu

pgsql: Teach fasthash_accum to use platform endianness for bytewise loa

Teach fasthash_accum to use platform endianness for bytewise loads This function previously used a mix of word-wise loads and bytewise loads. The bytewise loads happened to be little-endian regardless of platform. This in itself is not a problem. However, a future commit will require the same resu

pgsql: Speed up tail processing when hashing aligned C strings, take tw

Speed up tail processing when hashing aligned C strings, take two After encountering the NUL terminator, the word-at-a-time loop exits and we must hash the remaining bytes. Previously we calculated the terminator's position and re-loaded the remaining bytes from the input string. This was slower t

pgsql: Allow BufferAccessStrategy to limit pin count.

Allow BufferAccessStrategy to limit pin count. While pinning extra buffers to look ahead, users of strategies are in danger of using too many buffers. For some strategies, that means "escaping" from the ring, and in others it means forcing dirty data to disk very frequently with associated WAL fl

pgsql: Increase default vacuum_buffer_usage_limit to 2MB.

Increase default vacuum_buffer_usage_limit to 2MB. The BAS_VACUUM ring size has been 256kB since commit d526575f introduced the mechanism 17 years ago. Commit 1cbbee03 recently made it configurable but retained the traditional default. The correct default size has been debated for years, but 256