pgsql: Add WAL usage reporting to ANALYZE VERBOSE output.

2024-09-09 Thread Masahiko Sawada
analyze operations. Author: Anthonin Bonnefoy Reviewed-by: Masahiko Sawada Discussion: https://postgr.es/m/CAO6_Xqr__kTTCLkftqS0qSCm-J7_xbRG3Ge2rWhucxQJMJhcRA%40mail.gmail.com Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/bb7775234273268e8852068ee1a2eff9cd2a0020

pgsql: Clarify restrict_nonsystem_relation_kind description.

2024-08-30 Thread Masahiko Sawada
Clarify restrict_nonsystem_relation_kind description. This change improves the description of the restrict_nonsystem_relation_kind parameter in guc_table.c and the documentation for better clarity. Backpatch to 12, where this GUC parameter was introduced. Reviewed-by: Peter Eisentraut Discussion

pgsql: Clarify restrict_nonsystem_relation_kind description.

2024-08-30 Thread Masahiko Sawada
Clarify restrict_nonsystem_relation_kind description. This change improves the description of the restrict_nonsystem_relation_kind parameter in guc_table.c and the documentation for better clarity. Backpatch to 12, where this GUC parameter was introduced. Reviewed-by: Peter Eisentraut Discussion

pgsql: Clarify restrict_nonsystem_relation_kind description.

2024-08-30 Thread Masahiko Sawada
Clarify restrict_nonsystem_relation_kind description. This change improves the description of the restrict_nonsystem_relation_kind parameter in guc_table.c and the documentation for better clarity. Backpatch to 12, where this GUC parameter was introduced. Reviewed-by: Peter Eisentraut Discussion

pgsql: Clarify restrict_nonsystem_relation_kind description.

2024-08-30 Thread Masahiko Sawada
Clarify restrict_nonsystem_relation_kind description. This change improves the description of the restrict_nonsystem_relation_kind parameter in guc_table.c and the documentation for better clarity. Backpatch to 12, where this GUC parameter was introduced. Reviewed-by: Peter Eisentraut Discussion

pgsql: Clarify restrict_nonsystem_relation_kind description.

2024-08-30 Thread Masahiko Sawada
Clarify restrict_nonsystem_relation_kind description. This change improves the description of the restrict_nonsystem_relation_kind parameter in guc_table.c and the documentation for better clarity. Backpatch to 12, where this GUC parameter was introduced. Reviewed-by: Peter Eisentraut Discussion

pgsql: Clarify restrict_nonsystem_relation_kind description.

2024-08-30 Thread Masahiko Sawada
Clarify restrict_nonsystem_relation_kind description. This change improves the description of the restrict_nonsystem_relation_kind parameter in guc_table.c and the documentation for better clarity. Backpatch to 12, where this GUC parameter was introduced. Reviewed-by: Peter Eisentraut Discussion

pgsql: Clarify restrict_nonsystem_relation_kind description.

2024-08-30 Thread Masahiko Sawada
Clarify restrict_nonsystem_relation_kind description. This change improves the description of the restrict_nonsystem_relation_kind parameter in guc_table.c and the documentation for better clarity. Backpatch to 12, where this GUC parameter was introduced. Reviewed-by: Peter Eisentraut Discussion

Re: pgsql: Restrict accesses to non-system views and foreign tables during

2024-08-29 Thread Masahiko Sawada
On Thu, Aug 29, 2024 at 7:29 AM Peter Eisentraut wrote: > > On 27.08.24 22:02, Masahiko Sawada wrote: > > On Tue, Aug 27, 2024 at 6:29 AM Tom Lane wrote: > >> > >> Peter Eisentraut writes: > >>> Maybe in the documentation it would also be appropriate

Re: pgsql: Restrict accesses to non-system views and foreign tables during

2024-08-27 Thread Masahiko Sawada
other catalog-scanning tools might > like to use this too. > I've attached the patch to improve the descriptions while leaving this part. Regards, -- Masahiko Sawada Amazon Web Services: https://aws.amazon.com improve_description.patch Description: Binary data

pgsql: Fix identation.

2024-08-26 Thread Masahiko Sawada
Fix identation. Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/7229ebe011dff3f418251a4836f6d098923aa1e1 Modified Files -- src/backend/replication/logical/reorderbuffer.c | 18 +- 1 file changed, 9 insertions(+), 9 deletions(-)

pgsql: Fix identation.

2024-08-26 Thread Masahiko Sawada
Fix identation. Branch -- REL_17_STABLE Details --- https://git.postgresql.org/pg/commitdiff/c739ae9e288c095cfe1b91ce27a2f2c075ed5da4 Modified Files -- src/backend/replication/logical/reorderbuffer.c | 18 +- 1 file changed, 9 insertions(+), 9 deletions(-)

Re: pgsql: Restrict accesses to non-system views and foreign tables during

2024-08-26 Thread Masahiko Sawada
On Mon, Aug 26, 2024 at 7:14 AM Peter Eisentraut wrote: > > On 05.08.24 15:07, Masahiko Sawada wrote: > > To address this, pg_dump now utilizes the newly introduced > > restrict_nonsystem_relation_kind GUC parameter to restrict the > > accesses to non-system views and f

pgsql: Fix memory counter update in ReorderBuffer.

2024-08-26 Thread Masahiko Sawada
Fix memory counter update in ReorderBuffer. Commit 5bec1d6bc5e changed the memory usage updates of the ReorderBufferTXN to zero all at once by subtracting txn->size, rather than updating it for each change. However, if TOAST reconstruction data remained in the transaction when freeing it, there we

pgsql: Fix memory counter update in ReorderBuffer.

2024-08-26 Thread Masahiko Sawada
Fix memory counter update in ReorderBuffer. Commit 5bec1d6bc5e changed the memory usage updates of the ReorderBufferTXN to zero all at once by subtracting txn->size, rather than updating it for each change. However, if TOAST reconstruction data remained in the transaction when freeing it, there we

pgsql: Add resource statistics reporting to ANALYZE VERBOSE.

2024-08-13 Thread Masahiko Sawada
resource statistics reporting to ANALYZE VERBOSE by reusing the same logging code as autoanalyze. Author: Anthonin Bonnefoy Reviewed-by: Masahiko Sawada Discussion: https://postgr.es/m/CAO6_Xqr__kTTCLkftqS0qSCm-J7_xbRG3Ge2rWhucxQJMJhcRA%40mail.gmail.com Branch -- master Details --- https

pgsql: Use pgBufferUsage for buffer usage tracking in analyze.

2024-08-13 Thread Masahiko Sawada
message in both vacuum and analyze for better consistency. Author: Anthonin Bonnefoy Reviewed-by: Masahiko Sawada, Michael Paquier Discussion: https://postgr.es/m/CAO6_Xqr__kTTCLkftqS0qSCm-J7_xbRG3Ge2rWhucxQJMJhcRA%40mail.gmail.com Branch -- master Details --- https://git.postgresql.org/pg

pgsql: Restrict accesses to non-system views and foreign tables during

2024-08-09 Thread Masahiko Sawada
Restrict accesses to non-system views and foreign tables during pg_dump. When pg_dump retrieves the list of database objects and performs the data dump, there was possibility that objects are replaced with others of the same name, such as views, and access them. This vulnerability could result in

pgsql: Restrict accesses to non-system views and foreign tables during

2024-08-09 Thread Masahiko Sawada
Restrict accesses to non-system views and foreign tables during pg_dump. When pg_dump retrieves the list of database objects and performs the data dump, there was possibility that objects are replaced with others of the same name, such as views, and access them. This vulnerability could result in

pgsql: Restrict accesses to non-system views and foreign tables during

2024-08-09 Thread Masahiko Sawada
Restrict accesses to non-system views and foreign tables during pg_dump. When pg_dump retrieves the list of database objects and performs the data dump, there was possibility that objects are replaced with others of the same name, such as views, and access them. This vulnerability could result in

pgsql: Restrict accesses to non-system views and foreign tables during

2024-08-09 Thread Masahiko Sawada
Restrict accesses to non-system views and foreign tables during pg_dump. When pg_dump retrieves the list of database objects and performs the data dump, there was possibility that objects are replaced with others of the same name, such as views, and access them. This vulnerability could result in

pgsql: Restrict accesses to non-system views and foreign tables during

2024-08-09 Thread Masahiko Sawada
Restrict accesses to non-system views and foreign tables during pg_dump. When pg_dump retrieves the list of database objects and performs the data dump, there was possibility that objects are replaced with others of the same name, such as views, and access them. This vulnerability could result in

pgsql: Restrict accesses to non-system views and foreign tables during

2024-08-09 Thread Masahiko Sawada
Restrict accesses to non-system views and foreign tables during pg_dump. When pg_dump retrieves the list of database objects and performs the data dump, there was possibility that objects are replaced with others of the same name, such as views, and access them. This vulnerability could result in

pgsql: Restrict accesses to non-system views and foreign tables during

2024-08-09 Thread Masahiko Sawada
Restrict accesses to non-system views and foreign tables during pg_dump. When pg_dump retrieves the list of database objects and performs the data dump, there was possibility that objects are replaced with others of the same name, such as views, and access them. This vulnerability could result in

pgsql: Fix possibility of logical decoding partial transaction changes.

2024-07-11 Thread Masahiko Sawada
Fix possibility of logical decoding partial transaction changes. When creating and initializing a logical slot, the restart_lsn is set to the latest WAL insertion point (or the latest replay point on standbys). Subsequently, WAL records are decoded from that point to find the start point for extra

pgsql: Fix possibility of logical decoding partial transaction changes.

2024-07-11 Thread Masahiko Sawada
Fix possibility of logical decoding partial transaction changes. When creating and initializing a logical slot, the restart_lsn is set to the latest WAL insertion point (or the latest replay point on standbys). Subsequently, WAL records are decoded from that point to find the start point for extra

pgsql: Fix possibility of logical decoding partial transaction changes.

2024-07-11 Thread Masahiko Sawada
Fix possibility of logical decoding partial transaction changes. When creating and initializing a logical slot, the restart_lsn is set to the latest WAL insertion point (or the latest replay point on standbys). Subsequently, WAL records are decoded from that point to find the start point for extra

pgsql: Fix possibility of logical decoding partial transaction changes.

2024-07-11 Thread Masahiko Sawada
Fix possibility of logical decoding partial transaction changes. When creating and initializing a logical slot, the restart_lsn is set to the latest WAL insertion point (or the latest replay point on standbys). Subsequently, WAL records are decoded from that point to find the start point for extra

pgsql: Fix possibility of logical decoding partial transaction changes.

2024-07-11 Thread Masahiko Sawada
Fix possibility of logical decoding partial transaction changes. When creating and initializing a logical slot, the restart_lsn is set to the latest WAL insertion point (or the latest replay point on standbys). Subsequently, WAL records are decoded from that point to find the start point for extra

pgsql: Fix possibility of logical decoding partial transaction changes.

2024-07-11 Thread Masahiko Sawada
Fix possibility of logical decoding partial transaction changes. When creating and initializing a logical slot, the restart_lsn is set to the latest WAL insertion point (or the latest replay point on standbys). Subsequently, WAL records are decoded from that point to find the start point for extra

pgsql: Fix possibility of logical decoding partial transaction changes.

2024-07-11 Thread Masahiko Sawada
Fix possibility of logical decoding partial transaction changes. When creating and initializing a logical slot, the restart_lsn is set to the latest WAL insertion point (or the latest replay point on standbys). Subsequently, WAL records are decoded from that point to find the start point for extra

pgsql: Reintroduce dead tuple counter in pg_stat_progress_vacuum.

2024-06-13 Thread Masahiko Sawada
Reintroduce dead tuple counter in pg_stat_progress_vacuum. Commit 667e65aac3 changed both num_dead_tuples and max_dead_tuples columns to dead_tuple_bytes and max_dead_tuple_bytes columns, respectively. But as per discussion, the number of dead tuples collected still provides meaningful insights fo

pgsql: Fix segmentation fault in test_tidstore.

2024-06-11 Thread Masahiko Sawada
Fix segmentation fault in test_tidstore. The do_set_block_offsets() and other functions accessing the tidstore did not check if the tidstore was NULL. This led to a segmentation fault when these functions are called without calling the test_create(). This commit adds NULL checks in relevant funct

pgsql: Fix parallel vacuum buffer usage reporting.

2024-04-30 Thread Masahiko Sawada
ting for VACUUM command with the VERBOSE option was implemented in 15. So backpatch to 15. Reported-by: Anthonin Bonnefoy Author: Anthonin Bonnefoy Reviewed-by: Alena Rybakina, Masahiko Sawada Discussion: https://postgr.es/m/cao6_xqrqk+qzqcys_c6nk0cmfhuuwk85vt9crca1nfffbav...@mail.gmail.com Backpa

pgsql: Fix parallel vacuum buffer usage reporting.

2024-04-30 Thread Masahiko Sawada
ting for VACUUM command with the VERBOSE option was implemented in 15. So backpatch to 15. Reported-by: Anthonin Bonnefoy Author: Anthonin Bonnefoy Reviewed-by: Alena Rybakina, Masahiko Sawada Discussion: https://postgr.es/m/cao6_xqrqk+qzqcys_c6nk0cmfhuuwk85vt9crca1nfffbav...@mail.gmail.com Backpa

pgsql: Fix parallel vacuum buffer usage reporting.

2024-04-30 Thread Masahiko Sawada
ting for VACUUM command with the VERBOSE option was implemented in 15. So backpatch to 15. Reported-by: Anthonin Bonnefoy Author: Anthonin Bonnefoy Reviewed-by: Alena Rybakina, Masahiko Sawada Discussion: https://postgr.es/m/cao6_xqrqk+qzqcys_c6nk0cmfhuuwk85vt9crca1nfffbav...@mail.gmail.com Backpa

pgsql: radixtree: Fix SIGSEGV at update of embeddable value to non-embe

2024-04-25 Thread Masahiko Sawada
radixtree: Fix SIGSEGV at update of embeddable value to non-embeddable. Also, fix a memory leak when updating from non-embeddable to embeddable. Both were unreachable without adding C code. Reported-by: Noah Misch Author: Noah Misch Reviewed-by: Masahiko Sawada, John Naylor Discussion: https

pgsql: doc: Fix COPY ON_ERROR option syntax synopsis.

2024-04-17 Thread Masahiko Sawada
doc: Fix COPY ON_ERROR option syntax synopsis. ON_ERROR option values don't require quotations, which was inconsistent with the syntax synopsis in the documentation. Oversight in b725b7eec43. Author: Atsushi Torikoshi Reviewed-by: Masahiko Sawada Discussion: https://postgr.es/m/CAD

pgsql: Disallow specifying ON_ERROR option without value.

2024-04-16 Thread Masahiko Sawada
the ON_ERROR option, ensuring consistency across other non-boolean options and aligning with the documented syntax. Author: Atsushi Torikoshi Reviewed-by: Masahiko Sawada Discussion: https://postgr.es/m/a9770bf57646d90dedc3d54cf32634b2%40oss.nttdata.com Branch -- master Details --- https

pgsql: Revert indexed and enlargable binary heap implementation.

2024-04-11 Thread Masahiko Sawada
Revert indexed and enlargable binary heap implementation. This reverts commit b840508644 and bcb14f4abc. These commits were made for commit 5bec1d6bc5 (Improve eviction algorithm in ReorderBuffer using max-heap for many subtransactions). However, per discussion, commit efb8acc0d0 replaced binary h

pgsql: Replace binaryheap + index with pairingheap in reorderbuffer.c

2024-04-11 Thread Masahiko Sawada
d with the pairing heap seems to be small enough that it doesn't matter in practice. Reported-by: Jeff Davis Author: Heikki Linnakangas Reviewed-by: Michael Paquier, Hayato Kuroda, Masahiko Sawada Discussion: https://postgr.es/m/12747c15811d94efcc5cda72d6b35c80d7bf3443.camel%40j-da

pgsql: Add more tab completion support for ALTER DEFAULT PRIVILEGES in

2024-04-07 Thread Masahiko Sawada
, with cosmetic adjustments by me Reviewed-by: Shubham Khanna, Masahiko Sawada Discussion: https://postgr.es/m/CALDaNm1aEdJb-QJi%3DGWStkfj_%2BEDUK_VtDkn%2BTjQ2z7HyU0MBw%40mail.gmail.com Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/304b6b1a6b7116bae30ea24119

Re: pgsql: Implement pg_wal_replay_wait() stored procedure

2024-04-02 Thread Masahiko Sawada
Werror=maybe-uninitialized] 275 |delay_ms = (endtime - GetCurrentTimestamp()) / 1000; | ~^~~~ Regards, [1] https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=mamba&dt=2024-04-03%2005%3A33%3A24 -- Masahiko Sawada Amazon Web Services: https://aws.amazon.com

pgsql: Improve eviction algorithm in ReorderBuffer using max-heap for m

2024-04-02 Thread Masahiko Sawada
Improve eviction algorithm in ReorderBuffer using max-heap for many subtransactions. Previously, when selecting the transaction to evict during logical decoding, we check all transactions to find the largest transaction. This could lead to a significant replication lag especially in the case wher

pgsql: Add functions to binaryheap for efficient key removal and update

2024-04-02 Thread Masahiko Sawada
Add functions to binaryheap for efficient key removal and update. Previously, binaryheap didn't support updating a key and removing a node in an efficient way. For example, in order to remove a node from the binaryheap, the caller had to pass the node's position within the array that the binaryhea

pgsql: Make binaryheap enlargeable.

2024-04-02 Thread Masahiko Sawada
Make binaryheap enlargeable. The node array space of the binaryheap is doubled when there is no available space. Reviewed-by: Vignesh C, Peter Smith, Hayato Kuroda, Ajin Cherian, Tomas Vondra, Shubham Khanna Discussion: https://postgr.es/m/CAD21AoDffo37RC-eUuyHJKVEr017V2YYDLyn1xF_00ofptWbkg%40ma

pgsql: Use TidStore for dead tuple TIDs storage during lazy vacuum.

2024-04-01 Thread Masahiko Sawada
Use TidStore for dead tuple TIDs storage during lazy vacuum. Previously, we used a simple array for storing dead tuple IDs during lazy vacuum, which had a number of problems: * The array used a single allocation and so was limited to 1GB. * The allocation was pessimistically sized according to ta

pgsql: Add new COPY option LOG_VERBOSITY.

2024-03-31 Thread Masahiko Sawada
;verbose' is specified, a NOTICE message is emitted for each discarded row, providing additional information such as line number, column name, and the malformed value. This helps users to identify problematic rows that failed to load. Author: Bharath Rupireddy Reviewed-by: Michael Paquier, Atsu

pgsql: Improve tab completion for ALTER TABLE ALTER COLUMN SET in psql.

2024-03-28 Thread Masahiko Sawada
Improve tab completion for ALTER TABLE ALTER COLUMN SET in psql. The commit changes the tab completion to add DATA TYPE after ALTER TABLE ... ALTER COLUMN ... SET. Author: Vignesh C Reviewed-by: Shubham Khanna, Masahiko Sawada Discussion: https://postgr.es/m/CALDaNm1aEdJb-QJi%3DGWStkfj_

pgsql: Rethink create and attach APIs of shared TidStore.

2024-03-27 Thread Masahiko Sawada
Rethink create and attach APIs of shared TidStore. Previously, the behavior of TidStoreCreate() was inconsistent between local and shared TidStore instances in terms of memory limitation. For local TidStore, a memory context was created with initial and maximum memory block sizes, as well as a min

pgsql: Allow specifying initial and maximum segment sizes for DSA.

2024-03-26 Thread Masahiko Sawada
Allow specifying initial and maximum segment sizes for DSA. Previously, the DSA segment size always started with 1MB and grew up to DSA_MAX_SEGMENT_SIZE. It was inconvenient in certain scenarios, such as when the caller desired a soft constraint on the total DSA segment size, limiting it to less t

pgsql: Fix inconsistent function prototypes with function definitions.

2024-03-25 Thread Masahiko Sawada
Fix inconsistent function prototypes with function definitions. Introduced by 30e144287a. Reviewed-by: John Naylor Discussion: https://postgr.es/m/CAD21AoCaDT%2B-ZaVjbtvumms0tyyHPNLELK2UX-MLG9XCgioaNw%40mail.gmail.com Branch -- master Details --- https://git.postgresql.org/pg/commitdif

pgsql: Fix a calculation in TidStoreCreate().

2024-03-25 Thread Masahiko Sawada
Fix a calculation in TidStoreCreate(). Since we expect that the max_bytes is in bytes, not in kilobytes, it should not be multiplied by 1024. Introduced by 30e144287a. Reported-by: John Naylor, David Rowley Reviewed-by: John Naylor Discussion: https://postgr.es/m/CANWCAZZTE-14ofsucofTuhFsfuDGBN

Re: pgsql: Add TIDStore, to store sets of TIDs (ItemPointerData) efficientl

2024-03-25 Thread Masahiko Sawada
On Tue, Mar 26, 2024 at 12:22 PM John Naylor wrote: > > On Tue, Mar 26, 2024 at 9:36 AM Masahiko Sawada wrote: > > > > On Tue, Mar 26, 2024 at 7:16 AM David Rowley wrote: > > > > > > Either "max_bytes" is a bad name for this variable or the * 1024

Re: pgsql: Add TIDStore, to store sets of TIDs (ItemPointerData) efficientl

2024-03-25 Thread Masahiko Sawada
On Tue, Mar 26, 2024 at 7:16 AM David Rowley wrote: > > On Thu, 21 Mar 2024 at 14:10, Masahiko Sawada wrote: > > > > Add TIDStore, to store sets of TIDs (ItemPointerData) efficiently. > > > src/backend/access/common/tidstore.c | 463 > > +

pgsql: Fix potential integer handling issue in radixtree.h.

2024-03-24 Thread Masahiko Sawada
Fix potential integer handling issue in radixtree.h. Coverity complained about the integer handling issue; if we start with an arbitrary non-negative shift value, the loop may decrement it down to something less than zero before exiting. This commit adds an assertion to make sure the 'shift' is al

pgsql: Add TIDStore, to store sets of TIDs (ItemPointerData) efficientl

2024-03-20 Thread Masahiko Sawada
Add TIDStore, to store sets of TIDs (ItemPointerData) efficiently. TIDStore is a data structure designed to efficiently store large sets of TIDs. For TID storage, it employs a radix tree, where the key is a block number, and the value is a bitmap representing offset numbers. The TIDStore can be cr

pgsql: Fix meson installation of xid_wraparound test.

2024-02-08 Thread Masahiko Sawada
Fix meson installation of xid_wraparound test. Fix for e255b646a, to prevent installation of xid_wraparound test module during main install. Reported-by: Peter Eisentraut Discussion: https://postgr.es/m/84cd416a-0e37-4019-8380-1c8a3cdd8c5c%40eisentraut.org Branch -- master Details ---

pgsql: Remove ReorderBufferTupleBuf structure.

2024-01-28 Thread Masahiko Sawada
ucture makes little sense, the ReorderBufferTupleBuf is removed entirely. The code is refactored accordingly. No back-patching since these are ABI changes in an exposed structure and functions, and there would be some risk of breaking extensions. Author: Aleksander Alekseev Reviewed-by: Amit Kapila, Masahiko Sa

pgsql: Add progress reporting of skipped tuples during COPY FROM.

2024-01-24 Thread Masahiko Sawada
py view to report the number of tuples that were skipped because they contain malformed data. Bump catalog version. Author: Atsushi Torikoshi Reviewed-by: Masahiko Sawada Discussion: https://postgr.es/m/d12fd8c99adcae2744212cb23feff6ed%40oss.nttdata.com Branch -- master Details ---

pgsql: Show isCatalogRel in several rmgr descriptions.

2023-12-20 Thread Masahiko Sawada
: Michael Paquier, Masahiko Sawada Discussion: https://postgr.es/m/957dc8f9-2a02-4640-9c01-9dcbf97c4187%40gmail.com Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/bf6260b39da6ef34c7147d560c49f35d2bbf07c9 Modified Files -- src/backend/access/rmgrdesc

pgsql: Add tests for XID wraparound.

2023-11-29 Thread Masahiko Sawada
PG_TEST_EXTRA to run it. Reviewed-by: Daniel Gustafsson, John Naylor, Michael Paquier Reviewed-by: vignesh C Author: Heikki Linnakangas, Masahiko Sawada, Andres Freund Discussion: https://www.postgresql.org/message-id/CAD21AoDVhkXp8HjpFO-gp3TgL6tCKcZQNxn04m01VAtcSi-5sA%40mail.gmail.com Branch

pgsql: Add option to specify timeout seconds to BackgroundPsql.pm.

2023-11-29 Thread Masahiko Sawada
Add option to specify timeout seconds to BackgroundPsql.pm. Previously, a background psql session uses the default timeout and it cannot be overridden. This change adds a new option to set the timeout during start. There are no users of this new option. It is needed for an upcoming patch adding t

pgsql: Fix wrong description of BackgroundPsql's timeout.

2023-11-29 Thread Masahiko Sawada
Fix wrong description of BackgroundPsql's timeout. Backpatch through 16 where this was introduced by 664d757531e1. Reviewed-by: Daniel Gustafsson Backpatch-through: 16 Discussion: http://postgr.es/m/cad21aobxmeqdbloduawvwotlyb4ansxx4oynmyjjbhfq_vg...@mail.gmail.com Branch -- REL_16_STABLE

pgsql: Fix wrong description of BackgroundPsql's timeout.

2023-11-29 Thread Masahiko Sawada
Fix wrong description of BackgroundPsql's timeout. Backpatch through 16 where this was introduced by 664d757531e1. Reviewed-by: Daniel Gustafsson Backpatch-through: 16 Discussion: http://postgr.es/m/cad21aobxmeqdbloduawvwotlyb4ansxx4oynmyjjbhfq_vg...@mail.gmail.com Branch -- master Details

pgsql: Fix BackgroundPsql's set_query_timer_restart() issue without arg

2023-11-29 Thread Masahiko Sawada
Fix BackgroundPsql's set_query_timer_restart() issue without argument. The set_query_timer_restart() required an argument to define a value to query_timer_restart, but none of the existing callers passes an argument to this function. This changes the function to set a value without an argument.

pgsql: Fix BackgroundPsql's set_query_timer_restart() issue without arg

2023-11-29 Thread Masahiko Sawada
Fix BackgroundPsql's set_query_timer_restart() issue without argument. The set_query_timer_restart() required an argument to define a value to query_timer_restart, but none of the existing callers passes an argument to this function. This changes the function to set a value without an argument.

pgsql: Stabilize subscription stats test.

2023-09-08 Thread Masahiko Sawada
Stabilize subscription stats test. The new test added by commit 68a59f9e9 disables the subscription and manually drops the associated replication slot. However, since disabling the subsubscription doesn't wait for a walsender to release the replication slot and exit, pg_drop_replication_slot() cou

pgsql: Stabilize subscription stats test.

2023-09-08 Thread Masahiko Sawada
Stabilize subscription stats test. The new test added by commit 68a59f9e9 disables the subscription and manually drops the associated replication slot. However, since disabling the subsubscription doesn't wait for a walsender to release the replication slot and exit, pg_drop_replication_slot() cou

pgsql: Stabilize subscription stats test.

2023-09-08 Thread Masahiko Sawada
Stabilize subscription stats test. The new test added by commit 68a59f9e9 disables the subscription and manually drops the associated replication slot. However, since disabling the subsubscription doesn't wait for a walsender to release the replication slot and exit, pg_drop_replication_slot() cou

pgsql: Fix ReorderBufferCheckMemoryLimit() comment.

2023-08-01 Thread Masahiko Sawada
Fix ReorderBufferCheckMemoryLimit() comment. Commit 7259736a6 updated the comment but it was not correct since ReorderBufferLargestStreamableTopTXN() returns only top-level transactions. Reviewed-by: Amit Kapila Discussion: https://postgr.es/m/CAD21AoA9XB7OR86BqvrCe2dMYX%2BZv3-BvVmjF%3DGY2z6jN-k

pgsql: Fix ReorderBufferCheckMemoryLimit() comment.

2023-08-01 Thread Masahiko Sawada
Fix ReorderBufferCheckMemoryLimit() comment. Commit 7259736a6 updated the comment but it was not correct since ReorderBufferLargestStreamableTopTXN() returns only top-level transactions. Reviewed-by: Amit Kapila Discussion: https://postgr.es/m/CAD21AoA9XB7OR86BqvrCe2dMYX%2BZv3-BvVmjF%3DGY2z6jN-k

pgsql: Fix ReorderBufferCheckMemoryLimit() comment.

2023-08-01 Thread Masahiko Sawada
Fix ReorderBufferCheckMemoryLimit() comment. Commit 7259736a6 updated the comment but it was not correct since ReorderBufferLargestStreamableTopTXN() returns only top-level transactions. Reviewed-by: Amit Kapila Discussion: https://postgr.es/m/CAD21AoA9XB7OR86BqvrCe2dMYX%2BZv3-BvVmjF%3DGY2z6jN-k

pgsql: Fix ReorderBufferCheckMemoryLimit() comment.

2023-08-01 Thread Masahiko Sawada
Fix ReorderBufferCheckMemoryLimit() comment. Commit 7259736a6 updated the comment but it was not correct since ReorderBufferLargestStreamableTopTXN() returns only top-level transactions. Reviewed-by: Amit Kapila Discussion: https://postgr.es/m/CAD21AoA9XB7OR86BqvrCe2dMYX%2BZv3-BvVmjF%3DGY2z6jN-k

pgsql: Fix crash with RemoveFromWaitQueue() when detecting a deadlock.

2023-07-25 Thread Masahiko Sawada
Fix crash with RemoveFromWaitQueue() when detecting a deadlock. Commit 5764f611e used dclist_delete_from() to remove the proc from the wait queue. However, since it doesn't clear dist_node's next/prev to NULL, it could call RemoveFromWaitQueue() twice: when the process detects a deadlock and then

pgsql: Fix crash with RemoveFromWaitQueue() when detecting a deadlock.

2023-07-25 Thread Masahiko Sawada
Fix crash with RemoveFromWaitQueue() when detecting a deadlock. Commit 5764f611e used dclist_delete_from() to remove the proc from the wait queue. However, since it doesn't clear dist_node's next/prev to NULL, it could call RemoveFromWaitQueue() twice: when the process detects a deadlock and then

pgsql: Remove unnecessary checks for indexes for REPLICA IDENTITY FULL

2023-07-24 Thread Masahiko Sawada
Remove unnecessary checks for indexes for REPLICA IDENTITY FULL tables. Previously, when selecting an usable index for update/delete for the REPLICA IDENTITY FULL table, in IsIndexOnlyExpression(), we used to check if all index fields are not expressions. However, it was not necessary, because it

pgsql: Remove unnecessary checks for indexes for REPLICA IDENTITY FULL

2023-07-24 Thread Masahiko Sawada
Remove unnecessary checks for indexes for REPLICA IDENTITY FULL tables. Previously, when selecting an usable index for update/delete for the REPLICA IDENTITY FULL table, in IsIndexOnlyExpression(), we used to check if all index fields are not expressions. However, it was not necessary, because it

pgsql: Doc: clarify the conditions of usable indexes for REPLICA IDENTI

2023-07-12 Thread Masahiko Sawada
Doc: clarify the conditions of usable indexes for REPLICA IDENTITY FULL tables. Commit 89e46da5e allowed REPLICA IDENTITY FULL tables to use an index on the subscriber during apply of update/delete. This commit clarifies in the documentation that the leftmost field of candidate indexes must be a c

pgsql: Doc: clarify the conditions of usable indexes for REPLICA IDENTI

2023-07-12 Thread Masahiko Sawada
Doc: clarify the conditions of usable indexes for REPLICA IDENTITY FULL tables. Commit 89e46da5e allowed REPLICA IDENTITY FULL tables to use an index on the subscriber during apply of update/delete. This commit clarifies in the documentation that the leftmost field of candidate indexes must be a c

Re: pgsql: Report index vacuum progress.

2023-07-10 Thread Masahiko Sawada
On Tue, Jul 11, 2023 at 12:53 PM Michael Paquier wrote: > > On Tue, Jul 11, 2023 at 03:36:32AM +, Masahiko Sawada wrote: > > This uses the new parallel message type for progress reporting added > > by be06506e7. > > Seems like this should be f188972. Hmm, right. I for

pgsql: Report index vacuum progress.

2023-07-10 Thread Masahiko Sawada
new parallel message type for progress reporting added by be06506e7. Bump catversion because this changes the definition of pg_stat_progress_vacuum. Author: Sami Imseih Reviewed by: Masahiko Sawada, Michael Paquier, Nathan Bossart, Andres Freund Discussion: https://www.postgresql.org/message-id/fla

pgsql: Add new parallel message type to progress reporting.

2023-07-10 Thread Masahiko Sawada
f other backend progress APIs in the future. There are no users of this new message type as of this commit. That will follow in future commits. Idea from Andres Freund. Author: Sami Imseih Reviewed by: Michael Paquier, Masahiko Sawada Discussion: https://www.postgresql.org/message-id/flat/5478dfcd

pgsql: pgstat: fix subscription stats entry leak.

2023-07-04 Thread Masahiko Sawada
slots for both apply worker and tablesync worker. Also add a test to verify that the stats for slot-less subscription is removed at DROP SUBSCRIPTION time. Backpatch down to 15. Author: Masahiko Sawada Reviewed-by: Nathan Bossart, Hayato Kuroda, Melih Mutlu, Amit Kapila Discussion: https://postgr.es

pgsql: pgstat: fix subscription stats entry leak.

2023-07-04 Thread Masahiko Sawada
slots for both apply worker and tablesync worker. Also add a test to verify that the stats for slot-less subscription is removed at DROP SUBSCRIPTION time. Backpatch down to 15. Author: Masahiko Sawada Reviewed-by: Nathan Bossart, Hayato Kuroda, Melih Mutlu, Amit Kapila Discussion: https://postgr.es

pgsql: pgstat: fix subscription stats entry leak.

2023-07-04 Thread Masahiko Sawada
slots for both apply worker and tablesync worker. Also add a test to verify that the stats for slot-less subscription is removed at DROP SUBSCRIPTION time. Backpatch down to 15. Author: Masahiko Sawada Reviewed-by: Nathan Bossart, Hayato Kuroda, Melih Mutlu, Amit Kapila Discussion: https://postgr.es

pgsql: Add missing pg_basebackup TAP test for meson.

2023-06-15 Thread Masahiko Sawada
Add missing pg_basebackup TAP test for meson. 011_in_place_tablespace was missing from the list of pg_basebackup tests to run under meson, so add it. Oversight in 363e8f9115. Discussion: https://postgr.es/m/CAD21AoDTh1A8bvNBF3LQNQg=27xtpsgvpt+4_yyej6p4zv8...@mail.gmail.com Branch -- master

pgsql: Replace GUC_UNIT_MEMORY|GUC_UNIT_TIME with GUC_UNIT.

2023-06-15 Thread Masahiko Sawada
, Michael Paquier, Masahiko Sawada Discussion: https://postgr.es/m/meyp282mb1669ec0fed922f7a151673acb6...@meyp282mb1669.ausp282.prod.outlook.com Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/a54fc892ad0a5673d75b199128b1f5fcf5e1a41d Modified Files -- src

pgsql: Fix typo in comment.

2023-06-13 Thread Masahiko Sawada
Fix typo in comment. Introduced in 4d330a61bb1. Author: Masahiko Sawada Reviewed-by: Michael Paquier Discussion: https://postgr.es/m/cad21aodg8rtwjkrnjg9utp89vs8smfib2c55dvqkrcn8zr-...@mail.gmail.com Branch -- master Details --- https://git.postgresql.org/pg/commitdiff

pgsql: Honor run_as_owner option in tablesync worker.

2023-06-08 Thread Masahiko Sawada
he initial data copy. With this change, tablesync workers run with appropriate permissions based on "run_as_owner" option. Ajin Cherian, with changes and regression tests added by me. Reported-By: Amit Kapila Author: Ajin Cherian, Masahiko Sawada Reviewed-by: Ajin Cherian, Amit Kapil

pgsql: Add unit to vacuum_buffer_usage_limit value in postgresql.conf.s

2023-04-27 Thread Masahiko Sawada
Add unit to vacuum_buffer_usage_limit value in postgresql.conf.sample. Also adjust the indentation of the comment to the surrounding parameters. Author: Masahiko Sawada Reviewed-by: David Rowley, Daniel Gustafsson, Melanie Plageman Discussion: https://postgr.es/m

pgsql: Use elog to report unexpected action in handle_streamed_transact

2023-04-23 Thread Masahiko Sawada
Use elog to report unexpected action in handle_streamed_transaction(). An oversight in commit 216a784829c. Author: Masahiko Sawada Reviewed-by: Kyotaro Horiguchi, Amit Kapila Discussion: https://postgr.es/m/cad21aoddbm8_hjt-nmcvcjtk8k9hpzxwqjj7pyauvr4mm_n...@mail.gmail.com Branch -- master

Re: pgsql: Raise a warning if there is a possibility of data from multiple

2022-09-08 Thread Masahiko Sawada
On Thu, Sep 8, 2022 at 4:23 PM Amit Kapila wrote: > > On Thu, Sep 8, 2022 at 12:22 PM Masahiko Sawada wrote: > > > > On Thu, Sep 8, 2022 at 10:39 AM Amit Kapila wrote: > > > > > > Raise a warning if there is a possibility of data from multiple origins. >

Re: pgsql: Raise a warning if there is a possibility of data from multiple

2022-09-07 Thread Masahiko Sawada
"); Should the tables and the function in this query be schema-qualified? Looking at other code in subscriptioncmd.c, we use schema-qualified names. It works even without the schema names but it may be better to make them consistent. FYI, looking at tablesync.c, there are both styles; it seems that recently added codes use schema-unqualified names. Regards, -- Masahiko Sawada

Re: pgsql: Add wait_for_subscription_sync for TAP tests.

2022-08-03 Thread Masahiko Sawada
eally see the difficulty > > later. > > IMO we've seldom regretted back-patching testing infrastructure. > We have regretted not doing so. Okay, I'll submit patches for backbranches to the original thread on -hackers. Regards, -- Masahiko Sawada EDB: https://www.enterprisedb.com/

Re: pgsql: Move parallel vacuum code to vacuumparallel.c.

2021-12-22 Thread Masahiko Sawada
tate; ^ ../../../src/include/commands/vacuum.h:68:36: note: previous definition is here typedef struct ParallelVacuumState ParallelVacuumState; ^ 1 warning generated. It seems a C11 feature. The attached patch fixes this issue. Regards, -- Masahiko Sawada EDB: htt

Re: pgsql: Add a view to show the stats of subscription workers.

2021-11-29 Thread Masahiko Sawada
ption_workers by specifying the relation name *and* xid like the first check does. I've attached the patch. What do you think? FWIW, I'm still not investigating why "replication origin with OID 2 is already active for PID 23706" occurred. But regardle

Re: pgsql: pgstat: Bring up pgstat in BaseInit() to fix uninitialized use o

2021-08-12 Thread Masahiko Sawada
On Thu, Aug 12, 2021 at 3:16 PM Amit Kapila wrote: > > On Thu, Aug 12, 2021 at 11:38 AM Dilip Kumar wrote: > > > > On Thu, Aug 12, 2021 at 7:39 AM Masahiko Sawada > > wrote: > > > > > > On Wed, Aug 11, 2021 at 10:47 PM Amit Kapila > > > wro

Re: pgsql: pgstat: Bring up pgstat in BaseInit() to fix uninitialized use o

2021-08-11 Thread Masahiko Sawada
On Wed, Aug 11, 2021 at 10:47 PM Amit Kapila wrote: > > On Tue, Aug 10, 2021 at 4:37 PM Masahiko Sawada wrote: > > > > The apply worker registers SharedFileSetDeleteOnProcExit() when > > creating a file set to serialize the changes. When it raises an error > > due t

Re: pgsql: pgstat: Bring up pgstat in BaseInit() to fix uninitialized use o

2021-08-10 Thread Masahiko Sawada
teOnProcExit() when creating a file set to serialize the changes. When it raises an error due to conflict during applying the change, the callback eventually reports the temp file statistics but pgstat already shut down, resulting in this assertion failure. Regards, -- Masahiko Sawada EDB: https://www.enterprisedb.com/

  1   2   >