pgsql: Remove extra comment at TableAmRoutine.scan_analyze_next_block
Remove extra comment at TableAmRoutine.scan_analyze_next_block The extra comment was accidentally copied here by 6377e12a from heapam_scan_analyze_next_block(). Reported-by: Matthias van de Meent Discussion: https://postgr.es/m/CAEze2WjC5PiweG-4oe0hB_Zr59iF3tRE1gURm8w4Cg5b6JEBGw%40mail.gmail.com Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/70a845c04a47645b58f8276a6b3ab201ea8ec426 Modified Files -- src/include/access/tableam.h | 10 -- 1 file changed, 10 deletions(-)
pgsql: Add doc entry for the new GUC paramenter enable_group_by_reorder
Add doc entry for the new GUC paramenter enable_group_by_reordering 0452b461bc4 adds alternative orderings of group-by keys during the query optimization. This new feature is controlled by the new GUC parameter enable_group_by_reordering, which accidentally came without the documentation. This commit adds the missing documentation for that GUC. Reported-by: Bruce Momjian Discussion: https://postgr.es/m/ZnDx2FYlba_OafQd%40momjian.us Author: Andrei Lepikhov Reviewed-by: Pavel Borisov, Alexander Korotkov Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/82e79ee46b1c880cb7376cf4399c9883c1ddfaea Modified Files -- doc/src/sgml/config.sgml | 19 +++ 1 file changed, 19 insertions(+)
pgsql: Fix asymmetry in setting EquivalenceClass.ec_sortref
Fix asymmetry in setting EquivalenceClass.ec_sortref 0452b461bc made get_eclass_for_sort_expr() always set EquivalenceClass.ec_sortref if it's not done yet. This leads to an asymmetric situation when whoever first looks for the EquivalenceClass sets the ec_sortref. It is also counterintuitive that get_eclass_for_sort_expr() performs modification of data structures. This commit makes make_pathkeys_for_sortclauses_extended() responsible for setting EquivalenceClass.ec_sortref. Now we set the EquivalenceClass.ec_sortref's needed to explore alternative GROUP BY ordering specifically during building pathkeys by the list of grouping clauses. Discussion: https://postgr.es/m/17037754-f187-4138-8285-0e2bfebd0dea%40postgrespro.ru Reported-by: Tom Lane Author: Andrei Lepikhov Reviewed-by: Alexander Korotkov, Pavel Borisov Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/199012a3d844c6283e0ab4b1139440840a91433d Modified Files -- src/backend/optimizer/path/equivclass.c | 13 + src/backend/optimizer/path/pathkeys.c| 18 ++-- src/backend/optimizer/plan/planner.c | 16 --- src/include/optimizer/paths.h| 3 +- src/test/regress/expected/aggregates.out | 47 src/test/regress/sql/aggregates.sql | 14 ++ 6 files changed, 92 insertions(+), 19 deletions(-)
pgsql: Add invariants check to get_useful_group_keys_orderings()
Add invariants check to get_useful_group_keys_orderings() This commit introduces invariants checking of generated orderings in get_useful_group_keys_orderings() for assert-enabled builds. Discussion: https://postgr.es/m/a663f0f6-cbf6-49aa-af2e-234dc6768a07%40postgrespro.ru Reported-by: Tom Lane Author: Andrei Lepikhov Reviewed-by: Alexander Korotkov, Pavel Borisov Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/91143c03d4ca36406a53e05cd421b19e47d131d1 Modified Files -- src/backend/optimizer/path/pathkeys.c | 28 1 file changed, 28 insertions(+)
pgsql: Rename PathKeyInfo to GroupByOrdering
Rename PathKeyInfo to GroupByOrdering 0452b461bc made optimizer explore alternative orderings of group-by pathkeys. The PathKeyInfo data structure was used to store the particular ordering of group-by pathkeys and corresponding clauses. It turns out that PathKeyInfo is not the best name for that purpose. This commit renames this data structure to GroupByOrdering, and revises its comment. Discussion: https://postgr.es/m/db0fc3a4-966c-4cec-a136-94024d39212d%40postgrespro.ru Reported-by: Tom Lane Author: Andrei Lepikhov Reviewed-by: Alexander Korotkov, Pavel Borisov Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/0c1af2c35c7b456bd2fc76bbc9df5aa9c7911bde Modified Files -- src/backend/optimizer/path/pathkeys.c | 18 +- src/backend/optimizer/plan/planner.c | 8 src/include/nodes/pathnodes.h | 13 ++--- src/tools/pgindent/typedefs.list | 2 +- 4 files changed, 24 insertions(+), 17 deletions(-)
pgsql: Restore preprocess_groupclause()
Restore preprocess_groupclause() 0452b461bc made optimizer explore alternative orderings of group-by pathkeys. It eliminated preprocess_groupclause(), which was intended to match items between GROUP BY and ORDER BY. Instead, get_useful_group_keys_orderings() function generates orderings of GROUP BY elements at the time of grouping paths generation. The get_useful_group_keys_orderings() function takes into account 3 orderings of GROUP BY pathkeys and clauses: original order as written in GROUP BY, matching ORDER BY clauses as much as possible, and matching the input path as much as possible. Given that even before 0452b461b, preprocess_groupclause() could change the original order of GROUP BY clauses we don't need to consider it apart from ordering matching ORDER BY clauses. This commit restores preprocess_groupclause() to provide an ordering of GROUP BY elements matching ORDER BY before generation of paths. The new version of preprocess_groupclause() takes into account an incremental sort. The get_useful_group_keys_orderings() function now takes into 2 orderings of GROUP BY elements: the order generated preprocess_groupclause() and the order matching the input path as much as possible. Discussion: https://postgr.es/m/CAPpHfdvyWLMGwvxaf%3D7KAp-z-4mxbSH8ti2f6mNOQv5metZFzg%40mail.gmail.com Author: Alexander Korotkov Reviewed-by: Andrei Lepikhov, Pavel Borisov Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/505c008ca37c4f6f2fffcde370b5d8354c4d4dc3 Modified Files -- src/backend/optimizer/path/pathkeys.c | 55 +-- src/backend/optimizer/plan/planner.c | 108 +++--- src/include/nodes/pathnodes.h | 6 +- src/test/regress/expected/partition_aggregate.out | 6 +- 4 files changed, 108 insertions(+), 67 deletions(-)
pgsql: amcheck: Fixes for right page check during unique constraint che
amcheck: Fixes for right page check during unique constraint check * Don't forget to pfree() the right page when it's to be ignored. * Report error on unexpected non-leaf right page even if this page is not to be ignored. This restores the logic which was unintendedly changed in 97e5b0026f. Reported-by: Pavel Borisov Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/945ec4c4bca1e1c4347cd3f93135e96770ac1b4c Modified Files -- contrib/amcheck/verify_nbtree.c | 22 -- 1 file changed, 12 insertions(+), 10 deletions(-)
pgsql: Provide deterministic order for catalog queries in partition_spl
Provide deterministic order for catalog queries in partition_split.sql System catalog tables are subject to modification by parallel tests. This is the source of instability when querying them without explicit ORDER BY. This commit adds explicit ORDER BY to system catalog queries in partition_split.sql to stabilize the result. Reported-by: Tom Lane Discussion: https://postgr.es/m/695264.1716578979%40sss.pgh.pa.us Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/d53a4286d772c50ad7a8ff72ca637de613532592 Modified Files -- src/test/regress/expected/partition_split.out | 34 +-- src/test/regress/sql/partition_split.sql | 34 +-- 2 files changed, 34 insertions(+), 34 deletions(-)
pgsql: Don't copy extended statistics during MERGE/SPLIT partition oper
Don't copy extended statistics during MERGE/SPLIT partition operations When MERGE/SPLIT created new partitions, it was cloning the extended statistics of the parent table. However, extended stats on partitioned tables don't behave like indexes on partitioned tables (which exist only to create physical indexes on child tables). Rather, extended stats on a parent 1) cause extended stats to be collected and computed across the whole partition hierarchy, and 2) do not cause extended stats to be computed for the individual partitions. "CREATE TABLE ... PARTITION OF" command doesn't copy extended statistics. This commit makes createPartitionTable() behave consistently. Reported-by: Justin Pryzby Discussion: https://postgr.es/m/ZiJW1g2nbQs9ekwK%40pryzbyj2023 Author: Alexander Korotkov, Justin Pryzby Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/fbd4321fd5b4400fbbf356d686af6ad6d3208c66 Modified Files -- doc/src/sgml/ref/alter_table.sgml | 9 +++-- src/backend/commands/tablecmds.c | 8 +--- 2 files changed, 12 insertions(+), 5 deletions(-)
pgsql: Fix the name collision detection in MERGE/SPLIT partition operat
Fix the name collision detection in MERGE/SPLIT partition operations Both MERGE and SPLIT partition operations support the case when the name of the new partition matches the name of the existing partition to be merged/split. But the name collision detection doesn't always work as intended. The SPLIT partition operation finds the namespace to search for an existing partition without taking into account the parent's persistence. The MERGE partition operation checks for the name collision with simple equal() on RangeVar's simply ignoring the search_path. This commit fixes this behavior as follows. 1. The SPLIT partition operation now finds the namespace to search for an existing partition according to the parent's persistence. 2. The MERGE partition operation now checks for the name collision similarly to the SPLIT partition operation using RangeVarGetAndCheckCreationNamespace() and get_relname_relid(). Reported-by: Alexander Lakhin Discussion: https://postgr.es/m/86b4f1e3-0b5d-315c-9225-19860d64d685%40gmail.com Author: Dmitry Koval, Alexander Korotkov Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/3a82c689fd1be9bdf9d60135e5db7d352c051269 Modified Files -- src/backend/commands/tablecmds.c | 62 ++- src/test/regress/expected/partition_merge.out | 3 +- src/test/regress/expected/partition_split.out | 8 src/test/regress/sql/partition_merge.sql | 3 +- src/test/regress/sql/partition_split.sql | 9 5 files changed, 72 insertions(+), 13 deletions(-)
pgsql: amcheck: Don't load the right sibling page into BtreeCheckState
amcheck: Don't load the right sibling page into BtreeCheckState 5ae2087202 implemented a cross-page unique constraint check by loading the right sibling to the BtreeCheckState.target variable. This is wrong, because bt_target_page_check() shouldn't change the target page. Also, BtreeCheckState.target shouldn't be changed alone without BtreeCheckState.targetblock. However, the above didn't cause any visible bugs for the following reasons. 1. We do a cross-page unique constraint check only for leaf index pages. 2. The only way target page get accessed after a cross-page unique constraint check is loading of the lowkey. 3. The only place lowkey is used is bt_child_highkey_check(), and that applies only to non-leafs. The reasons above don't diminish the fact that changing BtreeCheckState.target for a cross-page unique constraint check is wrong. This commit changes this check to temporarily store the right sibling to the local variable. Reported-by: Peter Geoghegan Discussion: https://postgr.es/m/CAH2-Wzk%2B2116uOXdOViA27SHcr31WKPgmjsxXLBs_aTxAeThzg%40mail.gmail.com Author: Pavel Borisov Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/0b5c161248110b164a006004c78f9529a109 Modified Files -- contrib/amcheck/verify_nbtree.c | 14 +- 1 file changed, 9 insertions(+), 5 deletions(-)
pgsql: amcheck: Refactoring the storage of the last visible entry
amcheck: Refactoring the storage of the last visible entry This commit introduces a new data structure BtreeLastVisibleEntry comprising information about the last visible heap entry with the current value of key. Usage of this data structure allows us to avoid passing all this information as individual function arguments. Reported-by: Alexander Korotkov Discussion: https://www.postgresql.org/message-id/CAPpHfdsVbB9ToriaB1UHuOKwjKxiZmTFQcEF%3DjuzzC_nby31uA%40mail.gmail.com Author: Pavel Borisov, Alexander Korotkov Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/532d94fec32ac11263b53932365560491d1fd50a Modified Files -- contrib/amcheck/verify_nbtree.c | 117 --- src/tools/pgindent/typedefs.list | 1 + 2 files changed, 60 insertions(+), 58 deletions(-)
pgsql: amcheck: Report an error when the next page to a leaf is not a l
amcheck: Report an error when the next page to a leaf is not a leaf This is a very unlikely condition during checking a B-tree unique constraint, meaning that the index structure is violated badly, and we shouldn't continue checking to avoid endless loops, etc. So it's worth immediately throwing an error. Reported-by: Peter Geoghegan Discussion: https://postgr.es/m/CAH2-Wzk%2B2116uOXdOViA27SHcr31WKPgmjsxXLBs_aTxAeThzg%40mail.gmail.com Author: Pavel Borisov Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/97e5b0026fc276ab1bcde58ae98ae1fcd9c3acc3 Modified Files -- contrib/amcheck/verify_nbtree.c | 22 -- 1 file changed, 16 insertions(+), 6 deletions(-)
pgsql: Fix regression tests conflict in 3ca43dbbb6
Fix regression tests conflict in 3ca43dbbb6 3ca43dbbb6 adds regression tests with permission checks. The conflict has been observed at buildfarm member piculet. This commit fixes the conflict in the following way. 1. partition_split.sql now uses role names regress_partition_split_alice and regress_partition_split_bob (it mistakenly used regress_partition_merge_alice and regress_partition_merge_bob before). 2. Permissions on schemas partitions_merge_schema and partition_split_schema are granted to corresponding roles. Before, the lack of permissions led to the creation of objects in the public schema and potential conflict. Reported-by: Daniel Gustafsson Discussion: https://postgr.es/m/03A07EF6-98D2-419B-A3AA-A111C64CC207%40yesql.se Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/2a679ae94e46ea2fa1ff5461c9d0767faecc6c8c Modified Files -- src/test/regress/expected/partition_merge.out | 4 src/test/regress/expected/partition_split.out | 25 +++-- src/test/regress/sql/partition_merge.sql | 4 src/test/regress/sql/partition_split.sql | 25 +++-- 4 files changed, 38 insertions(+), 20 deletions(-)
pgsql: Add permission check for MERGE/SPLIT partition operations
Add permission check for MERGE/SPLIT partition operations Currently, we check only owner permission for the parent table before MERGE/SPLIT partition operations. This leads to a security hole when users can get access to the data of partitions without permission. This commit fixes this problem by requiring owner permission on all the partitions involved. Reported-by: Alexander Lakhin Discussion: https://postgr.es/m/0520c72e-8d97-245e-53f9-173beca2ab2e%40gmail.com Author: Dmitry Koval, Alexander Korotkov Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/3ca43dbbb67fbfb96dec8de2e268b96790555148 Modified Files -- src/backend/parser/parse_utilcmd.c| 5 src/test/regress/expected/partition_merge.out | 29 +++ src/test/regress/expected/partition_split.out | 29 +++ src/test/regress/sql/partition_merge.sql | 33 +++ src/test/regress/sql/partition_split.sql | 33 +++ 5 files changed, 129 insertions(+)
pgsql: Revert: Remove useless self-joins
Revert: Remove useless self-joins This commit reverts d3d55ce5713 and subsequent fixes 2b26a694554, 93c85db3b5b, b44a1708abe, b7f315c9d7d, 8a8ed916f73, b5fb6736ed3, 0a93f803f45, e0477837ce4, a7928a57b9f, 5ef34a8fc38, 30b4955a466, 8c441c08279, 028b15405b4, fe093994db4, 489072ab7a9, and 466979ef031. We are quite late in the release cycle and new bugs continue to appear. Even though we have fixes for all known bugs, there is a risk of throwing many bugs to end users. The plan for self-join elimination would be to do more review and testing, then re-commit in the early v18 cycle. Reported-by: Tom Lane Discussion: https://postgr.es/m/2422119.1714691974%40sss.pgh.pa.us Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/d1d286d83c0eed695910cb20d970ea9bea2e5001 Modified Files -- doc/src/sgml/config.sgml | 16 - src/backend/optimizer/path/indxpath.c | 39 - src/backend/optimizer/plan/analyzejoins.c | 1284 ++--- src/backend/optimizer/plan/planmain.c |5 - src/backend/utils/misc/guc_tables.c | 10 - src/include/nodes/pathnodes.h | 35 +- src/include/optimizer/paths.h |3 - src/include/optimizer/planmain.h |6 - src/test/regress/expected/equivclass.out | 30 - src/test/regress/expected/join.out| 1032 --- src/test/regress/expected/sysviews.out|3 +- src/test/regress/sql/equivclass.sql | 16 - src/test/regress/sql/join.sql | 470 --- src/tools/pgindent/typedefs.list |3 - 14 files changed, 69 insertions(+), 2883 deletions(-)
pgsql: Stabilize regression tests introduced by 259c96fa8f
Stabilize regression tests introduced by 259c96fa8f Add the ORDER BY clause to new queries to avoid ordering ambiguity. Per buildfarm member rorqual. Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/449cdcd486bfc6864e4fa6784cc1526a94fe69db Modified Files -- src/test/regress/expected/partition_merge.out | 5 +++-- src/test/regress/expected/partition_split.out | 3 ++- src/test/regress/sql/partition_merge.sql | 3 ++- src/test/regress/sql/partition_split.sql | 3 ++- 4 files changed, 9 insertions(+), 5 deletions(-)
pgsql: Inherit parent's AM for partition MERGE/SPLIT operations
Inherit parent's AM for partition MERGE/SPLIT operations This commit makes new partitions created by ALTER TABLE ... SPLIT PARTITION and ALTER TABLE ... MERGE PARTITIONS commands inherit the paret table access method. Reported-by: Alexander Lakhin Discussion: https://postgr.es/m/84ada05b-be5c-473e-6d1c-ebe5dd21b190%40gmail.com Reviewed-by: Pavel Borisov Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/259c96fa8f78c0446eba1880850ea6fbe5092120 Modified Files -- doc/src/sgml/ref/alter_table.sgml | 2 ++ src/backend/commands/tablecmds.c | 6 ++ src/test/regress/expected/partition_merge.out | 18 ++ src/test/regress/expected/partition_split.out | 23 +-- src/test/regress/sql/partition_merge.sql | 13 + src/test/regress/sql/partition_split.sql | 14 ++ 6 files changed, 74 insertions(+), 2 deletions(-)
pgsql: Change the way ATExecMergePartitions() handles the name collisio
Change the way ATExecMergePartitions() handles the name collision The name collision happens when the name of the new partition is the same as the name of one of the merging partitions. Currently, ATExecMergePartitions() first gives the new partition a temporary name and then renames it when old partitions are deleted. That negatively influences the naming of related objects like indexes and constrains, which could inherit a temporary name. This commit changes the implementation in the following way. A merging partition gets renamed first, then the new partition is created with the right name immediately. This resolves the issue of the naming of related objects. Reported-by: Alexander Lakhin Discussion: https://postgr.es/m/edfbd846-dcc1-42d1-ac26-715691b687d3%40postgrespro.ru Author: Dmitry Koval, Alexander Korotkov Reviewed-by: Robert Haas, Justin Pryzby, Pavel Borisov Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/885742b9f88b9386368ee94df8c94d154677ffba Modified Files -- src/backend/commands/tablecmds.c | 63 +-- src/test/regress/expected/partition_merge.out | 25 +++ src/test/regress/sql/partition_merge.sql | 18 3 files changed, 73 insertions(+), 33 deletions(-)
pgsql: Add tab completion for partition MERGE/SPLIT operations
Add tab completion for partition MERGE/SPLIT operations This commit implements psql tab completion for ALTER TABLE ... SPLIT PARTITION and ALTER TABLE ... MERGE PARTITIONS commands. Reported-by: Alexander Lakhin Discussion: https://postgr.es/m/5dee3937-8e9f-cca4-11fb-737709a92b37%40gmail.com Author: Dagfinn Ilmari Mannsåker, Pavel Borisov Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/60ae37a8bc02f7a4beef442ee7b4656fba0e4e71 Modified Files -- src/bin/psql/tab-complete.c | 18 -- 1 file changed, 16 insertions(+), 2 deletions(-)
pgsql: Document the way partition MERGE/SPLIT operations create new par
Document the way partition MERGE/SPLIT operations create new partitions Reported-by: Justin Pryzby Discussion: https://postgr.es/m/ZilrByTp-pbz6Mvf%40pryzbyj2023 Reviewed-by: Justin Pryzby Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/842c9b27057e8ecea02b816e3ec6c208779b3d39 Modified Files -- doc/src/sgml/ref/alter_table.sgml | 12 1 file changed, 12 insertions(+)
pgsql: Rename tables in tests of partition MERGE/SPLIT operations
Rename tables in tests of partition MERGE/SPLIT operations Replace "salesman" with "salesperson", "salesmen" with "salespeople". The names are both gramatically correct and gender-neutral. Reported-by: Alexander Lakhin Discussion: https://postgr.es/m/fdaa003e-919c-cbc9-4f0c-e4546e96bd65%40gmail.com Reviewed-by: Robert Haas, Pavel Borisov Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/f4fc7cb54b6a9041f7064de71cb3ee3c4f48e061 Modified Files -- src/test/regress/expected/partition_merge.out | 666 ++--- src/test/regress/expected/partition_split.out | 1266 - src/test/regress/sql/partition_merge.sql | 160 ++-- src/test/regress/sql/partition_split.sql | 280 +++--- 4 files changed, 1186 insertions(+), 1186 deletions(-)
pgsql: Fix error message in check_partition_bounds_for_split_range()
Fix error message in check_partition_bounds_for_split_range() Currently, the error message is produced by a system of complex substitutions making it quite untranslatable and hard to read. This commit splits this into 4 plain error messages suitable for translation. Reported-by: Kyotaro Horiguchi Discussion: https://postgr.es/m/20240408.152402.1485994009160660141.horikyota.ntt%40gmail.com Reviewed-by: Pavel Borisov Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/96c7381c4ceee2ef7c0a7327beb71dec47cf855e Modified Files -- src/backend/partitioning/partbounds.c | 70 --- 1 file changed, 49 insertions(+), 21 deletions(-)
pgsql: Make new partitions with parent's persistence during MERGE/SPLIT
Make new partitions with parent's persistence during MERGE/SPLIT The createPartitionTable() function is responsible for creating new partitions for ALTER TABLE ... MERGE PARTITIONS, and ALTER TABLE ... SPLIT PARTITION commands. It emulates the behaviour of CREATE TABLE ... (LIKE ...), where new table persistence should be specified by the user. In the table partitioning persistent of the partition and its parent must match. So, this commit makes createPartitionTable() copy the persistence of the parent partition. Also, this commit makes createPartitionTable() recheck the persistence after the new table creation. This is needed because persistence might be affected by pg_temp in search_path. This commit also changes the signature of createPartitionTable() making it take the parent's Relation itself instead of the name of the parent relation, and return the Relation of new partition. That doesn't lead to complications, because both callers have the parent table open and need to open the new partition. Reported-by: Alexander Lakhin Discussion: https://postgr.es/m/dbc8b96c-3cf0-d1ee-860d-0e491da20485%40gmail.com Author: Dmitry Koval Reviewed-by: Alexander Korotkov, Robert Haas, Justin Pryzby, Pavel Borisov Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/fcf80c5d5f0f3787e70fca8fd029d2e08a923f91 Modified Files -- doc/src/sgml/ref/alter_table.sgml | 6 ++ src/backend/commands/tablecmds.c | 75 +- src/test/regress/expected/partition_merge.out | 134 ++ src/test/regress/expected/partition_split.out | 75 +- src/test/regress/sql/partition_merge.sql | 88 +++-- src/test/regress/sql/partition_split.sql | 43 - 6 files changed, 364 insertions(+), 57 deletions(-)
Re: pgsql: Fix failure to track role dependencies of pg_init_privs entries.
On Tue, Apr 30, 2024 at 2:26 AM Tom Lane wrote: > Fix failure to track role dependencies of pg_init_privs entries. I just noticed that test_pg_dump checks don't pass for me. And for most of the buildfarm members too. You must be already aware of this, just in case. -- Regards, Alexander Korotkov
pgsql: Refactoring for CommitTransactionCommand()/AbortCurrentTransacti
Refactoring for CommitTransactionCommand()/AbortCurrentTransaction() fefd9a3fed turned tail recursion of CommitTransactionCommand() and AbortCurrentTransaction() into iteration. However, it splits the handling of cases between different functions. This commit puts the handling of all the cases into AbortCurrentTransactionInternal() and CommitTransactionCommandInternal(). Now CommitTransactionCommand() and AbortCurrentTransaction() are just doing the repeated calls of internal functions. Reported-by: Andres Freund Discussion: https://postgr.es/m/20240415224834.w6piwtefskoh32mv%40awork3.anarazel.de Author: Andres Freund Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/40126ac68f2ff96351cd6071350eb2d5cbd50145 Modified Files -- src/backend/access/transam/xact.c | 155 +- 1 file changed, 71 insertions(+), 84 deletions(-)
pgsql: revert: Generalize relation analyze in table AM interface
revert: Generalize relation analyze in table AM interface This commit reverts 27bc1772fc and dd1f6b0c17. Per review by Andres Freund. Discussion: https://postgr.es/m/20240415201057.khoyxbwwxfgzomeo%40awork3.anarazel.de Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/6377e12a5a5278446bb0d215439b4825ef8996d1 Modified Files -- src/backend/access/heap/heapam_handler.c | 29 ++--- src/backend/access/table/tableamapi.c| 2 + src/backend/commands/analyze.c | 55 + src/include/access/heapam.h | 8 --- src/include/access/tableam.h | 101 --- src/include/commands/vacuum.h| 69 - src/include/foreign/fdwapi.h | 8 ++- src/tools/pgindent/typedefs.list | 3 - 8 files changed, 107 insertions(+), 168 deletions(-)
pgsql: Grammar fixes for split/merge partitions code
Grammar fixes for split/merge partitions code The fixes relate to comments, error messages, and corresponding expected output of regression tests. Discussion: https://postgr.es/m/CAMbWs49DDsknxyoycBqiE72VxzL_sYHF6zqL8dSeNehKPJhkKg%40mail.gmail.com Discussion: https://postgr.es/m/86bfd241-a58c-479a-9a72-2c67a02becf8%40postgrespro.ru Discussion: https://postgr.es/m/CAHewXNkGMPU50QG7V6Q60JGFORfo8LfYO1_GCkCa0VWbmB-fEw%40mail.gmail.com Author: Richard Guo, Dmitry Koval, Tender Wang Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/9dfcac8e15acc3b4d4d5bc02383a56ccb07229fe Modified Files -- doc/src/sgml/ddl.sgml | 2 +- doc/src/sgml/ref/alter_table.sgml | 6 +-- src/backend/commands/tablecmds.c | 21 +- src/backend/parser/parse_utilcmd.c| 6 +-- src/backend/partitioning/partbounds.c | 35 src/test/isolation/specs/partition-merge.spec | 2 +- src/test/regress/expected/partition_merge.out | 14 +++ src/test/regress/expected/partition_split.out | 58 +-- src/test/regress/sql/partition_merge.sql | 12 +++--- src/test/regress/sql/partition_split.sql | 48 +++--- 10 files changed, 104 insertions(+), 100 deletions(-)
pgsql: Revert: Implement pg_wal_replay_wait() stored procedure
Revert: Implement pg_wal_replay_wait() stored procedure This commit reverts 06c418e163, e37662f221, bf1e650806, 25f42429e2, ee79928441, and 74eaf66f98 per review by Heikki Linnakangas. Discussion: https://postgr.es/m/b155606b-e744-4218-bda5-29379779da1a%40iki.fi Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/772faafca1b288c4dd66b7150a7831c27b768003 Modified Files -- doc/src/sgml/func.sgml | 113 src/backend/access/transam/xact.c | 6 - src/backend/access/transam/xlog.c | 7 - src/backend/access/transam/xlogrecovery.c | 11 - src/backend/catalog/system_functions.sql| 3 - src/backend/commands/Makefile | 3 +- src/backend/commands/meson.build| 1 - src/backend/commands/waitlsn.c | 337 src/backend/lib/pairingheap.c | 18 +- src/backend/storage/ipc/ipci.c | 3 - src/backend/storage/lmgr/proc.c | 6 - src/backend/utils/activity/wait_event_names.txt | 2 - src/include/catalog/catversion.h| 2 +- src/include/catalog/pg_proc.dat | 5 - src/include/commands/waitlsn.h | 77 -- src/include/lib/pairingheap.h | 3 - src/include/storage/lwlocklist.h| 1 - src/test/recovery/meson.build | 1 - src/test/recovery/t/043_wal_replay_wait.pl | 97 --- src/tools/pgindent/typedefs.list| 2 - 20 files changed, 4 insertions(+), 694 deletions(-)
pgsql: Revert: Let table AM insertion methods control index insertion
Revert: Let table AM insertion methods control index insertion This commit reverts b1484a3f19 per review by Andres Freund. Discussion: https://postgr.es/m/20240410165236.rwyrny7ihi4ddxw4%40awork3.anarazel.de Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/da841aa4dc279bb0053de56121c927ec943edff3 Modified Files -- src/backend/access/heap/heapam.c | 4 +--- src/backend/access/heap/heapam_handler.c | 4 +--- src/backend/access/table/tableam.c | 6 ++ src/backend/catalog/indexing.c | 4 +--- src/backend/commands/copyfrom.c | 13 - src/backend/commands/createas.c | 4 +--- src/backend/commands/matview.c | 4 +--- src/backend/commands/tablecmds.c | 22 +- src/backend/executor/execReplication.c | 6 ++ src/backend/executor/nodeModifyTable.c | 6 ++ src/include/access/heapam.h | 2 +- src/include/access/tableam.h | 25 +++-- 12 files changed, 28 insertions(+), 72 deletions(-)
pgsql: Revert: Allow table AM to store complex data structures in rd_am
Revert: Allow table AM to store complex data structures in rd_amcache This commit reverts 02eb07ea89 per review by Andres Freund. Discussion: https://postgr.es/m/20240410165236.rwyrny7ihi4ddxw4%40awork3.anarazel.de Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/922c4c461d213a422ee7eb6c38e399607539210a Modified Files -- src/backend/access/heap/heapam_handler.c | 1 - src/backend/utils/cache/relcache.c | 11 --- src/include/access/tableam.h | 34 src/include/utils/rel.h | 10 -- 4 files changed, 12 insertions(+), 44 deletions(-)
pgsql: Revert: Allow table AM tuple_insert() method to return the diffe
Revert: Allow table AM tuple_insert() method to return the different slot This commit reverts c35a3fb5e0 per review by Andres Freund. Discussion: https://postgr.es/m/20240410165236.rwyrny7ihi4ddxw4%40awork3.anarazel.de Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/8dd0bb84da7d56a9e41241b26bfbf6b79644d574 Modified Files -- src/backend/access/heap/heapam_handler.c | 4 +--- src/backend/commands/tablecmds.c | 8 src/backend/executor/nodeModifyTable.c | 6 +++--- src/include/access/tableam.h | 21 + 4 files changed, 17 insertions(+), 22 deletions(-)
pgsql: Revert: Allow locking updated tuples in tuple_update() and tuple
Revert: Allow locking updated tuples in tuple_update() and tuple_delete() This commit reverts 87985cc925 and 818861eb57 per review by Andres Freund. Discussion: https://postgr.es/m/20240410165236.rwyrny7ihi4ddxw4%40awork3.anarazel.de Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/193e6d18e553a104d315ff81892d509d89a30fd8 Modified Files -- src/backend/access/heap/heapam.c | 205 +- src/backend/access/heap/heapam_handler.c | 93 ++-- src/backend/access/table/tableam.c | 26 +-- src/backend/commands/trigger.c | 55 +++-- src/backend/executor/execReplication.c | 19 +- src/backend/executor/nodeModifyTable.c | 353 +-- src/include/access/heapam.h | 19 +- src/include/access/tableam.h | 73 ++- src/include/commands/trigger.h | 4 +- 9 files changed, 346 insertions(+), 501 deletions(-)
pgsql: Revert: Custom reloptions for table AM
Revert: Custom reloptions for table AM This commit reverts 9bd99f4c26 and 422041542f per review by Andres Freund. Discussion: https://postgr.es/m/20240410165236.rwyrny7ihi4ddxw4%40awork3.anarazel.de Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/bc1e2092ebb857802a9713d0d3588079e2f0216a Modified Files -- src/backend/access/common/reloptions.c | 149 + src/backend/access/heap/heapam.c | 4 +- src/backend/access/heap/heapam_handler.c | 13 -- src/backend/access/heap/heaptoast.c| 9 +- src/backend/access/heap/hio.c | 4 +- src/backend/access/heap/pruneheap.c| 4 +- src/backend/access/heap/rewriteheap.c | 4 +- src/backend/access/table/tableam.c | 2 +- src/backend/access/table/tableamapi.c | 25 src/backend/commands/createas.c| 13 +- src/backend/commands/tablecmds.c | 63 - src/backend/commands/vacuum.c | 8 +- src/backend/postmaster/autovacuum.c| 13 +- src/backend/tcop/utility.c | 28 +--- src/backend/utils/cache/relcache.c | 54 +--- src/include/access/reloptions.h| 11 +- src/include/access/tableam.h | 50 --- src/include/utils/rel.h| 148 ++-- src/test/modules/Makefile | 1 - src/test/modules/meson.build | 1 - src/test/modules/test_tam_options/.gitignore | 4 - src/test/modules/test_tam_options/Makefile | 23 .../test_tam_options/expected/test_tam_options.out | 36 - src/test/modules/test_tam_options/meson.build | 33 - .../test_tam_options/sql/test_tam_options.sql | 25 .../test_tam_options/test_tam_options--1.0.sql | 12 -- .../modules/test_tam_options/test_tam_options.c| 66 - .../test_tam_options/test_tam_options.control | 4 - src/tools/pgindent/typedefs.list | 4 +- 29 files changed, 161 insertions(+), 650 deletions(-)
pgsql: revert: Transform OR clauses to ANY expression
revert: Transform OR clauses to ANY expression This commit reverts 72bd38cc99 due to implementation and design issues. Reported-by: Tom Lane Discussion: https://postgr.es/m/3604469.1712628736%40sss.pgh.pa.us Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/ff9f72c68f678ded340b431c3e280fe56644a3e7 Modified Files -- doc/src/sgml/config.sgml | 57 src/backend/nodes/queryjumblefuncs.c | 27 -- src/backend/optimizer/prep/prepqual.c | 400 +- src/backend/utils/misc/guc_tables.c | 12 - src/backend/utils/misc/postgresql.conf.sample | 1 - src/include/nodes/queryjumble.h | 1 - src/include/optimizer/optimizer.h | 2 - src/test/regress/expected/create_index.out| 159 -- src/test/regress/expected/join.out| 50 src/test/regress/expected/partition_prune.out | 37 ++- src/test/regress/sql/create_index.sql | 45 --- src/test/regress/sql/join.sql | 11 - src/test/regress/sql/partition_prune.sql | 2 - src/tools/pgindent/typedefs.list | 2 - 14 files changed, 20 insertions(+), 786 deletions(-)
pgsql: Checks for ALTER TABLE ... SPLIT/MERGE PARTITIONS ... commands
Checks for ALTER TABLE ... SPLIT/MERGE PARTITIONS ... commands Check that the target partition actually belongs to the parent table. Reported-by: Alexander Lakhin Discussion: https://postgr.es/m/cd842601-cf1a-9806-f7b7-d2509b93ba61%40gmail.com Author: Dmitry Koval Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/c99ef1811a064a94fb78917f35bb5853059a92b7 Modified Files -- src/backend/commands/tablecmds.c | 12 - src/backend/parser/parse_utilcmd.c| 39 +++ src/test/regress/expected/partition_merge.out | 29 +++- src/test/regress/expected/partition_split.out | 13 + src/test/regress/sql/partition_merge.sql | 24 ++--- src/test/regress/sql/partition_split.sql | 15 +++ 6 files changed, 111 insertions(+), 21 deletions(-)
pgsql: Provide a way block-level table AMs could re-use acquire_sample_
Provide a way block-level table AMs could re-use acquire_sample_rows() While keeping API the same, this commit provides a way for block-level table AMs to re-use existing acquire_sample_rows() by providing custom callbacks for getting the next block and the next tuple. Reported-by: Andres Freund Discussion: https://postgr.es/m/20240407214001.jgpg5q3yv33ve6y3%40awork3.anarazel.de Reviewed-by: Pavel Borisov Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/dd1f6b0c172a643a73d6b71259fa2d10378b39eb Modified Files -- src/backend/access/heap/heapam_handler.c | 12 +++ src/backend/commands/analyze.c | 42 +++- src/include/access/tableam.h | 13 src/include/commands/vacuum.h| 56 ++-- src/tools/pgindent/typedefs.list | 2 ++ 5 files changed, 106 insertions(+), 19 deletions(-)
pgsql: Fix some grammer errors from error messages and codes comments
Fix some grammer errors from error messages and codes comments Discussion: https://postgr.es/m/CAHewXNkGMPU50QG7V6Q60JGFORfo8LfYO1_GCkCa0VWbmB-fEw%40mail.gmail.com Author: Tender Wang Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/df64c81ca9cba7b0fcb0ded5f735e99e676671c1 Modified Files -- src/backend/commands/tablecmds.c | 18 +- src/backend/parser/parse_utilcmd.c| 2 +- src/backend/partitioning/partbounds.c | 10 +++--- src/include/nodes/parsenodes.h| 4 ++-- src/test/regress/expected/partition_split.out | 4 ++-- 5 files changed, 21 insertions(+), 17 deletions(-)
pgsql: Fill CommonRdOptions with default values in extract_autovac_opts
Fill CommonRdOptions with default values in extract_autovac_opts() Reported-by: Thomas Munro Reported-by: Pavel Borisov Discussion: https://postgr.es/m/CA%2BhUKGLZzLR50RBvuqOO3MZ%3DF54ETz-rTp1PDX9uDGP_GqyYqA%40mail.gmail.com Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/422041542f313f23ca66cad26e9b2b99c4d1999a Modified Files -- src/backend/access/common/reloptions.c | 28 src/backend/postmaster/autovacuum.c| 1 + src/backend/utils/cache/relcache.c | 21 + src/include/access/reloptions.h| 1 + 4 files changed, 31 insertions(+), 20 deletions(-)
Re: pgsql: Custom reloptions for table AM
Hi, Thomas! On Mon, Apr 8, 2024 at 12:03 PM Thomas Munro wrote: > I think this is uninitialised memory: I'm already working on this. Will be fixed in 10-15 minutes. -- Regards, Alexander Korotkov
pgsql: Custom reloptions for table AM
Custom reloptions for table AM Let table AM define custom reloptions for its tables. This allows specifying AM-specific parameters by the WITH clause when creating a table. The reloptions, which could be used outside of table AM, are now extracted into the CommonRdOptions data structure. These options could be by decision of table AM directly specified by a user or calculated in some way. The new test module test_tam_options evaluates the ability to set up custom reloptions and calculate fields of CommonRdOptions on their base. The code may use some parts from prior work by Hao Wu. Discussion: https://postgr.es/m/CAPpHfdurb9ycV8udYqM%3Do0sPS66PJ4RCBM1g-bBpvzUfogY0EA%40mail.gmail.com Discussion: https://postgr.es/m/AMUA1wBBBxfc3tKRLLdU64rb.1.1683276279979.Hmail.wuhao%40hashdata.cn Reviewed-by: Reviewed-by: Pavel Borisov, Matthias van de Meent, Jess Davis Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/9bd99f4c26fe37b8ee2f199aa868a0e2fdba4c43 Modified Files -- src/backend/access/common/reloptions.c | 121 - src/backend/access/heap/heapam.c | 4 +- src/backend/access/heap/heapam_handler.c | 13 ++ src/backend/access/heap/heaptoast.c| 9 +- src/backend/access/heap/hio.c | 4 +- src/backend/access/heap/pruneheap.c| 4 +- src/backend/access/heap/rewriteheap.c | 4 +- src/backend/access/table/tableam.c | 2 +- src/backend/access/table/tableamapi.c | 25 src/backend/commands/createas.c| 13 +- src/backend/commands/tablecmds.c | 63 + src/backend/commands/vacuum.c | 8 +- src/backend/postmaster/autovacuum.c| 12 +- src/backend/tcop/utility.c | 28 +++- src/backend/utils/cache/relcache.c | 73 +- src/include/access/reloptions.h| 10 +- src/include/access/tableam.h | 50 +++ src/include/utils/rel.h| 148 +++-- src/test/modules/Makefile | 1 + src/test/modules/meson.build | 1 + src/test/modules/test_tam_options/.gitignore | 4 + src/test/modules/test_tam_options/Makefile | 23 .../test_tam_options/expected/test_tam_options.out | 36 + src/test/modules/test_tam_options/meson.build | 33 + .../test_tam_options/sql/test_tam_options.sql | 25 .../test_tam_options/test_tam_options--1.0.sql | 12 ++ .../modules/test_tam_options/test_tam_options.c| 66 + .../test_tam_options/test_tam_options.control | 4 + src/tools/pgindent/typedefs.list | 4 +- 29 files changed, 639 insertions(+), 161 deletions(-)
Re: pgsql: Transform OR clauses to ANY expression
On Mon, Apr 8, 2024 at 9:24 AM Kyotaro Horiguchi
wrote:
> At Mon, 08 Apr 2024 14:46:57 +0900 (JST), Kyotaro Horiguchi
> wrote in
> > At Sun, 07 Apr 2024 22:28:06 +0000, Alexander Korotkov
> > wrote in
> > > Transform OR clauses to ANY expression
> >
> > This commit introduces a message like this:
> >
> > > gettext_noop("Set the minimum length of the list of OR clauses to attempt
> > > the OR-to-ANY transformation."),
> >
> > Unlike the usual phrasing of similar messages in this context, which
> > use the form "Sets the minimum length of...", this message uses an
> > imperative form ("Set"). I believe it would be better to align the
> > tone of this message with the others by changing "Set" to "Sets".
> >
> > regards.
> >
> >
> > diff --git a/src/backend/utils/misc/guc_tables.c
> > b/src/backend/utils/misc/guc_tables.c
> > index 83e3a59d7e..a527ffe0b0 100644
> > --- a/src/backend/utils/misc/guc_tables.c
> > +++ b/src/backend/utils/misc/guc_tables.c
> > @@ -3670,7 +3670,7 @@ struct config_int ConfigureNamesInt[] =
> >
> > {
> > {"or_to_any_transform_limit", PGC_USERSET, QUERY_TUNING_OTHER,
> > - gettext_noop("Set the minimum length of the list of
> > OR clauses to attempt the OR-to-ANY transformation."),
> > + gettext_noop("Sets the minimum length of the list of
> > OR clauses to attempt the OR-to-ANY transformation."),
> > gettext_noop("Once the limit is reached, the planner
> > will try to replace expression like "
> >"'x=c1 OR x=c2 ..' to the
> > expression 'x = ANY(ARRAY[c1,c2,..])'"),
> > GUC_EXPLAIN
>
> Sorry for the sequential posts, but I found the following contruct in
> the same patch to be quite untranslatable.
No worries. But these are distinct patches.
> > errmsg("%s bound of partition \"%s\" is %s %s bound of split partition",
> > first ? "lower" : "upper",
> > relname,
> > defaultPart ? (first ? "less than" : "greater than") : "not
> > equals to",
> > first ? "lower" : "upper"),
> > parser_errposition(pstate, datum->location)));
>
> I might be misunderstanding this, but if the expressions are correct,
> the message could be divided into four fixed messages based on "first"
> and "defaultPart". Alternatively, it might be better to provide
> simpler explation of the situation.
Yes, splitting this into multiple messages helps both translating and
readability. Will be fixed in the next couple of days.
--
Regards,
Alexander Korotkov
pgsql: Fix the wording of or_to_any_transform_limit description
Fix the wording of or_to_any_transform_limit description Reported-by: Kyotaro Horiguchi Discussion: https://postgr.es/m/20240408.144657.1746688590065601661.horikyota.ntt%40gmail.com Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/b453a7a16a7ed2ba96522e521143bc652b74875f Modified Files -- src/backend/utils/misc/guc_tables.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
pgsql: Fix the value of or_to_any_transform_limit in postgresql.conf.sa
Fix the value of or_to_any_transform_limit in postgresql.conf.sample Reported-by: Justin Pryzby Discussion: https://postgr.es/m/ZhM8jH8gsKm5Q-9p%40pryzbyj2023 Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/2af75e1174786a02fc35755c0cb98c522d72a065 Modified Files -- src/backend/utils/misc/postgresql.conf.sample | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
pgsql: Fix usage of same ListCell transform_or_to_any()'s in nested loo
Fix usage of same ListCell transform_or_to_any()'s in nested loops Discussion: https://postgr.es/m/CAAKRu_b4SXNW4GAM0bv3e6wcL5ODSXg1ZdRCn6uyLLjSPbveBg%40mail.gmail.com Author: Melanie Plageman Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/beabea6c2063e583628c59d03102dba996975b4a Modified Files -- src/backend/optimizer/prep/prepqual.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-)
Re: pgsql: Transform OR clauses to ANY expression
On Mon, Apr 8, 2024 at 1:35 AM Melanie Plageman wrote: > On Sun, Apr 7, 2024 at 6:28 PM Alexander Korotkov > wrote: > > > > Transform OR clauses to ANY expression > > > > Replace (expr op C1) OR (expr op C2) ... with expr op ANY(ARRAY[C1, C2, > > ...]) > > on the preliminary stage of optimization when we are still working with the > > expression tree. > > > > Here Cn is a n-th constant expression, 'expr' is non-constant expression, > > 'op' > > is an operator which returns boolean result and has a commuter (for the case > > of reverse order of constant and non-constant parts of the expression, > > like 'Cn op expr'). > > > > Sometimes it can lead to not optimal plan. This is why there is a > > or_to_any_transform_limit GUC. It specifies a threshold value of length of > > arguments in an OR expression that triggers the OR-to-ANY transformation. > > Generally, more groupable OR arguments mean that transformation will be more > > likely to win than to lose. > > I'm getting this warning now > > /src/backend/optimizer/prep/prepqual.c:582:33: warning: declaration of > ‘lc__state’ shadows a previous local [-Wshadow=compatible-local] > 582 | foreach(lc, entry->consts) Thank you for catching. I'm fixing this now. -- Regards, Alexander Korotkov
pgsql: Transform OR clauses to ANY expression
Transform OR clauses to ANY expression Replace (expr op C1) OR (expr op C2) ... with expr op ANY(ARRAY[C1, C2, ...]) on the preliminary stage of optimization when we are still working with the expression tree. Here Cn is a n-th constant expression, 'expr' is non-constant expression, 'op' is an operator which returns boolean result and has a commuter (for the case of reverse order of constant and non-constant parts of the expression, like 'Cn op expr'). Sometimes it can lead to not optimal plan. This is why there is a or_to_any_transform_limit GUC. It specifies a threshold value of length of arguments in an OR expression that triggers the OR-to-ANY transformation. Generally, more groupable OR arguments mean that transformation will be more likely to win than to lose. Discussion: https://postgr.es/m/567ED6CA.2040504%40sigaev.ru Author: Alena Rybakina Author: Andrey Lepikhov Reviewed-by: Peter Geoghegan Reviewed-by: Ranier Vilela Reviewed-by: Alexander Korotkov Reviewed-by: Robert Haas Reviewed-by: Jian He Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/72bd38cc99a15da6f97373fae98027c908c398ea Modified Files -- doc/src/sgml/config.sgml | 57 src/backend/nodes/queryjumblefuncs.c | 27 ++ src/backend/optimizer/prep/prepqual.c | 399 +- src/backend/utils/misc/guc_tables.c | 12 + src/backend/utils/misc/postgresql.conf.sample | 1 + src/include/nodes/queryjumble.h | 1 + src/include/optimizer/optimizer.h | 2 + src/test/regress/expected/create_index.out| 159 ++ src/test/regress/expected/join.out| 50 src/test/regress/expected/partition_prune.out | 37 +-- src/test/regress/sql/create_index.sql | 45 +++ src/test/regress/sql/join.sql | 11 + src/test/regress/sql/partition_prune.sql | 2 + src/tools/pgindent/typedefs.list | 2 + 14 files changed, 785 insertions(+), 20 deletions(-)
pgsql: Implement ALTER TABLE ... MERGE PARTITIONS ... command
Implement ALTER TABLE ... MERGE PARTITIONS ... command This new DDL command merges several partitions into the one partition of the target table. The target partition is created using new createPartitionTable() function with parent partition as the template. This commit comprises quite naive implementation which works in single process and holds the ACCESS EXCLUSIVE LOCK on the parent table during all the operations including the tuple routing. This is why this new DDL command can't be recommended for large partitioned tables under a high load. However, this implementation come in handy in certain cases even as is. Also, it could be used as a foundation for future implementations with lesser locking and possibly parallel. Discussion: https://postgr.es/m/c73a1746-0cd0-6bdd-6b23-3ae0b7c0c582%40postgrespro.ru Author: Dmitry Koval Reviewed-by: Matthias van de Meent, Laurenz Albe, Zhihong Yu, Justin Pryzby Reviewed-by: Alvaro Herrera, Robert Haas, Stephane Tachoires Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/1adf16b8fba45f77056d91573cd7138ed9da4ebf Modified Files -- doc/src/sgml/ddl.sgml | 19 + doc/src/sgml/ref/alter_table.sgml | 77 ++- src/backend/commands/tablecmds.c | 354 +- src/backend/parser/gram.y | 22 +- src/backend/parser/parse_utilcmd.c | 89 +++ src/backend/partitioning/partbounds.c | 207 ++ src/include/nodes/parsenodes.h | 14 + src/include/parser/kwlist.h| 1 + src/include/partitioning/partbounds.h | 6 + src/test/isolation/expected/partition-merge.out| 199 ++ src/test/isolation/isolation_schedule | 1 + src/test/isolation/specs/partition-merge.spec | 54 ++ .../modules/test_ddl_deparse/test_ddl_deparse.c| 3 + src/test/regress/expected/partition_merge.out | 732 + src/test/regress/parallel_schedule | 2 +- src/test/regress/sql/partition_merge.sql | 430 src/tools/pgindent/typedefs.list | 1 + 17 files changed, 2189 insertions(+), 22 deletions(-)
pgsql: Implement ALTER TABLE ... SPLIT PARTITION ... command
Implement ALTER TABLE ... SPLIT PARTITION ... command This new DDL command splits a single partition into several parititions. Just like ALTER TABLE ... MERGE PARTITIONS ... command, new patitions are created using createPartitionTable() function with parent partition as the template. This commit comprises quite naive implementation which works in single process and holds the ACCESS EXCLUSIVE LOCK on the parent table during all the operations including the tuple routing. This is why this new DDL command can't be recommended for large partitioned tables under a high load. However, this implementation come in handy in certain cases even as is. Also, it could be used as a foundation for future implementations with lesser locking and possibly parallel. Discussion: https://postgr.es/m/c73a1746-0cd0-6bdd-6b23-3ae0b7c0c582%40postgrespro.ru Author: Dmitry Koval Reviewed-by: Matthias van de Meent, Laurenz Albe, Zhihong Yu, Justin Pryzby Reviewed-by: Alvaro Herrera, Robert Haas, Stephane Tachoires Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/87c21bb9412c8ba2727dec5ebcd74d44c2232d11 Modified Files -- doc/src/sgml/ddl.sgml | 19 + doc/src/sgml/ref/alter_table.sgml | 65 +- src/backend/commands/tablecmds.c | 411 ++ src/backend/parser/gram.y | 38 +- src/backend/parser/parse_utilcmd.c | 62 +- src/backend/partitioning/partbounds.c | 657 + src/backend/utils/adt/ruleutils.c | 18 + src/include/nodes/parsenodes.h |1 + src/include/parser/kwlist.h|1 + src/include/partitioning/partbounds.h |5 + src/include/utils/ruleutils.h |2 + src/test/isolation/expected/partition-split.out| 190 +++ src/test/isolation/isolation_schedule |1 + src/test/isolation/specs/partition-split.spec | 54 + .../modules/test_ddl_deparse/test_ddl_deparse.c|3 + src/test/regress/expected/partition_split.out | 1417 src/test/regress/parallel_schedule |2 +- src/test/regress/sql/partition_split.sql | 833 src/tools/pgindent/typedefs.list |1 + 19 files changed, 3766 insertions(+), 14 deletions(-)
pgsql: Use an LWLock instead of a spinlock in waitlsn.c
Use an LWLock instead of a spinlock in waitlsn.c This should prevent busy-waiting when number of waiting processes is high. Discussion: https://postgr.es/m/202404030658.hhj3vfxeyhft%40alvherre.pgsql Author: Alvaro Herrera Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/25f42429e2ff2acca35c9154fc2e36b75c79227a Modified Files -- src/backend/commands/waitlsn.c | 15 +++ src/backend/utils/activity/wait_event_names.txt | 1 + src/include/commands/waitlsn.h | 5 + src/include/storage/lwlocklist.h| 1 + 4 files changed, 10 insertions(+), 12 deletions(-)
pgsql: Call WaitLSNCleanup() in AbortTransaction()
Call WaitLSNCleanup() in AbortTransaction() Even though waiting for replay LSN happens without explicit transaction, AbortTransaction() is responsible for the cleanup of the shared memory if the error is thrown in a stored procedure. So, we need to do WaitLSNCleanup() there to clean up after some unexpected error happened while waiting for replay LSN. Discussion: https://postgr.es/m/202404051815.eri4u5q6oj26%40alvherre.pgsql Author: Alvaro Herrera Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/74eaf66f988c868deb0816bae4dd184eedae1448 Modified Files -- src/backend/access/transam/xact.c | 6 ++ 1 file changed, 6 insertions(+)
pgsql: Clarify what is protected by WaitLSNLock
Clarify what is protected by WaitLSNLock Not just WaitLSNState.waitersHeap, but also WaitLSNState.procInfos and updating of WaitLSNState.minWaitedLSN is protected by WaitLSNLock. There is one now documented exclusion on fast-path checking of WaitLSNProcInfo.inHeap flag. Discussion: https://postgr.es/m/202404030658.hhj3vfxeyhft%40alvherre.pgsql Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/ee79928441e7e291532b833455ebfee27d7cab5c Modified Files -- src/backend/commands/waitlsn.c | 10 -- src/include/commands/waitlsn.h | 7 +-- 2 files changed, 13 insertions(+), 4 deletions(-)
pgsql: Fix the parameters order for TableAmRoutine.relation_copy_for_cl
Fix the parameters order for TableAmRoutine.relation_copy_for_cluster() Specify OldTable first, NewTable second as used by table_relation_copy_for_cluster() and as implemented in heapam_relation_copy_for_cluster(). Backpatch to PostgreSQL 12, where TableAmRoutine was introduced. Discussion: https://postgr.es/m/ME3P282MB3166860D4911AE82F92DF7C5B63F2%40ME3P282MB3166.AUSP282.PROD.OUTLOOK.COM Author: Japin Li Reviewed-by: Pavel Borisov Backpatch-through: 12 Branch -- REL_16_STABLE Details --- https://git.postgresql.org/pg/commitdiff/bafad43c37bf531f392e65ba726c253040b608f2 Modified Files -- src/include/access/tableam.h | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
pgsql: Fix the parameters order for TableAmRoutine.relation_copy_for_cl
Fix the parameters order for TableAmRoutine.relation_copy_for_cluster() Specify OldTable first, NewTable second as used by table_relation_copy_for_cluster() and as implemented in heapam_relation_copy_for_cluster(). Backpatch to PostgreSQL 12, where TableAmRoutine was introduced. Discussion: https://postgr.es/m/ME3P282MB3166860D4911AE82F92DF7C5B63F2%40ME3P282MB3166.AUSP282.PROD.OUTLOOK.COM Author: Japin Li Reviewed-by: Pavel Borisov Backpatch-through: 12 Branch -- REL_15_STABLE Details --- https://git.postgresql.org/pg/commitdiff/c2faf48fa3b262f57cb999a3ab1e00e8d46176cd Modified Files -- src/include/access/tableam.h | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
pgsql: Fix the parameters order for TableAmRoutine.relation_copy_for_cl
Fix the parameters order for TableAmRoutine.relation_copy_for_cluster() Specify OldTable first, NewTable second as used by table_relation_copy_for_cluster() and as implemented in heapam_relation_copy_for_cluster(). Backpatch to PostgreSQL 12, where TableAmRoutine was introduced. Discussion: https://postgr.es/m/ME3P282MB3166860D4911AE82F92DF7C5B63F2%40ME3P282MB3166.AUSP282.PROD.OUTLOOK.COM Author: Japin Li Reviewed-by: Pavel Borisov Backpatch-through: 12 Branch -- REL_14_STABLE Details --- https://git.postgresql.org/pg/commitdiff/19b8481b42efd29ac265e0d14566a7609b8d474b Modified Files -- src/include/access/tableam.h | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
pgsql: Fix the parameters order for TableAmRoutine.relation_copy_for_cl
Fix the parameters order for TableAmRoutine.relation_copy_for_cluster() Specify OldTable first, NewTable second as used by table_relation_copy_for_cluster() and as implemented in heapam_relation_copy_for_cluster(). Backpatch to PostgreSQL 12, where TableAmRoutine was introduced. Discussion: https://postgr.es/m/ME3P282MB3166860D4911AE82F92DF7C5B63F2%40ME3P282MB3166.AUSP282.PROD.OUTLOOK.COM Author: Japin Li Reviewed-by: Pavel Borisov Backpatch-through: 12 Branch -- REL_13_STABLE Details --- https://git.postgresql.org/pg/commitdiff/68044d15b737fe567c9c2e00a1c8466530ed02ce Modified Files -- src/include/access/tableam.h | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
pgsql: Fix the parameters order for TableAmRoutine.relation_copy_for_cl
Fix the parameters order for TableAmRoutine.relation_copy_for_cluster() Specify OldTable first, NewTable second as used by table_relation_copy_for_cluster() and as implemented in heapam_relation_copy_for_cluster(). Backpatch to PostgreSQL 12, where TableAmRoutine was introduced. Discussion: https://postgr.es/m/ME3P282MB3166860D4911AE82F92DF7C5B63F2%40ME3P282MB3166.AUSP282.PROD.OUTLOOK.COM Author: Japin Li Reviewed-by: Pavel Borisov Backpatch-through: 12 Branch -- REL_12_STABLE Details --- https://git.postgresql.org/pg/commitdiff/38585549a3ef96fbbd8a1e7c86dc1b4b8ee5faa8 Modified Files -- src/include/access/tableam.h | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
pgsql: Fix the parameters order for TableAmRoutine.relation_copy_for_cl
Fix the parameters order for TableAmRoutine.relation_copy_for_cluster() Specify OldTable first, NewTable second as used by table_relation_copy_for_cluster() and as implemented in heapam_relation_copy_for_cluster(). Backpatch to PostgreSQL 12, where TableAmRoutine was introduced. Discussion: https://postgr.es/m/ME3P282MB3166860D4911AE82F92DF7C5B63F2%40ME3P282MB3166.AUSP282.PROD.OUTLOOK.COM Author: Japin Li Reviewed-by: Pavel Borisov Backpatch-through: 12 Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/97ce821e3e171ce99fa7c398889ac08432cd0264 Modified Files -- src/include/access/tableam.h | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
pgsql: Use the pairing heap instead of a flat array for LSN replay wait
Use the pairing heap instead of a flat array for LSN replay waiters 06c418e163 introduced pg_wal_replay_wait() procedure allowing to wait for the particular LSN to be replayed on standby. The waiters were stored in the flat array. Even though scanning small arrays is fast, that might be a problem at scale (a lot of waiting processes). This commit replaces the flat shared memory array with the pairing heap, which holds the waiter with the least LSN at the top. This gives us O(log N) complexity for both inserting and removing waiters. Reported-by: Alvaro Herrera Discussion: https://postgr.es/m/202404030658.hhj3vfxeyhft%40alvherre.pgsql Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/bf1e65080629e2b0ac47ffe245576da96eff8420 Modified Files -- src/backend/access/transam/xlogrecovery.c | 2 +- src/backend/commands/waitlsn.c| 168 ++ src/backend/lib/pairingheap.c | 18 +++- src/include/commands/waitlsn.h| 44 +++- src/include/lib/pairingheap.h | 3 + 5 files changed, 140 insertions(+), 95 deletions(-)
pgsql: Minor improvements for waitlsn.c
Minor improvements for waitlsn.c * Remove extra includes * Fill 'cur' in addLSNWaiter() before taking the spinlock * Initialize 'endtime' with zero in WaitForLSN() to avoid compiler warning Reported-by: Alvaro Herrera, Masahiko Sawada, Daniel Gustafsson Discussion: https://postgr.es/m/202404030658.hhj3vfxeyhft%40alvherre.pgsql Discussion: https://postgr.es/m/CAD21AoAx7irptnPH1OkkkNh9E0M6X-phfX7sYZfwoMsc1qV1sQ%40mail.gmail.com Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/e37662f22158c29bc55eda4eda1757f444cf701a Modified Files -- src/backend/commands/waitlsn.c | 18 -- 1 file changed, 4 insertions(+), 14 deletions(-)
pgsql: Move WaitLSNShmemInit() to CreateOrAttachShmemStructs()
Move WaitLSNShmemInit() to CreateOrAttachShmemStructs() Thanks to Andres Freund, Thomas Munrom and David Rowley for investigating this issue. Discussion: https://postgr.es/m/CAPpHfdvap5mMLikt8CUjA0osAvCJHT0qnYeR3f84EJ_Kvse0mg%40mail.gmail.com Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/2c91e13013414cf77bb8026a19a926e08f4e9e7a Modified Files -- src/backend/storage/ipc/ipci.c | 6 +- 1 file changed, 1 insertion(+), 5 deletions(-)
Re: pgsql: Allow SIGINT to cancel psql database reconnections.
On Tue, Apr 2, 2024 at 5:30 PM Robert Haas wrote: > Allow SIGINT to cancel psql database reconnections. > > After installing the SIGINT handler in psql, SIGINT can no longer cancel > database reconnections. For instance, if the user starts a reconnection > and then needs to do some form of interaction (ie psql is polling), > there is no way to cancel the reconnection process currently. > > Use PQconnectStartParams() in order to insert a cancel_pressed check > into the polling loop. > > Tristan Partin, reviewed by Gurjeet Singh, Heikki Linnakangas, Jelte > Fennema-Nio, and me. > > Discussion: http://postgr.es/m/d08wwcpvhkhn.3qelikzj2d...@neon.tech This appears to have missing pgindent. https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=koel=2024-04-02%2021%3A19%3A02 -- Regards, Alexander Korotkov
Re: pgsql: Implement pg_wal_replay_wait() stored procedure
On Tue, Apr 2, 2024 at 10:58 PM Alexander Korotkov wrote: > Implement pg_wal_replay_wait() stored procedure I'm trying to figure out if this failure could be related to this commit... https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=culicidae=2024-04-02%2020%3A24%3A55 -- Regards, Alexander Korotkov
pgsql: Implement pg_wal_replay_wait() stored procedure
Implement pg_wal_replay_wait() stored procedure pg_wal_replay_wait() is to be used on standby and specifies waiting for the specific WAL location to be replayed before starting the transaction. This option is useful when the user makes some data changes on primary and needs a guarantee to see these changes on standby. The queue of waiters is stored in the shared memory array sorted by LSN. During replay of WAL waiters whose LSNs are already replayed are deleted from the shared memory array and woken up by setting of their latches. pg_wal_replay_wait() needs to wait without any snapshot held. Otherwise, the snapshot could prevent the replay of WAL records implying a kind of self-deadlock. This is why it is only possible to implement pg_wal_replay_wait() as a procedure working in a non-atomic context, not a function. Catversion is bumped. Discussion: https://postgr.es/m/eb12f9b03851bb2583adab5df9579b4b%40postgrespro.ru Author: Kartyshov Ivan, Alexander Korotkov Reviewed-by: Michael Paquier, Peter Eisentraut, Dilip Kumar, Amit Kapila Reviewed-by: Alexander Lakhin, Bharath Rupireddy, Euler Taveira Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/06c418e163e913966e17cb2d3fb1c5f8a8d58308 Modified Files -- doc/src/sgml/func.sgml | 113 src/backend/access/transam/xlog.c | 7 + src/backend/access/transam/xlogrecovery.c | 11 + src/backend/catalog/system_functions.sql| 3 + src/backend/commands/Makefile | 3 +- src/backend/commands/meson.build| 1 + src/backend/commands/waitlsn.c | 348 src/backend/storage/ipc/ipci.c | 7 + src/backend/storage/lmgr/proc.c | 6 + src/backend/utils/activity/wait_event_names.txt | 1 + src/include/catalog/catversion.h| 2 +- src/include/catalog/pg_proc.dat | 5 + src/include/commands/waitlsn.h | 43 +++ src/test/recovery/meson.build | 1 + src/test/recovery/t/043_wal_replay_wait.pl | 97 +++ src/tools/pgindent/typedefs.list| 2 + 16 files changed, 648 insertions(+), 2 deletions(-)
pgsql: Revert "Custom reloptions for table AM"
Revert "Custom reloptions for table AM" This reverts commit c95c25f9af4bc77f2f66a587735c50da08c12b37 due to multiple design issues spotted after commit. Reported-by: Jeff Davis Discussion: https://postgr.es/m/11550b536211d5748bb2865ed6cb3502ff073bf7.camel%40j-davis.com Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/867cc7b6ddb9d998103688a56048fe9a1ddd972a Modified Files -- src/backend/access/common/reloptions.c | 6 ++-- src/backend/access/heap/heapam_handler.c | 12 --- src/backend/access/table/tableamapi.c| 25 --- src/backend/commands/tablecmds.c | 55 +--- src/backend/postmaster/autovacuum.c | 4 +-- src/backend/utils/cache/relcache.c | 6 +--- src/include/access/reloptions.h | 2 -- src/include/access/tableam.h | 43 - 8 files changed, 27 insertions(+), 126 deletions(-)
pgsql: Custom reloptions for table AM
Custom reloptions for table AM Let table AM define custom reloptions for its tables. This allows to specify AM-specific parameters by WITH clause when creating a table. The code may use some parts from prior work by Hao Wu. Discussion: https://postgr.es/m/CAPpHfdurb9ycV8udYqM%3Do0sPS66PJ4RCBM1g-bBpvzUfogY0EA%40mail.gmail.com Discussion: https://postgr.es/m/AMUA1wBBBxfc3tKRLLdU64rb.1.1683276279979.Hmail.wuhao%40hashdata.cn Reviewed-by: Reviewed-by: Pavel Borisov, Matthias van de Meent Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/c95c25f9af4bc77f2f66a587735c50da08c12b37 Modified Files -- src/backend/access/common/reloptions.c | 6 ++-- src/backend/access/heap/heapam_handler.c | 12 +++ src/backend/access/table/tableamapi.c| 25 +++ src/backend/commands/tablecmds.c | 55 +++- src/backend/postmaster/autovacuum.c | 4 ++- src/backend/utils/cache/relcache.c | 6 +++- src/include/access/reloptions.h | 2 ++ src/include/access/tableam.h | 43 + 8 files changed, 126 insertions(+), 27 deletions(-)
pgsql: Generalize relation analyze in table AM interface
Generalize relation analyze in table AM interface Currently, there is just one algorithm for sampling tuples from a table written in acquire_sample_rows(). Custom table AM can just redefine the way to get the next block/tuple by implementing scan_analyze_next_block() and scan_analyze_next_tuple() API functions. This approach doesn't seem general enough. For instance, it's unclear how to sample this way index-organized tables. This commit allows table AM to encapsulate the whole sampling algorithm (currently implemented in acquire_sample_rows()) into the relation_analyze() API function. Discussion: https://postgr.es/m/CAPpHfdurb9ycV8udYqM%3Do0sPS66PJ4RCBM1g-bBpvzUfogY0EA%40mail.gmail.com Reviewed-by: Pavel Borisov, Matthias van de Meent Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/27bc1772fc814946918a5ac8ccb9b5c5ad0380aa Modified Files -- src/backend/access/heap/heapam_handler.c | 29 ++--- src/backend/access/table/tableamapi.c| 2 - src/backend/commands/analyze.c | 54 src/include/access/heapam.h | 9 +++ src/include/access/tableam.h | 106 +++ src/include/commands/vacuum.h| 19 ++ src/include/foreign/fdwapi.h | 6 +- 7 files changed, 100 insertions(+), 125 deletions(-)
pgsql: Let table AM insertion methods control index insertion
Let table AM insertion methods control index insertion Previously, the executor did index insert unconditionally after calling table AM interface methods tuple_insert() and multi_insert(). This commit introduces the new parameter insert_indexes for these two methods. Setting '*insert_indexes' to true saves the current logic. Setting it to false indicates that table AM cares about index inserts itself and doesn't want the caller to do that. Discussion: https://postgr.es/m/CAPpHfdurb9ycV8udYqM%3Do0sPS66PJ4RCBM1g-bBpvzUfogY0EA%40mail.gmail.com Reviewed-by: Pavel Borisov, Matthias van de Meent, Mark Dilger Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/b1484a3f1910bfd0e254afe40085dfc3351bda8c Modified Files -- src/backend/access/heap/heapam.c | 4 +++- src/backend/access/heap/heapam_handler.c | 4 +++- src/backend/access/table/tableam.c | 6 -- src/backend/catalog/indexing.c | 4 +++- src/backend/commands/copyfrom.c | 13 + src/backend/commands/createas.c | 4 +++- src/backend/commands/matview.c | 4 +++- src/backend/commands/tablecmds.c | 6 +- src/backend/executor/execReplication.c | 6 -- src/backend/executor/nodeModifyTable.c | 6 -- src/include/access/heapam.h | 2 +- src/include/access/tableam.h | 25 ++--- 12 files changed, 60 insertions(+), 24 deletions(-)
pgsql: Fix some typos and grammar issues from commit 87985cc92522
Fix some typos and grammar issues from commit 87985cc92522 Reported-by: Alexander Lakhin Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/818861eb578663a0d4d8d7dc4e18c96a148b3c75 Modified Files -- src/backend/access/heap/heapam.c | 2 +- src/backend/commands/trigger.c | 4 ++-- 2 files changed, 3 insertions(+), 3 deletions(-)
pgsql: Add comments on some MinimalTupleSlots methods usage
Add comments on some MinimalTupleSlots methods usage Discussion: https://postgr.es/m/CALT9ZEHNeagO5PLb4Nv9J_ZaCtp%2BArdVmbSLc0RHUzx_RPAa4w%40mail.gmail.com Author: Pavel Borisov Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/10baee0c95d15f70c0c2b0e52640651777ce806d Modified Files -- src/backend/executor/execTuples.c | 9 + 1 file changed, 9 insertions(+)
pgsql: Improve error message for tts_(virtual|minimal)_is_current_xact_
Improve error message for tts_(virtual|minimal)_is_current_xact_tuple Discussion: https://postgr.es/m/CALT9ZEHNeagO5PLb4Nv9J_ZaCtp%2BArdVmbSLc0RHUzx_RPAa4w%40mail.gmail.com Author: Pavel Borisov Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/41d3780d3d29acd7e0a7a6922f2757243e9186d0 Modified Files -- src/backend/executor/execTuples.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
pgsql: Allow locking updated tuples in tuple_update() and tuple_delete(
Allow locking updated tuples in tuple_update() and tuple_delete() Currently, in read committed transaction isolation mode (default), we have the following sequence of actions when tuple_update()/tuple_delete() finds the tuple updated by the concurrent transaction. 1. Attempt to update/delete tuple with tuple_update()/tuple_delete(), which returns TM_Updated. 2. Lock tuple with tuple_lock(). 3. Re-evaluate plan qual (recheck if we still need to update/delete and calculate the new tuple for update). 4. Second attempt to update/delete tuple with tuple_update()/tuple_delete(). This attempt should be successful, since the tuple was previously locked. This commit eliminates step 2 by taking the lock during the first tuple_update()/tuple_delete() call. The heap table access method saves some effort by checking the updated tuple once instead of twice. Future undo-based table access methods, which will start from the latest row version, can immediately place a lock there. Also, this commit makes tuple_update()/tuple_delete() optionally save the old tuple into the dedicated slot. That saves efforts on re-fetching tuples in certain cases. The code in nodeModifyTable.c is simplified by removing the nested switch/case. Discussion: https://postgr.es/m/CAPpHfdua-YFw3XTprfutzGp28xXLigFtzNbuFY8yPhqeq6X5kg%40mail.gmail.com Reviewed-by: Aleksander Alekseev, Pavel Borisov, Vignesh C, Mason Sharp Reviewed-by: Andres Freund, Chris Travers Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/87985cc9252296f11db3cacc155e8d710d2e9b9c Modified Files -- src/backend/access/heap/heapam.c | 205 +- src/backend/access/heap/heapam_handler.c | 94 ++-- src/backend/access/table/tableam.c | 26 ++- src/backend/commands/trigger.c | 55 ++--- src/backend/executor/execReplication.c | 19 +- src/backend/executor/nodeModifyTable.c | 353 ++- src/include/access/heapam.h | 19 +- src/include/access/tableam.h | 73 +-- src/include/commands/trigger.h | 4 +- 9 files changed, 502 insertions(+), 346 deletions(-)
pgsql: Add EvalPlanQual delete returning isolation test
Add EvalPlanQual delete returning isolation test Author: Andres Freund Reviewed-by: Pavel Borisov Discussion: https://www.postgresql.org/message-id/flat/CAPpHfdua-YFw3XTprfutzGp28xXLigFtzNbuFY8yPhqeq6X5kg%40mail.gmail.com Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/8ffc2aa720a2bdef33b16609ed4f8ed07932c141 Modified Files -- src/test/isolation/expected/eval-plan-qual.out | 30 ++ src/test/isolation/specs/eval-plan-qual.spec | 4 2 files changed, 34 insertions(+)
pgsql: reindexdb: Fix warning about uninitialized indices_tables_cell
reindexdb: Fix warning about uninitialized indices_tables_cell Initialize indices_tables_cell with NULL to silence the warning. Also, refactor the place of the first assignment of indices_tables_cell. Reported-by: Thomas Munro, David Rowley, Tom Lane, Richard Guo Discussion: https://postgr.es/m/2348025.1711332418%40sss.pgh.pa.us Discussion: https://postgr.es/m/E1roXs4-005UdX-1V%40gemulon.postgresql.org Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/cc0e7ebd304a24815c0531000a4c3693878cd96c Modified Files -- src/bin/scripts/reindexdb.c | 20 ++-- 1 file changed, 14 insertions(+), 6 deletions(-)
pgsql: reindexdb: Add the index-level REINDEX with multiple jobs
reindexdb: Add the index-level REINDEX with multiple jobs Straight-forward index-level REINDEX is not supported with multiple jobs as we cannot control the concurrent processing of multiple indexes depending on the same relation. Instead, we dedicate the whole table to certain reindex job. Thus, if indexes in the lists belong to different tables, that gives us a fair level of parallelism. This commit teaches get_parallel_object_list() to fetch table names for indexes in the case of index-level REINDEX. The same tables are grouped together in the output order, and the list of indexes is also rebuilt to match that order. Later during processingof that list, we push indexes belonging to the same table into the same job. Discussion: https://postgr.es/m/CACG%3DezZU_VwDi-1PN8RUSE6mcYG%2BYx1NH_rJO4%2BKe-mKqLp%3DNw%40mail.gmail.com Author: Maxim Orlov, Svetlana Derevyanko, Alexander Korotkov Reviewed-by: Michael Paquier Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/47f99a407de224df6f9c43697d0a9c0a5598b250 Modified Files -- doc/src/sgml/ref/reindexdb.sgml| 3 +- src/bin/scripts/reindexdb.c| 146 +++-- src/bin/scripts/t/090_reindexdb.pl | 8 +- 3 files changed, 130 insertions(+), 27 deletions(-)
pgsql: amcheck: Support for different header sizes of short varlena dat
amcheck: Support for different header sizes of short varlena datum In the heap, tuples may contain short varlena datum with both 1B header and 4B headers. But the corresponding index tuple should always have such varlena's with 1B headers. So, for fingerprinting, we need to convert. Backpatch to all supported versions. Discussion: https://postgr.es/m/flat/7bdbe559-d61a-4ae4-a6e1-48abdf3024cc%40postgrespro.ru Author: Michael Zhilin Reviewed-by: Alexander Lakhin, Andrey Borodin, Jian He, Alexander Korotkov Backpatch-through: 12 Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/ab65dfb0fb2908989c0ef3330464b7af171e9728 Modified Files -- contrib/amcheck/expected/check_btree.out | 13 contrib/amcheck/sql/check_btree.sql | 11 ++ contrib/amcheck/verify_nbtree.c | 36 +++- 3 files changed, 55 insertions(+), 5 deletions(-)
pgsql: amcheck: Normalize index tuples containing uncompressed varlena
amcheck: Normalize index tuples containing uncompressed varlena It might happen that the varlena value wasn't compressed by index_form_tuple() due to current storage parameters. If compression is currently enabled, we need to compress such values to match index tuple coming from the heap. Backpatch to all supported versions. Discussion: https://postgr.es/m/flat/7bdbe559-d61a-4ae4-a6e1-48abdf3024cc%40postgrespro.ru Author: Andrey Borodin Reviewed-by: Alexander Lakhin, Michael Zhilin, Jian He, Alexander Korotkov Backpatch-through: 12 Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/b1fe8efdf17eb4877f7c4c31c85111ec740ad872 Modified Files -- contrib/amcheck/expected/check_btree.out | 10 ++ contrib/amcheck/sql/check_btree.sql | 6 ++ contrib/amcheck/verify_nbtree.c | 13 + 3 files changed, 29 insertions(+)
pgsql: amcheck: Support for different header sizes of short varlena dat
amcheck: Support for different header sizes of short varlena datum In the heap, tuples may contain short varlena datum with both 1B header and 4B headers. But the corresponding index tuple should always have such varlena's with 1B headers. So, for fingerprinting, we need to convert. Backpatch to all supported versions. Discussion: https://postgr.es/m/flat/7bdbe559-d61a-4ae4-a6e1-48abdf3024cc%40postgrespro.ru Author: Michael Zhilin Reviewed-by: Alexander Lakhin, Andrey Borodin, Jian He, Alexander Korotkov Backpatch-through: 12 Branch -- REL_16_STABLE Details --- https://git.postgresql.org/pg/commitdiff/a6ddb8ad0ad0947bf86e8cd99c87b156f0956228 Modified Files -- contrib/amcheck/expected/check_btree.out | 13 contrib/amcheck/sql/check_btree.sql | 11 ++ contrib/amcheck/verify_nbtree.c | 36 +++- 3 files changed, 55 insertions(+), 5 deletions(-)
pgsql: amcheck: Normalize index tuples containing uncompressed varlena
amcheck: Normalize index tuples containing uncompressed varlena It might happen that the varlena value wasn't compressed by index_form_tuple() due to current storage parameters. If compression is currently enabled, we need to compress such values to match index tuple coming from the heap. Backpatch to all supported versions. Discussion: https://postgr.es/m/flat/7bdbe559-d61a-4ae4-a6e1-48abdf3024cc%40postgrespro.ru Author: Andrey Borodin Reviewed-by: Alexander Lakhin, Michael Zhilin, Jian He, Alexander Korotkov Backpatch-through: 12 Branch -- REL_15_STABLE Details --- https://git.postgresql.org/pg/commitdiff/0d466bce9ee19a5f7ef6c2964376aa6572ef76fd Modified Files -- contrib/amcheck/expected/check_btree.out | 10 ++ contrib/amcheck/sql/check_btree.sql | 6 ++ contrib/amcheck/verify_nbtree.c | 13 + 3 files changed, 29 insertions(+)
pgsql: amcheck: Normalize index tuples containing uncompressed varlena
amcheck: Normalize index tuples containing uncompressed varlena It might happen that the varlena value wasn't compressed by index_form_tuple() due to current storage parameters. If compression is currently enabled, we need to compress such values to match index tuple coming from the heap. Backpatch to all supported versions. Discussion: https://postgr.es/m/flat/7bdbe559-d61a-4ae4-a6e1-48abdf3024cc%40postgrespro.ru Author: Andrey Borodin Reviewed-by: Alexander Lakhin, Michael Zhilin, Jian He, Alexander Korotkov Backpatch-through: 12 Branch -- REL_16_STABLE Details --- https://git.postgresql.org/pg/commitdiff/3676b846b129b975034927cdf5f57297211d6f15 Modified Files -- contrib/amcheck/expected/check_btree.out | 10 ++ contrib/amcheck/sql/check_btree.sql | 6 ++ contrib/amcheck/verify_nbtree.c | 13 + 3 files changed, 29 insertions(+)
pgsql: amcheck: Support for different header sizes of short varlena dat
amcheck: Support for different header sizes of short varlena datum In the heap, tuples may contain short varlena datum with both 1B header and 4B headers. But the corresponding index tuple should always have such varlena's with 1B headers. So, for fingerprinting, we need to convert. Backpatch to all supported versions. Discussion: https://postgr.es/m/flat/7bdbe559-d61a-4ae4-a6e1-48abdf3024cc%40postgrespro.ru Author: Michael Zhilin Reviewed-by: Alexander Lakhin, Andrey Borodin, Jian He, Alexander Korotkov Backpatch-through: 12 Branch -- REL_15_STABLE Details --- https://git.postgresql.org/pg/commitdiff/54e6184db3613e868ca7f042aebe69393fd73b96 Modified Files -- contrib/amcheck/expected/check_btree.out | 13 contrib/amcheck/sql/check_btree.sql | 11 ++ contrib/amcheck/verify_nbtree.c | 36 +++- 3 files changed, 55 insertions(+), 5 deletions(-)
pgsql: amcheck: Normalize index tuples containing uncompressed varlena
amcheck: Normalize index tuples containing uncompressed varlena It might happen that the varlena value wasn't compressed by index_form_tuple() due to current storage parameters. If compression is currently enabled, we need to compress such values to match index tuple coming from the heap. Backpatch to all supported versions. Discussion: https://postgr.es/m/flat/7bdbe559-d61a-4ae4-a6e1-48abdf3024cc%40postgrespro.ru Author: Andrey Borodin Reviewed-by: Alexander Lakhin, Michael Zhilin, Jian He, Alexander Korotkov Backpatch-through: 12 Branch -- REL_14_STABLE Details --- https://git.postgresql.org/pg/commitdiff/4e8529da48baf780f2a2c7e0255a0c9ea02b0d11 Modified Files -- contrib/amcheck/expected/check_btree.out | 10 ++ contrib/amcheck/sql/check_btree.sql | 6 ++ contrib/amcheck/verify_nbtree.c | 13 + 3 files changed, 29 insertions(+)
pgsql: amcheck: Support for different header sizes of short varlena dat
amcheck: Support for different header sizes of short varlena datum In the heap, tuples may contain short varlena datum with both 1B header and 4B headers. But the corresponding index tuple should always have such varlena's with 1B headers. So, for fingerprinting, we need to convert. Backpatch to all supported versions. Discussion: https://postgr.es/m/flat/7bdbe559-d61a-4ae4-a6e1-48abdf3024cc%40postgrespro.ru Author: Michael Zhilin Reviewed-by: Alexander Lakhin, Andrey Borodin, Jian He, Alexander Korotkov Backpatch-through: 12 Branch -- REL_13_STABLE Details --- https://git.postgresql.org/pg/commitdiff/e2c241416515ec9dd711296385ab420e1afc4864 Modified Files -- contrib/amcheck/expected/check_btree.out | 13 contrib/amcheck/sql/check_btree.sql | 11 ++ contrib/amcheck/verify_nbtree.c | 36 +++- 3 files changed, 55 insertions(+), 5 deletions(-)
pgsql: amcheck: Support for different header sizes of short varlena dat
amcheck: Support for different header sizes of short varlena datum In the heap, tuples may contain short varlena datum with both 1B header and 4B headers. But the corresponding index tuple should always have such varlena's with 1B headers. So, for fingerprinting, we need to convert. Backpatch to all supported versions. Discussion: https://postgr.es/m/flat/7bdbe559-d61a-4ae4-a6e1-48abdf3024cc%40postgrespro.ru Author: Michael Zhilin Reviewed-by: Alexander Lakhin, Andrey Borodin, Jian He, Alexander Korotkov Backpatch-through: 12 Branch -- REL_12_STABLE Details --- https://git.postgresql.org/pg/commitdiff/50f8611d073e255c9d5b3792965595879c22c610 Modified Files -- contrib/amcheck/expected/check_btree.out | 13 contrib/amcheck/sql/check_btree.sql | 11 ++ contrib/amcheck/verify_nbtree.c | 36 +++- 3 files changed, 55 insertions(+), 5 deletions(-)
pgsql: amcheck: Normalize index tuples containing uncompressed varlena
amcheck: Normalize index tuples containing uncompressed varlena It might happen that the varlena value wasn't compressed by index_form_tuple() due to current storage parameters. If compression is currently enabled, we need to compress such values to match index tuple coming from the heap. Backpatch to all supported versions. Discussion: https://postgr.es/m/flat/7bdbe559-d61a-4ae4-a6e1-48abdf3024cc%40postgrespro.ru Author: Andrey Borodin Reviewed-by: Alexander Lakhin, Michael Zhilin, Jian He, Alexander Korotkov Backpatch-through: 12 Branch -- REL_12_STABLE Details --- https://git.postgresql.org/pg/commitdiff/d603e674462d7c4df0e88b7a61d4ae14ae3ed191 Modified Files -- contrib/amcheck/expected/check_btree.out | 10 ++ contrib/amcheck/sql/check_btree.sql | 6 ++ contrib/amcheck/verify_nbtree.c | 13 + 3 files changed, 29 insertions(+)
pgsql: amcheck: Support for different header sizes of short varlena dat
amcheck: Support for different header sizes of short varlena datum In the heap, tuples may contain short varlena datum with both 1B header and 4B headers. But the corresponding index tuple should always have such varlena's with 1B headers. So, for fingerprinting, we need to convert. Backpatch to all supported versions. Discussion: https://postgr.es/m/flat/7bdbe559-d61a-4ae4-a6e1-48abdf3024cc%40postgrespro.ru Author: Michael Zhilin Reviewed-by: Alexander Lakhin, Andrey Borodin, Jian He, Alexander Korotkov Backpatch-through: 12 Branch -- REL_14_STABLE Details --- https://git.postgresql.org/pg/commitdiff/5df5d9cd7ea4f40f6e3efec650bdc7615b6d16ed Modified Files -- contrib/amcheck/expected/check_btree.out | 13 contrib/amcheck/sql/check_btree.sql | 11 ++ contrib/amcheck/verify_nbtree.c | 36 +++- 3 files changed, 55 insertions(+), 5 deletions(-)
pgsql: amcheck: Normalize index tuples containing uncompressed varlena
amcheck: Normalize index tuples containing uncompressed varlena It might happen that the varlena value wasn't compressed by index_form_tuple() due to current storage parameters. If compression is currently enabled, we need to compress such values to match index tuple coming from the heap. Backpatch to all supported versions. Discussion: https://postgr.es/m/flat/7bdbe559-d61a-4ae4-a6e1-48abdf3024cc%40postgrespro.ru Author: Andrey Borodin Reviewed-by: Alexander Lakhin, Michael Zhilin, Jian He, Alexander Korotkov Backpatch-through: 12 Branch -- REL_13_STABLE Details --- https://git.postgresql.org/pg/commitdiff/5cc1f2626369536c191009bf084c9d1f456abc7a Modified Files -- contrib/amcheck/expected/check_btree.out | 10 ++ contrib/amcheck/sql/check_btree.sql | 6 ++ contrib/amcheck/verify_nbtree.c | 13 + 3 files changed, 29 insertions(+)
pgsql: Fix an oversight in refactoring in 06b10f80ba4.
Fix an oversight in refactoring in 06b10f80ba4. It was against intended skipping prechecking keys optimization in the first page of range queries to not influence point queries performance. Reported-by: Anton Melnikov Discussion: https://postgr.es/m/30cd7524-b9f1-4cf8-9c4a-223eb2e34441%40postgrespro.ru Author: Pavel Borisov Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/b670b93a66fc554714e0fe8e51a944912bb9fd68 Modified Files -- src/backend/access/nbtree/nbtsearch.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
pgsql: Allow table AM tuple_insert() method to return the different slo
Allow table AM tuple_insert() method to return the different slot This allows table AM to return a native tuple slot even if VirtualTupleTableSlot is given as an input. Native tuple slots have knowledge about system attributes, which could be accessed in the future. table_multi_insert() method already can modify the input 'slots' array. Discussion: https://postgr.es/m/CAPpHfdurb9ycV8udYqM%3Do0sPS66PJ4RCBM1g-bBpvzUfogY0EA%40mail.gmail.com Reviewed-by: Matthias van de Meent, Mark Dilger, Pavel Borisov Reviewed-by: Nikita Malakhov, Japin Li Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/c35a3fb5e067fc95f13206418e3785d2cb059da1 Modified Files -- src/backend/access/heap/heapam_handler.c | 4 +++- src/backend/executor/nodeModifyTable.c | 6 +++--- src/include/access/tableam.h | 21 - 3 files changed, 18 insertions(+), 13 deletions(-)
pgsql: Add TupleTableSlotOps.is_current_xact_tuple() method
Add TupleTableSlotOps.is_current_xact_tuple() method This allows us to abstract how/whether table AM uses transaction identifiers. A custom table AM can use a custom slot, which may not store xmin directly, but determine the tuple belonging to the current transaction in the other way. Discussion: https://postgr.es/m/CAPpHfdurb9ycV8udYqM%3Do0sPS66PJ4RCBM1g-bBpvzUfogY0EA%40mail.gmail.com Reviewed-by: Matthias van de Meent, Mark Dilger, Pavel Borisov Reviewed-by: Nikita Malakhov, Japin Li Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/0997e0af273d80add75bcf5616eee000d0a78397 Modified Files -- src/backend/executor/execTuples.c | 79 + src/backend/utils/adt/ri_triggers.c | 8 +--- src/include/executor/tuptable.h | 21 ++ 3 files changed, 101 insertions(+), 7 deletions(-)
pgsql: Allow table AM to store complex data structures in rd_amcache
Allow table AM to store complex data structures in rd_amcache The new table AM method free_rd_amcache is responsible for freeing all the memory related to rd_amcache and setting free_rd_amcache to NULL. If the new method is not specified, we still assume rd_amcache to be a single chunk of memory, which could be just pfree'd. Discussion: https://postgr.es/m/CAPpHfdurb9ycV8udYqM%3Do0sPS66PJ4RCBM1g-bBpvzUfogY0EA%40mail.gmail.com Reviewed-by: Matthias van de Meent, Mark Dilger, Pavel Borisov Reviewed-by: Nikita Malakhov, Japin Li Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/02eb07ea89d27f1d05a5055bf779042d2953b4e7 Modified Files -- src/backend/access/heap/heapam_handler.c | 1 + src/backend/utils/cache/relcache.c | 11 +++ src/include/access/tableam.h | 34 src/include/utils/rel.h | 10 ++ 4 files changed, 44 insertions(+), 12 deletions(-)
pgsql: psql: fix variable existence tab completion
psql: fix variable existence tab completion
psql has the :{?name} syntax for testing for a psql variable existence. This
commit implements a tab completion for this syntax. Notably, in order to
implement this we have to remove '{' from WORD_BREAKS. It appears that
'{' here from the very beginning and it comes from the default value of
rl_basic_word_break_characters. And :{?name} is the only psql syntax using
the '{' sign. So, removing it from WORD_BREAKS shouldn't break anything.
Discussion:
https://postgr.es/m/CAGRrpzZU48F2oV3d8eDLr%3D4TU9xFH5Jt9ED%2BqU1%2BX91gMH68Sw%40mail.gmail.com
Author: Steve Chavez
Reviewed-by: Erik Wienhold
Branch
--
master
Details
---
https://git.postgresql.org/pg/commitdiff/927332b95e778c0d15a9fbf96e3efeab0d3d937c
Modified Files
--
src/bin/psql/t/010_tab_completion.pl | 8
src/bin/psql/tab-complete.c | 4 +++-
2 files changed, 11 insertions(+), 1 deletion(-)
pgsql: Use locale-aware value for \watch in 005_timeouts.pl
Use locale-aware value for \watch in 005_timeouts.pl Reported-by: Alexander Lakhin Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/605062227fdc8331f2ffa0f60e7a8724b6e56480 Modified Files -- src/test/modules/test_misc/t/005_timeouts.pl | 18 +++--- 1 file changed, 11 insertions(+), 7 deletions(-)
Re: pgsql: Add TAP tests for timeouts
On Fri, Mar 15, 2024 at 2:25 PM Andrey M. Borodin wrote: > > On 15 Mar 2024, at 16:30, Alexander Korotkov wrote: > > > > Maybe, but do you see any negative side effects of the unconditionally > > unset of flags? > > Nope, just expressed possible option. > > > If not, I would prefer to keep the code simple. > > IMO that's fine. Let's incorporate wording improvement from Kyotaro to the > same patch? PFA draft for this. Thank you, I've pushed this, but split into two commits for bugfix and for improved wordings. > Maybe it worth to wait a little to see if some other failures will pop up? I prefer to push now because we're quite confident this fix will improve the situation. It should increase the chances of catching other bugs if any, because buildfarm which was failing on this bug will go ahead and have a chance to spot more. -- Regards, Alexander Korotkov
pgsql: Fix wordings in timeouts TAP test
Fix wordings in timeouts TAP test Reported-by: Kyotaro Horiguchi Discussion: https://postgr.es/m/20240315.104235.1835366724413653745.horikyota.ntt%40gmail.com Author: Andrey Borodin Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/7a65cc079e717c5d87f1920929affe7138c31eee Modified Files -- src/test/modules/test_misc/t/005_timeouts.pl | 12 ++-- 1 file changed, 6 insertions(+), 6 deletions(-)
pgsql: Fix race condition in transaction timeout TAP tests
Fix race condition in transaction timeout TAP tests The interruption handler within the injection point can get stuck in an infinite loop while handling transaction timeout. To avoid this situation we reset the timeout flag before invoking the injection point. Author: Alexander Korotkov Reviewed-by: Andrey Borodin Discussion: https://postgr.es/m/ZfPchPC6oNN71X2J%40paquier.xyz Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/4c2eda67f55a8a263820d12aaeaa7e1dfe7406ee Modified Files -- src/backend/tcop/postgres.c | 12 +--- 1 file changed, 5 insertions(+), 7 deletions(-)
Re: pgsql: Add TAP tests for timeouts
On Fri, Mar 15, 2024 at 1:27 PM Andrey M. Borodin wrote: > > > On 15 Mar 2024, at 15:44, Alexander Korotkov wrote: > > > > We loop in the interrupt checking, given that the injection point handler > > checks for interrupts internally. I propose to unset the timeout flag > > before the injection point (see the attached patch). > > Oh, cool. > As far as I understand, this is only necessary for the test with injection > point. > So, maybe unset it only when injection points are enabled? Something like > this is already used in GIN. > > #ifdef USE_INJECTION_POINTS > > #endif Maybe, but do you see any negative side effects of the unconditionally unset of flags? If not, I would prefer to keep the code simple. -- Regards, Alexander Korotkov
Re: pgsql: Add TAP tests for timeouts
On Fri, Mar 15, 2024 at 9:44 AM Michael Paquier wrote: > On Fri, Mar 15, 2024 at 11:20:49AM +0500, Andrey M. Borodin wrote: > > And hachi sometimes pass this test too [0]. > > > > I’ll look more on this. Do I understand right that we have only 2 buildfarm members with injection points? > > I have added --enable-injection-points to three of them: hachi, > gokiburi and batta. The switch is also enabled in the CI by defaultm > where you may be able to capture a bit more information than the > buildfarm. I managed to reproduce the issue locally by setting the following extra config via TEMP_CONFIG: log_line_prefix = '%m [%p:%l] %q%a ' log_connections = 'true' log_disconnections = 'true' log_checkpoints = 'true' log_statement = 'all' wal_compression = 'on' debug_parallel_query = regress wal_compression = lz4 default_toast_compression = lz4 The backtrace is as follows. We loop in the interrupt checking, given that the injection point handler checks for interrupts internally. I propose to unset the timeout flag before the injection point (see the attached patch). * thread #1, name = 'postgres', stop reason = signal SIGSTOP * frame #0: 0x7f3756f289e7 libc.so.6`epoll_wait(epfd=9, events=0x556a302cabc8, maxevents=1, timeout=-1) at epoll_wait.c:30:10 frame #1: 0x556a2fc41aab postgres`WaitEventSetWait at latch.c:1570:7 frame #2: 0x556a2fc41e95 postgres`WaitLatch(latch=, wakeEvents=, timeout=, wait_event_info=117440513) at latch.c:538:6 frame #3: 0x556a2fc4f611 postgres`ConditionVariableTimedSleep(cv=0x7f3757b0e224, timeout=-1, wait_event_info=117440513) at condition_variable.c:163:10 frame #4: 0x7f3757b10628 injection_points.so`injection_wait at injection_points.c:153:3 frame #5: 0x556a2fdd4f54 postgres`InjectionPointRun at injection_point.c:313:2 frame #6: 0x556a2fc6a8b3 postgres`ProcessInterrupts.part.0 at postgres.c:3430:4 frame #7: 0x556a2fc4f70b postgres`ConditionVariableTimedSleep(cv=0x7f3757b0e224, timeout=-1, wait_event_info=117440513) at condition_variable.c:196:3 frame #8: 0x7f3757b10628 injection_points.so`injection_wait at injection_points.c:153:3 frame #9: 0x556a2fdd4f54 postgres`InjectionPointRun at injection_point.c:313:2 frame #10: 0x556a2fc6a8b3 postgres`ProcessInterrupts.part.0 at postgres.c:3430:4 frame #11: 0x556a2fc4f70b postgres`ConditionVariableTimedSleep(cv=0x7f3757b0e224, timeout=-1, wait_event_info=117440513) at condition_variable.c:196:3 frame #12: 0x7f3757b10628 injection_points.so`injection_wait at injection_points.c:153:3 frame #13: 0x556a2fdd4f54 postgres`InjectionPointRun at injection_point.c:313:2 frame #14: 0x556a2fc6a8b3 postgres`ProcessInterrupts.part.0 at postgres.c:3430:4 frame #15: 0x556a2fabe63d postgres`ExecGather at nodeGather.c:315:3 frame #16: 0x556a2fabe574 postgres`ExecGather at nodeGather.c:269:10 frame #17: 0x556a2fabe549 postgres`ExecGather(pstate=0x556a303d8138) at nodeGather.c:222:9 frame #18: 0x556a2faa21da postgres`standard_ExecutorRun at executor.h:274:9 frame #19: 0x556a2fc6f3cf postgres`PortalRunSelect at pquery.c:924:4 frame #20: 0x556a2fc70b7e postgres`PortalRun at pquery.c:768:18 frame #21: 0x556a2fc6c996 postgres`exec_simple_query at postgres.c:1274:10 frame #22: 0x556a2fc6e378 postgres`PostgresMain at postgres.c:4682:7 frame #23: 0x556a2fbcefa0 postgres`ServerLoop.isra.0 at postmaster.c:4452:2 frame #24: 0x556a2fbcff8b postgres`PostmasterMain at postmaster.c:1478:11 frame #25: 0x556a2f8cb5a7 postgres`main(argc=4, argv=0x556a302ca340) at main.c:197:3 frame #26: 0x7f3756e28150 libc.so.6`__libc_start_call_main(main=(postgres`main at main.c:59:1), argc=4, argv=0x7c17e678) at libc_start_call_main.h:58:16 frame #27: 0x7f3756e28209 libc.so.6`__libc_start_main_impl(main=(postgres`main at main.c:59:1), argc=4, argv=0x7c17e678, init=, fini=, rtld_fini=, stack_end=0x7c17e668) at libc-start.c:360:3 frame #28: 0x556a2f8cbb75 postgres`_start + 37 -- Regards, Alexander Korotkov transaction_timeout_patch_fix.patch Description: Binary data
pgsql: Fix contrib/pg_visibility/meson.build
Fix contrib/pg_visibility/meson.build I broke that in e85662df44ff by oversight. Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/c20d90a41ca869f9c6dd4058ad1c7f5c9ee9d912 Modified Files -- contrib/pg_visibility/meson.build | 1 + 1 file changed, 1 insertion(+)
pgsql: Add TAP tests for timeouts
Add TAP tests for timeouts This commit adds new tests to verify that transaction_timeout, idle_session_timeout, and idle_in_transaction_session_timeout work as expected. We introduce new injection points in before throwing a timeout FATAL error and check these injection points are reached. Discussion: https://postgr.es/m/CAAhFRxiQsRs2Eq5kCo9nXE3HTugsAAJdSQSmxncivebAxdmBjQ%40mail.gmail.com Author: Andrey Borodin Reviewed-by: Alexander Korotkov Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/eeefd4280f6e5167d70efabb89586b7d38922d95 Modified Files -- src/backend/tcop/postgres.c | 10 +++ src/test/modules/test_misc/Makefile | 4 + src/test/modules/test_misc/meson.build | 4 + src/test/modules/test_misc/t/005_timeouts.pl | 129 +++ 4 files changed, 147 insertions(+)
