pgsql: make dist uses git archive
make dist uses git archive This changes "make dist" to directly use "git archive", rather than the custom shell script it currently runs. This is to make the creation of the distribution tarball more directly traceable to the git repository. That is why we removed the "make distprep" step. "make dist" continues to produce a .gz and a .bz2 tarball as before. The archives produced this way are deterministic and reproducible, meaning for a given commit the result file should always be bit-for-bit identical. The exception is that if you use a git version older than 2.38.0, gzip records the platform in the archive, so you'd get a different output on Windows vs. macOS vs. "UNIX" (everything else). In git 2.38.0, this was changed so that everything is recorded as "UNIX" now. This is just something to keep in mind. This issue is specific to the gzip format, it does not affect other compression formats. Meson has its own distribution building command (meson dist), but we are not using that at this point. The main problem is that, the way they have implemented it, it is not deterministic in the above sense. Also, we want a "make" version for the time being. But the target name "dist" in meson is reserved for that reason, so we call the custom target "pgdist" (so call something like "meson compile -C build pgdist"). Reviewed-by: Tristan Partin Discussion: https://www.postgresql.org/message-id/flat/40e80f77-a294-4f29-a16f-e21bc7bc75fc%40eisentraut.org Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/619bc23a1a2f3750ac3668fe5a7564bc51e01684 Modified Files -- GNUmakefile.in | 32 + meson.build| 65 ++ 2 files changed, 79 insertions(+), 18 deletions(-)
pgsql: Fix potential integer handling issue in radixtree.h.
Fix potential integer handling issue in radixtree.h. Coverity complained about the integer handling issue; if we start with an arbitrary non-negative shift value, the loop may decrement it down to something less than zero before exiting. This commit adds an assertion to make sure the 'shift' is always 0 after the loop, and uses 0 as the shift to get the key chunk in the following operation. Introduced by ee1b30f12. Reported-by: Tom Lane as per coverity Reviewed-by: Tom Lane Discussion: https://postgr.es/m/2089517.1711299216%40sss.pgh.pa.us Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/80d5d4937c168af46c976feebe24765ad76eb540 Modified Files -- src/include/lib/radixtree.h | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
Re: pgsql: reindexdb: Add the index-level REINDEX with multiple jobs
On Mon, 25 Mar 2024 at 13:07, Alexander Korotkov wrote: > reindexdb: Add the index-level REINDEX with multiple jobs This seems to cause a new compiler warning: reindexdb.c: In function ‘reindex_one_database’: reindexdb.c:437:45: warning: ‘indices_tables_cell’ may be used uninitialized [-Wmaybe-uninitialized] 437 | indices_tables_cell = indices_tables_cell->next; | ^~~ reindexdb.c:280:31: note: ‘indices_tables_cell’ was declared here 280 | SimpleStringListCell *indices_tables_cell; | ^~~ which is causing problems in -Werror buildfarm members like mamba [1] David [1] https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=mamba&dt=2024-03-25%2001%3A03%3A35
pgsql: Allow planner to use Merge Append to efficiently implement UNION
Allow planner to use Merge Append to efficiently implement UNION Until now, UNION queries have often been suboptimal as the planner has only ever considered using an Append node and making the results unique by either using a Hash Aggregate, or by Sorting the entire Append result and running it through the Unique operator. Both of these methods always require reading all rows from the union subqueries. Here we adjust the union planner so that it can request that each subquery produce results in target list order so that these can be Merge Appended together and made unique with a Unique node. This can improve performance significantly as the union child can make use of the likes of btree indexes and/or Merge Joins to provide the top-level UNION with presorted input. This is especially good if the top-level UNION contains a LIMIT node that limits the output rows to a small subset of the unioned rows as cheap startup plans can be used. Author: David Rowley Reviewed-by: Richard Guo, Andy Fan Discussion: https://postgr.es/m/caaphdvpb_63xqodmxkuf8vb9m7cxyuyt4swvegqequ-gb7q...@mail.gmail.com Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/66c0185a3d14bbbf51d0fc9d267093ffec735231 Modified Files -- contrib/postgres_fdw/expected/postgres_fdw.out | 7 + contrib/postgres_fdw/sql/postgres_fdw.sql | 9 + src/backend/optimizer/path/equivclass.c| 61 +++ src/backend/optimizer/path/pathkeys.c | 19 + src/backend/optimizer/plan/planner.c | 85 ++- src/backend/optimizer/prep/prepunion.c | 711 + src/backend/parser/analyze.c | 3 +- src/include/nodes/pathnodes.h | 2 + src/include/optimizer/paths.h | 4 + src/include/optimizer/prep.h | 2 +- src/test/regress/expected/collate.icu.utf8.out | 2 + src/test/regress/expected/incremental_sort.out | 13 +- src/test/regress/expected/union.out| 32 +- src/test/regress/sql/collate.icu.utf8.sql | 2 + src/test/regress/sql/union.sql | 8 +- 15 files changed, 707 insertions(+), 253 deletions(-)
pgsql: reindexdb: Add the index-level REINDEX with multiple jobs
reindexdb: Add the index-level REINDEX with multiple jobs Straight-forward index-level REINDEX is not supported with multiple jobs as we cannot control the concurrent processing of multiple indexes depending on the same relation. Instead, we dedicate the whole table to certain reindex job. Thus, if indexes in the lists belong to different tables, that gives us a fair level of parallelism. This commit teaches get_parallel_object_list() to fetch table names for indexes in the case of index-level REINDEX. The same tables are grouped together in the output order, and the list of indexes is also rebuilt to match that order. Later during processingof that list, we push indexes belonging to the same table into the same job. Discussion: https://postgr.es/m/CACG%3DezZU_VwDi-1PN8RUSE6mcYG%2BYx1NH_rJO4%2BKe-mKqLp%3DNw%40mail.gmail.com Author: Maxim Orlov, Svetlana Derevyanko, Alexander Korotkov Reviewed-by: Michael Paquier Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/47f99a407de224df6f9c43697d0a9c0a5598b250 Modified Files -- doc/src/sgml/ref/reindexdb.sgml| 3 +- src/bin/scripts/reindexdb.c| 146 +++-- src/bin/scripts/t/090_reindexdb.pl | 8 +- 3 files changed, 130 insertions(+), 27 deletions(-)
pgsql: Fix convert_case(), introduced in 5c40364dd6.
Fix convert_case(), introduced in 5c40364dd6. Check source length before checking for NUL terminator to avoid reading one byte past the string end. Also fix unreachable bug when caller does not expect NUL-terminated result. Add unit test coverage of convert_case() in case_test.c, which makes it easier to reproduce the valgrind failure. Discussion: https://postgr.es/m/7a9fd36d-7a38-4dc2-e676-fc939491a...@gmail.com Reported-by: Alexander Lakhin Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/503c0ad976f5e4ada2d1b47ae05aa7133351b645 Modified Files -- src/common/unicode/case_test.c | 101 +++-- src/common/unicode_case.c | 6 +-- 2 files changed, 101 insertions(+), 6 deletions(-)
pgsql: doc: Clarify requirements for SET ROLE.
doc: Clarify requirements for SET ROLE. Since commit 3d14e171e9, SET ROLE has required the current session user to have membership with the SET option in the target role, but the SET ROLE documentation only mentions the membership requirement. This commit adds this important detail to the SET ROLE page. Reviewed-by: Robert Haas Discussion: https://postgr.es/m/CA%2BRLCQysHtME0znk2KUMJN343ksboSRQSU-hCnOjesX6VK300Q%40mail.gmail.com Backpatch-through: 16 Branch -- REL_16_STABLE Details --- https://git.postgresql.org/pg/commitdiff/d82dc79ba03c783fa41be768c1bf13cb8200a340 Modified Files -- doc/src/sgml/ref/set_role.sgml | 6 -- 1 file changed, 4 insertions(+), 2 deletions(-)
pgsql: doc: Clarify requirements for SET ROLE.
doc: Clarify requirements for SET ROLE. Since commit 3d14e171e9, SET ROLE has required the current session user to have membership with the SET option in the target role, but the SET ROLE documentation only mentions the membership requirement. This commit adds this important detail to the SET ROLE page. Reviewed-by: Robert Haas Discussion: https://postgr.es/m/CA%2BRLCQysHtME0znk2KUMJN343ksboSRQSU-hCnOjesX6VK300Q%40mail.gmail.com Backpatch-through: 16 Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/3330a8d1b792375ae3f7dffb7eb0a45c907cbc72 Modified Files -- doc/src/sgml/ref/set_role.sgml | 6 -- 1 file changed, 4 insertions(+), 2 deletions(-)
pgsql: Allow more cases to pass the unsafe-use-of-new-enum-value restri
Allow more cases to pass the unsafe-use-of-new-enum-value restriction. Up to now we've rejected cases like BEGIN; CREATE TYPE rainbow AS ENUM (); ALTER TYPE rainbow ADD VALUE 'red'; -- use the value 'red', perhaps in a constraint or index COMMIT; The concern is that the uncommitted enum value 'red' might get into an index and then break the index if we roll back the ALTER ADD. If the ALTER is in the same transaction as the CREATE then it's really perfectly safe, but we weren't taking the trouble to identify that. pg_dump in binary-upgrade mode will emit enum definitions that look like the above, which up to now didn't fall foul of the unsafe-usage check because we processed each restore command as a separate transaction. However an upcoming patch proposes to bundle the restore commands into large transactions to reduce XID consumption during pg_upgrade, and that makes this behavior a problem. To fix, remember the OIDs of enum types created in the current transaction, and allow use of enum values that are added to one later in the same transaction. To do this fully correctly in the presence of subtransactions, we'd have to track subtransaction nesting level of the CREATE and do maintenance work at every subsequent subtransaction exit. That seems expensive, and we don't need it to satisfy pg_dump's usage. Hence, apply the additional optimization only when the CREATE and ALTER are at outermost transaction level. Patch by me, reviewed by Andrew Dunstan Discussion: https://postgr.es/m/1548468.1711220...@sss.pgh.pa.us Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/af1d3958432c9438c4ec168bd16bdf9a91066a9f Modified Files -- src/backend/catalog/pg_enum.c | 208 + src/backend/utils/adt/enum.c | 19 ++-- src/test/regress/expected/enum.out | 12 ++- src/test/regress/sql/enum.sql | 5 +- 4 files changed, 181 insertions(+), 63 deletions(-)
pgsql: Release PQconninfoOptions array in GetDbnameFromConnectionOption
Release PQconninfoOptions array in GetDbnameFromConnectionOptions(). It wasn't getting freed in one code path, which Coverity identified as a resource leak. It's probably of little consequence, but re-ordering the code into the correct sequence is no more work than dismissing the complaint. Minor oversight in commit a145f424d. While here, improve the unreasonably clunky coding of FindDbnameInConnParams: use of an output parameter is unnecessary and prone to uninitialized-variable problems. Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/d37e0d0c50ee560fa2e7dc2a4dc904a885957985 Modified Files -- src/bin/pg_basebackup/streamutil.c | 29 ++--- 1 file changed, 14 insertions(+), 15 deletions(-)
pgsql: Release temporary array in check_for_data_types_usage().
Release temporary array in check_for_data_types_usage(). Coverity identified this as a resource leak. It's surely of no consequence given that the function is called only once per run, but freeing the storage is no more work than dismissing the complaint. Minor oversight in commit 347758b12. Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/225e1dde463ae312714d71a929f45cfd153df2be Modified Files -- src/bin/pg_upgrade/check.c | 2 ++ 1 file changed, 2 insertions(+)
pgsql: ci: freebsd repartition script didn't copy .git directory
ci: freebsd repartition script didn't copy .git directory We need a slightly different "cp" incantation to make sure top-level "dot" files, such as ".git", are also copied. This is relevant for example if a script wants to execute a git command. This currently does not happen, but it has come up while testing other patches. Reviewed-by: Tristan Partin Discussion: https://www.postgresql.org/message-id/flat/40e80f77-a294-4f29-a16f-e21bc7bc75fc%40eisentraut.org Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/fc2d260c7e6236fe2447dad0f8415e72f4be66a2 Modified Files -- src/tools/ci/gcp_freebsd_repartition.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
