[COMMITTERS] pgsql: Update comment.
Update comment. mergepreread()/mergeprereadone() don't exist anymore, the function that does roughly the same is now called mergereadnext(). Branch -- master Details --- http://git.postgresql.org/pg/commitdiff/c86c2d9d57c1e6c5fec860873734cccaf31df5b0 Modified Files -- src/backend/utils/sort/tuplesort.c | 16 1 file changed, 8 insertions(+), 8 deletions(-) -- Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-committers
[COMMITTERS] pgsql: Correct logical decoding restore behaviour for subtransactions.
Correct logical decoding restore behaviour for subtransactions. Before initializing iteration over a subtransaction's changes, the last few changes were not spilled to disk. That's correct if the transaction didn't spill to disk, but otherwise... This bug can lead to missed or misorderd subtransaction contents when they were spilled to disk. Move spilling of the remaining in-memory changes to ReorderBufferIterTXNInit(), where it can easily be applied to the top transaction and, if present, subtransactions. Since this code had too many bugs already, noticeably increase test coverage. Fixes: #14319 Reported-By: Huan Ruan Discussion: <20160909012610.20024.58...@wrigleys.postgresql.org> Backport: 9,4-, where logical decoding was added Branch -- REL9_5_STABLE Details --- http://git.postgresql.org/pg/commitdiff/ce603a34a4568e876c0d9c826325d9b12c9e5497 Modified Files -- contrib/test_decoding/Makefile | 2 +- contrib/test_decoding/expected/spill.out| 256 contrib/test_decoding/sql/spill.sql | 179 + src/backend/replication/logical/reorderbuffer.c | 13 +- 4 files changed, 444 insertions(+), 6 deletions(-) -- Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-committers
[COMMITTERS] pgsql: Correct logical decoding restore behaviour for subtransactions.
Correct logical decoding restore behaviour for subtransactions. Before initializing iteration over a subtransaction's changes, the last few changes were not spilled to disk. That's correct if the transaction didn't spill to disk, but otherwise... This bug can lead to missed or misorderd subtransaction contents when they were spilled to disk. Move spilling of the remaining in-memory changes to ReorderBufferIterTXNInit(), where it can easily be applied to the top transaction and, if present, subtransactions. Since this code had too many bugs already, noticeably increase test coverage. Fixes: #14319 Reported-By: Huan Ruan Discussion: <20160909012610.20024.58...@wrigleys.postgresql.org> Backport: 9,4-, where logical decoding was added Branch -- master Details --- http://git.postgresql.org/pg/commitdiff/61633f79048d040aefeed68dcac0e0b996a06189 Modified Files -- contrib/test_decoding/Makefile | 3 +- contrib/test_decoding/expected/spill.out| 256 contrib/test_decoding/sql/spill.sql | 179 + src/backend/replication/logical/reorderbuffer.c | 13 +- 4 files changed, 445 insertions(+), 6 deletions(-) -- Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-committers
[COMMITTERS] pgsql: Correct logical decoding restore behaviour for subtransactions.
Correct logical decoding restore behaviour for subtransactions. Before initializing iteration over a subtransaction's changes, the last few changes were not spilled to disk. That's correct if the transaction didn't spill to disk, but otherwise... This bug can lead to missed or misorderd subtransaction contents when they were spilled to disk. Move spilling of the remaining in-memory changes to ReorderBufferIterTXNInit(), where it can easily be applied to the top transaction and, if present, subtransactions. Since this code had too many bugs already, noticeably increase test coverage. Fixes: #14319 Reported-By: Huan Ruan Discussion: <20160909012610.20024.58...@wrigleys.postgresql.org> Backport: 9,4-, where logical decoding was added Branch -- REL9_6_STABLE Details --- http://git.postgresql.org/pg/commitdiff/76c0b73df6c481a29cf6e45bebeafd6d71957f08 Modified Files -- contrib/test_decoding/Makefile | 3 +- contrib/test_decoding/expected/spill.out| 256 contrib/test_decoding/sql/spill.sql | 179 + src/backend/replication/logical/reorderbuffer.c | 13 +- 4 files changed, 445 insertions(+), 6 deletions(-) -- Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-committers
[COMMITTERS] pgsql: Correct logical decoding restore behaviour for subtransactions.
Correct logical decoding restore behaviour for subtransactions. Before initializing iteration over a subtransaction's changes, the last few changes were not spilled to disk. That's correct if the transaction didn't spill to disk, but otherwise... This bug can lead to missed or misorderd subtransaction contents when they were spilled to disk. Move spilling of the remaining in-memory changes to ReorderBufferIterTXNInit(), where it can easily be applied to the top transaction and, if present, subtransactions. Since this code had too many bugs already, noticeably increase test coverage. Fixes: #14319 Reported-By: Huan Ruan Discussion: <20160909012610.20024.58...@wrigleys.postgresql.org> Backport: 9,4-, where logical decoding was added Branch -- REL9_4_STABLE Details --- http://git.postgresql.org/pg/commitdiff/07172b20f1a0c4e3a24b0b8f6781de4d2c4866cb Modified Files -- contrib/test_decoding/Makefile | 2 +- contrib/test_decoding/expected/spill.out| 256 contrib/test_decoding/sql/spill.sql | 179 + src/backend/replication/logical/reorderbuffer.c | 13 +- 4 files changed, 444 insertions(+), 6 deletions(-) -- Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-committers
[COMMITTERS] pgsql: Convert contrib/hstore_plpython to not use direct linking to oth
Convert contrib/hstore_plpython to not use direct linking to other modules. Previously, on most platforms, we allowed hstore_plpython's references to hstore and plpython to be unresolved symbols at link time, trusting the dynamic linker to resolve them when the module is loaded. This has a number of problems, the worst being that the dynamic linker does not know where the references come from and can do nothing but fail if those other modules haven't been loaded. We've more or less gotten away with that for the limited use-case of datatype transform modules, but even there, it requires some awkward hacks, most recently commit 83c249200. Instead, let's not treat these references as linker-resolvable at all, but use function pointers that are manually filled in by the module's _PG_init function. There are few enough contact points that this doesn't seem unmaintainable, at least for these use-cases. (Note that the same technique wouldn't work at all for decoupling from libpython itself, but fortunately that's just a standard shared library and can be linked to normally.) This is an initial patch that just converts hstore_plpython. If the buildfarm doesn't find any fatal problems, I'll work on the other transform modules soon. Tom Lane, per an idea of Andres Freund's. Discussion: <2652.1475512...@sss.pgh.pa.us> Branch -- master Details --- http://git.postgresql.org/pg/commitdiff/d51924be886c2a05e691fa05b16cb6b30ab8370f Modified Files -- contrib/hstore_plpython/Makefile | 19 +++ contrib/hstore_plpython/hstore_plpython.c | 65 ++ contrib/hstore_plpython/hstore_plpython2u--1.0.sql | 5 -- contrib/hstore_plpython/hstore_plpython3u--1.0.sql | 5 -- contrib/hstore_plpython/hstore_plpythonu--1.0.sql | 5 -- src/tools/msvc/Mkvcbuild.pm| 3 +- 6 files changed, 76 insertions(+), 26 deletions(-) -- Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-committers
[COMMITTERS] pgsql: Show a sensible value in pg_settings.unit for GUC_UNIT_XSEGS var
Show a sensible value in pg_settings.unit for GUC_UNIT_XSEGS variables. Commit 88e982302 invented GUC_UNIT_XSEGS for min_wal_size and max_wal_size, but neglected to make it display sensibly in pg_settings.unit (by adding a case to the switch in GetConfigOptionByNum). Fix that, and adjust said switch to throw a run-time error the next time somebody forgets. In passing, avoid using a static buffer for the output string --- the rest of this function pstrdup's from a local buffer, and I see no very good reason why the units code should do it differently and less safely. Per report from Otar Shavadze. Back-patch to 9.5 where the new unit type was added. Report: Branch -- REL9_6_STABLE Details --- http://git.postgresql.org/pg/commitdiff/993d94c59038fb80068a2b7780a1062dcaa2bb4f Modified Files -- src/backend/utils/misc/guc.c | 20 ++-- src/include/utils/guc.h | 4 ++-- 2 files changed, 16 insertions(+), 8 deletions(-) -- Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-committers
[COMMITTERS] pgsql: Show a sensible value in pg_settings.unit for GUC_UNIT_XSEGS var
Show a sensible value in pg_settings.unit for GUC_UNIT_XSEGS variables. Commit 88e982302 invented GUC_UNIT_XSEGS for min_wal_size and max_wal_size, but neglected to make it display sensibly in pg_settings.unit (by adding a case to the switch in GetConfigOptionByNum). Fix that, and adjust said switch to throw a run-time error the next time somebody forgets. In passing, avoid using a static buffer for the output string --- the rest of this function pstrdup's from a local buffer, and I see no very good reason why the units code should do it differently and less safely. Per report from Otar Shavadze. Back-patch to 9.5 where the new unit type was added. Report: Branch -- REL9_5_STABLE Details --- http://git.postgresql.org/pg/commitdiff/f50fa46cca9a1e5bfd517f7f1d979ef4db55ce03 Modified Files -- src/backend/utils/misc/guc.c | 20 ++-- src/include/utils/guc.h | 4 ++-- 2 files changed, 16 insertions(+), 8 deletions(-) -- Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-committers
[COMMITTERS] pgsql: Show a sensible value in pg_settings.unit for GUC_UNIT_XSEGS var
Show a sensible value in pg_settings.unit for GUC_UNIT_XSEGS variables. Commit 88e982302 invented GUC_UNIT_XSEGS for min_wal_size and max_wal_size, but neglected to make it display sensibly in pg_settings.unit (by adding a case to the switch in GetConfigOptionByNum). Fix that, and adjust said switch to throw a run-time error the next time somebody forgets. In passing, avoid using a static buffer for the output string --- the rest of this function pstrdup's from a local buffer, and I see no very good reason why the units code should do it differently and less safely. Per report from Otar Shavadze. Back-patch to 9.5 where the new unit type was added. Report: Branch -- master Details --- http://git.postgresql.org/pg/commitdiff/6bc811c992a804bdb8d228ce0be9f0a8e7198df6 Modified Files -- src/backend/utils/misc/guc.c | 20 ++-- src/include/utils/guc.h | 4 ++-- 2 files changed, 16 insertions(+), 8 deletions(-) -- Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-committers
[COMMITTERS] pgsql: Fix RLS with COPY (col1, col2) FROM tab
Fix RLS with COPY (col1, col2) FROM tab Attempting to COPY a subset of columns from a table with RLS enabled would fail due to an invalid query being constructed (using a single ColumnRef with the list of fields to exact in 'fields', but that's for the different levels of an indirection for a single column, not for specifying multiple columns). Correct by building a ColumnRef and then RestTarget for each column being requested and then adding those to the targetList for the select query. Include regression tests to hopefully catch if this is broken again in the future. Patch-By: Adam Brightwell Reviewed-By: Michael Paquier Branch -- master Details --- http://git.postgresql.org/pg/commitdiff/814b9e9b8edf36cac65e0d8fcef17e50a04b1617 Modified Files -- src/backend/commands/copy.c | 63 -- src/test/regress/expected/copy2.out | 78 + src/test/regress/sql/copy2.sql | 63 ++ 3 files changed, 192 insertions(+), 12 deletions(-) -- Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-committers
[COMMITTERS] pgsql: Fix RLS with COPY (col1, col2) FROM tab
Fix RLS with COPY (col1, col2) FROM tab Attempting to COPY a subset of columns from a table with RLS enabled would fail due to an invalid query being constructed (using a single ColumnRef with the list of fields to exact in 'fields', but that's for the different levels of an indirection for a single column, not for specifying multiple columns). Correct by building a ColumnRef and then RestTarget for each column being requested and then adding those to the targetList for the select query. Include regression tests to hopefully catch if this is broken again in the future. Patch-By: Adam Brightwell Reviewed-By: Michael Paquier Branch -- REL9_5_STABLE Details --- http://git.postgresql.org/pg/commitdiff/647a86e374d2eb01d8cdda2fc759a770b38cc232 Modified Files -- src/backend/commands/copy.c | 63 -- src/test/regress/expected/copy2.out | 78 + src/test/regress/sql/copy2.sql | 63 ++ 3 files changed, 192 insertions(+), 12 deletions(-) -- Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-committers
[COMMITTERS] pgsql: Fix RLS with COPY (col1, col2) FROM tab
Fix RLS with COPY (col1, col2) FROM tab Attempting to COPY a subset of columns from a table with RLS enabled would fail due to an invalid query being constructed (using a single ColumnRef with the list of fields to exact in 'fields', but that's for the different levels of an indirection for a single column, not for specifying multiple columns). Correct by building a ColumnRef and then RestTarget for each column being requested and then adding those to the targetList for the select query. Include regression tests to hopefully catch if this is broken again in the future. Patch-By: Adam Brightwell Reviewed-By: Michael Paquier Branch -- REL9_6_STABLE Details --- http://git.postgresql.org/pg/commitdiff/190765a059cc5e8226cdb18b9715741b32b12255 Modified Files -- src/backend/commands/copy.c | 63 -- src/test/regress/expected/copy2.out | 78 + src/test/regress/sql/copy2.sql | 63 ++ 3 files changed, 192 insertions(+), 12 deletions(-) -- Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-committers
[COMMITTERS] pgsql: Enforce a specific order for probing library loadability in pg_u
Enforce a specific order for probing library loadability in pg_upgrade. pg_upgrade checks whether all the shared libraries used in the old cluster are also available in the new one by issuing LOAD for each library name. Previously, it cared not what order it did the LOADs in. Ideally it should not have to care, but currently the transform modules in contrib fail unless both the language and datatype modules they depend on are loaded first. A backend-side solution for that looks possible but probably not back-patchable, so as a stopgap measure, let's do the LOAD tests in order by library name length. That should fix the problem for reasonably-named transform modules, eg "hstore_plpython" will be loaded after both "hstore" and "plpython". (Yeah, it's a hack.) In a larger sense, having a predictable order of these probes is a good thing, since it will make upgrades predictably work or not work in the face of inter-library dependencies. Also, this patch replaces O(N^2) de-duplication logic with O(N log N) logic, which could matter in installations with very many databases. So I don't foresee reverting this even after we have a proper fix for the library-dependency problem. In passing, improve a couple of SQL queries used here. Per complaint from Andrew Dunstan that pg_upgrade'ing the transform contrib modules failed. Back-patch to 9.5 where transform modules were introduced. Discussion: Branch -- master Details --- http://git.postgresql.org/pg/commitdiff/83c2492002162bf79d2a0811bff5724e395909d7 Modified Files -- src/bin/pg_upgrade/function.c | 97 ++- 1 file changed, 67 insertions(+), 30 deletions(-) -- Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-committers
[COMMITTERS] pgsql: Enforce a specific order for probing library loadability in pg_u
Enforce a specific order for probing library loadability in pg_upgrade. pg_upgrade checks whether all the shared libraries used in the old cluster are also available in the new one by issuing LOAD for each library name. Previously, it cared not what order it did the LOADs in. Ideally it should not have to care, but currently the transform modules in contrib fail unless both the language and datatype modules they depend on are loaded first. A backend-side solution for that looks possible but probably not back-patchable, so as a stopgap measure, let's do the LOAD tests in order by library name length. That should fix the problem for reasonably-named transform modules, eg "hstore_plpython" will be loaded after both "hstore" and "plpython". (Yeah, it's a hack.) In a larger sense, having a predictable order of these probes is a good thing, since it will make upgrades predictably work or not work in the face of inter-library dependencies. Also, this patch replaces O(N^2) de-duplication logic with O(N log N) logic, which could matter in installations with very many databases. So I don't foresee reverting this even after we have a proper fix for the library-dependency problem. In passing, improve a couple of SQL queries used here. Per complaint from Andrew Dunstan that pg_upgrade'ing the transform contrib modules failed. Back-patch to 9.5 where transform modules were introduced. Discussion: Branch -- REL9_6_STABLE Details --- http://git.postgresql.org/pg/commitdiff/bac56dbe090670408551bd68690e91f7b0207ccf Modified Files -- src/bin/pg_upgrade/function.c | 97 ++- 1 file changed, 67 insertions(+), 30 deletions(-) -- Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-committers
[COMMITTERS] pgsql: Enforce a specific order for probing library loadability in pg_u
Enforce a specific order for probing library loadability in pg_upgrade. pg_upgrade checks whether all the shared libraries used in the old cluster are also available in the new one by issuing LOAD for each library name. Previously, it cared not what order it did the LOADs in. Ideally it should not have to care, but currently the transform modules in contrib fail unless both the language and datatype modules they depend on are loaded first. A backend-side solution for that looks possible but probably not back-patchable, so as a stopgap measure, let's do the LOAD tests in order by library name length. That should fix the problem for reasonably-named transform modules, eg "hstore_plpython" will be loaded after both "hstore" and "plpython". (Yeah, it's a hack.) In a larger sense, having a predictable order of these probes is a good thing, since it will make upgrades predictably work or not work in the face of inter-library dependencies. Also, this patch replaces O(N^2) de-duplication logic with O(N log N) logic, which could matter in installations with very many databases. So I don't foresee reverting this even after we have a proper fix for the library-dependency problem. In passing, improve a couple of SQL queries used here. Per complaint from Andrew Dunstan that pg_upgrade'ing the transform contrib modules failed. Back-patch to 9.5 where transform modules were introduced. Discussion: Branch -- REL9_5_STABLE Details --- http://git.postgresql.org/pg/commitdiff/0f259bd178f64b8e351a62d5479e3565e84b956e Modified Files -- src/bin/pg_upgrade/function.c | 97 ++- 1 file changed, 67 insertions(+), 30 deletions(-) -- Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-committers
[COMMITTERS] pgsql: Change the way pre-reading in external sort's merge phase works.
Change the way pre-reading in external sort's merge phase works. Don't pre-read tuples into SortTuple slots during merge. Instead, use the memory for larger read buffers in logtape.c. We're doing the same number of READTUP() calls either way, but managing the pre-read SortTuple slots is much more complicated. Also, the on-tape representation is more compact than SortTuples, so we can fit more pre-read tuples into the same amount of memory this way. And we have better cache-locality, when we use just a small number of SortTuple slots. Now that we only hold one tuple from each tape in the SortTuple slots, we can greatly simplify the "batch memory" management. We now maintain a small set of fixed-sized slots, to hold the tuples, and fall back to palloc() for larger tuples. We use this method during all merge phases, not just the final merge, and also when randomAccess is requested, and also in the TSS_SORTEDONTAPE case. In other words, it's used whenever we do an external sort. Reviewed by Peter Geoghegan and Claudio Freire. Discussion: Branch -- master Details --- http://git.postgresql.org/pg/commitdiff/e94568ecc10f2638e542ae34f2990b821bbf90ac Modified Files -- src/backend/utils/sort/logtape.c | 153 - src/backend/utils/sort/tuplesort.c | 1216 +--- src/include/utils/logtape.h|2 + 3 files changed, 574 insertions(+), 797 deletions(-) -- Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-committers
