pgsql: Fix parallel-safety check of expressions and predicate for index
Fix parallel-safety check of expressions and predicate for index builds As coded, the planner logic that calculates the number of parallel workers to use for a parallel index build uses expressions and predicates from the relcache, which are flattened for the planner by eval_const_expressions(). As reported in the bug, an immutable parallel-unsafe function flattened in the relcache would become a Const, which would be considered as parallel-safe, even if the predicate or the expressions including the function are not safe in parallel workers. Depending on the expressions or predicate used, this could cause the parallel build to fail. Tests are included that check parallel index builds with parallel-unsafe predicate and expressions. Two routines are added to lsyscache.h to be able to retrieve expressions and predicate of an index from its pg_index data. Reported-by: Alexander Lakhin Author: Tender Wang Reviewed-by: Jian He, Michael Paquier Discussion: https://postgr.es/m/CAHewXN=UaAaNn9ruHDH3Os8kxLVmtWqbssnf=dZN_s9=evh...@mail.gmail.com Backpatch-through: 12 Branch -- REL_15_STABLE Details --- https://git.postgresql.org/pg/commitdiff/50b5913a87c9c2d67330d82429f6dda61441d40c Modified Files -- src/backend/optimizer/plan/planner.c | 12 +- src/backend/utils/cache/lsyscache.c | 68 +++ src/include/utils/lsyscache.h | 2 + src/test/regress/expected/btree_index.out | 19 + src/test/regress/sql/btree_index.sql | 22 ++ 5 files changed, 121 insertions(+), 2 deletions(-)
pgsql: Fix parallel-safety check of expressions and predicate for index
Fix parallel-safety check of expressions and predicate for index builds As coded, the planner logic that calculates the number of parallel workers to use for a parallel index build uses expressions and predicates from the relcache, which are flattened for the planner by eval_const_expressions(). As reported in the bug, an immutable parallel-unsafe function flattened in the relcache would become a Const, which would be considered as parallel-safe, even if the predicate or the expressions including the function are not safe in parallel workers. Depending on the expressions or predicate used, this could cause the parallel build to fail. Tests are included that check parallel index builds with parallel-unsafe predicate and expressions. Two routines are added to lsyscache.h to be able to retrieve expressions and predicate of an index from its pg_index data. Reported-by: Alexander Lakhin Author: Tender Wang Reviewed-by: Jian He, Michael Paquier Discussion: https://postgr.es/m/CAHewXN=UaAaNn9ruHDH3Os8kxLVmtWqbssnf=dZN_s9=evh...@mail.gmail.com Backpatch-through: 12 Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/eae7be600be715b2f393b018fc4b98c5b89296da Modified Files -- src/backend/optimizer/plan/planner.c | 12 +- src/backend/utils/cache/lsyscache.c | 68 +++ src/include/utils/lsyscache.h | 2 + src/test/regress/expected/btree_index.out | 19 + src/test/regress/sql/btree_index.sql | 22 ++ 5 files changed, 121 insertions(+), 2 deletions(-)
pgsql: Fix parallel-safety check of expressions and predicate for index
Fix parallel-safety check of expressions and predicate for index builds As coded, the planner logic that calculates the number of parallel workers to use for a parallel index build uses expressions and predicates from the relcache, which are flattened for the planner by eval_const_expressions(). As reported in the bug, an immutable parallel-unsafe function flattened in the relcache would become a Const, which would be considered as parallel-safe, even if the predicate or the expressions including the function are not safe in parallel workers. Depending on the expressions or predicate used, this could cause the parallel build to fail. Tests are included that check parallel index builds with parallel-unsafe predicate and expressions. Two routines are added to lsyscache.h to be able to retrieve expressions and predicate of an index from its pg_index data. Reported-by: Alexander Lakhin Author: Tender Wang Reviewed-by: Jian He, Michael Paquier Discussion: https://postgr.es/m/CAHewXN=UaAaNn9ruHDH3Os8kxLVmtWqbssnf=dZN_s9=evh...@mail.gmail.com Backpatch-through: 12 Branch -- REL_13_STABLE Details --- https://git.postgresql.org/pg/commitdiff/b60d71c202cde344e0b447c2bb301addf95fa3b5 Modified Files -- src/backend/optimizer/plan/planner.c | 12 +- src/backend/utils/cache/lsyscache.c | 68 +++ src/include/utils/lsyscache.h | 2 + src/test/regress/expected/btree_index.out | 19 + src/test/regress/sql/btree_index.sql | 22 ++ 5 files changed, 121 insertions(+), 2 deletions(-)
pgsql: Fix parallel-safety check of expressions and predicate for index
Fix parallel-safety check of expressions and predicate for index builds As coded, the planner logic that calculates the number of parallel workers to use for a parallel index build uses expressions and predicates from the relcache, which are flattened for the planner by eval_const_expressions(). As reported in the bug, an immutable parallel-unsafe function flattened in the relcache would become a Const, which would be considered as parallel-safe, even if the predicate or the expressions including the function are not safe in parallel workers. Depending on the expressions or predicate used, this could cause the parallel build to fail. Tests are included that check parallel index builds with parallel-unsafe predicate and expressions. Two routines are added to lsyscache.h to be able to retrieve expressions and predicate of an index from its pg_index data. Reported-by: Alexander Lakhin Author: Tender Wang Reviewed-by: Jian He, Michael Paquier Discussion: https://postgr.es/m/CAHewXN=UaAaNn9ruHDH3Os8kxLVmtWqbssnf=dZN_s9=evh...@mail.gmail.com Backpatch-through: 12 Branch -- REL_12_STABLE Details --- https://git.postgresql.org/pg/commitdiff/c0e0174c216bedcd3cfeb6b0bb5eead50713c772 Modified Files -- src/backend/optimizer/plan/planner.c | 12 +- src/backend/utils/cache/lsyscache.c | 68 +++ src/include/utils/lsyscache.h | 2 + src/test/regress/expected/btree_index.out | 19 + src/test/regress/sql/btree_index.sql | 22 ++ 5 files changed, 121 insertions(+), 2 deletions(-)
pgsql: Fix parallel-safety check of expressions and predicate for index
Fix parallel-safety check of expressions and predicate for index builds As coded, the planner logic that calculates the number of parallel workers to use for a parallel index build uses expressions and predicates from the relcache, which are flattened for the planner by eval_const_expressions(). As reported in the bug, an immutable parallel-unsafe function flattened in the relcache would become a Const, which would be considered as parallel-safe, even if the predicate or the expressions including the function are not safe in parallel workers. Depending on the expressions or predicate used, this could cause the parallel build to fail. Tests are included that check parallel index builds with parallel-unsafe predicate and expressions. Two routines are added to lsyscache.h to be able to retrieve expressions and predicate of an index from its pg_index data. Reported-by: Alexander Lakhin Author: Tender Wang Reviewed-by: Jian He, Michael Paquier Discussion: https://postgr.es/m/CAHewXN=UaAaNn9ruHDH3Os8kxLVmtWqbssnf=dZN_s9=evh...@mail.gmail.com Backpatch-through: 12 Branch -- REL_14_STABLE Details --- https://git.postgresql.org/pg/commitdiff/56a8ab2fc6f76ec2ca7a257f36ef8138a3fd473a Modified Files -- src/backend/optimizer/plan/planner.c | 12 +- src/backend/utils/cache/lsyscache.c | 68 +++ src/include/utils/lsyscache.h | 2 + src/test/regress/expected/btree_index.out | 19 + src/test/regress/sql/btree_index.sql | 22 ++ 5 files changed, 121 insertions(+), 2 deletions(-)
pgsql: Fix parallel-safety check of expressions and predicate for index
Fix parallel-safety check of expressions and predicate for index builds As coded, the planner logic that calculates the number of parallel workers to use for a parallel index build uses expressions and predicates from the relcache, which are flattened for the planner by eval_const_expressions(). As reported in the bug, an immutable parallel-unsafe function flattened in the relcache would become a Const, which would be considered as parallel-safe, even if the predicate or the expressions including the function are not safe in parallel workers. Depending on the expressions or predicate used, this could cause the parallel build to fail. Tests are included that check parallel index builds with parallel-unsafe predicate and expressions. Two routines are added to lsyscache.h to be able to retrieve expressions and predicate of an index from its pg_index data. Reported-by: Alexander Lakhin Author: Tender Wang Reviewed-by: Jian He, Michael Paquier Discussion: https://postgr.es/m/CAHewXN=UaAaNn9ruHDH3Os8kxLVmtWqbssnf=dZN_s9=evh...@mail.gmail.com Backpatch-through: 12 Branch -- REL_16_STABLE Details --- https://git.postgresql.org/pg/commitdiff/4ec8f7708b6b4c494bd0ccd937fc28d34a872b49 Modified Files -- src/backend/optimizer/plan/planner.c | 12 +- src/backend/utils/cache/lsyscache.c | 68 +++ src/include/utils/lsyscache.h | 2 + src/test/regress/expected/btree_index.out | 19 + src/test/regress/sql/btree_index.sql | 22 ++ 5 files changed, 121 insertions(+), 2 deletions(-)
pgsql: Fix signedness error in 9f225e992 for gcc
Fix signedness error in 9f225e992 for gcc The first argument of vshrq_n_s8 needs to be a signed vector type, but it was passed unsigned. Clang is more lax with conversion, but gcc needs a cast. Fix by me, tested by Masahiko Sawada Per buildfarm members splitfin, batta, widowbird, snakefly, parula, massasauga Discussion: https://postgr.es/m/20240306074106.mg6w4koohdlworbs%40alap3.anarazel.de Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/de7c6fe8347ab726c80ebbfcdb57f4b714d5243d Modified Files -- src/include/port/simd.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
pgsql: Fix type-checking of RECORD-returning functions in FROM.
Fix type-checking of RECORD-returning functions in FROM. In the corner case where a function returning RECORD has been simplified to a RECORD constant or an inlined ROW() expression, ExecInitFunctionScan failed to cross-check the function's result rowtype against the coldeflist provided by the calling query. That happened because get_expr_result_type is able to extract a tupdesc from such expressions, which led ExecInitFunctionScan to ignore the coldeflist. (Instead, it used the extracted tupdesc to check the function's output, which of course always succeeds.) I have not been able to demonstrate any really serious consequences from this, because if some column of the result is of the wrong type and is directly referenced by a Var of the calling query, CheckVarSlotCompatibility will catch it. However, we definitely do fail to report the case where the function returns more columns than the coldeflist expects, and in the converse case where it returns fewer columns, we get an assert failure (but, seemingly, no worse results in non-assert builds). To fix, always build the expected tupdesc from the coldeflist if there is one, and consult get_expr_result_type only when there isn't one. Also remove the failing Assert, even though it is no longer reached after this fix. It doesn't seem to be adding anything useful, since later checking will deal with cases with the wrong number of columns. The only other place I could find that is doing something similar is inline_set_returning_function. There's no live bug there because we cannot be looking at a Const or RowExpr, but for consistency change that code to agree with ExecInitFunctionScan. Per report from PetSerAl. After some debate I've concluded that this should be back-patched. There is a small risk that somebody has been relying on such a case not throwing an error, but I judge this outweighed by the risk that I've missed some way in which the failure to cross-check has worse consequences than sketched above. Discussion: https://postgr.es/m/cakygshsera1exsjhr9wft3gn3wfhq5rfp8xhbzf70_qcrrr...@mail.gmail.com Branch -- REL_15_STABLE Details --- https://git.postgresql.org/pg/commitdiff/3b671dcf53d1e1f5a08f8a4cd204207a8c1a065e Modified Files -- src/backend/executor/nodeFunctionscan.c | 81 ++-- src/backend/optimizer/util/clauses.c | 14 -- src/test/regress/expected/rangefuncs.out | 13 + src/test/regress/sql/rangefuncs.sql | 9 4 files changed, 76 insertions(+), 41 deletions(-)
pgsql: Fix type-checking of RECORD-returning functions in FROM.
Fix type-checking of RECORD-returning functions in FROM. In the corner case where a function returning RECORD has been simplified to a RECORD constant or an inlined ROW() expression, ExecInitFunctionScan failed to cross-check the function's result rowtype against the coldeflist provided by the calling query. That happened because get_expr_result_type is able to extract a tupdesc from such expressions, which led ExecInitFunctionScan to ignore the coldeflist. (Instead, it used the extracted tupdesc to check the function's output, which of course always succeeds.) I have not been able to demonstrate any really serious consequences from this, because if some column of the result is of the wrong type and is directly referenced by a Var of the calling query, CheckVarSlotCompatibility will catch it. However, we definitely do fail to report the case where the function returns more columns than the coldeflist expects, and in the converse case where it returns fewer columns, we get an assert failure (but, seemingly, no worse results in non-assert builds). To fix, always build the expected tupdesc from the coldeflist if there is one, and consult get_expr_result_type only when there isn't one. Also remove the failing Assert, even though it is no longer reached after this fix. It doesn't seem to be adding anything useful, since later checking will deal with cases with the wrong number of columns. The only other place I could find that is doing something similar is inline_set_returning_function. There's no live bug there because we cannot be looking at a Const or RowExpr, but for consistency change that code to agree with ExecInitFunctionScan. Per report from PetSerAl. After some debate I've concluded that this should be back-patched. There is a small risk that somebody has been relying on such a case not throwing an error, but I judge this outweighed by the risk that I've missed some way in which the failure to cross-check has worse consequences than sketched above. Discussion: https://postgr.es/m/cakygshsera1exsjhr9wft3gn3wfhq5rfp8xhbzf70_qcrrr...@mail.gmail.com Branch -- REL_12_STABLE Details --- https://git.postgresql.org/pg/commitdiff/466376c9f848645a7391e9f1b19faca5bcab1360 Modified Files -- src/backend/executor/nodeFunctionscan.c | 81 ++-- src/test/regress/expected/rangefuncs.out | 13 + src/test/regress/sql/rangefuncs.sql | 9 3 files changed, 67 insertions(+), 36 deletions(-)
pgsql: Fix type-checking of RECORD-returning functions in FROM.
Fix type-checking of RECORD-returning functions in FROM. In the corner case where a function returning RECORD has been simplified to a RECORD constant or an inlined ROW() expression, ExecInitFunctionScan failed to cross-check the function's result rowtype against the coldeflist provided by the calling query. That happened because get_expr_result_type is able to extract a tupdesc from such expressions, which led ExecInitFunctionScan to ignore the coldeflist. (Instead, it used the extracted tupdesc to check the function's output, which of course always succeeds.) I have not been able to demonstrate any really serious consequences from this, because if some column of the result is of the wrong type and is directly referenced by a Var of the calling query, CheckVarSlotCompatibility will catch it. However, we definitely do fail to report the case where the function returns more columns than the coldeflist expects, and in the converse case where it returns fewer columns, we get an assert failure (but, seemingly, no worse results in non-assert builds). To fix, always build the expected tupdesc from the coldeflist if there is one, and consult get_expr_result_type only when there isn't one. Also remove the failing Assert, even though it is no longer reached after this fix. It doesn't seem to be adding anything useful, since later checking will deal with cases with the wrong number of columns. The only other place I could find that is doing something similar is inline_set_returning_function. There's no live bug there because we cannot be looking at a Const or RowExpr, but for consistency change that code to agree with ExecInitFunctionScan. Per report from PetSerAl. After some debate I've concluded that this should be back-patched. There is a small risk that somebody has been relying on such a case not throwing an error, but I judge this outweighed by the risk that I've missed some way in which the failure to cross-check has worse consequences than sketched above. Discussion: https://postgr.es/m/cakygshsera1exsjhr9wft3gn3wfhq5rfp8xhbzf70_qcrrr...@mail.gmail.com Branch -- REL_16_STABLE Details --- https://git.postgresql.org/pg/commitdiff/1b3029be5df0b0c9abd72fad13283d6d5f4c6a9f Modified Files -- src/backend/executor/nodeFunctionscan.c | 81 ++-- src/backend/optimizer/util/clauses.c | 14 -- src/test/regress/expected/rangefuncs.out | 13 + src/test/regress/sql/rangefuncs.sql | 9 4 files changed, 76 insertions(+), 41 deletions(-)
pgsql: Fix type-checking of RECORD-returning functions in FROM.
Fix type-checking of RECORD-returning functions in FROM. In the corner case where a function returning RECORD has been simplified to a RECORD constant or an inlined ROW() expression, ExecInitFunctionScan failed to cross-check the function's result rowtype against the coldeflist provided by the calling query. That happened because get_expr_result_type is able to extract a tupdesc from such expressions, which led ExecInitFunctionScan to ignore the coldeflist. (Instead, it used the extracted tupdesc to check the function's output, which of course always succeeds.) I have not been able to demonstrate any really serious consequences from this, because if some column of the result is of the wrong type and is directly referenced by a Var of the calling query, CheckVarSlotCompatibility will catch it. However, we definitely do fail to report the case where the function returns more columns than the coldeflist expects, and in the converse case where it returns fewer columns, we get an assert failure (but, seemingly, no worse results in non-assert builds). To fix, always build the expected tupdesc from the coldeflist if there is one, and consult get_expr_result_type only when there isn't one. Also remove the failing Assert, even though it is no longer reached after this fix. It doesn't seem to be adding anything useful, since later checking will deal with cases with the wrong number of columns. The only other place I could find that is doing something similar is inline_set_returning_function. There's no live bug there because we cannot be looking at a Const or RowExpr, but for consistency change that code to agree with ExecInitFunctionScan. Per report from PetSerAl. After some debate I've concluded that this should be back-patched. There is a small risk that somebody has been relying on such a case not throwing an error, but I judge this outweighed by the risk that I've missed some way in which the failure to cross-check has worse consequences than sketched above. Discussion: https://postgr.es/m/cakygshsera1exsjhr9wft3gn3wfhq5rfp8xhbzf70_qcrrr...@mail.gmail.com Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/2ed8f9a01e74bfc63a56b2108f44237bc5eab9f5 Modified Files -- src/backend/executor/nodeFunctionscan.c | 81 ++-- src/backend/optimizer/util/clauses.c | 14 -- src/test/regress/expected/rangefuncs.out | 13 + src/test/regress/sql/rangefuncs.sql | 9 4 files changed, 76 insertions(+), 41 deletions(-)
pgsql: Fix type-checking of RECORD-returning functions in FROM.
Fix type-checking of RECORD-returning functions in FROM. In the corner case where a function returning RECORD has been simplified to a RECORD constant or an inlined ROW() expression, ExecInitFunctionScan failed to cross-check the function's result rowtype against the coldeflist provided by the calling query. That happened because get_expr_result_type is able to extract a tupdesc from such expressions, which led ExecInitFunctionScan to ignore the coldeflist. (Instead, it used the extracted tupdesc to check the function's output, which of course always succeeds.) I have not been able to demonstrate any really serious consequences from this, because if some column of the result is of the wrong type and is directly referenced by a Var of the calling query, CheckVarSlotCompatibility will catch it. However, we definitely do fail to report the case where the function returns more columns than the coldeflist expects, and in the converse case where it returns fewer columns, we get an assert failure (but, seemingly, no worse results in non-assert builds). To fix, always build the expected tupdesc from the coldeflist if there is one, and consult get_expr_result_type only when there isn't one. Also remove the failing Assert, even though it is no longer reached after this fix. It doesn't seem to be adding anything useful, since later checking will deal with cases with the wrong number of columns. The only other place I could find that is doing something similar is inline_set_returning_function. There's no live bug there because we cannot be looking at a Const or RowExpr, but for consistency change that code to agree with ExecInitFunctionScan. Per report from PetSerAl. After some debate I've concluded that this should be back-patched. There is a small risk that somebody has been relying on such a case not throwing an error, but I judge this outweighed by the risk that I've missed some way in which the failure to cross-check has worse consequences than sketched above. Discussion: https://postgr.es/m/cakygshsera1exsjhr9wft3gn3wfhq5rfp8xhbzf70_qcrrr...@mail.gmail.com Branch -- REL_13_STABLE Details --- https://git.postgresql.org/pg/commitdiff/d769f9d97feb4cd4d3bf485cd3fa80916d1e484f Modified Files -- src/backend/executor/nodeFunctionscan.c | 81 ++-- src/backend/optimizer/util/clauses.c | 14 -- src/test/regress/expected/rangefuncs.out | 13 + src/test/regress/sql/rangefuncs.sql | 9 4 files changed, 76 insertions(+), 41 deletions(-)
pgsql: Fix type-checking of RECORD-returning functions in FROM.
Fix type-checking of RECORD-returning functions in FROM. In the corner case where a function returning RECORD has been simplified to a RECORD constant or an inlined ROW() expression, ExecInitFunctionScan failed to cross-check the function's result rowtype against the coldeflist provided by the calling query. That happened because get_expr_result_type is able to extract a tupdesc from such expressions, which led ExecInitFunctionScan to ignore the coldeflist. (Instead, it used the extracted tupdesc to check the function's output, which of course always succeeds.) I have not been able to demonstrate any really serious consequences from this, because if some column of the result is of the wrong type and is directly referenced by a Var of the calling query, CheckVarSlotCompatibility will catch it. However, we definitely do fail to report the case where the function returns more columns than the coldeflist expects, and in the converse case where it returns fewer columns, we get an assert failure (but, seemingly, no worse results in non-assert builds). To fix, always build the expected tupdesc from the coldeflist if there is one, and consult get_expr_result_type only when there isn't one. Also remove the failing Assert, even though it is no longer reached after this fix. It doesn't seem to be adding anything useful, since later checking will deal with cases with the wrong number of columns. The only other place I could find that is doing something similar is inline_set_returning_function. There's no live bug there because we cannot be looking at a Const or RowExpr, but for consistency change that code to agree with ExecInitFunctionScan. Per report from PetSerAl. After some debate I've concluded that this should be back-patched. There is a small risk that somebody has been relying on such a case not throwing an error, but I judge this outweighed by the risk that I've missed some way in which the failure to cross-check has worse consequences than sketched above. Discussion: https://postgr.es/m/cakygshsera1exsjhr9wft3gn3wfhq5rfp8xhbzf70_qcrrr...@mail.gmail.com Branch -- REL_14_STABLE Details --- https://git.postgresql.org/pg/commitdiff/a595c3075fb4b8ab4211b074485eb0a17062ec7f Modified Files -- src/backend/executor/nodeFunctionscan.c | 81 ++-- src/backend/optimizer/util/clauses.c | 14 -- src/test/regress/expected/rangefuncs.out | 13 + src/test/regress/sql/rangefuncs.sql | 9 4 files changed, 76 insertions(+), 41 deletions(-)
pgsql: Add Unicode property tables.
Add Unicode property tables. Provide functions to test for Unicode properties, such as Alphabetic or Cased. These functions use tables derived from Unicode data files, similar to the tables for Unicode normalization or general category, and those tables can be updated with the 'update-unicode' build target. Use Unicode properties to provide functions to test for regex character classes, like 'punct' or 'alnum'. Infrastructure in preparation for a builtin collation provider, and may also be useful for other callers. Discussion: https://postgr.es/m/ff4c2f2f9c8fc7ca27c1c24ae37ecaeaeaff6b53.camel%40j-davis.com Reviewed-by: Daniel Verite, Peter Eisentraut, Jeremy Schneider Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/ad49994538c57270c244fab4e1de040568066313 Modified Files -- src/common/unicode/Makefile|6 +- src/common/unicode/README | 45 +- src/common/unicode/category_test.c | 222 +- .../unicode/generate-unicode_category_table.pl | 390 ++- src/common/unicode/meson.build |4 +- src/common/unicode_category.c | 318 +- src/include/common/unicode_category.h | 27 +- src/include/common/unicode_category_table.h| 3694 +++- 8 files changed, 4604 insertions(+), 102 deletions(-)
pgsql: Revert "Fix parallel-safety check of expressions and predicate f
Revert "Fix parallel-safety check of expressions and predicate for index builds" This reverts commit eae7be600be7, following a discussion with Tom Lane, due to concerns that this impacts the decisions made by the planner for the number of workers spawned based on the inlining and const-folding of index expressions and predicate for cases that would have worked until this commit. Discussion: https://postgr.es/m/162802.1709746...@sss.pgh.pa.us Backpatch-through: 12 Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/099ca50bd41c1a72fe5a086bb095858ea74c88de Modified Files -- src/backend/optimizer/plan/planner.c | 12 +- src/backend/utils/cache/lsyscache.c | 68 --- src/include/utils/lsyscache.h | 2 - src/test/regress/expected/btree_index.out | 19 - src/test/regress/sql/btree_index.sql | 22 -- 5 files changed, 2 insertions(+), 121 deletions(-)
pgsql: Revert "Fix parallel-safety check of expressions and predicate f
Revert "Fix parallel-safety check of expressions and predicate for index builds" This reverts commit eae7be600be7, following a discussion with Tom Lane, due to concerns that this impacts the decisions made by the planner for the number of workers spawned based on the inlining and const-folding of index expressions and predicate for cases that would have worked until this commit. Discussion: https://postgr.es/m/162802.1709746...@sss.pgh.pa.us Backpatch-through: 12 Branch -- REL_13_STABLE Details --- https://git.postgresql.org/pg/commitdiff/70a31629aede159a99226da3edc4b1c10192cc44 Modified Files -- src/backend/optimizer/plan/planner.c | 12 +- src/backend/utils/cache/lsyscache.c | 68 --- src/include/utils/lsyscache.h | 2 - src/test/regress/expected/btree_index.out | 19 - src/test/regress/sql/btree_index.sql | 22 -- 5 files changed, 2 insertions(+), 121 deletions(-)
pgsql: Revert "Fix parallel-safety check of expressions and predicate f
Revert "Fix parallel-safety check of expressions and predicate for index builds" This reverts commit eae7be600be7, following a discussion with Tom Lane, due to concerns that this impacts the decisions made by the planner for the number of workers spawned based on the inlining and const-folding of index expressions and predicate for cases that would have worked until this commit. Discussion: https://postgr.es/m/162802.1709746...@sss.pgh.pa.us Backpatch-through: 12 Branch -- REL_12_STABLE Details --- https://git.postgresql.org/pg/commitdiff/69e8f9dadb01377b8b757aa892ef00ce377f047e Modified Files -- src/backend/optimizer/plan/planner.c | 12 +- src/backend/utils/cache/lsyscache.c | 68 --- src/include/utils/lsyscache.h | 2 - src/test/regress/expected/btree_index.out | 19 - src/test/regress/sql/btree_index.sql | 22 -- 5 files changed, 2 insertions(+), 121 deletions(-)
pgsql: Revert "Fix parallel-safety check of expressions and predicate f
Revert "Fix parallel-safety check of expressions and predicate for index builds" This reverts commit eae7be600be7, following a discussion with Tom Lane, due to concerns that this impacts the decisions made by the planner for the number of workers spawned based on the inlining and const-folding of index expressions and predicate for cases that would have worked until this commit. Discussion: https://postgr.es/m/162802.1709746...@sss.pgh.pa.us Backpatch-through: 12 Branch -- REL_15_STABLE Details --- https://git.postgresql.org/pg/commitdiff/e82b64a92cf5650e618b26b85ca5c0f090286bb4 Modified Files -- src/backend/optimizer/plan/planner.c | 12 +- src/backend/utils/cache/lsyscache.c | 68 --- src/include/utils/lsyscache.h | 2 - src/test/regress/expected/btree_index.out | 19 - src/test/regress/sql/btree_index.sql | 22 -- 5 files changed, 2 insertions(+), 121 deletions(-)
pgsql: Revert "Fix parallel-safety check of expressions and predicate f
Revert "Fix parallel-safety check of expressions and predicate for index builds" This reverts commit eae7be600be7, following a discussion with Tom Lane, due to concerns that this impacts the decisions made by the planner for the number of workers spawned based on the inlining and const-folding of index expressions and predicate for cases that would have worked until this commit. Discussion: https://postgr.es/m/162802.1709746...@sss.pgh.pa.us Backpatch-through: 12 Branch -- REL_14_STABLE Details --- https://git.postgresql.org/pg/commitdiff/49b971298a9b4271a9d5f91dcc969c5cb1acfbc1 Modified Files -- src/backend/optimizer/plan/planner.c | 12 +- src/backend/utils/cache/lsyscache.c | 68 --- src/include/utils/lsyscache.h | 2 - src/test/regress/expected/btree_index.out | 19 - src/test/regress/sql/btree_index.sql | 22 -- 5 files changed, 2 insertions(+), 121 deletions(-)
pgsql: Revert "Fix parallel-safety check of expressions and predicate f
Revert "Fix parallel-safety check of expressions and predicate for index builds" This reverts commit eae7be600be7, following a discussion with Tom Lane, due to concerns that this impacts the decisions made by the planner for the number of workers spawned based on the inlining and const-folding of index expressions and predicate for cases that would have worked until this commit. Discussion: https://postgr.es/m/162802.1709746...@sss.pgh.pa.us Backpatch-through: 12 Branch -- REL_16_STABLE Details --- https://git.postgresql.org/pg/commitdiff/c46817ee512afa156561cce2dd6dc8440fe98d55 Modified Files -- src/backend/optimizer/plan/planner.c | 12 +- src/backend/utils/cache/lsyscache.c | 68 --- src/include/utils/lsyscache.h | 2 - src/test/regress/expected/btree_index.out | 19 - src/test/regress/sql/btree_index.sql | 22 -- 5 files changed, 2 insertions(+), 121 deletions(-)
pgsql: Revert "Add recovery TAP test for race condition with slot inval
Revert "Add recovery TAP test for race condition with slot invalidations" This reverts commit 08a52ab151ca, due to some sporadic instability in the test. Getting the test right should require some redesign with a second injection point, but let's revert it for now to avoid these issues in the CI as a lot of patches are under discussion in this last commit fest. Per buildfarm members hachi and gokiburi. Discussion: https://postgr.es/m/zekqqhcriqlvp...@paquier.xyz Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/65db0cfb4c036b14520a22dba5a858185b713643 Modified Files -- src/backend/replication/slot.c | 9 -- .../recovery/t/035_standby_logical_decoding.pl | 116 + 2 files changed, 2 insertions(+), 123 deletions(-)
pgsql: Add template for adaptive radix tree
Add template for adaptive radix tree This implements a radix tree data structure based on the design in "The Adaptive Radix Tree: ARTful Indexing for Main-Memory Databases" by Viktor Leis, Alfons Kemper, and ThomasNeumann, 2013. The main technique that makes it adaptive is using several different node types, each with a different capacity of elements, and a different algorithm for accessing them. The nodes start small and grow/shrink as needed. The main advantage over hash tables is efficient sorted iteration and better memory locality when successive keys are lexicographically close together. The implementation currently assumes 64-bit integer keys, and traversing the tree is in general slower than a linear probing hash table, so this is not a general-purpose associative array. The paper describes two other techniques not implemented here, namely "path compression" and "lazy expansion". These can further reduce memory usage and speed up traversal, but the former would add significant complexity and the latter requires storing the full key with the value. We do trivially compress the path when leading bytes of the key are zeros, however. For value storage, we use "combined pointer/value slots", as recommended in the paper. Values of size equal or smaller than the the platform's pointer type are stored in the array of child pointers in the last level node, while larger values are each stored in a separate allocation. This is for now fixed at compile time, but it would be fairly trivial to allow determining at runtime how variable-length values are stored. One innovation in our implementation compared to the ART paper is decoupling the notion of node "size class" from "kind". The size classes within a given node kind have the same underlying type, but a variable capacity for children, so we can introduce additional node sizes with little additional code. To enable different use cases to specialize for different value types and for shared/local memory, we use macro-templatized code generation in the same manner as simplehash.h and sort_template.h. Future commits will use this infrastructure for storing TIDs. Patch by Masahiko Sawada and John Naylor, but a substantial amount of credit is due to Andres Freund, whose proof-of-concept was a valuable source of coding idioms and awareness of performance pitfalls, and who reviewed earlier versions. Discussion: https://postgr.es/m/CAD21AoAfOZvmfR0j8VmZorZjL7RhTiQdVttNuC4W-Shdc2a-AA%40mail.gmail.com Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/ee1b30f128d8f63a5184d2bcf1c48a3efc3fcbf9 Modified Files -- src/backend/utils/mmgr/dsa.c | 13 + src/include/lib/radixtree.h| 3009 src/include/utils/dsa.h|1 + src/test/modules/Makefile |1 + src/test/modules/meson.build |1 + src/test/modules/test_radixtree/.gitignore |4 + src/test/modules/test_radixtree/Makefile | 23 + .../test_radixtree/expected/test_radixtree.out | 41 + src/test/modules/test_radixtree/meson.build| 34 + .../modules/test_radixtree/sql/test_radixtree.sql |7 + .../modules/test_radixtree/test_radixtree--1.0.sql |8 + src/test/modules/test_radixtree/test_radixtree.c | 473 +++ .../modules/test_radixtree/test_radixtree.control |4 + src/tools/pgindent/typedefs.list |1 + 14 files changed, 3620 insertions(+)
Re: pgsql: Add template for adaptive radix tree
John Naylor writes: > This implements a radix tree data structure based on the design in > "The Adaptive Radix Tree: ARTful Indexing for Main-Memory Databases" > by Viktor Leis, Alfons Kemper, and ThomasNeumann, 2013. Hm ... do we know that this is not patent-encumbered technology? Commits citing such specific prior art make me nervous. regards, tom lane
Re: pgsql: Add template for adaptive radix tree
On Thu, Mar 7, 2024 at 1:08 PM Tom Lane wrote: > > John Naylor writes: > > This implements a radix tree data structure based on the design in > > "The Adaptive Radix Tree: ARTful Indexing for Main-Memory Databases" > > by Viktor Leis, Alfons Kemper, and ThomasNeumann, 2013. > > Hm ... do we know that this is not patent-encumbered technology? > Commits citing such specific prior art make me nervous. There are several open source implementations in a variety of languages, so I assumed not.
Re: pgsql: Add template for adaptive radix tree
John Naylor writes: > On Thu, Mar 7, 2024 at 1:08 PM Tom Lane wrote: >> Hm ... do we know that this is not patent-encumbered technology? >> Commits citing such specific prior art make me nervous. > There are several open source implementations in a variety of > languages, so I assumed not. Are any of those implementations used in places that might entice a patent troll to come after them? (If you think Postgres isn't an inviting target for patent trolls, you're wrong. We've avoided getting sued so far, but man this topic scares me.) regards, tom lane
Re: pgsql: Add template for adaptive radix tree
On Thu, Mar 7, 2024 at 1:32 PM Tom Lane wrote: > > John Naylor writes: > > On Thu, Mar 7, 2024 at 1:08 PM Tom Lane wrote: > >> Hm ... do we know that this is not patent-encumbered technology? > >> Commits citing such specific prior art make me nervous. > > > There are several open source implementations in a variety of > > languages, so I assumed not. > > Are any of those implementations used in places that might > entice a patent troll to come after them? > > (If you think Postgres isn't an inviting target for patent > trolls, you're wrong. We've avoided getting sued so far, > but man this topic scares me.) Understandably so. FWIW, the use case proposed by the authors was for secondary indexes within in-memory databases, not as a type of associative array. I'm unable to find patents for the thing itself, but IANAL. I believe I've been in contact with some of the same authors about a different subject, and they seemed open to people trying to implement their ideas (it was a different paper, to be sure, and unfortunately I no longer email account).
pgsql: Fix redefinition of typedefs
Fix redefinition of typedefs Per buildfarm members sifaka and longfin, clang with -Wtypedef-redefinition warns of duplicate typedefs unless building with C11. Follow-up to ee1b30f12. Masahiko Sawada Discussion: https://postgr.es/m/CANWCAZauSg%3DLUbBbXhpeQtBuPifmzQNTYS6O8NsoAPz1zL-Txg%40mail.gmail.com Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/ac234e6377dd8bccc83f256fce043bf4ecf9d1f7 Modified Files -- src/include/lib/radixtree.h | 8 1 file changed, 4 insertions(+), 4 deletions(-)
pgsql: Fix incorrect format specifier for int64
Fix incorrect format specifier for int64 Follow-up to ee1b30f12, per buildfarm member mamba. Discussion: https://postgr.es/m/CANWCAZYwyRMU%2BOTVOjK%3Dno1hm-W3ZQ5vrSFM1MFAaLtLydvwzA%40mail.gmail.com Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/e444ebcb85c0b55b1ccf7bcb785ad2708090a2a2 Modified Files -- src/include/lib/radixtree.h | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-)
