pgsql: Ignore extended statistics for inheritance trees

2022-01-14 Thread Tomas Vondra
Ignore extended statistics for inheritance trees Since commit 859b3003de we only build extended statistics for individual relations, ignoring the child relations. This resolved the issue with updating catalog tuple twice, but we still tried to use the statistics when calculating estimates for the

pgsql: Consider fractional paths in generate_orderedappend_paths

2022-01-12 Thread Tomas Vondra
Consider fractional paths in generate_orderedappend_paths When building append paths, we've been looking only at startup and total costs for the paths. When building fractional paths that may eliminate the cheapest one, because it may be dominated by two separate paths (one for startup, one for

pgsql: Move test for BRIN HOT behavior to stats.sql

2021-12-10 Thread Tomas Vondra
Move test for BRIN HOT behavior to stats.sql The test added by 5753d4ee32 relies on statistics collector, and so it may occasionally fail when the UDP packet gets lost. Some machines may be susceptible to this, probably depending on load etc. Move the test to stats.sql, which is known to already

pgsql: Add bool to btree_gist documentation

2021-12-10 Thread Tomas Vondra
Add bool to btree_gist documentation Commit 57e3c516 added bool opclass to btree_gist, but update the list of data types in docs to reflect this change. Reported-by: Pavel Luzanov Discussion: https://postgr.es/m/CAE2gYzyDKJBZngssR84VGZEN=Ux=v9fv23qfpgo+7-yynkk...@mail.gmail.com Branch --

pgsql: Ignore BRIN indexes when checking for HOT udpates

2021-11-30 Thread Tomas Vondra
on visibility info. This also removes rd_indexattr list, and replaces it with rd_attrsvalid flag. The list was not used anywhere, and a simple flag is sufficient. Patch by Josef Simanek, various fixes and improvements by me. Author: Josef Simanek Reviewed-by: Tomas Vondra, Alvaro Herrera

pgsql: Fix gist_bool_ops to use gbtreekey2

2021-11-07 Thread Tomas Vondra
Fix gist_bool_ops to use gbtreekey2 Commit 57e3c5160b added a new GiST bool opclass, but it used gbtreekey4 to store the data, which left two bytes undefined, as reported by skink, our valgrind animal. There was a bit more confusion, because the opclass also used gbtreekey8 in the definition.

Re: pgsql: Introduce 'bbstreamer' abstraction to modularize pg_basebackup.

2021-11-06 Thread Tomas Vondra
about the MSVC issue. regards -- Tomas Vondra EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company

pgsql: Add bool GiST opclass to btree_gist

2021-11-06 Thread Tomas Vondra
Add bool GiST opclass to btree_gist Adds bool opclass to btree_gist extension, to allow creating GiST indexes on bool columns. GiST indexes on a single bool column don't seem particularly useful, but this allows defining exclusion constraings involving a bool column, for example. Author: Emre

pgsql: Mark mystreamer variable as PG_USED_FOR_ASSERTS_ONLY

2021-11-06 Thread Tomas Vondra
Mark mystreamer variable as PG_USED_FOR_ASSERTS_ONLY Silences warnings about unused variable, when built without asserts. Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/dafcf887daa472b0a49bee7e07042372bc37cee4 Modified Files --

pgsql: Fix handling of NaN values in BRIN minmax multi

2021-11-06 Thread Tomas Vondra
Fix handling of NaN values in BRIN minmax multi When calculating distance between float4/float8 values, we need to be a bit more careful about NaN values in order not to trigger assert. We consider NaN values to be equal (distace 0.0) and in infinite distance from all other values. On builds

pgsql: Fix handling of NaN values in BRIN minmax multi

2021-11-06 Thread Tomas Vondra
Fix handling of NaN values in BRIN minmax multi When calculating distance between float4/float8 values, we need to be a bit more careful about NaN values in order not to trigger assert. We consider NaN values to be equal (distace 0.0) and in infinite distance from all other values. On builds

pgsql: Release memory allocated by dependency_degree

2021-09-23 Thread Tomas Vondra
Release memory allocated by dependency_degree Calculating degree of a functional dependency may allocate a lot of memory - we have released mot of the explicitly allocated memory, but e.g. detoasted varlena values were left behind. That may be an issue, because we consider a lot of dependencies

pgsql: Free memory after building each statistics object

2021-09-23 Thread Tomas Vondra
: Justin Pryzby Reported-by: Justin Pryzby Reviewed-by: Tomas Vondra Backpatch-through: 10 Discussion: https://www.postgresql.org/message-id/20210915200928.GP831%40telsasoft.com Branch -- REL_10_STABLE Details --- https://git.postgresql.org/pg/commitdiff

pgsql: Free memory after building each statistics object

2021-09-23 Thread Tomas Vondra
: Justin Pryzby Reported-by: Justin Pryzby Reviewed-by: Tomas Vondra Backpatch-through: 10 Discussion: https://www.postgresql.org/message-id/20210915200928.GP831%40telsasoft.com Branch -- REL_11_STABLE Details --- https://git.postgresql.org/pg/commitdiff

pgsql: Release memory allocated by dependency_degree

2021-09-23 Thread Tomas Vondra
Release memory allocated by dependency_degree Calculating degree of a functional dependency may allocate a lot of memory - we have released mot of the explicitly allocated memory, but e.g. detoasted varlena values were left behind. That may be an issue, because we consider a lot of dependencies

pgsql: Release memory allocated by dependency_degree

2021-09-23 Thread Tomas Vondra
Release memory allocated by dependency_degree Calculating degree of a functional dependency may allocate a lot of memory - we have released mot of the explicitly allocated memory, but e.g. detoasted varlena values were left behind. That may be an issue, because we consider a lot of dependencies

pgsql: Free memory after building each statistics object

2021-09-23 Thread Tomas Vondra
: Justin Pryzby Reported-by: Justin Pryzby Reviewed-by: Tomas Vondra Backpatch-through: 10 Discussion: https://www.postgresql.org/message-id/20210915200928.GP831%40telsasoft.com Branch -- REL_12_STABLE Details --- https://git.postgresql.org/pg/commitdiff

pgsql: Release memory allocated by dependency_degree

2021-09-23 Thread Tomas Vondra
Release memory allocated by dependency_degree Calculating degree of a functional dependency may allocate a lot of memory - we have released mot of the explicitly allocated memory, but e.g. detoasted varlena values were left behind. That may be an issue, because we consider a lot of dependencies

pgsql: Free memory after building each statistics object

2021-09-23 Thread Tomas Vondra
: Justin Pryzby Reported-by: Justin Pryzby Reviewed-by: Tomas Vondra Backpatch-through: 10 Discussion: https://www.postgresql.org/message-id/20210915200928.GP831%40telsasoft.com Branch -- REL_13_STABLE Details --- https://git.postgresql.org/pg/commitdiff

pgsql: Free memory after building each statistics object

2021-09-23 Thread Tomas Vondra
: Justin Pryzby Reported-by: Justin Pryzby Reviewed-by: Tomas Vondra Backpatch-through: 10 Discussion: https://www.postgresql.org/message-id/20210915200928.GP831%40telsasoft.com Branch -- REL_14_STABLE Details --- https://git.postgresql.org/pg/commitdiff

pgsql: Release memory allocated by dependency_degree

2021-09-23 Thread Tomas Vondra
Release memory allocated by dependency_degree Calculating degree of a functional dependency may allocate a lot of memory - we have released mot of the explicitly allocated memory, but e.g. detoasted varlena values were left behind. That may be an issue, because we consider a lot of dependencies

pgsql: Free memory after building each statistics object

2021-09-23 Thread Tomas Vondra
: Justin Pryzby Reported-by: Justin Pryzby Reviewed-by: Tomas Vondra Backpatch-through: 10 Discussion: https://www.postgresql.org/message-id/20210915200928.GP831%40telsasoft.com Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/83772cc78e0392a247231ba510c61b6612b93b3f

pgsql: Release memory allocated by dependency_degree

2021-09-23 Thread Tomas Vondra
Release memory allocated by dependency_degree Calculating degree of a functional dependency may allocate a lot of memory - we have released mot of the explicitly allocated memory, but e.g. detoasted varlena values were left behind. That may be an issue, because we consider a lot of dependencies

pgsql: Disallow extended statistics on system columns

2021-09-19 Thread Tomas Vondra
Disallow extended statistics on system columns Since introduction of extended statistics, we've disallowed references to system columns. So for example CREATE STATISTICS s ON ctid FROM t; would fail. But with extended statistics on expressions, it was possible to work around this limitation

pgsql: Disallow extended statistics on system columns

2021-09-19 Thread Tomas Vondra
Disallow extended statistics on system columns Since introduction of extended statistics, we've disallowed references to system columns. So for example CREATE STATISTICS s ON ctid FROM t; would fail. But with extended statistics on expressions, it was possible to work around this limitation

pgsql: Identify simple column references in extended statistics

2021-09-01 Thread Tomas Vondra
Identify simple column references in extended statistics Until now, when defining extended statistics, everything except a plain column reference was treated as complex expression. So for example "a" was a column reference, but "(a)" would be an expression. In most cases this does not matter

pgsql: Identify simple column references in extended statistics

2021-09-01 Thread Tomas Vondra
Identify simple column references in extended statistics Until now, when defining extended statistics, everything except a plain column reference was treated as complex expression. So for example "a" was a column reference, but "(a)" would be an expression. In most cases this does not matter

pgsql: Don't print extra parens around expressions in extended stats

2021-08-31 Thread Tomas Vondra
Don't print extra parens around expressions in extended stats The code printing expressions for extended statistics doubled the parens, producing results like ((a+1)), which is unnecessary and not consistent with how we print expressions elsewhere. Fixed by tweaking the code to produce just a

pgsql: Don't print extra parens around expressions in extended stats

2021-08-31 Thread Tomas Vondra
Don't print extra parens around expressions in extended stats The code printing expressions for extended statistics doubled the parens, producing results like ((a+1)), which is unnecessary and not consistent with how we print expressions elsewhere. Fixed by tweaking the code to produce just a

pgsql: Rename the role in stats_ext to have regress_ prefix

2021-08-31 Thread Tomas Vondra
Rename the role in stats_ext to have regress_ prefix Commit 5be8ce82e8 added a new role to the stats_ext regression suite, but the role name did not start with regress_ causing failures when running with ENFORCE_REGRESSION_TEST_NAME_RESTRICTIONS. Fixed by renaming the role to start with the

pgsql: Rename the role in stats_ext to have regress_ prefix

2021-08-31 Thread Tomas Vondra
Rename the role in stats_ext to have regress_ prefix Commit 5be8ce82e8 added a new role to the stats_ext regression suite, but the role name did not start with regress_ causing failures when running with ENFORCE_REGRESSION_TEST_NAME_RESTRICTIONS. Fixed by renaming the role to start with the

pgsql: Rename the role in stats_ext to have regress_ prefix

2021-08-31 Thread Tomas Vondra
Rename the role in stats_ext to have regress_ prefix Commit 5be8ce82e8 added a new role to the stats_ext regression suite, but the role name did not start with regress_ causing failures when running with ENFORCE_REGRESSION_TEST_NAME_RESTRICTIONS. Fixed by renaming the role to start with the

pgsql: Rename the role in stats_ext to have regress_ prefix

2021-08-31 Thread Tomas Vondra
Rename the role in stats_ext to have regress_ prefix Commit 5be8ce82e8 added a new role to the stats_ext regression suite, but the role name did not start with regress_ causing failures when running with ENFORCE_REGRESSION_TEST_NAME_RESTRICTIONS. Fixed by renaming the role to start with the

pgsql: Rename the role in stats_ext to have regress_ prefix

2021-08-31 Thread Tomas Vondra
Rename the role in stats_ext to have regress_ prefix Commit 5be8ce82e8 added a new role to the stats_ext regression suite, but the role name did not start with regress_ causing failures when running with ENFORCE_REGRESSION_TEST_NAME_RESTRICTIONS. Fixed by renaming the role to start with the

pgsql: Rename the role in stats_ext to have regress_ prefix

2021-08-31 Thread Tomas Vondra
Rename the role in stats_ext to have regress_ prefix Commit 5be8ce82e8 added a new role to the stats_ext regression suite, but the role name did not start with regress_ causing failures when running with ENFORCE_REGRESSION_TEST_NAME_RESTRICTIONS. Fixed by renaming the role to start with the

pgsql: Fix lookup error in extended stats ownership check

2021-08-31 Thread Tomas Vondra
Fix lookup error in extended stats ownership check When an ownership check on extended statistics object failed, the code was calling aclcheck_error_type to report the failure, which is clearly wrong, resulting in cache lookup errors. Fix by calling aclcheck_error. This issue exists since the

pgsql: Fix lookup error in extended stats ownership check

2021-08-31 Thread Tomas Vondra
Fix lookup error in extended stats ownership check When an ownership check on extended statistics object failed, the code was calling aclcheck_error_type to report the failure, which is clearly wrong, resulting in cache lookup errors. Fix by calling aclcheck_error. This issue exists since the

pgsql: Fix lookup error in extended stats ownership check

2021-08-31 Thread Tomas Vondra
Fix lookup error in extended stats ownership check When an ownership check on extended statistics object failed, the code was calling aclcheck_error_type to report the failure, which is clearly wrong, resulting in cache lookup errors. Fix by calling aclcheck_error. This issue exists since the

pgsql: Fix lookup error in extended stats ownership check

2021-08-31 Thread Tomas Vondra
Fix lookup error in extended stats ownership check When an ownership check on extended statistics object failed, the code was calling aclcheck_error_type to report the failure, which is clearly wrong, resulting in cache lookup errors. Fix by calling aclcheck_error. This issue exists since the

pgsql: Fix lookup error in extended stats ownership check

2021-08-31 Thread Tomas Vondra
Fix lookup error in extended stats ownership check When an ownership check on extended statistics object failed, the code was calling aclcheck_error_type to report the failure, which is clearly wrong, resulting in cache lookup errors. Fix by calling aclcheck_error. This issue exists since the

pgsql: Fix lookup error in extended stats ownership check

2021-08-31 Thread Tomas Vondra
Fix lookup error in extended stats ownership check When an ownership check on extended statistics object failed, the code was calling aclcheck_error_type to report the failure, which is clearly wrong, resulting in cache lookup errors. Fix by calling aclcheck_error. This issue exists since the

pgsql: Use appropriate tuple descriptor in FDW batching

2021-08-12 Thread Tomas Vondra
Use appropriate tuple descriptor in FDW batching The FDW batching code was using the same tuple descriptor both for all slots (regular and plan slots), but that's incorrect - the subplan may use a different descriptor. Currently this is benign, because batching is used only for INSERTs, and in

pgsql: Use appropriate tuple descriptor in FDW batching

2021-08-12 Thread Tomas Vondra
Use appropriate tuple descriptor in FDW batching The FDW batching code was using the same tuple descriptor both for all slots (regular and plan slots), but that's incorrect - the subplan may use a different descriptor. Currently this is benign, because batching is used only for INSERTs, and in

pgsql: psql \dX: check schema when listing statistics objects

2021-07-26 Thread Tomas Vondra
psql \dX: check schema when listing statistics objects Commit ad600bba04 added psql command \dX listing extended statistics objects, but it failed to consider search_path when selecting the elements so some of the returned elements might be invisible. The visibility was already considered for

pgsql: psql \dX: check schema when listing statistics objects

2021-07-26 Thread Tomas Vondra
psql \dX: check schema when listing statistics objects Commit ad600bba04 added psql command \dX listing extended statistics objects, but it failed to consider search_path when selecting the elements so some of the returned elements might be invisible. The visibility was already considered for

pgsql: Fix copying data into slots with FDW batching

2021-06-16 Thread Tomas Vondra
Fix copying data into slots with FDW batching Commit b676ac443b optimized handling of tuple slots with bulk inserts into foreign tables, so that the slots are initialized only once and reused for all batches. The data was however copied into the slots only after the initialization, inserting

pgsql: Optimize creation of slots for FDW bulk inserts

2021-06-11 Thread Tomas Vondra
Optimize creation of slots for FDW bulk inserts Commit b663a41363 introduced bulk inserts for FDW, but the handling of tuple slots turned out to be problematic for two reasons. Firstly, the slots were re-created for each individual batch. Secondly, all slots referenced the same tuple descriptor -

pgsql: Adjust batch size in postgres_fdw to not use too many parameters

2021-06-08 Thread Tomas Vondra
Adjust batch size in postgres_fdw to not use too many parameters The FE/BE protocol identifies parameters with an Int16 index, which limits the maximum number of parameters per query to 65535. With batching added to postges_fdw this limit is much easier to hit, as the whole batch is essentially a

pgsql: Fix pg_visibility regression failure with CLOBBER_CACHE_ALWAYS

2021-06-08 Thread Tomas Vondra
Fix pg_visibility regression failure with CLOBBER_CACHE_ALWAYS Commit 8e03eb92e9 reverted a bit too much code, reintroducing one of the issues fixed by 39b66a91bd - a page might have been left partially empty after relcache invalidation. Reported-By: Tom Lane Author: Masahiko Sawada Discussion:

pgsql: Add transformed flag to nodes/*funcs.c for CREATE STATISTICS

2021-06-06 Thread Tomas Vondra
Add transformed flag to nodes/*funcs.c for CREATE STATISTICS Commit a4d75c86bf added a new flag, tracking if the statement was processed by transformStatsStmt(), but failed to add this flag to nodes/*funcs.c. Catversion bump, due to adding a flag to copy/equal/out functions. Reported-by: Noah

pgsql: Revert most of 39b66a91bd

2021-06-02 Thread Tomas Vondra
Revert most of 39b66a91bd Reverts most of commit 39b66a91bd, which was found to cause significant regression for REFRESH MATERIALIZED VIEW. This means only rows inserted by heap_multi_insert will benefit from the optimization, implemented in commit 7db0cd2145. Reported-by: Masahiko Sawada

pgsql: Copy the INSERT query in postgres_fdw

2021-05-07 Thread Tomas Vondra
Copy the INSERT query in postgres_fdw When executing the INSERT with batching, we may need to rebuild the query when the batch size changes, in which case we pfree the current string. We must not release the original string, stored in fdw_private, because that may be needed in EXPLAIN ANALYZE. So

pgsql: Make pg_get_statisticsobjdef_expressions return NULL

2021-05-07 Thread Tomas Vondra
Make pg_get_statisticsobjdef_expressions return NULL The usual behavior for functions in ruleutils.c is to return NULL when the object does not exist. pg_get_statisticsobjdef_expressions raised an error instead, so correct that. Reported-by: Justin Pryzby Discussion:

pgsql: Fix typos in comments about extended statistics

2021-05-07 Thread Tomas Vondra
Fix typos in comments about extended statistics Reported-by: Justin Pryzby Discussion: https://postgr.es/m/20210505210947.GA27406%40telsasoft.com Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/93f9af138795a7d12366187de95f4961fb07ed98 Modified Files --

pgsql: Mention statistics objects in maintenance.sgml

2021-05-07 Thread Tomas Vondra
Mention statistics objects in maintenance.sgml The docs mentioned expression indexes as a way to improve selectivity estimates for functions, but we have a second option to improve that by creating extended statistics. So mention that too. Reported-by: Justin Pryzby Discussion:

pgsql: Don't add non-existent pages to bitmap from BRIN

2021-04-07 Thread Tomas Vondra
Don't add non-existent pages to bitmap from BRIN The code in bringetbitmap() simply added the whole matching page range to the TID bitmap, as determined by pages_per_range, even if some of the pages were beyond the end of the heap. The query then might fail with an error like this: ERROR:

pgsql: Don't add non-existent pages to bitmap from BRIN

2021-04-07 Thread Tomas Vondra
Don't add non-existent pages to bitmap from BRIN The code in bringetbitmap() simply added the whole matching page range to the TID bitmap, as determined by pages_per_range, even if some of the pages were beyond the end of the heap. The query then might fail with an error like this: ERROR:

pgsql: Don't add non-existent pages to bitmap from BRIN

2021-04-07 Thread Tomas Vondra
Don't add non-existent pages to bitmap from BRIN The code in bringetbitmap() simply added the whole matching page range to the TID bitmap, as determined by pages_per_range, even if some of the pages were beyond the end of the heap. The query then might fail with an error like this: ERROR:

pgsql: Don't add non-existent pages to bitmap from BRIN

2021-04-07 Thread Tomas Vondra
Don't add non-existent pages to bitmap from BRIN The code in bringetbitmap() simply added the whole matching page range to the TID bitmap, as determined by pages_per_range, even if some of the pages were beyond the end of the heap. The query then might fail with an error like this: ERROR:

pgsql: Don't add non-existent pages to bitmap from BRIN

2021-04-07 Thread Tomas Vondra
Don't add non-existent pages to bitmap from BRIN The code in bringetbitmap() simply added the whole matching page range to the TID bitmap, as determined by pages_per_range, even if some of the pages were beyond the end of the heap. The query then might fail with an error like this: ERROR:

pgsql: Don't add non-existent pages to bitmap from BRIN

2021-04-07 Thread Tomas Vondra
Don't add non-existent pages to bitmap from BRIN The code in bringetbitmap() simply added the whole matching page range to the TID bitmap, as determined by pages_per_range, even if some of the pages were beyond the end of the heap. The query then might fail with an error like this: ERROR:

Re: pgsql: Extended statistics on expressions

2021-04-06 Thread Tomas Vondra
On 4/1/21 4:24 AM, David Rowley wrote: > On Thu, 1 Apr 2021 at 14:49, Tomas Vondra > wrote: >> I think that's probably sufficient for now. More testing may be useful >> once I refactor the code a bit to make it more readable. > > Just to let you know, I left it runnin

pgsql: Fix handling of clauses incompatible with extended statistics

2021-04-06 Thread Tomas Vondra
Fix handling of clauses incompatible with extended statistics Handling of incompatible clauses while applying extended statistics was a bit confused - while handling a mix of compatible and incompatible clauses it sometimes incorrectly treated the incompatible clauses as compatible, resulting in

pgsql: Fix bug in brin_minmax_multi_union

2021-04-04 Thread Tomas Vondra
Fix bug in brin_minmax_multi_union When calling sort_expanded_ranges() we need to remember the return value, because the function sorts and also deduplicates the ranges. So the number of ranges may decrease. brin_minmax_multi_union failed to do that, which resulted in crashes due to bogus ranges

pgsql: Add regression test for minmax-multi macaddr8 type

2021-04-04 Thread Tomas Vondra
Add regression test for minmax-multi macaddr8 type The regression test for BRIN minmax-multi opclasses tested almost all supported data types, with the exception of macaddr8. So this adds it. Branch -- master Details ---

pgsql: Fix BRIN minmax-multi distance for inet type

2021-04-04 Thread Tomas Vondra
Fix BRIN minmax-multi distance for inet type The distance calculation ignored the mask, unlike the inet comparator, which resulted in negative distance in some cases. Fixed by applying the mask in brin_minmax_multi_distance_inet. I've considered simply calling inetmi() to calculate the delta, but

pgsql: Fix BRIN minmax-multi distance for timetz type

2021-04-04 Thread Tomas Vondra
Fix BRIN minmax-multi distance for timetz type The distance calculation ignored the time zone, so the result of (b-a) might have ended negative even if (b > a). Fixed by considering the time zone difference. Reported-by: Jaime Casanova Discussion:

pgsql: Fix BRIN minmax-multi distance for interval type

2021-04-04 Thread Tomas Vondra
Fix BRIN minmax-multi distance for interval type The distance calculation for interval type was treating months as having 31 days, which is inconsistent with the interval comparator (using 30 days). Due to this it was possible to get negative distance (b-a) when

Re: pgsql: Extended statistics on expressions

2021-03-31 Thread Tomas Vondra
On 4/1/21 3:43 AM, David Rowley wrote: > On Thu, 1 Apr 2021 at 11:07, Tomas Vondra > wrote: >> The attached patch fixes this for me. David, can you check if this >> resolves the issue for you? > > Thanks for taking a look at this. > > I applied your patch a

Re: pgsql: Add Result Cache executor node

2021-03-31 Thread Tomas Vondra
worker. Not sure why some of the numbers were not replaced by N, though. regards -- Tomas Vondra EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company

Re: pgsql: Extended statistics on expressions

2021-03-31 Thread Tomas Vondra
On 3/31/21 7:54 PM, Tomas Vondra wrote: > On 3/31/21 7:08 PM, Alvaro Herrera wrote: >> On 2021-Mar-31, Tomas Vondra wrote: >> >>> Thanks for the report, I'll take a look. You're right this seems like an >>> out-of-bounds access, but mcv_match_expressi

Re: pgsql: Extended statistics on expressions

2021-03-31 Thread Tomas Vondra
On 3/31/21 7:08 PM, Alvaro Herrera wrote: > On 2021-Mar-31, Tomas Vondra wrote: > >> Thanks for the report, I'll take a look. You're right this seems like an >> out-of-bounds access, but mcv_match_expression is only expected to be >> run on expressions we know are in th

Re: pgsql: Extended statistics on expressions

2021-03-31 Thread Tomas Vondra
h not finding > anything since it checks idx >= bms_num_members(keys), which is 2. > > I'm not quite sure how this is all meant to work. Are you able to look > further? > Thanks for the report, I'll take a look. You're right this seems like an out-of-bounds access, but mcv_match_expression is only expected to be run on expressions we know are in the statistics (because we pick the statistics like that). Clearly, that does not happen here, not sure why. It's quite weird that we end up running textlike(), when the statistics is on (double precision, boolean) columns ... regards -- Tomas Vondra EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company

pgsql: Stabilize stats_ext test with other collations

2021-03-27 Thread Tomas Vondra
Stabilize stats_ext test with other collations The tests used string concatenation to test statistics on expressions, but that made the tests locale-dependent, e.g. because the ordering of '11' and '1X' depends on the collation. This affected both the estimated and actual row couts, breaking some

pgsql: Extended statistics on expressions

2021-03-26 Thread Tomas Vondra
at least the functional dependencies. This should not be a major issue in practice, as the functional dependencies actually rely on per-column statistics, which were always reset anyway. Author: Tomas Vondra Reviewed-by: Justin Pryzby, Dean Rasheed, Zhihong Yu Discussion: https://postgr.es/m

pgsql: Reduce duration of stats_ext regression tests

2021-03-26 Thread Tomas Vondra
. So reduce the size of some of the tests that will be used to test expressions, to keep the duration under control. Done in a separate commit before adding the statistics on expressions, to make it clear which estimates are expected to change. Author: Tomas Vondra Discussion: https://postgr.es/m

pgsql: Fix ndistinct estimates with system attributes

2021-03-26 Thread Tomas Vondra
to pg_class.reltuples) than SELECT 1 FROM t GROUP BY a, b, ctid; with extended statistics on (a,b). Fixed by retaining information about the system attribute. Backpatch all the way to 10, where extended statistics were introduced. Author: Tomas Vondra Backpatch-through: 10 Branch -- REL_10_STABLE

pgsql: Fix ndistinct estimates with system attributes

2021-03-26 Thread Tomas Vondra
to pg_class.reltuples) than SELECT 1 FROM t GROUP BY a, b, ctid; with extended statistics on (a,b). Fixed by retaining information about the system attribute. Backpatch all the way to 10, where extended statistics were introduced. Author: Tomas Vondra Backpatch-through: 10 Branch -- REL_11_STABLE

pgsql: Fix ndistinct estimates with system attributes

2021-03-26 Thread Tomas Vondra
to pg_class.reltuples) than SELECT 1 FROM t GROUP BY a, b, ctid; with extended statistics on (a,b). Fixed by retaining information about the system attribute. Backpatch all the way to 10, where extended statistics were introduced. Author: Tomas Vondra Backpatch-through: 10 Branch -- REL_12_STABLE

pgsql: Fix ndistinct estimates with system attributes

2021-03-26 Thread Tomas Vondra
to pg_class.reltuples) than SELECT 1 FROM t GROUP BY a, b, ctid; with extended statistics on (a,b). Fixed by retaining information about the system attribute. Backpatch all the way to 10, where extended statistics were introduced. Author: Tomas Vondra Backpatch-through: 10 Branch -- REL_13_STABLE

pgsql: Fix ndistinct estimates with system attributes

2021-03-26 Thread Tomas Vondra
to pg_class.reltuples) than SELECT 1 FROM t GROUP BY a, b, ctid; with extended statistics on (a,b). Fixed by retaining information about the system attribute. Backpatch all the way to 10, where extended statistics were introduced. Author: Tomas Vondra Backpatch-through: 10 Branch -- master

pgsql: Fix alignment in BRIN minmax-multi deserialization

2021-03-26 Thread Tomas Vondra
Fix alignment in BRIN minmax-multi deserialization The deserialization failed to ensure correct alignment, as it assumed it can simply point into the serialized value. The serialization however ignores alignment and copies just the significant bytes in order to make the result as small as

pgsql: BRIN bloom indexes

2021-03-26 Thread Tomas Vondra
, but compute a 32-bit hash first, and the Bloom filter is built on the hash value. Collisions should not be a huge issue though, as the number of distinct values in a page ranges is usually fairly small. Bump catversion, due to various catalog changes. Author: Tomas Vondra Reviewed-by: Alvaro Herrera

pgsql: Support the old signature of BRIN consistent function

2021-03-26 Thread Tomas Vondra
make backpatching simpler. Catversion bump, because of changes in pg_amproc. Author: Tomas Vondra Author: Nikita Glukhov Reviewed-by: Mark Dilger Reviewed-by: Alexander Korotkov Reviewed-by: Masahiko Sawada Reviewed-by: John Naylor Discussion: https://postgr.es/m/c1138ead-7668-f0e1-0638

pgsql: Remove unnecessary pg_amproc BRIN minmax entries

2021-03-26 Thread Tomas Vondra
equivalent BRIN operator families in an extension. Catversion bump, because of pg_amproc changes. Author: Tomas Vondra Reviewed-by: Alvaro Herrera Discussion: https://postgr.es/m/78c357ab-3395-8433-e7b3-b2cfcc9fdc23%40enterprisedb.com Branch -- master Details --- https://git.postgresql.org/pg

pgsql: Allow composite types in catalog bootstrap

2021-03-23 Thread Tomas Vondra
Allow composite types in catalog bootstrap When resolving types during catalog bootstrap, try to reload the pg_type contents if a type is not found. That allows catalogs to contain composite types, e.g. row types for other catalogs. Author: Justin Pryzby Reviewed-by: Dean Rasheed, Tomas Vondra

pgsql: Convert Typ from array to list in bootstrap

2021-03-23 Thread Tomas Vondra
Convert Typ from array to list in bootstrap It's a bit easier and more convenient to free and reload a List, compared to a plain array. This will be helpful when allowing catalogs to contain composite types. Author: Justin Pryzby Reviewed-by: Dean Rasheed, Tomas Vondra Discussion: https

pgsql: Use correct spelling of statistics kind

2021-03-22 Thread Tomas Vondra
Use correct spelling of statistics kind A couple error messages and comments used 'statistic kind', not the correct 'statistics kind'. Fix and backpatch all the way back to 10, where extended statistics were introduced. Backpatch-through: 10 Branch -- master Details ---

pgsql: Use correct spelling of statistics kind

2021-03-22 Thread Tomas Vondra
Use correct spelling of statistics kind A couple error messages and comments used 'statistic kind', not the correct 'statistics kind'. Fix and backpatch all the way back to 10, where extended statistics were introduced. Backpatch-through: 10 Branch -- REL_13_STABLE Details ---

pgsql: Use correct spelling of statistics kind

2021-03-22 Thread Tomas Vondra
Use correct spelling of statistics kind A couple error messages and comments used 'statistic kind', not the correct 'statistics kind'. Fix and backpatch all the way back to 10, where extended statistics were introduced. Backpatch-through: 10 Branch -- REL_12_STABLE Details ---

pgsql: Use correct spelling of statistics kind

2021-03-22 Thread Tomas Vondra
Use correct spelling of statistics kind A couple error messages and comments used 'statistic kind', not the correct 'statistics kind'. Fix and backpatch all the way back to 10, where extended statistics were introduced. Backpatch-through: 10 Branch -- REL_11_STABLE Details ---

pgsql: Use correct spelling of statistics kind

2021-03-22 Thread Tomas Vondra
Use correct spelling of statistics kind A couple error messages and comments used 'statistic kind', not the correct 'statistics kind'. Fix and backpack all the way back to 10, where extended statistics were introduced. Backpatch-through: 10 Branch -- REL_10_STABLE Details ---

pgsql: Pass all scan keys to BRIN consistent function at once

2021-03-22 Thread Tomas Vondra
. Bump catversion, because of pg_proc changes. Author: Tomas Vondra Reviewed-by: Alvaro Herrera Reviewed-by: Mark Dilger Reviewed-by: Alexander Korotkov Reviewed-by: John Naylor Reviewed-by: Nikita Glukhov Discussion: https://postgr.es/m/c1138ead-7668-f0e1-0638-c3be3237e...@2ndquadrant.com

pgsql: Move bsearch_arg to src/port

2021-03-22 Thread Tomas Vondra
Move bsearch_arg to src/port Until now the bsearch_arg function was used only in extended statistics code, so it was defined in that code. But we already have qsort_arg in src/port, so let's move it next to it. Branch -- master Details ---

pgsql: Optimize allocations in bringetbitmap

2021-03-22 Thread Tomas Vondra
the pieces have the same lifespan, and it saves quite a bit of CPU and memory overhead. Author: Tomas Vondra Reviewed-by: Alvaro Herrera Reviewed-by: Mark Dilger Reviewed-by: Alexander Korotkov Reviewed-by: Masahiko Sawada Reviewed-by: John Naylor Discussion: https://postgr.es/m/c1138ead-7668

pgsql: Move IS [NOT] NULL handling from BRIN support functions

2021-03-22 Thread Tomas Vondra
the support procedures - quite a bit, as they don't need to care about NULL values and flags at all. It also means the IS [NOT] NULL clauses can be evaluated without invoking the support procedure. Author: Tomas Vondra Author: Nikita Glukhov Reviewed-by: Nikita Glukhov Reviewed-by: Mark Dilger

pgsql: Use valid compression method in brin_form_tuple

2021-03-20 Thread Tomas Vondra
the default one. Author: Tomas Vondra Reviewed-by: Justin Pryzby Discussion: https://postgr.es/m/e0367f27-392c-321a-7411-a58e1a7e4817%40enterprisedb.com Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/882b2cdc08c4100e273f24742e2118be98708a07 Modified Files

pgsql: Use lfirst_int in cmp_list_len_contents_asc

2021-03-19 Thread Tomas Vondra
Use lfirst_int in cmp_list_len_contents_asc The function added in be45be9c33 is comparing integer lists (IntList) by length and contents, but there were two bugs. Firstly, it used intVal() to extract the value, but that's for Value nodes, not for extracting int values from IntList. Secondly, it

pgsql: Fix race condition in remove_temp_files_after_crash TAP test

2021-03-19 Thread Tomas Vondra
Fix race condition in remove_temp_files_after_crash TAP test The TAP test was written so that it was not waiting for the correct SQL command, but for output from the preceding one. This resulted in race conditions, allowing the commands to run in a different order, not block as expected and so

pgsql: Fix TAP test for remove_temp_files_after_crash

2021-03-18 Thread Tomas Vondra
Reviewed-by: Tomas Vondra Discussion: https://postgr.es/m/CAH503wDKdYzyq7U-QJqGn%3DGm6XmoK%2B6_6xTJ-Yn5WSvoHLY1Ww%40mail.gmail.com Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/a16b2b960f0eec2fe367e86017b3c24ed688ba2b Modified Files -- src/test/recovery

<    1   2   3   4   5   6   >