This is an automated email from the ASF dual-hosted git repository. maxyang pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/cloudberry.git
commit e92b599526e268f7315025ebf96357f0c5786819 Author: Tom Lane <[email protected]> AuthorDate: Tue Aug 2 18:05:34 2022 -0400 Be more wary about 32-bit integer overflow in pg_stat_statements. We've heard a couple of reports of people having trouble with multi-gigabyte-sized query-texts files. It occurred to me that on 32-bit platforms, there could be an issue with integer overflow of calculations associated with the total query text size. Address that with several changes: 1. Limit pg_stat_statements.max to INT_MAX / 2 not INT_MAX. The hashtable code will bound it to that anyway unless "long" is 64 bits. We still need overflow guards on its use, but this helps. 2. Add a check to prevent extending the query-texts file to more than MaxAllocHugeSize. If it got that big, qtext_load_file would certainly fail, so there's not much point in allowing it. Without this, we'd need to consider whether extent, query_offset, and related variables shouldn't be off_t not size_t. 3. Adjust the comparisons in need_gc_qtexts() to be done in 64-bit arithmetic on all platforms. It appears possible that under duress those multiplications could overflow 32 bits, yielding a false conclusion that we need to garbage-collect the texts file, which could lead to repeatedly garbage-collecting after every hash table insertion. Per report from Bruno da Silva. I'm not convinced that these issues fully explain his problem; there may be some other bug that's contributing to the query-texts file becoming so large in the first place. But it did get that big, so #2 is a reasonable defense, and #3 could explain the reported performance difficulties. (See also commit 8bbe4cbd9, which addressed some related bugs. The second Discussion: link is the thread that led up to that.) This issue is old, and is primarily a problem for old platforms, so back-patch. Discussion: https://postgr.es/m/cab+nuk93fl1q9elocotvlp07g7rav4vbdrkm0cvqohdvmpa...@mail.gmail.com Discussion: https://postgr.es/m/[email protected] --- contrib/pg_stat_statements/pg_stat_statements.c | 26 +++++++++++++++++++++---- 1 file changed, 22 insertions(+), 4 deletions(-) diff --git a/contrib/pg_stat_statements/pg_stat_statements.c b/contrib/pg_stat_statements/pg_stat_statements.c index 1e3981c7eb..e3534f247d 100644 --- a/contrib/pg_stat_statements/pg_stat_statements.c +++ b/contrib/pg_stat_statements/pg_stat_statements.c @@ -385,7 +385,7 @@ _PG_init(void) &pgss_max, 5000, 100, - INT_MAX, + INT_MAX / 2, PGC_POSTMASTER, 0, NULL, @@ -2061,6 +2061,18 @@ qtext_store(const char *query, int query_len, *query_offset = off; + /* + * Don't allow the file to grow larger than what qtext_load_file can + * (theoretically) handle. This has been seen to be reachable on 32-bit + * platforms. + */ + if (unlikely(query_len >= MaxAllocHugeSize - off)) + { + errno = EFBIG; /* not quite right, but it'll do */ + fd = -1; + goto error; + } + /* Now write the data into the successfully-reserved part of the file */ fd = OpenTransientFile(PGSS_TEXT_FILE, O_RDWR | O_CREAT | PG_BINARY); if (fd < 0) @@ -2246,8 +2258,14 @@ need_gc_qtexts(void) SpinLockRelease(&s->mutex); } - /* Don't proceed if file does not exceed 512 bytes per possible entry */ - if (extent < 512 * pgss_max) + /* + * Don't proceed if file does not exceed 512 bytes per possible entry. + * + * Here and in the next test, 32-bit machines have overflow hazards if + * pgss_max and/or mean_query_len are large. Force the multiplications + * and comparisons to be done in uint64 arithmetic to forestall trouble. + */ + if ((uint64) extent < (uint64) 512 * pgss_max) return false; /* @@ -2257,7 +2275,7 @@ need_gc_qtexts(void) * query length in order to prevent garbage collection from thrashing * uselessly. */ - if (extent < pgss->mean_query_len * pgss_max * 2) + if ((uint64) extent < (uint64) pgss->mean_query_len * pgss_max * 2) return false; return true; --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
