Re: Doc update for pg_stat_statements normalization

2023-02-28 Thread Michael Paquier
On Wed, Mar 01, 2023 at 12:43:40AM +, Imseih (AWS), Sami wrote: > Yes, that makes sense. Okay, thanks. Done that now on HEAD. -- Michael signature.asc Description: PGP signature

Re: Doc update for pg_stat_statements normalization

2023-02-28 Thread Imseih (AWS), Sami
> + > + Queries on which normalization can be applied may be observed with constant > + values in pg_stat_statements, especially when there > + is a high rate of entry deallocations. To reduce the likelihood of this > + happening, consider increasing pg_stat_statements.max. > + The

Re: Doc update for pg_stat_statements normalization

2023-02-28 Thread Michael Paquier
atements_info view, discussed below + in , + provides statistics about entry deallocations. + Are you OK with this text? -- Michael From dd8938e4ba1b1f29d14b3fa2dc76301f42592cad Mon Sep 17 00:00:00 2001 From: Michael Paquier Date: Wed, 1 Mar 2023 09:05:08 +0900 Subject: [PATCH v3] do

Re: Doc update for pg_stat_statements normalization

2023-02-28 Thread Imseih (AWS), Sami
> I am OK with an addition to the documentation to warn that one may > have to increase the maximum number of entries that can be stored if > seeing a non-normalized entry that should have been normalized. I agree. We introduce the concept of a plannable statement in a previous section and we

Re: Doc update for pg_stat_statements normalization

2023-02-27 Thread Michael Paquier
On Fri, Feb 24, 2023 at 08:54:00PM +, Imseih (AWS), Sami wrote: > I think the only thing to do here is to call this out in docs with a > suggestion to increase > pg_stat_statements.max to reduce the likelihood. I also attached the suggested > doc enhancement as well. + + A query text may

Re: Doc update for pg_stat_statements normalization

2023-02-27 Thread Michael Paquier
On Mon, Feb 27, 2023 at 10:53:26PM +, Imseih (AWS), Sami wrote: > Wouldn't be less in terms of memory usage to just store the required > JumbleState fields in Query[Desc]? > > clocations_count, > highest_extern_param_id, > clocations_locations, > clocations_length Yes, these would be enough

Re: Doc update for pg_stat_statements normalization

2023-02-27 Thread Imseih (AWS), Sami
>On Sat, Feb 25, 2023 at 01:59:04PM +, Imseih (AWS), Sami wrote:> >> The overhead of storing this additional private data for the life of the > query >> execution may not be desirable. >Okay, but why? Additional memory to maintain the JumbleState data in other structs, and

Re: Doc update for pg_stat_statements normalization

2023-02-26 Thread Michael Paquier
On Sat, Feb 25, 2023 at 01:59:04PM +, Imseih (AWS), Sami wrote: > The overhead of storing this additional private data for the life of the query > execution may not be desirable. Okay, but why? > I think we also will need to copy the > private data to QueryDesc as well to make it available

Re: Doc update for pg_stat_statements normalization

2023-02-25 Thread Imseih (AWS), Sami
>> Could things be done in a more stable way? For example, imagine that >> we have an extra Query field called void *private_data that extensions >> can use to store custom data associated to a query ID, then we could >> do something like that: >> - In the post-analyze hook,

Re: Doc update for pg_stat_statements normalization

2023-02-25 Thread Julien Rouhaud
On Sat, Feb 25, 2023 at 02:58:36PM +0900, Michael Paquier wrote: > On Fri, Feb 24, 2023 at 08:54:00PM +, Imseih (AWS), Sami wrote: > > I think the only thing to do here is to call this out in docs with a > > suggestion to increase pg_stat_statements.max to reduce the > > likelihood. I also

Re: Doc update for pg_stat_statements normalization

2023-02-24 Thread Michael Paquier
On Fri, Feb 24, 2023 at 08:54:00PM +, Imseih (AWS), Sami wrote: > I think the only thing to do here is to call this out in docs with a > suggestion to increase pg_stat_statements.max to reduce the > likelihood. I also attached the suggested doc enhancement as well. Improving the docs about

Doc update for pg_stat_statements normalization

2023-02-24 Thread Imseih (AWS), Sami
Replacing constants in pg_stat_statements is on a best effort basis. It is not unlikely that on a busy workload with heavy entry deallocation, the user may observe the query with the constants in pg_stat_statements. From what I can see, this is because the only time an entry is normalized is