Re: Log a sample of transactions

2019-04-04 Thread Adrien NAYRAT
On 4/3/19 11:45 PM, Alvaro Herrera wrote: On 2019-Mar-29, Adrien NAYRAT wrote: On 3/29/19 3:06 AM, Kuroda, Hayato wrote: Dear Adrien, Hello Kuroda-san, I understood the cost of randomizing is very low. Maybe it's OK.. I'll change the status to "Ready For Committer." Thanks, I hope it wi

Re: Log a sample of transactions

2019-04-03 Thread Alvaro Herrera
On 2019-Mar-29, Adrien NAYRAT wrote: > On 3/29/19 3:06 AM, Kuroda, Hayato wrote: > > Dear Adrien, > > Hello Kuroda-san, > > > > > I understood the cost of randomizing is very low. Maybe it's OK.. > > I'll change the status to "Ready For Committer." > > Thanks, I hope it will be commited in PG1

Re: Log a sample of transactions

2019-03-29 Thread Adrien NAYRAT
On 3/29/19 3:06 AM, Kuroda, Hayato wrote: Dear Adrien, Hello Kuroda-san, I understood the cost of randomizing is very low. Maybe it's OK.. I'll change the status to "Ready For Committer." Thanks, I hope it will be commited in PG12 as the feature is closed to log_statement_sample_rate.

RE: Log a sample of transactions

2019-03-28 Thread Kuroda, Hayato
Dear Adrien, I understood the cost of randomizing is very low. Maybe it's OK.. I'll change the status to "Ready For Committer." Finally, I apologize for having delayed review. Thank you for your good opportunities. Best Regards, Hayato Kuroda Fujitsu LIMITED

Re: Log a sample of transactions

2019-03-28 Thread Adrien NAYRAT
On 3/28/19 10:45 AM, Christoph Berg wrote: Re: Adrien NAYRAT 2019-03-28 Sorry, I changed that, someone suggest using either "0" and "1", or "0.0" and "1.0" but not mixing both. Agreed with using "0.0" and "1.0". I will remove this change. +#log_transaction_sample_rate = 0 # Fractio

Re: Log a sample of transactions

2019-03-28 Thread Christoph Berg
Re: Adrien NAYRAT 2019-03-28 > > > > Sorry, I changed that, someone suggest using either "0" and "1", or > > > > "0.0" and "1.0" but not mixing both. > > > > Agreed with using "0.0" and "1.0". > > > > > > I will remove this change. > +#log_transaction_sample_rate = 0 # Fraction of transacti

Re: Log a sample of transactions

2019-03-28 Thread Masahiko Sawada
On Thu, Mar 28, 2019 at 6:33 PM Adrien NAYRAT wrote: > > Hello, > > On 3/28/19 3:18 AM, Masahiko Sawada wrote: > >>> > >>> Sorry, I changed that, someone suggest using either "0" and "1", or > >>> "0.0" and "1.0" but not mixing both. > > > > Agreed with using "0.0" and "1.0". > > > >>> I will remo

Re: Log a sample of transactions

2019-03-28 Thread Adrien NAYRAT
Hello, On 3/28/19 3:18 AM, Masahiko Sawada wrote: Sorry, I changed that, someone suggest using either "0" and "1", or "0.0" and "1.0" but not mixing both. Agreed with using "0.0" and "1.0". I will remove this change. --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/u

Re: Log a sample of transactions

2019-03-27 Thread Masahiko Sawada
Thank you for updating the patch! On Wed, Mar 27, 2019 at 10:28 PM Adrien NAYRAT wrote: > > On 3/27/19 10:22 AM, Adrien NAYRAT wrote: > > Hello, > > > > On 3/27/19 7:05 AM, Masahiko Sawada wrote: > > Why are both of these checked? ISTM once xact_is_sampled is set, we > > ought not to resp

Re: Log a sample of transactions

2019-03-27 Thread Adrien NAYRAT
On 3/27/19 10:22 AM, Adrien NAYRAT wrote: Hello, On 3/27/19 7:05 AM, Masahiko Sawada wrote: Why are both of these checked? ISTM once xact_is_sampled is set, we ought not to respect a different value of log_xact_sample_rate for the rest of that transaction. I added theses checks to allow to dis

Re: Log a sample of transactions

2019-03-27 Thread Adrien NAYRAT
Hello, On 3/27/19 7:05 AM, Masahiko Sawada wrote: Why are both of these checked? ISTM once xact_is_sampled is set, we ought not to respect a different value of log_xact_sample_rate for the rest of that transaction. I added theses checks to allow to disable logging during a sampled transaction,

Re: Log a sample of transactions

2019-03-26 Thread Masahiko Sawada
Thank you for updating the patch! On Sat, Mar 2, 2019 at 3:18 AM Adrien NAYRAT wrote: > > Hello, > > On 2/15/19 3:24 PM, Adrien NAYRAT wrote: > > On 2/14/19 9:14 PM, Andres Freund wrote: > >> I wonder if this doesn't need a warning, explaining that using this when > >> there are large transaction

Re: Log a sample of transactions

2019-03-26 Thread Adrien NAYRAT
On 3/26/19 1:54 AM, Kuroda, Hayato wrote: Dear David, I have a will and already read the patch, but I thought it's not my turn. Sorry. Hello, Adrien, I did not find any test for log_min_duration that could help me. LCOV indicate there is no tests on this part (look check_log_duration()

RE: Re: Log a sample of transactions

2019-03-25 Thread Kuroda, Hayato
Dear David, I have a will and already read the patch, but I thought it's not my turn. Sorry. Adrien, > I did not find any test for log_min_duration that could help me. LCOV > indicate > there is no tests on this part (look check_log_duration()): > https://coverage.postgresql.org/src/backend/t

Re: Re: Log a sample of transactions

2019-03-25 Thread David Steele
Hi Hayato, On 3/1/19 10:17 PM, Adrien NAYRAT wrote: Patch attached with fix mentioned above, but without xact_is_sampled reset. Are you planning to review this patch? If not I remove you from the reviewers so perhaps somebody else will have a look. Regards, -- -David da...@pgmasters.net

Re: Log a sample of transactions

2019-03-01 Thread Adrien NAYRAT
Hello, On 2/15/19 3:24 PM, Adrien NAYRAT wrote: On 2/14/19 9:14 PM, Andres Freund wrote: I wonder if this doesn't need a warning, explaining that using this when there are large transactions could lead to slowdowns. Yes, I will add some wording Warning added. It seems pretty weird to have

Re: Log a sample of transactions

2019-02-15 Thread Adrien NAYRAT
On 2/14/19 9:14 PM, Andres Freund wrote: Hi, Hello Andres, On 2019-01-26 11:44:58 +0100, Adrien NAYRAT wrote: + + log_transaction_sample_rate (real) + + log_transaction_sample_rate configuration parameter + + + + + Set the fraction o

Re: Log a sample of transactions

2019-02-14 Thread Andres Freund
Hi, On 2019-01-26 11:44:58 +0100, Adrien NAYRAT wrote: > + xreflabel="log_transaction_sample_rate"> > + log_transaction_sample_rate > (real) > + > + log_transaction_sample_rate configuration > parameter > + > + > + > + > + Set the fraction

Re: Log a sample of transactions

2019-02-04 Thread Michael Paquier
On Mon, Feb 04, 2019 at 04:32:05PM +0100, Adrien NAYRAT wrote: > Could it be acceptable if I set log_min_duration_statement = 0? At least this has the merit to make the behavior not rely on randomness. I have not looked at the patch in details so I cannot say for sure, but I am not sure if it is

Re: Log a sample of transactions

2019-02-04 Thread Adrien NAYRAT
On 2/4/19 3:26 AM, Michael Paquier wrote: These would take time to execute, even if you need to measure one second, and may be hard to make stable on slow machines. Could it be acceptable if I set log_min_duration_statement = 0? Moved to next CF. Thanks

Re: Log a sample of transactions

2019-02-03 Thread Michael Paquier
On Sun, Feb 03, 2019 at 12:23:00PM +0100, Adrien Nayrat wrote: > I did not find any test for log_min_duration that could help me. LCOV indicate > there is no tests on this part (look check_log_duration()): > https://coverage.postgresql.org/src/backend/tcop/postgres.c.gcov.html These would take tim

Re: Log a sample of transactions

2019-02-03 Thread Adrien Nayrat
On 1/28/19 2:53 AM, Kuroda, Hayato wrote: > BTW, I give you a suggestion about a test. > This parameter enables users to log statements randomly, hence adding some > tests is very difficult. > Perhaps Only three cases are available: > > * When log_transaction_sample_rate is set to 1, all statemen

RE: Log a sample of transactions

2019-01-27 Thread Kuroda, Hayato
Dear Adrien, >> * xact_is_sampled is left at the end of a transaction. >> Should the parameter be set to false at the lowest layer of the transaction >> system? >> I understand it is unnecessary for the functionality, but it have more >> symmetry. > > Yes, it is not necessary. I wonder what is m

Re: Log a sample of transactions

2019-01-26 Thread Adrien NAYRAT
On 1/23/19 3:12 AM, Kuroda, Hayato wrote: Dear Adrien, Hello Kuroda-san config.sgml You must document the behavior when users change the parameter during a transaction. あやしい・・・ Agreed, I added a wording. postgres.c I give you three comments. /* flag for logging

Re: Log a sample of transactions

2019-01-24 Thread Robert Haas
On Fri, Jan 18, 2019 at 8:23 AM Adrien NAYRAT wrote: > On 1/18/19 9:03 AM, Peter Eisentraut wrote: > > But if you have trouble with a specific transaction, how will a setting > > help that randomly logs transactions, not necessarily the one you are > > concerned about? > > Yes, it assumes your app

RE: Log a sample of transactions

2019-01-22 Thread Kuroda, Hayato
Dear Adrien, > Hum, I am not an english native speaker, I choosed "rate" because it is > already used for auto_explain.sample_rate and for log_statement_sample_rate If the community agree those name, renaming parameters are not needed. Consistency is the most important in a DBMS. :-) I read you

Re: Log a sample of transactions

2019-01-18 Thread Adrien NAYRAT
On 1/18/19 9:03 AM, Peter Eisentraut wrote: But if you have trouble with a specific transaction, how will a setting help that randomly logs transactions, not necessarily the one you are concerned about? Yes, it assumes your application performs same type of transaction. Maybe the use case is t

Re: Log a sample of transactions

2019-01-18 Thread Peter Eisentraut
On 15/01/2019 18:03, Adrien NAYRAT wrote: > The goal is not to find slow queries in a transaction, but troubleshoot > applicative issue when you have short queries. > > Sometimes you want to understand what happens in a transaction, either > you perfectly know your application, either you have t

Re: Log a sample of transactions

2019-01-17 Thread Adrien NAYRAT
On 1/16/19 10:09 AM, Masahiko Sawada wrote: Since we set xact_is_sampled only when transaction starts and see it during transaction we cannot disable logging during transaction and vice versa. I can imagine the use case where user wants to disable logging during transaction. So it might be better

Re: Log a sample of transactions

2019-01-16 Thread Masahiko Sawada
On Wed, Jan 16, 2019 at 2:03 AM Adrien NAYRAT wrote: > > On 1/15/19 11:42 AM, Masahiko Sawada wrote: > >> When you troubleshoot applicative issues with multi-statements > >> transaction, you may have to log all queries to find all statements of one > >> transaction. With high throughput, it coul

Re: Log a sample of transactions

2019-01-16 Thread Adrien NAYRAT
On 1/16/19 1:40 AM, Kuroda, Hayato wrote: By the way, I think the name of this parameter should be changed. In the Cambridge dictionary, the word "rate" means as follows: the speed at which something happens or changes, or the amount or number of times it happens or changes in a particular perio

Re: Log a sample of transactions

2019-01-15 Thread Michael Paquier
On Tue, Jan 15, 2019 at 06:03:36PM +0100, Adrien NAYRAT wrote: > The goal is not to find slow queries in a transaction, but troubleshoot > applicative issue when you have short queries. > > Sometimes you want to understand what happens in a transaction, either you > perfectly know your application

RE: Log a sample of transactions

2019-01-15 Thread Kuroda, Hayato
Dear Adrien, > Your messages looks normal to me, you will have same messages if you put > log_min_duration to 0. That was my lack of understanding. Sorry. By the way, I think the name of this parameter should be changed. In the Cambridge dictionary, the word "rate" means as follows: the speed

Re: Log a sample of transactions

2019-01-15 Thread Adrien NAYRAT
On 1/15/19 11:42 AM, Masahiko Sawada wrote: When you troubleshoot applicative issues with multi-statements transaction, you may have to log all queries to find all statements of one transaction. With high throughput, it could be hard to log all queries without causing troubles. Hm, can we use

Re: Log a sample of transactions

2019-01-15 Thread Masahiko Sawada
On Sat, Jan 5, 2019 at 12:57 AM Adrien Mobile wrote: > > Le 4 janvier 2019 13:20:09 GMT+01:00, Peter Eisentraut > a écrit : > >On 12/12/2018 22:32, Adrien Nayrat wrote: > >> Per idea of Nikolay Samokhvalov[1] I propose this patch to add the > >possibility > >> to log all statements from a fracti

Re: Log a sample of transactions

2019-01-15 Thread Adrien NAYRAT
On 1/15/19 9:00 AM, Kuroda, Hayato wrote: Dear Adrien, I applied your file and faced a bug-candidate. (I have not read your source yet. Sorry.) When I tried to use tab-complition, some dirty messages were appeared. Messages I faced are attached in this mail. Best Regards, Hayato Kuroda Fujits

RE: Log a sample of transactions

2019-01-15 Thread Kuroda, Hayato
Dear Adrien, I applied your file and faced a bug-candidate. (I have not read your source yet. Sorry.) When I tried to use tab-complition, some dirty messages were appeared. Messages I faced are attached in this mail. Best Regards, Hayato Kuroda Fujitsu LIMITED postgres=# SET client_min_message

Re: Log a sample of transactions

2019-01-04 Thread Adrien Mobile
Le 4 janvier 2019 13:20:09 GMT+01:00, Peter Eisentraut a écrit : >On 12/12/2018 22:32, Adrien Nayrat wrote: >> Per idea of Nikolay Samokhvalov[1] I propose this patch to add the >possibility >> to log all statements from a fraction of transactions. >> >> I have several questions: >> * Do we wa

Re: Log a sample of transactions

2019-01-04 Thread Peter Eisentraut
On 12/12/2018 22:32, Adrien Nayrat wrote: > Per idea of Nikolay Samokhvalov[1] I propose this patch to add the possibility > to log all statements from a fraction of transactions. > > I have several questions: > * Do we want this feature? It's not clear to me what the use is. The per-statement