Re: Service Engine Logging

2013-06-19 Thread Jacques Le Roux
Ha forgot, done, please Adrian have a look if you did not already https://cwiki.apache.org/confluence/display/OFBTECH/Metrics Jacques From: Jacques Le Roux jacques.le.r...@les7arts.com Great, thanks Adrian! Maybe a wiki page using http://markmail.org/message/x4lzvda66ju6gdg5 would help to

Re: Service Engine Logging

2013-06-19 Thread Adrian Crum
It looks good! -Adrian On 6/19/2013 9:26 AM, Jacques Le Roux wrote: Ha forgot, done, please Adrian have a look if you did not already https://cwiki.apache.org/confluence/display/OFBTECH/Metrics Jacques From: Jacques Le Roux jacques.le.r...@les7arts.com Great, thanks Adrian! Maybe a wiki

Re: Performance Metrics Configuration (was: Service Engine Logging)

2013-05-20 Thread Jacques Le Roux
Hi Adrian, OK, let's try to be constructive. I must say my 1st botched tentative had side effects on the Metrics framework. I have created https://issues.apache.org/jira/browse/OFBIZ-5198 and attached a patch. Please review and if you are still against explain why in details, thanks! I

Re: Service Engine Logging

2013-05-18 Thread Adrian Crum
I'm going to try one more time to ask nicely, and if that doesn't work, then I'm prepared to veto your commit. You added redundant methods to the existing code that do nothing more than duplicate existing functionality. You are not taking the time to understand how to use the feature

Re: Service Engine Logging

2013-05-18 Thread Jacques Le Roux
From: Adrian Crum adrian.c...@sandglass-software.com I'm going to try one more time to ask nicely, and if that doesn't work, then I'm prepared to veto your commit. You added redundant methods to the existing code that do nothing more than duplicate existing functionality. duplicate

Re: Service Engine Logging

2013-05-18 Thread Adrian Crum
No, you do not want a simple average - because it is meaningless. You don't have to be an expert in statistics to understand why. Your design averages all requests since server start. A server could run for weeks, months, years... so your average includes response times from so long ago that

Performance Metrics Configuration (was: Service Engine Logging)

2013-05-18 Thread Adrian Crum
Okay, I reverted your commit and made some changes to the performance metrics feature. Those changes should accommodate your requirements. The discussion will go smoother if you list your requirements, and allow me to show you how to configure the metrics to do what you want. -Adrian On

Re: Service Engine Logging

2013-05-17 Thread Jacques Le Roux
Hi Adrian, At r1483822 I have improved it a bit for requests. I was more interested in chained ones, it works for all. It's a new colums in Metrics page which measure the request event duration. Just thinking I did not use the I wonder if the use of recordServiceRate() in doRequest() makes

Re: Service Engine Logging

2013-05-17 Thread Adrian Crum
There was no need to change the Metrics code. If you don't want smoothing, then just set the smoothing factor to 1. This is explained in the schema. Also, please don't put request-specific code in Metrics - that breaks the abstraction. -Adrian On 5/17/2013 3:30 PM, Jacques Le Roux wrote:

Re: Service Engine Logging

2013-05-17 Thread Jacques Le Roux
From: Adrian Crum adrian.c...@sandglass-software.com There was no need to change the Metrics code. If you don't want smoothing, then just set the smoothing factor to 1. This is explained in the schema. Yes I tried that. But it did not work for my need. See where I put the

Re: Service Engine Logging

2013-05-17 Thread Jacques Le Roux
What I wanted to add by Just thinking I did not use the was that I did not use the moving average (I forgot to remove this part after). Actually I had no needs for it and it's already covered by recordServiceRate() anyway. The pb with current position of recordServiceRate() in RequestHandler

Re: Service Engine Logging

2013-05-17 Thread Adrian Crum
The real request duration is meaningless. If a garbage collection occurs during the request, the measurement will be artificially high. That's the whole point of the smoothing - it factors out transient performance differences. Please don't break the design. If you don't like the way it

Re: Service Engine Logging

2013-05-17 Thread Jacques Le Roux
recordServiceRate() is still working the same. Note that after around 1000 (Law of large Numbers) garbage collection or other noises should not have much impact, and actually they also exist... I could do one thing if you want: use the same algo for recordRequestRate() than is in

Re: Service Engine Logging

2013-05-17 Thread Jacques Le Roux
From: Adrian Crum adrian.c...@sandglass-software.com The real request duration is meaningless. If a garbage collection occurs during the request, the measurement will be artificially high. That's the whole point of the smoothing - it factors out transient performance differences. Please

Re: Service Engine Logging

2013-05-17 Thread Adrian Crum
What you believe users want and what the feature was designed to do seem to be entirely different. The metric is intended to measure the entire request. If you want something different, then feel free to modify your local copy. -Adrian On 5/17/2013 5:25 PM, Jacques Le Roux wrote: From:

Re: Service Engine Logging

2013-05-17 Thread Jacques Le Roux
From: Adrian Crum adrian.c...@sandglass-software.com What you believe users want and what the feature was designed to do seem to be entirely different. The metric is intended to measure the entire request. If you want something different, then feel free to modify your local copy. I

Re: Service Engine Logging

2013-05-17 Thread Adrian Crum
Agreed. You added something that is unnecessary. Please remove it. Metrics.java does not need to be changed. If you want to change what is being measured, then apply Metrics java in a different way, but do not change it. -Adrian On 5/17/2013 5:45 PM, Jacques Le Roux wrote: From: Adrian

Re: Service Engine Logging

2013-05-17 Thread Jacques Le Roux
From: Adrian Crum adrian.c...@sandglass-software.com Agreed. You added something that is unnecessary. Please remove it. I don't agree it's unnecessary, it does measure the request events durations for chained request and that's quite useful. Without this new column the measure for chained

Re: Service Engine Logging

2013-04-19 Thread Jacques Le Roux
Actually r1361725 was also needed for services metrics to work. For services I get slightly higher results than in log. It's more consistent than for chained requests which are plain false. I did not get a chance to look at code yet... Jacques Jacques Le Roux wrote: From: Jacques Le Roux

Re: Service Engine Logging

2013-04-19 Thread Adrian Crum
Keep in mind the metrics provide a moving average. Also, there is a smoothing factor that can affect the value. The smoothing factor is important. Let's say you are using a monitoring tool to monitor a particular request. If that request has a very brief increase in response time, you don't

Re: Service Engine Logging

2013-04-19 Thread Jacques Le Roux
OK, I guess the smoothing factor explains the tiny differences for services But for chained requests it makes no sense. The metrics report ten more than in log. I will have a look because it's a fine tool for chained requests duration which are not availabe in the stats feature Thanks Jacques

Re: Service Engine Logging

2013-04-19 Thread Adrian Crum
Anything you can do to improve the implementation will be appreciated. Sorry I haven't had more time to document it. -Adrian On 4/19/2013 12:01 PM, Jacques Le Roux wrote: OK, I guess the smoothing factor explains the tiny differences for services But for chained requests it makes no sense.

Re: Service Engine Logging

2013-04-15 Thread Jacques Le Roux
From: Jacques Le Roux jacques.le.r...@les7arts.com Thanks Adrian, Yes I saw the threshold idea in Java comments Looks like an interesting tool BTW for the other statistics (I never used it), do you know if chained requests are taken into account? Hi Adrian, I can answer now that

Re: Service Engine Logging

2013-04-11 Thread Jacques Le Roux
Thanks Atul, So you mean you simply need to put a token (like webapp name) before the request name in this metric name? Did you try it with another request? I see it works on trunk demo and OOTB locally. But I can't get it to work on a custom app I have patched, so maybe I miss something I

Re: Service Engine Logging

2013-04-11 Thread Adrian Crum
I will create a Wiki page this weekend. The metric name can be anything - the example has a URL: prefix to help distinguish request metrics from service metrics. You can put anything you want there. The request-map can have an additional response element - which will direct the servlet to

Re: Service Engine Logging

2013-04-11 Thread Jacques Le Roux
Sent: Wednesday, April 10, 2013 9:17 PM Subject: Re: Service Engine Logging Hi Adrian, Let me know if you will, else I might do it... BTW, are 1361722 and 1361724 the only commit? Is there a Jira? Could you explain just a bit more the request syntax, sorry but the OOTB example is a bit

Re: Service Engine Logging

2013-04-11 Thread Adrian Crum
Welsh and The Regents of the University of California. All rights reserved. But nothing in NOTICE Jacques - Original Message - From: Jacques Le Roux jacques.le.r...@les7arts.com To: dev@ofbiz.apache.org Sent: Wednesday, April 10, 2013 9:17 PM Subject: Re: Service Engine Logging Hi Adrian

Re: Service Engine Logging

2013-04-11 Thread Jacques Le Roux
@ofbiz.apache.org Sent: Wednesday, April 10, 2013 9:17 PM Subject: Re: Service Engine Logging Hi Adrian, Let me know if you will, else I might do it... BTW, are 1361722 and 1361724 the only commit? Is there a Jira? Could you explain just a bit more the request syntax, sorry but the OOTB

Re: Service Engine Logging

2013-04-11 Thread Jacques Le Roux
Thanks Adrian, Yes I saw the threshold idea in Java comments Looks like an interesting tool BTW for the other statistics (I never used it), do you know if chained requests are taken into account? Still, can you confirm only if 1361722 and 1361724 the only commits? I guess there is no Jira?

Re: Service Engine Logging

2013-04-11 Thread Adrian Crum
The metrics commits included 136172, 1361724, and 1361727. -Adrian On 4/11/2013 12:17 PM, Jacques Le Roux wrote: Thanks Adrian, Yes I saw the threshold idea in Java comments Looks like an interesting tool BTW for the other statistics (I never used it), do you know if chained requests are

Re: Service Engine Logging

2013-04-11 Thread Jacques Le Roux
Thanks Adrian! Jacques From: Adrian Crum adrian.c...@sandglass-software.com The metrics commits included 136172, 1361724, and 1361727. -Adrian On 4/11/2013 12:17 PM, Jacques Le Roux wrote: Thanks Adrian, Yes I saw the threshold idea in Java comments Looks like an interesting tool

Re: Service Engine Logging

2013-04-10 Thread Jacques Le Roux
Hi Adrian, Let me know if you will, else I might do it... BTW, are 1361722 and 1361724 the only commit? Is there a Jira? Could you explain just a bit more the request syntax, sorry but the OOTB example is a bit confusing. +request-map uri=ViewMetrics +security https=true

Re: Service Engine Logging

2013-04-10 Thread Atul Vani
I think that's just the name, to recognize it among list of several others, when there will be lots of them. A certain convention is used to specify that it is for URL or any service. Then again same names request mappings can exist in separate webapps, so a leading appname, in this case

Re: Service Engine Logging

2013-04-08 Thread Jacques Le Roux
Great, thanks Adrian! Maybe a wiki page using http://markmail.org/message/x4lzvda66ju6gdg5 would help to remember the commands? Jacques From: Adrian Crum adrian.c...@sandglass-software.com On 4/7/2013 6:38 PM, Adrian Crum wrote: On 4/7/2013 12:11 PM, Jacques Le Roux wrote: Jacques Le Roux

Re: Service Engine Logging

2013-04-07 Thread Jacques Le Roux
This seems over complicated to me. Don't arbitrarily slow OFBiz by storing (IOs) services durations at each service call. Let's keep it simple and easy tunable. The way it was before ( 50 = slow; 200 = very slow) was there for years. I did not code it, and after a decade we want to change

Re: Service Engine Logging

2013-04-07 Thread Adrian Crum
From my perspective, you are the one complicating things. If you don't want to slow down the system with logging I/O, then turn logging off. We've always done it that way is not a valid argument. You still haven't given us any requirements that demonstrate the need to put complicated decision

Re: Service Engine Logging

2013-04-07 Thread Adrian Crum
I have a problem with a single threshold value for all services too. The metrics feature is something I added a while ago. You can assign a metric to a service you are interested in and then monitor that service's performance. Just follow the instructions in the schema. -Adrian On 4/7/2013

Re: Service Engine Logging

2013-04-07 Thread Jacques Le Roux
From: Adrian Crum adrian.c...@sandglass-software.com From my perspective, you are the one complicating things. If you don't want to slow down the system with logging I/O, then turn logging off. When you are continously developping a system, removing all log info/timing from production can be

Re: Service Engine Logging

2013-04-07 Thread Adrian Crum
On 4/7/2013 10:30 AM, Jacques Le Roux wrote: From: Adrian Crum adrian.c...@sandglass-software.com From my perspective, you are the one complicating things. If you don't want to slow down the system with logging I/O, then turn logging off. When you are continously developping a system,

Re: Service Engine Logging

2013-04-07 Thread Jacques Le Roux
From: Adrian Crum adrian.c...@sandglass-software.com On 4/7/2013 10:30 AM, Jacques Le Roux wrote: From: Adrian Crum adrian.c...@sandglass-software.com From my perspective, you are the one complicating things. If you don't want to slow down the system with logging I/O, then turn logging off.

Re: Service Engine Logging

2013-04-07 Thread Jacques Le Roux
Jacques Le Roux wrote: From: Adrian Crum adrian.c...@sandglass-software.com On 4/7/2013 10:30 AM, Jacques Le Roux wrote: From: Adrian Crum adrian.c...@sandglass-software.com From my perspective, you are the one complicating things. If you don't want to slow down the system with logging I/O,

Re: Service Engine Logging

2013-04-07 Thread Adrian Crum
On 4/7/2013 12:11 PM, Jacques Le Roux wrote: Jacques Le Roux wrote: From: Adrian Crum adrian.c...@sandglass-software.com Have you considered using the metrics feature? Ha forgot to ask about it, now I see and remember this thread and I will digg in

Re: Service Engine Logging

2013-04-07 Thread Adrian Crum
On 4/7/2013 6:38 PM, Adrian Crum wrote: On 4/7/2013 12:11 PM, Jacques Le Roux wrote: Jacques Le Roux wrote: From: Adrian Crum adrian.c...@sandglass-software.com Have you considered using the metrics feature? Ha forgot to ask about it, now I see and remember this thread and I will digg in

Re: Service Engine Logging (was: svn commit: r1463863 - /ofbiz/trunk/framework/service/src/org/ofbiz/service/ServiceDispatcher.java )

2013-04-06 Thread Jacques Le Roux
Hi Adrian, Thanks for asking, I committed and commented at revision: 1465223 Atul, It was not easy to read your patch in the email (cut at 80 chars). Please open a Jira if you want to improve my commit. Thanks Jacques From: Adrian Crum adrian.c...@sandglass-software.com Jacques, What

Re: Service Engine Logging (was: svn commit: r1463863 - /ofbiz/trunk/framework/service/src/org/ofbiz/service/ServiceDispatcher.java )

2013-04-06 Thread Atul Vani
Jacques, Thanks. I will do it soon. On Sat, 06 Apr 2013 16:30:30 +0530, Jacques Le Roux jacques.le.r...@les7arts.com wrote: Hi Adrian, Thanks for asking, I committed and commented at revision: 1465223 Atul, It was not easy to read your patch in the email (cut at 80 chars). Please

Re: Service Engine Logging

2013-04-06 Thread Adrian Crum
Please don't change the timing logging - there should not be any conditions placed on it. You didn't answer my question. I was hoping we could avoid a commit war by discussing your requirements and designing a solution that makes everyone happy. -Adrian On 4/6/2013 12:00 PM, Jacques Le

Re: Service Engine Logging

2013-04-06 Thread Jacques Le Roux
I thought the commit comment answered your question OOTB, there is only 1 change from what you committed: now services longer than 1 sec will also show as slow in log Jacques From: Adrian Crum adrian.c...@sandglass-software.com Please don't change the timing logging - there should not be any

Re: Service Engine Logging

2013-04-06 Thread Adrian Crum
No, a commit is NOT a reply to an email. Please, let's discuss this. You seem to be forcing your perception of how logging should be done on the rest of the community. I would prefer that we all participate in a discussion and come up with a design that works for everyone. -Adrian On

Re: Service Engine Logging

2013-04-06 Thread Atul Vani
Here's what I think, it's all raw though :) * As suggested by Jacopo, we maintain stats in some kind of entity. Let's say average running time. * We use this average running time to decide if a timing log should be printed. The thing is, not all services are same, some are complex and are

Service Engine Logging (was: svn commit: r1463863 - /ofbiz/trunk/framework/service/src/org/ofbiz/service/ServiceDispatcher.java)

2013-04-05 Thread Adrian Crum
Jacques, What are your requirements? What are you looking for in the logs? -Adrian On 4/4/2013 10:54 PM, Jacques Le Roux wrote: These numbers are from experience of hours and hours staring at clusters logs, but yes it's arbitrary and depend on context (as I guess were picked the initial