subject:"Access Log Sampler"

Writing a filter for Access Log Sampler

2011-08-03 Thread Will Roden


I want to write a custom filter for Access Log Sampler, and I am having
trouble finding how to do it.  I see SessionFilter and LogFilter in the
jmeter source code, so I can figure out how to code it, but I don't know
where to put my new class.  Can I just put it in a jar in the ext and have
it picked up like a plugin, or is there something else to it?

Thanks,

Will

Re: access log sampler, state thereof

2011-02-25 Thread Andrey Pohilko

Hi, Will,

I've uploaded current snapshot here: http://code.google.com/p/jmeter-
plugins/downloads/detail?name=JMeterPlugins-0.4.1-
snapshot1.zipcan=1q=#makechanges

Raw Sampler now is pretty simple, you just provide full HTTP request to send 
on wire. 

Massive way of using Raw Sampler is Raw Data Source pre-processor. Add it to 
your test and prepare a file for it. File has simple format:
request length in bytes\n
request data f specified length[newlines possible]next request length\n
next request data ... etc...

Then use ${rawData} as Raw Sampler request data. 

Make note that your request data contains \r\n as end of lines.

Again, this is very early stage for Raw concept, so your feedback will help 
to make it more transparent and easy to use. I've attached sample test plan 
and data file to start with.

Stay in touch,
Andrey

В сообщении от 25 февраля 2011 01:02:10 автор Will Milspec написал:
 Some comments:
 to recreate actual load from a very busy system.
 
 Actual load is usually concurrent. In most cases you dont get these values
 from a log file (and its important that the same session use the same
 thread).
 
 Right. We desire session recreation. From a functional perspective
 (recreating actual load), we find that our production servers can get a lot
 of bot crawls. Each bot request uses its own session. (5000 requests=5000
 sessions). These smal sessions add overhead when those sessions expire.
 If all 5000 requests came from the same session, we would not see the
 session expiration overhead.
 
 Technically I've read that Jmeter basically tries to 'recreate sessions' by
 lumping together requests from the same IP address, but have not confirmed
 how the access log sampler simulates these sessions.
 
 Ideally the access log sampler would work as follows:
 -recognize 'session id' to  the access log
 -access log sampler groups requests from the same sessions into the same
 session. (it can't re-use the exact same session ID as the servlet
 container generates it automatically. ).
 
 If you can generate all POST and GET data, you could use HTTP Raw Sampler
 (custom sampler, not shipped with JMeter). It allows you have full control
 
 Where can I find the raw sampler.
 
 Addendum
 
 fwiw, here's (pretty trivial) example code to add the post parameter to the
 tomcat access log. It adds overhead, mostly when the access log valve
 prints out the parameters.
 
 A. Access log format to get a request attribute, here called rpf params.
 e.g.:
 '%{rpfParams}r'
 
 The tomcat access log valve will call
 request.getAttribute(rpfParams).toString()
 
 B. Add a filter that adds sets a 'parameter wrapper' as a request
 attribute. public void doFilter(ServletRequest pRequest, ServletResponse
 pResponse, FilterChain pChain) throws IOException, ServletException
{
   pRequest.setAttribute(rpfParams,new
 RecordParamsFilter.ParameterWrapper(pRequest.getParameterMap()));
   pChain.doFilter(pRequest, pResponse);
}
 
/**
 * Class to wrap  parameter map. Main work occurs in the toString.
 *
 * Exists to allow printing of both get and post parameters.
 */
public class ParameterWrapper
{
   Map fMap;
 
   public ParameterWrapper(Map pMap)
   {
  fMap = pMap;
   }
 
   @Override
   public String toString()
   {
  StringBuffer sb = new StringBuffer();
  for (Iterator it = fMap.entrySet().iterator(); it.hasNext(); )
  {
 Map.Entry entry = (Map.Entry) it.next();
 String[] values = (String[]) entry.getValue();
 for (String value : values)
 {
sb.append(entry.getKey()).append(=);
sb.append(value);
 }
 if (it.hasNext())
 {
sb.append();
 }
  }
  return sb.toString();
   }
}


RawRequestSample.jmx
Description: XML document
773
GET /blog/247/comment/ HTTP/1.1
Host: apc.kg
Connection: keep-alive
Referer: http://apc.kg/blog/247/
Cache-Control: max-age=0
Accept: 
application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5
User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US) AppleWebKit/534.13 (KHTML, 
like Gecko) Ubuntu/10.10 Chromium/9.0.597.94 Chrome/9.0.597.94 Safari/534.13
Accept-Encoding: gzip,deflate,sdch
Accept-Language: ru-RU,ru;q=0.8,en-US;q=0.6,en;q=0.4
Accept-Charset: windows-1251,utf-8;q=0.7,*;q=0.3
Cookie: 
__utmz=50250230.1297072279.1.1.utmcsr=(direct)|utmccn=(direct)|utmcmd=(none); 
PHPSESSID=a8df1f5d8a623f8c62e5e4e849c64a14; 
__utma=50250230.838534113.1297072279.1297238127.1298628733.5; __utmc=50250230; 
__utmb=50250230.10.10.1298628733

675
GET /img/spaw2/js/spaw.js.php HTTP/1.1
Host: apc.kg
Connection: keep-alive
Referer: http://apc.kg/blog/247/comment/
Accept: */*
User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US) AppleWebKit/534.13 (KHTML, 
like Gecko) Ubuntu/10.10 Chromium/9.0.597.94

Re: access log sampler, state thereof

2011-02-25 Thread Deepak Shetty

Hi
so i looked through some of it , AccessLogSampler was designed to replay the
requests , not simulate the load. So while it seems that it does implement
some of the below (e.g. A map of CookieManagers is kept so that each thread
should correctly pick up the session when it makes a request when you use a
sessionfilter) - the concurrency part must be implemented by you. - Maybe
you should raise an enhancement request in bugzilla - what you are asking
for certainly seems to be something that a lot of us could use.

regards
deepak


On Thu, Feb 24, 2011 at 2:15 PM, Deepak Shetty shet...@gmail.com wrote:

 Ideally the access log sampler would work as follows:
 -recognize 'session id' to  the access log
 -access log sampler groups requests from the same sessions into the same
 session. (it can't re-use the exact same session ID as the servlet
 container
 generates it automatically. ).
 But you are missing one part. A thread in JMeter must wait till the
 response returns. therefore you also need to account for concurrent sessions
 and concurrent requests.

 If I had to implement this I'd do something like
 a. Configure tomcat to also output out SessionIds into the log (along with
 post data) - IP address might work as well
 b.  Pre process your log file to
  1. Figure out when a session began and when it ended (first and last
 request)
  2. calculate the maximum number of concurrent sessions (this is the
 number of threads you will need in JMeter).
  3. Generate new log files that each thread will use by splitting up
 your original log files , based on concurrent sessions(i.e. IP1 - request1
 should go into a file for Thread1 , if now you get IP2 - request2 , then the
 same thread cannot execute this request and you need to make this request2
 from a new thread. If now IP1 session has finished then Thread1 can clear
 all its cookies and start making requests again).
 c. Now use your sampler (either normal + BSH or a customised version) so
 that each thread reads its file and plays the request that it has to(taking
 into account that some variables are dynamic so you cannot always replay the
 test exactly).

 I havent seen the AccessLogSampler code so Im not sure how much of this is
 already implemented. Some of the above may be unneccessarily complicated ,
 need to think about it a bit more.

 regards
 deepak



 On Thu, Feb 24, 2011 at 2:02 PM, Will Milspec will.mils...@gmail.comwrote:

 Some comments:

 to recreate actual load from a very busy system.
 Actual load is usually concurrent. In most cases you dont get these
 values
 from a log file (and its important that the same session use the same
 thread).

 Right. We desire session recreation. From a functional perspective
 (recreating actual load), we find that our production servers can get a
 lot
 of bot crawls. Each bot request uses its own session. (5000
 requests=5000
 sessions). These smal sessions add overhead when those sessions expire.
 If
 all 5000 requests came from the same session, we would not see the
 session
 expiration overhead.

 Technically I've read that Jmeter basically tries to 'recreate sessions'
 by
 lumping together requests from the same IP address, but have not confirmed
 how the access log sampler simulates these sessions.

 Ideally the access log sampler would work as follows:
 -recognize 'session id' to  the access log
 -access log sampler groups requests from the same sessions into the same
 session. (it can't re-use the exact same session ID as the servlet
 container
 generates it automatically. ).


 If you can generate all POST and GET data, you could use HTTP Raw Sampler
 (custom sampler, not shipped with JMeter). It allows you have full
 control
 Where can I find the raw sampler.

 Addendum
 
 fwiw, here's (pretty trivial) example code to add the post parameter to
 the
 tomcat access log. It adds overhead, mostly when the access log valve
 prints
 out the parameters.

 A. Access log format to get a request attribute, here called rpf params.
 e.g.:
 '%{rpfParams}r'

 The tomcat access log valve will call
 request.getAttribute(rpfParams).toString()

 B. Add a filter that adds sets a 'parameter wrapper' as a request
 attribute.
   public void doFilter(ServletRequest pRequest, ServletResponse pResponse,
 FilterChain pChain) throws IOException, ServletException
   {
  pRequest.setAttribute(rpfParams,new
 RecordParamsFilter.ParameterWrapper(pRequest.getParameterMap()));
  pChain.doFilter(pRequest, pResponse);
   }

   /**
* Class to wrap  parameter map. Main work occurs in the toString.
*
* Exists to allow printing of both get and post parameters.
*/
   public class ParameterWrapper
   {
  Map fMap;

  public ParameterWrapper(Map pMap)
  {
 fMap = pMap;
  }

  @Override
  public String toString()
  {
 StringBuffer sb = new StringBuffer();
 for (Iterator it = fMap.entrySet().iterator(); it.hasNext

Re: access log sampler, state thereof

2011-02-24 Thread Andrey Pohilko

Hi,

If you can generate all POST and GET data, you could use HTTP Raw Sampler
(custom sampler, not shipped with JMeter). It allows you have full control
over request data - cookies, headers etc, but requires far more efforts ro
prepare data.

If you interested, we could discuss it with more details, Raw Sampler concept
is at very early development stage now.

Best regards,
Andrey Pohilko,
JMeter Plugins at Google Code
Project Owner
http://code.google.com/p/jmeter-plugins/

В сообщении от 24 февраля 2011 01:34:12 автор Will Milspec написал:
hi all,

I'm new to jmeter and have interest in using the access log sampler to
recreate actual load from a very busy system.

Anyone on the list using it currently? Can you share your insights
recommendations?

We really would like it to replay 'posts' (i.e. as well as 'gets'), but
reading the documentation and code, it appears that no posts.

For our application, posting content results in cleared caches, i.e. which
require cache reloading...versus mere gets which fetch from the cache. So
post-less access logs really will not tax the application to its full
extent.

I've looked at adding a new parser class (i.e. for a custom format which
includes post data--it's pretty easy in tomcat to log this information
using a servlet filter and the tomcat access log valve formatting
strings)however..reviewing the code I'm not sure if the HttpSampler
will actually replay a post ...but I digress.. This is fodder for
another question

45 matches

Mail list logo