Hi CP, After looking at the sampler - one question in mind: I might want to get more details on specific action inside the script, i.e: browse into google.com search for a value
I'd like to know how much time the search took - and I can't find an easy solution for this. Any ideas? Regards, Shmuel. On Mon, Apr 23, 2012 at 3:38 AM, Cheen-Pin Lim <[email protected]>wrote: > Hi Shmuel, > > I've yet to try the WebSampler implementation on our test environment. > Unfortunately I'm away for the next 2 weeks, but when I get back to work I > can try out the WebSampler and see if I can replicate the statistics I'm > getting in our production environment. > > regards, > CP > > On 22 April 2012 23:49, Shmuel Krakower <[email protected]> wrote: > > > I'd suggest to HP to drop C language scripting in order to make scripting > > easier / shorter. > > > > As Chaitanya said - browser response times is overrated. > > > > Anyhow, I agree that implementing load tests with hundreds / thousands of > > virtual users with that kind of a sampler(which running a browser) will > > require much more hardware but as CP said, with today's cloud services > this > > is very affordable, thus this is not the issue. > > I think that there are services on the web which already doing this ( > > https://browsermob.com/website-load-testing-features), but I never tried > > those. > > > > CP, if you get different numbers between you monitoring solution and > JMeter > > reports, you should try to understand why that happens. > > It might be for many reasons. Did developing this sampler resolved your > > original problem? Do you get more close numbers now? > > > > Shmuel. > > > > > > On Sun, Apr 22, 2012 at 5:27 AM, Cheen-Pin Lim <[email protected] > > >wrote: > > > > > Hi Chaitanya, > > > > > > I guess I am looking at this from a browser view rather than from the > > > server side. What I tend to find is that current load testing does not > > > properly reflect the browser load times on our site. The reports from > > > Google (in webmasters tools) and newrelic.com give a very different > > > picture > > > from that done during load/performance testing. > > > > > > We use HP Load Runner for our in-house load testing and the numbers we > > get > > > (even when we download 'all' external resources) does not come close to > > the > > > numbers reported by the other tools. I guess I am attempting to > address > > a > > > gap that I see in our in-house load/performance testing, and the > > > integration of webdriver into JMeter could solve this problem. At this > > > stage it is an attempt to try to provide a means to simulate what our > > > production servers see (i.e. using a real browser that will spend time > > > executing JS/CSS). > > > > > > Yes, I agree that the complexity and cost of running a browser instance > > is > > > higher than using the standard HTTP Sampler, but with modern services > > such > > > as Amazon AWS, I would think that these problems would be > > > addressable/solvable. > > > > > > regards, > > > CP > > > > > > On 22 April 2012 06:30, chaitanya bhatt <[email protected]> > > wrote: > > > > > > > *On Sat, Apr 21, 2012 at 4:07 AM, Cheen-Pin Lim < > > [email protected] > > > > >wrote: > > > > >Once you have more load > > > > >applied to the servers/services different characteristics tend to > > > emerge, > > > > >which is why I was thinking that the use of JMeter would be useful. > > > > > > > > *I may totally sound cynical while saying this but honestly the > reason > > > why > > > > I am criticising your idea is mainly due to the week rational behind > > your > > > > motivation to create this feature: > > > > > > > > 1. First of all browser induced latency on the total workload is > > grossly > > > > overrated. Having said that, all that an accurate simulation of > browser > > > > latency would do to your workload is that it would marginally reduce > > the > > > > total hits made on the server, which I think is bad from a > fundamental > > > load > > > > testing perspective. > > > > 2.To accommodate in-memory containers to hold 100s-1000s of virutal > > user > > > > browser sessions is practically impossible without testers having to > > > > significantly size up their test lab resources. Think about the cost > > > > factor. > > > > 3. Looking at todays trend of browser adoption your web driver would > > have > > > > to be equipped with simulation capability multiple types of browsers > > such > > > > as IE, Chromer, Safari etc. etc. and also you would have to consider > > > > simulating various device characteristics on the generated load. > Think > > > > about the complexity. > > > > > > > > @Shmuel: The loadrunner protocol you are talking about is Ajax > > > > TrueClient. HP came out with Ajax TrueClient not with an intention > > > > of generating real browser traffic but it was mainly developed to > > reduce > > > > the complexity of scripting apps that employ unintelligible data > > > > serialization like Google GWT. With this HP's protocol you wouldn't > > have > > > a > > > > necessity of correlating session variables at all and there by you > > would > > > be > > > > significantly reducing the time to develop scripts. However, this > > > protocol > > > > currently has poor user adoption due to various limitations. > > > > > > > > Thanks > > > > Chaitanya M Bhatt > > > > http://www.performancecompetence.com > > > > > > > > On Sat, Apr 21, 2012 at 4:07 AM, Cheen-Pin Lim < > [email protected] > > > > >wrote: > > > > > > > > > Hi, > > > > > > > > > > The tools mentioned help a great deal, however they generally look > at > > > it > > > > > from a single browser to a single server scenario. Once you have > > more > > > > load > > > > > applied to the servers/services different characteristics tend to > > > emerge, > > > > > which is why I was thinking that the use of JMeter would be useful. > > > > > > > > > > Furthermore, if our primary audience is hitting the site using a > > > browser, > > > > > then would it not make sense to use the same tool to simulate the > > load? > > > > > > > > > > Just my thoughts. > > > > > > > > > > regards, > > > > > CP > > > > > > > > > > On 21 April 2012 17:26, chaitanya bhatt <[email protected] > > > > > > wrote: > > > > > > > > > > > Pleanty of awesome tools are already available for free to meet > > your > > > > > goals. > > > > > > > > > > > > Free: > > > > > > https://developers.google.com/pagespeed/ > > > > > > http://yslow.org/ > > > > > > > > > > > > Free or Premium: > > > > > > http://ajax.dynatrace.com/ajax/en/content/c-speed-page-load.aspx > > > > > > > > > > > > > > > > > > Selenium has a webdriver which you can plan on extending - if you > > > will. > > > > > > IMHO Jmeter is not the right platform for this. > > > > > > Thanks > > > > > > Chaitnaya M Bhatt > > > > > > http://www.performancecompetence.com > > > > > > On Fri, Apr 20, 2012 at 11:55 PM, Cheen-Pin Lim < > > > > [email protected] > > > > > > >wrote: > > > > > > > > > > > > > Hi, > > > > > > > > > > > > > > After looking at the current HTTP Sampler, I would like to > > propose > > > a > > > > > > > WebDriver based implementation. The problem is that with a lot > > of > > > > > modern > > > > > > > websites using AJAX, advanced CSS3 (transitions/transforms) to > do > > > the > > > > > > > rendering, the performance problem may not be 'visible' to the > > > > standard > > > > > > > HTTP Sampler. The idea is that if we could use JMeter to > > control a > > > > > > browser > > > > > > > and collect the time it takes to render each 'page' it would > give > > > the > > > > > > users > > > > > > > an understanding of how long a page would take to render. > > > > > > > > > > > > > > Some of the high level goals are as follows: > > > > > > > > > > > > > > 1. Allow any WebDriver supported browser to be used to > perform > > > the > > > > > > > Sampling. > > > > > > > 2. Provide a BeanShell/BSF style pane where the user can > script > > > the > > > > > > > behaviour of WebDriver. > > > > > > > > > > > > > > I've got a very basic implementation working here: > > > > > > > https://github.com/cplim/JMeter - you'll have to switch to the > > > > > > 'webmeter' > > > > > > > branch to see the changes. There are limitations with this > proof > > > of > > > > > > > concept, mainly: > > > > > > > > > > > > > > 1. It only creates Firefox instances > > > > > > > 2. I've only provided Javascript lang support using BSF. > > > > > > > > > > > > > > Some outstanding questions: > > > > > > > > > > > > > > 1. How to support Phone (Android/iOS) based WebDriver, as > these > > > > seem > > > > > to > > > > > > > expect to only connect to a single phone. > > > > > > > > > > > > > > I would be happy to contribute the changes back to JMeter. > Can I > > > get > > > > > > some > > > > > > > feedback on this proposal, and what I need to do to contribute > my > > > > > changes > > > > > > > back to the main JMeter codebase? > > > > > > > > > > > > > > regards, > > > > > > > CP Lim > > > > > > > > > > > > > > > > > > > > > > > > > > > >
