Hi, +1 to the previous responses, in short you're going to have to experiment with things and decide what works best for you.
I can only share my experience with rest APIs, though my example is on the extreme. In the case I will present, the roundtrip of a single request takes 2-10-20ms and the added size of the request + it's response is smaller than 1kb. But the app needs to handle many concurrent active users. To generate and handle the kind of load that was required for such a test, we used actual physical machines (2x quad core, with 24Gb RAM servers) or EC2 instances (the extra large memory type of instance). And we were able to generate a maximum throughput of 30 thousands requests per second, on a single test client (using the physical machines). The test was able to sustain this value for long periods of time (but it required supervision). Assertions and logging done by the test script was reduced to a bare minimum, enough to make sure that the script is actually executing what we think it does. After experimentations, the test script was configured to run smoothly in a 2Gb Max Heap memory instance of jmeter. It spawned 1000-3000 threads and it ran smoothly (was able to generate constant throughput, when the test target was stable, for long periods of time, 12-48h and the amount of errors that were related to the test client was negligible). To generate 30k requests, 10-12 instances of jmeter were used and the machine CPU usage was 70-90%, but there were many tweaks on the OS side we had to make, mostly related to the network configuration (after we ran into various issues). The idea of a 2Gb instance was to be able to scale horizontally though. In this case, scaling vertically meant using more jmeter instances and this kind of setup proved more stable and easier to maintain. So that was the long version of: jmeter can handle as many threads as the JVM can on that particular machine, which is quite a lot. But with more complex applications, things will get complicated. The bigger the response, the more processing you require within jmeter, the more resources it will require and the number of threads you'll be able to spawn in a stable setup will decrease dramatically. For something like a website, you can overwhelm 1 frontend instance of the target server with 50-100 threads (but they will consume a lot of CPU on your test client). --Adrian On Tue, Aug 12, 2014 at 6:24 PM, Deepak Shetty <shet...@gmail.com> wrote: > Hi > >"docs mention there is no hard & fast rule, it depends on system > hardware/software config & how soon the server responds for APIs under > test." > This is the technically correct answer > > >" lists a number between 200-300 for number of threads" > This is most likely a rule of thumb or a starting point (e.g. I will rarely > exceed 40 threads on my client PC) > > >We now need to put more load on it > It depends on how you define "load" - *Usually* you mean the concurrent > usage that the server sees in which case option B is the correct one.(It's > possible you might be running a long running test , just to see whether > there are memory leaks or something like that and you are using "load" in > that context in which case you probably only need option A. - happens , but > rare) > However lets say you want to test out 10 users accessing your system at the > same time, for an hour , then you might want to use different users (rather > than using the same user or 10 users which means you might get better than > expected results due to server caching user details or something like that) > - in which case you do also need Option A to make Option B realistic. > However you cannot just increase the number of threads without limit , > because at some point of time , either the JMeter machine or the network > might start slowing down your tests which would not happen in a real life > scenario - in which case you now need to add more JMeter machines (either > distributed or standalone) - What number of threads your system supports is > dependent on multiple factors and there is no way out of testing and > validating your results - This has probably been discussed numerous times > in the archives of these mailing lists. > > > > regards > deepak > > > > > > > > > On Mon, Aug 11, 2014 at 6:06 PM, Shilpa Kulkarni <s...@payasonline.com> > wrote: > > > Hi > > > > I have a question about thread count limit in jmeter. Some of the - > > slightly older documentation on the internet - lists a number between > > 200-300 for number of threads. Some docs mention there is no hard & fast > > rule, it depends on system hardware/software config & how soon the server > > responds for APIs under test. > > > > I have built a basic API test plan for a mobile/server type of > application > > & tested for 2 users in csv file & thread count = 10. We now need to put > > more load on it & I was checking out for any guide lines here. What are > my > > options? A + B or just one of them? Are there any other options? > > A. Increase number of users in csv file > > B. Increase thread count > > > > Thanks > > SK > > >
