Hello, I've been following JMeter for a while now, even did a separate project based on JMeter (YaMeter.com ) so I figured I should mention some (technical) concerns of mine about JMeter.
I thought about sending multiple emails, each about a single aspect, but I will just list everything here and people are invited to only answer the subset they care about. 1. Userbase It seems to me there are no clear metrics about the JMeter userbase. We have no phone-home, no plugin portal, no download statistics(!?) so we can only use indirect means of estimating the userbase: stackoverflow questions and such. This seems a very poor state of affairs. Resources are scarce: how do you know what to focus on if you have very little insight into who your users are, what are they doing and where are they struggling? 1.1 Metrics I believe it is possible to add usage metrics to JMeter while respecting the ASF policy on privacy, etc. It's also unclear to me how download stats are not shared by the Apache mirrors (any ASF document explaining this policy?). 2. Speed of development Compared to some of the plugins, JMeter seems to move very slowly. This might create the impression of stability but I believe it's just a sign of being resource constrained. Are any companies putting resources into JMeter? I see a lot of companies / startups trying to make money off JMeter but how many are contributing back? BTW, one reason I decided to do YaMeter.com instead of trying to push into JMeter proper is the approval speed. Getting a patch in seems to take a lot of effort and I can only imagine how long it would have take to try something as large as what I did with YaMeter. 3. UI The JMeter UI needs some improving. The problem is not that it uses Java / Swing because even Swing can be made to look good but we need people to look into this. Is it "good enough" for people being forced to use JMeter in a corporate setting? Would an user pick it just based on the UI? 3.1 Swing If Swing remains the future of JMeter, I suggest looking into a better framework like Apache NetBeans Platform which provides a lot of useful things for desktop Java apps. Generally speaking, it would be good to split the UI from the "core" so we have some path where we evolve to some other UI. 3.2 Web based Alternatively, a web-based interface could be made for JMeter. Any interest in this? Just like companies use local GitLab installations, I am certain a lot of companies would install a local Apache JMeter Web Portal where all the load testing would be centralised. 4. Plugin Manager It seems unacceptable to me that JMeter does not have a Plugin Manager and a plugin portal with some lower-usage plugins (eg. mongodb). It's acceptable for other plugins to exist elsewhere, but those should be just another "PPA" (see Ubuntu PPA) or another "plugin repository" URL to register. 5. Out of the box experience The way I see it, Apache JMeter is not just a "platform" with some core files and external plugins but something usable out of the box. So, except niche situations, users should be catered by the default JMeter install. 5.1 HTTP2 A major problem being the lack of an official HTTP2 plugin, which people have to take from elsewhere. Blazemeter is probably quite happy with their implementation being used since it funnels people into their services. 6. Project based I find the whole configuration .properties file a very poor solution to the notion of a "project". If a JMX needs a .properties change, then those two belong together. JMeter should have the notion of a 'project' and maybe even be capable of having two projects open at once. And executing two unrelated tests at once. (Same for servers perhaps?) 6.1 Too many .properties I also believe JMeter tries to be too configurable sometimes with flags for everything. 7. Module system JMeter is using a lot of reflection to reinvent its own quirky module system (which provides the way the plugins work). This needs some standardization work. The NetBeans Platform, for example, comes with its own module system that also supports OSGI bundles (another module system). Then, there's the JVM service locator (for services) and the Java 9 modules. 8. Remote is using old technologies RMI is a killer in modern setups. It needs open ports on both the slave and master for communication which only makes sense within the same intranet and it's a pain to configure when using Cloud resources. The whole communication between JMeter client and servers should be rewritten using more modern protocols. 9. Cloud aware The notion that the user tries to configure N servers then gives JMeter a list of IPs for remote load testing seems... primitive. This is something I've been trying to solve with YaMeter.com 10. Packaging JMeter should be everywhere. Official Docker files, AMI images, etc. 11. Pipeline aware / agents JMeter should think more about making it easy to be hooked up into a CI / CD pipeline. 12. Script files instead of JMX The XML files are a pain to diff and people like to track / compare changes in modern devops shops. Maybe a switch to YAML might suffice, but I believe a new scripting support (based on Groovy or whatever) should be developed. --emi
