Hello Emilian,
Find my 2 cents below.
Regards
On Mon, Oct 30, 2017 at 1:27 PM, Emilian Bold <emilian.b...@protonmail.ch>
wrote:
> Hello,
>
> I've been following JMeter for a while now, even did a separate project
> based on JMeter (YaMeter.com ) so I figured I should mention some
> (technical) concerns of mine about JMeter.
>
> I thought about sending multiple emails, each about a single aspect, but I
> will just list everything here and people are invited to only answer the
> subset they care about.
>
> 1. Userbase
>
> It seems to me there are no clear metrics about the JMeter userbase. We
> have no phone-home, no plugin portal, no download statistics(!?) so we can
> only use indirect means of estimating the userbase: stackoverflow questions
> and such.
>
> This seems a very poor state of affairs. Resources are scarce: how do you
> know what to focus on if you have very little insight into who your users
> are, what are they doing and where are they struggling?
>
1.1 Metrics
>
> I believe it is possible to add usage metrics to JMeter while respecting
> the ASF policy on privacy, etc.
>
> It's also unclear to me how download stats are not shared by the Apache
> mirrors (any ASF document explaining this policy?).
>
I agree with this.
I have asked Apache infra if we are able to get some metrics but it seems
it's not the case.
I am also very unhappy with this and have to default to:
- stackoverflow questions
- downloads of plugins
- user mails
>
> 2. Speed of development
>
> Compared to some of the plugins, JMeter seems to move very slowly. This
> might create the impression of stability but I believe it's just a sign of
> being resource constrained.
>
Partly true IMO:
- I think we're a bit slower partly because we are maybe more exigent on
quality and have to handle more problems than plugins project:
- backward compat / not breaking plugins
- more use cases
- I find personally the release process too slow:
- I work on JMeter Maven Plugin, whenever I want to release a
version, it takes me 10 minutes max thanks to rultor/maven.(
https://github.com/jmeter-maven-plugin/jmeter-maven-plugin/issues/245)
- On JMeter, we need a release manager to be available, then it's a
manual process partly that takes at least 4 hours and a lot of
configuration for a new release manager (where is svnmucc on Mac OSX) but
more a day I think and finally we have 3 days of vote process, I
think the
last one is Apache process related, I don't think anything can be done.
- Still there are some benefits, I think we find more bugs in this
process, but we could also be releasing more frequently.
But another thing, did you have a look at the frequency at which user
upgrade ?
I still see a lot of people using version that are 5 year old, so if we
released more often, would user upgrade their versions ?
> Are any companies putting resources into JMeter? I see a lot of companies
> / startups trying to make money off JMeter but how many are contributing
> back?
>
Not enough involvement in Core IMO:
- My company Ubik-Ingenierie contributes in 2 ways:
- Patches, bug fixes
- Bigger features like JMeter Web Report , JSON Plugin
- Sponsoring partly my time for working on project
I wonder for example why HTTP2 was not donated immediately to Core.
>
> BTW, one reason I decided to do YaMeter.com instead of trying to push into
> JMeter proper is the approval speed. Getting a patch in seems to take a lot
> of effort and I can only imagine how long it would have take to try
> something as large as what I did with YaMeter.
>
I think we have made improvements in that field, have a look at PR and
Patches taken into account, look at thanks section of every released.
I don't think we are slow to take into account PR provided they are
complete. We give feedback in max 1 days AFAIK. For bugs it is even less.
I don't know statistics of other projects , but from my experience we are
good here.
Regarding YaMeter, I thing we would have integrated it very quickly. I
wrote to you about it.
Regarding Undo, I was commited after the release, but as you know it is
still not satisfying .
But I agree that donating some important piece of work is not simple as the
IP Clearance process is not a light one, we had to use it for integrating
Web Report, it is clearly
something that refrains other donations IMO.
> 3. UI
>
> The JMeter UI needs some improving. The problem is not that it uses Java /
> Swing because even Swing can be made to look good but we need people to
> look into this. Is it "good enough" for people being forced to use JMeter
> in a corporate setting?
Yes because the Core provides the feature you want for that.
Should people only look at UI for such a tool ? No IMO, it would be a
mistake.
I know a lot of Great UIs for "limited (to be polite)" cores, I prefer the
countrary for now :-) .
> Would an user pick it just based on the UI?
>
No
JMeter UI is clearly a bad publicity for the core. I am always disappointed
to read tweet about JMeter looking very 90, being ugly ...
The IDE is powerful but the Swing look is not great and some LAF are even
more awful , for example the Windows one are very ugly on some OSes.
> 3.1 Swing
>
> If Swing remains the future of JMeter, I suggest looking into a better
> framework like Apache NetBeans Platform which provides a lot of useful
> things for desktop Java apps.
>
> Generally speaking, it would be good to split the UI from the "core" so we
> have some path where we evolve to some other UI.
>
If we had time , I would go for a full rewriting either in WEB or Using
Java FX, why not netbeans but I am not an expert of it. I am also very
impressed by IntelliJ look while it is in Swing AFAIK
But I must say that developing UI in JMeter is rather easy and maintainable
from my little experience with such UIs
>
> 3.2 Web based
>
> Alternatively, a web-based interface could be made for JMeter. Any
> interest in this?
>
Yes , but always the same problem , no time
>
> Just like companies use local GitLab installations, I am certain a lot of
> companies would install a local Apache JMeter Web Portal where all the load
> testing would be centralised.
>
> 4. Plugin Manager
>
> It seems unacceptable to me that JMeter does not have a Plugin Manager and
> a plugin portal with some lower-usage plugins (eg. mongodb).
>
> It's acceptable for other plugins to exist elsewhere, but those should be
> just another "PPA" (see Ubuntu PPA) or another "plugin repository" URL to
> register.
>
I absolutely agree. When Andrei started his work on Plugin Manager, I wrote
that this feature should have been in JMeter core.
> 5. Out of the box experience
>
> The way I see it, Apache JMeter is not just a "platform" with some core
> files and external plugins but something usable out of the box.
>
> So, except niche situations, users should be catered by the default JMeter
> install.
>
Yes
>
> 5.1 HTTP2
>
> A major problem being the lack of an official HTTP2 plugin, which people
> have to take from elsewhere. Blazemeter is probably quite happy with their
> implementation being used since it funnels people into their services.
>
Did you get a confirmation from them that they would be ready to donate it ?
> 6. Project based
>
> I find the whole configuration .properties file a very poor solution to
> the notion of a "project". If a JMX needs a .properties change, then those
> two belong together.
>
> JMeter should have the notion of a 'project' and maybe even be capable of
> having two projects open at once. And executing two unrelated tests at
> once. (Same for servers perhaps?)
>
Yes
>
> 6.1 Too many .properties
>
> I also believe JMeter tries to be too configurable sometimes with flags
> for everything.
>
Yes. There has been some cleanup but go ahead for proposing more cleanup.
>
> 7. Module system
>
> JMeter is using a lot of reflection to reinvent its own quirky module
> system (which provides the way the plugins work).
>
> This needs some standardization work.
>
> The NetBeans Platform, for example, comes with its own module system that
> also supports OSGI bundles (another module system). Then, there's the JVM
> service locator (for services) and the Java 9 modules.
>
Probably. OSGI is not a model for me. Too much headache.
JBoss modules was interesting but not enough documented.
Java 9 modules might be a good option.
>
> 8. Remote is using old technologies
>
> RMI is a killer in modern setups. It needs open ports on both the slave
> and master for communication which only makes sense within the same
> intranet and it's a pain to configure when using Cloud resources.
>
> The whole communication between JMeter client and servers should be
> rewritten using more modern protocols.
>
Absolutely, I think the future should be a rest webservice.
> 9. Cloud aware
>
> The notion that the user tries to configure N servers then gives JMeter a
> list of IPs for remote load testing seems... primitive. This is something
> I've been trying to solve with YaMeter.com
>
> 10. Packaging
>
> JMeter should be everywhere. Official Docker files, AMI images, etc.
>
Not sure, that's a too big scope. Why AMI and not other clouds ? Why docker
and not other technologies ?
I think we need to limit the scope and there is enough communauty to help
here as we already see that.
>
> 11. Pipeline aware / agents
>
> JMeter should think more about making it easy to be hooked up into a CI /
> CD pipeline.
>
What do you propose ?
>
> 12. Script files instead of JMX
>
> The XML files are a pain to diff and people like to track / compare
> changes in modern devops shops.
>
> Maybe a switch to YAML might suffice, but I believe a new scripting
> support (based on Groovy or whatever) should be developed.
>
This has been a subject of a lot of discussions, but unfortunately too much
talk and no code.
> --emi
>
> Thanks for your feedback and insights on all these points.
--
Cordialement.
Philippe Mouawad.
Ubik-Ingénierie
UBIK LOAD PACK Web Site <http://www.ubikloadpack.com/>
UBIK LOAD PACK on TWITTER <https://twitter.com/ubikloadpack>
