buildbot success in on jmeter-trunk
The Buildbot has detected a restored build on builder jmeter-trunk while building . Full details are available at: https://ci.apache.org/builders/jmeter-trunk/builds/3112 Buildbot URL: https://ci.apache.org/ Buildslave for this Build: bb_slave1_ubuntu Build Reason: The AnyBranchScheduler scheduler named 'on-jmeter-commit' triggered this build Build Source Stamp: [branch jmeter/trunk] 1813831 Blamelist: pmouawad Build succeeded! Sincerely, -The Buildbot
buildbot failure in on jmeter-trunk
The Buildbot has detected a new failure on builder jmeter-trunk while building . Full details are available at: https://ci.apache.org/builders/jmeter-trunk/builds/3111 Buildbot URL: https://ci.apache.org/ Buildslave for this Build: bb_slave1_ubuntu Build Reason: The AnyBranchScheduler scheduler named 'on-jmeter-commit' triggered this build Build Source Stamp: [branch jmeter/trunk] 1813826 Blamelist: pmouawad BUILD FAILED: failed shell_3 Sincerely, -The Buildbot
Re: Some JMeter (technical) concerns
>>>Can I start a discussion with ASF legal@, infra@ and the rest about how >> an Apache project could get some metrics or add "telemetry". >>Yes go ahead I've opened https://issues.apache.org/jira/browse/INFRA-15406 I'll also open another issue with legal@ about metrics. 2. Speed of development [...] But another thing, did you have a look at the frequency at which user upgrade ? >>>Where would I see such info? >> >Not that simple, it is more something I see from our work and the answers > we provide on : > - stackoverflow questions > > - the user mailing list > > And also https://jmeter-plugins.org/ => Metrics are better IMO since user > see that upgrades exist I see. It's sad to depend on jmeter-plugins.org, a non-Apache site, for such important information, no? >>>I have to prepare multiple patches from YaMeter. The code is also using >> some NetBeans Platform frameworks so I would have to un-tangle those and >> use simpler things if I have to port it to JMeter as a plugin or something. >> >I'd be happy if you could but anyway good luck with it. I'm not trying to make money off YaMeter, it's all free, no ads, no nothing. In a way, it's how I envision JMeter should look like. Not just as an UI, but as a modular app with a plugin manager, updates, cloud friendly, etc. Preparing individual patches for PRs just takes an ungodly amount of time. >>>This might be an interesting exercise. >>What sub-set of JMeter could we provide in the 1st release of a web UI? >> Just some HTTP request and a visualization would be a great start for users >> (this is where some metrics would help to see what users use most). >> >I don't know about that, what framework would you be using ? What > architecture ? > I think this needs some architectural work to handle Plugins. > How would be provide the 2 UIs together ? > I feel it's a huge piece of work, but I am ready to go for it provided we > have some strong engagement from volunteers for some time. It's huge only if you want to port the entire JMeter to the web in a single release. The 1st web release has to be a super-lite JMeter. It's important to start it and give a direction. If the users like it and it's any good we can grow more rapidly. >>>How about JMeter provides a Plugin Manager out of the box in the next >> release? >> What's so complex about it? >> >Nothing, if you have a plan, go ahead, I'll be happy to merge PRs. OK, I will try to do this. > Very bad thinking IMO, we could for example work on more frequent releases > and an improvements of the release process. I agree. >For me it's not a prioritary problem frankly. I see the current > architecture which is simple provides enough power and this is proven by > the number of plugins in the market. > Unless you have a full backward compat solution, I'm for now not convinced > about the benefit vs the sum of problems we'll face. Backward compatibility seems doable. Even if it's not perfect, it will be a good way to see which plugins the users have :-) > If we go for providing a Cloud solution, I think JCloud should be used. > I don't want to privilege one Cloud provider unless we have some > involvement from that provider of course. I'm unsure about JCloud. Even the official client SDKs are under development sometimes so what are the chances of JCloud being better? OTOH, JCloud seems to just talk to the REST endpoints which is something that sounds logical. So, they are not a wrapper on top of multiple SDKs, just REST clients. --emi
Re: Integrating Darcula LAF as JMeter main LAF
Yes, sadly the project is not in Maven Central, which is annoying. I use Maven for YaMeter and the Darcula JAR needs special treatment since it's local... --emi > Original Message >Subject: Integrating Darcula LAF as JMeter main LAF >Local Time: October 30, 2017 8:28 PM >UTC Time: October 30, 2017 6:28 PM >From: philippe.moua...@gmail.com >To: dev@jmeter.apache.org > >Hello, > Following Emilian Bold suggestion about Darcula LAF, I've had a look at it. > Integration seems easy. > > But I see 2 minor to major problems: > - Library does not seem to have a versioning system: > - https://github.com/bulenkov/Darcula/issues/40 > - There is no maven artifact : > - https://github.com/bulenkov/Darcula/issues/7 > - There seems to be an issue with Java9, but I think this one is not a > problem > > But I'll move towards integration by for now managing a download as we do > for Checkstyle. > The JAR is available at: > - https://github.com/bulenkov/Darcula/raw/master/build/darcula.jar > > My main concern is that it's not versioned clearly but I think we can work > with commits ids: > - >https://github.com/bulenkov/Darcula/blob/e208efb96f70e4be9dc362fbb46f6e181ef501dd/build/darcula.jar > > > If I am wrong, please correct me here. > Thanks > >Cordialement. > Philippe Mouawad. >
Integrating Darcula LAF as JMeter main LAF
Hello, Following Emilian Bold suggestion about Darcula LAF, I've had a look at it. Integration seems easy. But I see 2 minor to major problems: - Library does not seem to have a versioning system: - https://github.com/bulenkov/Darcula/issues/40 - There is no maven artifact : - https://github.com/bulenkov/Darcula/issues/7 - There seems to be an issue with Java9, but I think this one is not a problem But I'll move towards integration by for now managing a download as we do for Checkstyle. The JAR is available at: - https://github.com/bulenkov/Darcula/raw/master/build/darcula.jar My main concern is that it's not versioned clearly but I think we can work with commits ids: - https://github.com/bulenkov/Darcula/blob/e208efb96f70e4be9dc362fbb46f6e181ef501dd/build/darcula.jar If I am wrong, please correct me here. Thanks -- Cordialement. Philippe Mouawad.
HC4.5.X Migration
Hello I've started few weeks ago a migration to last 4.5.X APIs. I am close to something that looks usable now but I'd like some early feedback on this if possible. Code is located here: - https://github.com/ubikloadpack/jmeter/tree/HC4_FULL_MIGRATION PRs , comments ... welcome. While working on this I noticed the following: - We don't really support Digest Scheme, this PR will do that - Kerberos although supported is not tested, what can be done to improve ? - Coverage of HTTP is not enough , for example access through proxy is not tested. Anybody knows a simple implementation of a proxy that offers auth through Basic or Digest ? Thanks -- Regards. Philippe
Re: Some JMeter (technical) concerns
Hi, I've tested it with a ugly POC , wow ! I love it ! Thanks Emilian !!! On Mac OSX, it seems to work fine. Do you know what it looks like on Linux and Windows ? In the past I faced issues when plugging existing lafs, mainly "No Copy / Paste". On Mac it seems ok. Regards On Mon, Oct 30, 2017 at 6:13 PM, Antonio Gomes Rodrigues wrote: > 2017-10-30 18:07 GMT+01:00 Philippe Mouawad : > > > On Mon, Oct 30, 2017 at 5:10 PM, Emilian Bold < > emilian.b...@protonmail.ch> > > wrote: > > > > > Answers bellow: > > > > > > >1.1 Metrics > > > >>I believe it is possible to add usage metrics to JMeter while > > respecting > > > >> the ASF policy on privacy, etc. > > > >>It's also unclear to me how download stats are not shared by the > Apache > > > >> mirrors (any ASF document explaining this policy?). > > > >> > > > >I agree with this. > > > > I have asked Apache infra if we are able to get some metrics but it > > seems > > > > it's not the case. > > > > I am also very unhappy with this and have to default to: > > > > - stackoverflow questions > > > > - downloads of plugins > > > > - user mails > > > > > > > > > Can I start a discussion with ASF legal@, infra@ and the rest about > how > > > an Apache project could get some metrics or add "telemetry". > > > > > Yes go ahead > > > > > > > > > > > >>2. Speed of development > > > [...] > > > > But another thing, did you have a look at the frequency at which user > > > > upgrade ? > > > > > > Where would I see such info? > > > > > > > Not that simple, it is more something I see from our work and the answers > > we provide on : > > > >- stackoverflow questions > > > > > >- the user mailing list > > > > And also https://jmeter-plugins.org/ => Metrics are better IMO since > user > > see that upgrades exist > > > > > > > > > > > I still see a lot of people using version that are 5 year old, so if > we > > > > released more often, would user upgrade their versions ? > > > > > > They would, if JMeter would tell them about this and provide a simple > > > one-click way to auto-update (think how Google Chrome autoupdates, > etc). > > > > > Yes > > > > > > > > >>Are any companies putting resources into JMeter? I see a lot of > > companies > > > >> / startups trying to make money off JMeter but how many are > > contributing > > > >> back? > > > >> > > > >Not enough involvement in Core IMO: > > > > - My company Ubik-Ingenierie contributes in 2 ways: > > > > - Patches, bug fixes > > > > - Bigger features like JMeter Web Report , JSON Plugin > > > > - Sponsoring partly my time for working on project > > > > > > > > > > I wonder for example why HTTP2 was not donated immediately to Core. > > > > > > Because it's better for them to control the plugin. > > > > > > > I am not sure. > > As you can tell from your own discussion, being late on providing HTTP2 > is > > not a good point for JMeter and as such for any commercial product > relying > > on it IMO. > > So in a way, providing it as a plugin and not in core is a mistake IMO. > > I also find it a bit discouraging as a member of PMC to see so few > > contributions from big players, and on that side also I find it's a big > > mistake from > > commercial project that make money from it not to contribute back to > core. > > > > I still hope this will change. > > > > Hopefully we have contributions from individuals and it's great. > > > > > > > > > > >>BTW, one reason I decided to do YaMeter.com instead of trying to push > > > into > > > >> JMeter proper is the approval speed. Getting a patch in seems to > take > > a > > > lot > > > >> of effort and I can only imagine how long it would have take to try > > > >> something as large as what I did with YaMeter. > > > >> > > > >I think we have made improvements in that field, have a look at PR and > > > > Patches taken into account, look at thanks section of every released. > > > > I don't think we are slow to take into account PR provided they are > > > > complete. We give feedback in max 1 days AFAIK. For bugs it is even > > less. > > > > I don't know statistics of other projects , but from my experience we > > are > > > > good here. > > > > > > I guess the speed matters less if it's a one-off bugfix that you want > to > > > upstream. > > > > > > In my case, I had the time to contribute and felt it would have taken > me > > > forever to wait for each fix. Due to bad luck, this also happened near > a > > > JMeter release which implied a code freeze of sorts... > > > > > Yes it was that cause > > > > > > > > > Regarding YaMeter, I think we would have integrated it very quickly. > I > > > > wrote to you about it. > > > > > > I have to prepare multiple patches from YaMeter. The code is also using > > > some NetBeans Platform frameworks so I would have to un-tangle those > and > > > use simpler things if I have to port it to JMeter as a plugin or > > something. > > > > > > > I'd be happy if you could but anyway good luck with it. > > > > > > > > > Regarding Undo, I was commited aft
Re: Some JMeter (technical) concerns
2017-10-30 18:07 GMT+01:00 Philippe Mouawad : > On Mon, Oct 30, 2017 at 5:10 PM, Emilian Bold > wrote: > > > Answers bellow: > > > > >1.1 Metrics > > >>I believe it is possible to add usage metrics to JMeter while > respecting > > >> the ASF policy on privacy, etc. > > >>It's also unclear to me how download stats are not shared by the Apache > > >> mirrors (any ASF document explaining this policy?). > > >> > > >I agree with this. > > > I have asked Apache infra if we are able to get some metrics but it > seems > > > it's not the case. > > > I am also very unhappy with this and have to default to: > > > - stackoverflow questions > > > - downloads of plugins > > > - user mails > > > > > > Can I start a discussion with ASF legal@, infra@ and the rest about how > > an Apache project could get some metrics or add "telemetry". > > > Yes go ahead > > > > > > > >>2. Speed of development > > [...] > > > But another thing, did you have a look at the frequency at which user > > > upgrade ? > > > > Where would I see such info? > > > > Not that simple, it is more something I see from our work and the answers > we provide on : > >- stackoverflow questions > > >- the user mailing list > > And also https://jmeter-plugins.org/ => Metrics are better IMO since user > see that upgrades exist > > > > > > > I still see a lot of people using version that are 5 year old, so if we > > > released more often, would user upgrade their versions ? > > > > They would, if JMeter would tell them about this and provide a simple > > one-click way to auto-update (think how Google Chrome autoupdates, etc). > > > Yes > > > > > >>Are any companies putting resources into JMeter? I see a lot of > companies > > >> / startups trying to make money off JMeter but how many are > contributing > > >> back? > > >> > > >Not enough involvement in Core IMO: > > > - My company Ubik-Ingenierie contributes in 2 ways: > > > - Patches, bug fixes > > > - Bigger features like JMeter Web Report , JSON Plugin > > > - Sponsoring partly my time for working on project > > > > > > > I wonder for example why HTTP2 was not donated immediately to Core. > > > > Because it's better for them to control the plugin. > > > > I am not sure. > As you can tell from your own discussion, being late on providing HTTP2 is > not a good point for JMeter and as such for any commercial product relying > on it IMO. > So in a way, providing it as a plugin and not in core is a mistake IMO. > I also find it a bit discouraging as a member of PMC to see so few > contributions from big players, and on that side also I find it's a big > mistake from > commercial project that make money from it not to contribute back to core. > > I still hope this will change. > > Hopefully we have contributions from individuals and it's great. > > > > > > >>BTW, one reason I decided to do YaMeter.com instead of trying to push > > into > > >> JMeter proper is the approval speed. Getting a patch in seems to take > a > > lot > > >> of effort and I can only imagine how long it would have take to try > > >> something as large as what I did with YaMeter. > > >> > > >I think we have made improvements in that field, have a look at PR and > > > Patches taken into account, look at thanks section of every released. > > > I don't think we are slow to take into account PR provided they are > > > complete. We give feedback in max 1 days AFAIK. For bugs it is even > less. > > > I don't know statistics of other projects , but from my experience we > are > > > good here. > > > > I guess the speed matters less if it's a one-off bugfix that you want to > > upstream. > > > > In my case, I had the time to contribute and felt it would have taken me > > forever to wait for each fix. Due to bad luck, this also happened near a > > JMeter release which implied a code freeze of sorts... > > > Yes it was that cause > > > > > > Regarding YaMeter, I think we would have integrated it very quickly. I > > > wrote to you about it. > > > > I have to prepare multiple patches from YaMeter. The code is also using > > some NetBeans Platform frameworks so I would have to un-tangle those and > > use simpler things if I have to port it to JMeter as a plugin or > something. > > > > I'd be happy if you could but anyway good luck with it. > > > > > > Regarding Undo, I was commited after the release, but as you know it is > > > still not satisfying . > > > > Yes, but not because of my patch which fixed quite a few things. > > > True. Your patch improved a lot but still feature is not mature enough yet. > > > > > >>3. UI > > > > > JMeter UI is clearly a bad publicity for the core. I am always > > disappointed > > > to read tweet about JMeter looking very 90, being ugly ... > > > > > > The IDE is powerful but the Swing look is not great and some LAF are > even > > > more awful , for example the Windows one are very ugly on some OSes. > > > > I believe YaMeter looks great and it's just a LnF (Darcula from > IntelliJ). > > > > How about we add that to J
Re: Some JMeter (technical) concerns
On Mon, Oct 30, 2017 at 5:10 PM, Emilian Bold wrote: > Answers bellow: > > >1.1 Metrics > >>I believe it is possible to add usage metrics to JMeter while respecting > >> the ASF policy on privacy, etc. > >>It's also unclear to me how download stats are not shared by the Apache > >> mirrors (any ASF document explaining this policy?). > >> > >I agree with this. > > I have asked Apache infra if we are able to get some metrics but it seems > > it's not the case. > > I am also very unhappy with this and have to default to: > > - stackoverflow questions > > - downloads of plugins > > - user mails > > > Can I start a discussion with ASF legal@, infra@ and the rest about how > an Apache project could get some metrics or add "telemetry". > Yes go ahead > > > >>2. Speed of development > [...] > > But another thing, did you have a look at the frequency at which user > > upgrade ? > > Where would I see such info? > Not that simple, it is more something I see from our work and the answers we provide on : - stackoverflow questions - the user mailing list And also https://jmeter-plugins.org/ => Metrics are better IMO since user see that upgrades exist > > > I still see a lot of people using version that are 5 year old, so if we > > released more often, would user upgrade their versions ? > > They would, if JMeter would tell them about this and provide a simple > one-click way to auto-update (think how Google Chrome autoupdates, etc). > Yes > > >>Are any companies putting resources into JMeter? I see a lot of companies > >> / startups trying to make money off JMeter but how many are contributing > >> back? > >> > >Not enough involvement in Core IMO: > > - My company Ubik-Ingenierie contributes in 2 ways: > > - Patches, bug fixes > > - Bigger features like JMeter Web Report , JSON Plugin > > - Sponsoring partly my time for working on project > > > > I wonder for example why HTTP2 was not donated immediately to Core. > > Because it's better for them to control the plugin. > I am not sure. As you can tell from your own discussion, being late on providing HTTP2 is not a good point for JMeter and as such for any commercial product relying on it IMO. So in a way, providing it as a plugin and not in core is a mistake IMO. I also find it a bit discouraging as a member of PMC to see so few contributions from big players, and on that side also I find it's a big mistake from commercial project that make money from it not to contribute back to core. I still hope this will change. Hopefully we have contributions from individuals and it's great. > > >>BTW, one reason I decided to do YaMeter.com instead of trying to push > into > >> JMeter proper is the approval speed. Getting a patch in seems to take a > lot > >> of effort and I can only imagine how long it would have take to try > >> something as large as what I did with YaMeter. > >> > >I think we have made improvements in that field, have a look at PR and > > Patches taken into account, look at thanks section of every released. > > I don't think we are slow to take into account PR provided they are > > complete. We give feedback in max 1 days AFAIK. For bugs it is even less. > > I don't know statistics of other projects , but from my experience we are > > good here. > > I guess the speed matters less if it's a one-off bugfix that you want to > upstream. > > In my case, I had the time to contribute and felt it would have taken me > forever to wait for each fix. Due to bad luck, this also happened near a > JMeter release which implied a code freeze of sorts... > Yes it was that cause > > > Regarding YaMeter, I think we would have integrated it very quickly. I > > wrote to you about it. > > I have to prepare multiple patches from YaMeter. The code is also using > some NetBeans Platform frameworks so I would have to un-tangle those and > use simpler things if I have to port it to JMeter as a plugin or something. > I'd be happy if you could but anyway good luck with it. > > > Regarding Undo, I was commited after the release, but as you know it is > > still not satisfying . > > Yes, but not because of my patch which fixed quite a few things. > True. Your patch improved a lot but still feature is not mature enough yet. > > >>3. UI > > > JMeter UI is clearly a bad publicity for the core. I am always > disappointed > > to read tweet about JMeter looking very 90, being ugly ... > > > > The IDE is powerful but the Swing look is not great and some LAF are even > > more awful , for example the Windows one are very ugly on some OSes. > > I believe YaMeter looks great and it's just a LnF (Darcula from IntelliJ). > > How about we add that to JMeter? > I'd love it ! Is it possible ? > >>3.1 Swing > >>If Swing remains the future of JMeter, I suggest looking into a better > >> framework like Apache NetBeans Platform which provides a lot of useful > >> things for desktop Java apps. > >>Generally speaking, it would be good to split the UI from the "core" so > we > >> have som
Re: Some JMeter (technical) concerns
Answers bellow: >1.1 Metrics >>I believe it is possible to add usage metrics to JMeter while respecting >> the ASF policy on privacy, etc. >>It's also unclear to me how download stats are not shared by the Apache >> mirrors (any ASF document explaining this policy?). >> >I agree with this. > I have asked Apache infra if we are able to get some metrics but it seems > it's not the case. > I am also very unhappy with this and have to default to: > - stackoverflow questions > - downloads of plugins > - user mails Can I start a discussion with ASF legal@, infra@ and the rest about how an Apache project could get some metrics or add "telemetry". >>2. Speed of development [...] > But another thing, did you have a look at the frequency at which user > upgrade ? Where would I see such info? > I still see a lot of people using version that are 5 year old, so if we > released more often, would user upgrade their versions ? They would, if JMeter would tell them about this and provide a simple one-click way to auto-update (think how Google Chrome autoupdates, etc). >>Are any companies putting resources into JMeter? I see a lot of companies >> / startups trying to make money off JMeter but how many are contributing >> back? >> >Not enough involvement in Core IMO: > - My company Ubik-Ingenierie contributes in 2 ways: > - Patches, bug fixes > - Bigger features like JMeter Web Report , JSON Plugin > - Sponsoring partly my time for working on project > I wonder for example why HTTP2 was not donated immediately to Core. Because it's better for them to control the plugin. >>BTW, one reason I decided to do YaMeter.com instead of trying to push into >> JMeter proper is the approval speed. Getting a patch in seems to take a lot >> of effort and I can only imagine how long it would have take to try >> something as large as what I did with YaMeter. >> >I think we have made improvements in that field, have a look at PR and > Patches taken into account, look at thanks section of every released. > I don't think we are slow to take into account PR provided they are > complete. We give feedback in max 1 days AFAIK. For bugs it is even less. > I don't know statistics of other projects , but from my experience we are > good here. I guess the speed matters less if it's a one-off bugfix that you want to upstream. In my case, I had the time to contribute and felt it would have taken me forever to wait for each fix. Due to bad luck, this also happened near a JMeter release which implied a code freeze of sorts... > Regarding YaMeter, I thing we would have integrated it very quickly. I > wrote to you about it. I have to prepare multiple patches from YaMeter. The code is also using some NetBeans Platform frameworks so I would have to un-tangle those and use simpler things if I have to port it to JMeter as a plugin or something. > Regarding Undo, I was commited after the release, but as you know it is > still not satisfying . Yes, but not because of my patch which fixed quite a few things. >>3. UI > JMeter UI is clearly a bad publicity for the core. I am always disappointed > to read tweet about JMeter looking very 90, being ugly ... > > The IDE is powerful but the Swing look is not great and some LAF are even > more awful , for example the Windows one are very ugly on some OSes. I believe YaMeter looks great and it's just a LnF (Darcula from IntelliJ). How about we add that to JMeter? >>3.1 Swing >>If Swing remains the future of JMeter, I suggest looking into a better >> framework like Apache NetBeans Platform which provides a lot of useful >> things for desktop Java apps. >>Generally speaking, it would be good to split the UI from the "core" so we >> have some path where we evolve to some other UI. >> >If we had time , I would go for a full rewriting either in WEB or Using > Java FX, why not netbeans but I am not an expert of it. I am also very > impressed by IntelliJ look while it is in Swing AFAIK IntelliJ also has the IntelliJ Platform, just like Apache NetBeans Platform. Both are Apache 2.0. >>3.2 Web based >>Alternatively, a web-based interface could be made for JMeter. Any >> interest in this? >> >Yes , but always the same problem , no time This might be an interesting exercise. What sub-set of JMeter could we provide in the 1st release of a web UI? Just some HTTP request and a visualization would be a great start for users (this is where some metrics would help to see what users use most). >>4. Plugin Manager >>It seems unacceptable to me that JMeter does not have a Plugin Manager and >> a plugin portal with some lower-usage plugins (eg. mongodb). >>It's acceptable for other plugins to exist elsewhere, but those should be >> just another "PPA" (see Ubuntu PPA) or another "plugin repository" URL to >> register. >> > >I absolutely agree. When Andrei started his work on Plugin Manager, I wrote > that this feature should have been in JMeter core. How about JMeter provides a Plugin Manager out of
svnmucc on macOS (was: Some JMeter (technical) concerns)
On 30 October 2017 at 13:11, Philippe Mouawad wrote: > Hello Emilian, > Find my 2 cents below. > > > configuration for a new release manager (where is svnmucc on Mac OSX) > but I have: $ svnmucc --version svnmucc, version 1.9.7 (r1800392) compiled Aug 10 2017, 21:32:31 on x86_64-apple-darwin16.3.0 Not all distros have the tools, but they are available.
Re: Some JMeter (technical) concerns
Hi My opinion We are aware and a lot of them have been already discuted The problem is the lack of time Antonio Le 30 oct. 2017 14:11, "Philippe Mouawad" a écrit : > Hello Emilian, > Find my 2 cents below. > > > Regards > > On Mon, Oct 30, 2017 at 1:27 PM, Emilian Bold > wrote: > > > Hello, > > > > I've been following JMeter for a while now, even did a separate project > > based on JMeter (YaMeter.com ) so I figured I should mention some > > (technical) concerns of mine about JMeter. > > > > I thought about sending multiple emails, each about a single aspect, but > I > > will just list everything here and people are invited to only answer the > > subset they care about. > > > > 1. Userbase > > > > It seems to me there are no clear metrics about the JMeter userbase. We > > have no phone-home, no plugin portal, no download statistics(!?) so we > can > > only use indirect means of estimating the userbase: stackoverflow > questions > > and such. > > > > This seems a very poor state of affairs. Resources are scarce: how do you > > know what to focus on if you have very little insight into who your users > > are, what are they doing and where are they struggling? > > > > > 1.1 Metrics > > > > I believe it is possible to add usage metrics to JMeter while respecting > > the ASF policy on privacy, etc. > > > > It's also unclear to me how download stats are not shared by the Apache > > mirrors (any ASF document explaining this policy?). > > > > I agree with this. > I have asked Apache infra if we are able to get some metrics but it seems > it's not the case. > I am also very unhappy with this and have to default to: > >- stackoverflow questions >- downloads of plugins >- user mails > > > > > > > > 2. Speed of development > > > > Compared to some of the plugins, JMeter seems to move very slowly. This > > might create the impression of stability but I believe it's just a sign > of > > being resource constrained. > > > > Partly true IMO: > >- I think we're a bit slower partly because we are maybe more exigent on >quality and have to handle more problems than plugins project: > - backward compat / not breaking plugins > - more use cases >- I find personally the release process too slow: > - I work on JMeter Maven Plugin, whenever I want to release a > version, it takes me 10 minutes max thanks to rultor/maven.( > https://github.com/jmeter-maven-plugin/jmeter-maven- > plugin/issues/245) > - On JMeter, we need a release manager to be available, then it's a > manual process partly that takes at least 4 hours and a lot of > configuration for a new release manager (where is svnmucc on Mac > OSX) but > more a day I think and finally we have 3 days of vote process, I > think the > last one is Apache process related, I don't think anything can be > done. > - Still there are some benefits, I think we find more bugs in this > process, but we could also be releasing more frequently. > > But another thing, did you have a look at the frequency at which user > upgrade ? > > I still see a lot of people using version that are 5 year old, so if we > released more often, would user upgrade their versions ? > > > > > Are any companies putting resources into JMeter? I see a lot of companies > > / startups trying to make money off JMeter but how many are contributing > > back? > > > > Not enough involvement in Core IMO: > >- My company Ubik-Ingenierie contributes in 2 ways: > - Patches, bug fixes > - Bigger features like JMeter Web Report , JSON Plugin > - Sponsoring partly my time for working on project > > I wonder for example why HTTP2 was not donated immediately to Core. > > > > > BTW, one reason I decided to do YaMeter.com instead of trying to push > into > > JMeter proper is the approval speed. Getting a patch in seems to take a > lot > > of effort and I can only imagine how long it would have take to try > > something as large as what I did with YaMeter. > > > > I think we have made improvements in that field, have a look at PR and > Patches taken into account, look at thanks section of every released. > I don't think we are slow to take into account PR provided they are > complete. We give feedback in max 1 days AFAIK. For bugs it is even less. > I don't know statistics of other projects , but from my experience we are > good here. > Regarding YaMeter, I thing we would have integrated it very quickly. I > wrote to you about it. > > Regarding Undo, I was commited after the release, but as you know it is > still not satisfying . > > But I agree that donating some important piece of work is not simple as the > IP Clearance process is not a light one, we had to use it for integrating > Web Report, it is clearly > something that refrains other donations IMO. > > > > > 3. UI > > > > The JMeter UI needs some improving. The problem is not that it uses Java > / > > Swing because even Swing can be made to look good bu
Re: Some JMeter (technical) concerns
Hello Emilian, Find my 2 cents below. Regards On Mon, Oct 30, 2017 at 1:27 PM, Emilian Bold wrote: > Hello, > > I've been following JMeter for a while now, even did a separate project > based on JMeter (YaMeter.com ) so I figured I should mention some > (technical) concerns of mine about JMeter. > > I thought about sending multiple emails, each about a single aspect, but I > will just list everything here and people are invited to only answer the > subset they care about. > > 1. Userbase > > It seems to me there are no clear metrics about the JMeter userbase. We > have no phone-home, no plugin portal, no download statistics(!?) so we can > only use indirect means of estimating the userbase: stackoverflow questions > and such. > > This seems a very poor state of affairs. Resources are scarce: how do you > know what to focus on if you have very little insight into who your users > are, what are they doing and where are they struggling? > 1.1 Metrics > > I believe it is possible to add usage metrics to JMeter while respecting > the ASF policy on privacy, etc. > > It's also unclear to me how download stats are not shared by the Apache > mirrors (any ASF document explaining this policy?). > I agree with this. I have asked Apache infra if we are able to get some metrics but it seems it's not the case. I am also very unhappy with this and have to default to: - stackoverflow questions - downloads of plugins - user mails > > 2. Speed of development > > Compared to some of the plugins, JMeter seems to move very slowly. This > might create the impression of stability but I believe it's just a sign of > being resource constrained. > Partly true IMO: - I think we're a bit slower partly because we are maybe more exigent on quality and have to handle more problems than plugins project: - backward compat / not breaking plugins - more use cases - I find personally the release process too slow: - I work on JMeter Maven Plugin, whenever I want to release a version, it takes me 10 minutes max thanks to rultor/maven.( https://github.com/jmeter-maven-plugin/jmeter-maven-plugin/issues/245) - On JMeter, we need a release manager to be available, then it's a manual process partly that takes at least 4 hours and a lot of configuration for a new release manager (where is svnmucc on Mac OSX) but more a day I think and finally we have 3 days of vote process, I think the last one is Apache process related, I don't think anything can be done. - Still there are some benefits, I think we find more bugs in this process, but we could also be releasing more frequently. But another thing, did you have a look at the frequency at which user upgrade ? I still see a lot of people using version that are 5 year old, so if we released more often, would user upgrade their versions ? > Are any companies putting resources into JMeter? I see a lot of companies > / startups trying to make money off JMeter but how many are contributing > back? > Not enough involvement in Core IMO: - My company Ubik-Ingenierie contributes in 2 ways: - Patches, bug fixes - Bigger features like JMeter Web Report , JSON Plugin - Sponsoring partly my time for working on project I wonder for example why HTTP2 was not donated immediately to Core. > > BTW, one reason I decided to do YaMeter.com instead of trying to push into > JMeter proper is the approval speed. Getting a patch in seems to take a lot > of effort and I can only imagine how long it would have take to try > something as large as what I did with YaMeter. > I think we have made improvements in that field, have a look at PR and Patches taken into account, look at thanks section of every released. I don't think we are slow to take into account PR provided they are complete. We give feedback in max 1 days AFAIK. For bugs it is even less. I don't know statistics of other projects , but from my experience we are good here. Regarding YaMeter, I thing we would have integrated it very quickly. I wrote to you about it. Regarding Undo, I was commited after the release, but as you know it is still not satisfying . But I agree that donating some important piece of work is not simple as the IP Clearance process is not a light one, we had to use it for integrating Web Report, it is clearly something that refrains other donations IMO. > 3. UI > > The JMeter UI needs some improving. The problem is not that it uses Java / > Swing because even Swing can be made to look good but we need people to > look into this. Is it "good enough" for people being forced to use JMeter > in a corporate setting? Yes because the Core provides the feature you want for that. Should people only look at UI for such a tool ? No IMO, it would be a mistake. I know a lot of Great UIs for "limited (to be polite)" cores, I prefer the countrary for now :-) . > Would an user pick it just based on the UI? > No JMeter
Some JMeter (technical) concerns
Hello, I've been following JMeter for a while now, even did a separate project based on JMeter (YaMeter.com ) so I figured I should mention some (technical) concerns of mine about JMeter. I thought about sending multiple emails, each about a single aspect, but I will just list everything here and people are invited to only answer the subset they care about. 1. Userbase It seems to me there are no clear metrics about the JMeter userbase. We have no phone-home, no plugin portal, no download statistics(!?) so we can only use indirect means of estimating the userbase: stackoverflow questions and such. This seems a very poor state of affairs. Resources are scarce: how do you know what to focus on if you have very little insight into who your users are, what are they doing and where are they struggling? 1.1 Metrics I believe it is possible to add usage metrics to JMeter while respecting the ASF policy on privacy, etc. It's also unclear to me how download stats are not shared by the Apache mirrors (any ASF document explaining this policy?). 2. Speed of development Compared to some of the plugins, JMeter seems to move very slowly. This might create the impression of stability but I believe it's just a sign of being resource constrained. Are any companies putting resources into JMeter? I see a lot of companies / startups trying to make money off JMeter but how many are contributing back? BTW, one reason I decided to do YaMeter.com instead of trying to push into JMeter proper is the approval speed. Getting a patch in seems to take a lot of effort and I can only imagine how long it would have take to try something as large as what I did with YaMeter. 3. UI The JMeter UI needs some improving. The problem is not that it uses Java / Swing because even Swing can be made to look good but we need people to look into this. Is it "good enough" for people being forced to use JMeter in a corporate setting? Would an user pick it just based on the UI? 3.1 Swing If Swing remains the future of JMeter, I suggest looking into a better framework like Apache NetBeans Platform which provides a lot of useful things for desktop Java apps. Generally speaking, it would be good to split the UI from the "core" so we have some path where we evolve to some other UI. 3.2 Web based Alternatively, a web-based interface could be made for JMeter. Any interest in this? Just like companies use local GitLab installations, I am certain a lot of companies would install a local Apache JMeter Web Portal where all the load testing would be centralised. 4. Plugin Manager It seems unacceptable to me that JMeter does not have a Plugin Manager and a plugin portal with some lower-usage plugins (eg. mongodb). It's acceptable for other plugins to exist elsewhere, but those should be just another "PPA" (see Ubuntu PPA) or another "plugin repository" URL to register. 5. Out of the box experience The way I see it, Apache JMeter is not just a "platform" with some core files and external plugins but something usable out of the box. So, except niche situations, users should be catered by the default JMeter install. 5.1 HTTP2 A major problem being the lack of an official HTTP2 plugin, which people have to take from elsewhere. Blazemeter is probably quite happy with their implementation being used since it funnels people into their services. 6. Project based I find the whole configuration .properties file a very poor solution to the notion of a "project". If a JMX needs a .properties change, then those two belong together. JMeter should have the notion of a 'project' and maybe even be capable of having two projects open at once. And executing two unrelated tests at once. (Same for servers perhaps?) 6.1 Too many .properties I also believe JMeter tries to be too configurable sometimes with flags for everything. 7. Module system JMeter is using a lot of reflection to reinvent its own quirky module system (which provides the way the plugins work). This needs some standardization work. The NetBeans Platform, for example, comes with its own module system that also supports OSGI bundles (another module system). Then, there's the JVM service locator (for services) and the Java 9 modules. 8. Remote is using old technologies RMI is a killer in modern setups. It needs open ports on both the slave and master for communication which only makes sense within the same intranet and it's a pain to configure when using Cloud resources. The whole communication between JMeter client and servers should be rewritten using more modern protocols. 9. Cloud aware The notion that the user tries to configure N servers then gives JMeter a list of IPs for remote load testing seems... primitive. This is something I've been trying to solve with YaMeter.com 10. Packaging JMeter should be everywhere. Official Docker files, AMI images, etc. 11. Pipeline aware / agents JMeter should think more about making it easy to b
