[jira] [Commented] (LANG-1110) Implement HashSetvBitSetTest using JMH

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/LANG-1110?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15862642#comment-15862642
 ] 

ASF GitHub Bot commented on LANG-1110:
--

Github user kinow commented on the issue:

https://github.com/apache/commons-lang/pull/182
  
Hi @C0rWin 

Some time ago I found this pull request and had to learn what was JMH :-) 
added a comment on 
[LANG-1110](https://issues.apache.org/jira/browse/LANG-1110), and today had 
time to revisit the issue and take a look at the replies.

So [Commons 
CSV](https://github.com/apache/commons-csv/blob/a7757849d3e9b2a37967ce208ab22f6d14567c98/pom.xml#L389)
 and [Commons 
RNG](https://github.com/apache/commons-rng/tree/688f3611131e05a072d998edb93e13492c3ed256/commons-rng-jmh)
 use JMH too. Both projects apply a similar approach, with the main difference 
being that Commons RNG - maybe for being a multi-module project - contains a 
jmh Maven module.

However, the dependencies and plug-in execution are part of a benchmark 
profile. Following the suggestions in 
[LANG-1110](https://issues.apache.org/jira/browse/LANG-1110), would be nice if 
we:

* Used a similar approach, such as profile, class names, etc
* Update [jmh dependencies to the 
latest](http://search.maven.org/#search%7Cga%7C1%7Cjmh-core)

I wanted to check with you before I opened a separate pull request. Would 
you be willing to update this pull request again with these changes? If so, 
just ping me and I'll reply here as soon as possible. Have plenty of time to 
work on Open Source projects in the next two months :-)

Cheers
Bruno


> Implement HashSetvBitSetTest using JMH
> --
>
> Key: LANG-1110
> URL: https://issues.apache.org/jira/browse/LANG-1110
> Project: Commons Lang
>  Issue Type: Test
>  Components: lang.*
>Reporter: Benedikt Ritter
> Fix For: Patch Needed, 3.6
>
> Attachments: HashSetvBitSetTest.java
>
>
> HashSetvBitSetTest currently writes to Std out, which is bad practice for 
> unit tests. Since HashSetvBitSetTest is really a performance test and not a 
> unit test, we should reimplement it using OpenJMH. The results could then be 
> published on our website.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (LANG-1110) Implement HashSetvBitSetTest using JMH

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/LANG-1110?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15862726#comment-15862726
 ] 

ASF GitHub Bot commented on LANG-1110:
--

Github user C0rWin commented on the issue:

https://github.com/apache/commons-lang/pull/182
  
Hi Bruno,


On Sun, Feb 12, 2017 at 6:35 AM, Bruno P. Kinoshita <
notificati...@github.com> wrote:

> Hi @C0rWin 
>
> Some time ago I found this pull request and had to learn what was JMH :-)
> added a comment on LANG-1110
> , and today had time to
> revisit the issue and take a look at the replies.
>
> So Commons CSV
> 

> and Commons RNG
> 

> use JMH too. Both projects apply a similar approach, with the main
> difference being that Commons RNG - maybe for being a multi-module project
> - contains a jmh Maven module.
>
> However, the dependencies and plug-in execution are part of a benchmark
> profile. Following the suggestions in LANG-1110
> , would be nice if we:
>
>- Used a similar approach, such as profile, class names, etc
>- Update jmh dependencies to the latest
>
>
> I wanted to check with you before I opened a separate pull request. Would
> you be willing to update this pull request again with these changes? If 
so,
> just ping me and I'll reply here as soon as possible. Have plenty of time
> to work on Open Source projects in the next two months :-)
>

I can update my pull request to align it with according to your request.

Thanks,
Artem Barger.



> Implement HashSetvBitSetTest using JMH
> --
>
> Key: LANG-1110
> URL: https://issues.apache.org/jira/browse/LANG-1110
> Project: Commons Lang
>  Issue Type: Test
>  Components: lang.*
>Reporter: Benedikt Ritter
> Fix For: Patch Needed, 3.6
>
> Attachments: HashSetvBitSetTest.java
>
>
> HashSetvBitSetTest currently writes to Std out, which is bad practice for 
> unit tests. Since HashSetvBitSetTest is really a performance test and not a 
> unit test, we should reimplement it using OpenJMH. The results could then be 
> published on our website.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (LANG-1110) Implement HashSetvBitSetTest using JMH

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/LANG-1110?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15878869#comment-15878869
 ] 

ASF GitHub Bot commented on LANG-1110:
--

Github user kinow commented on the issue:

https://github.com/apache/commons-lang/pull/182
  
@C0rWin what about trying to add the test from 
[LANG-1110](https://issues.apache.org/jira/browse/LANG-1110) to the pull 
request? Look at the attachments section.


> Implement HashSetvBitSetTest using JMH
> --
>
> Key: LANG-1110
> URL: https://issues.apache.org/jira/browse/LANG-1110
> Project: Commons Lang
>  Issue Type: Test
>  Components: lang.*
>Reporter: Benedikt Ritter
> Fix For: Patch Needed, 3.6
>
> Attachments: HashSetvBitSetTest.java
>
>
> HashSetvBitSetTest currently writes to Std out, which is bad practice for 
> unit tests. Since HashSetvBitSetTest is really a performance test and not a 
> unit test, we should reimplement it using OpenJMH. The results could then be 
> published on our website.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (LANG-1110) Implement HashSetvBitSetTest using JMH

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/LANG-1110?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15879475#comment-15879475
 ] 

ASF GitHub Bot commented on LANG-1110:
--

Github user C0rWin commented on the issue:

https://github.com/apache/commons-lang/pull/182
  
I'm fine with it, will add the test from LANG-1110.


> Implement HashSetvBitSetTest using JMH
> --
>
> Key: LANG-1110
> URL: https://issues.apache.org/jira/browse/LANG-1110
> Project: Commons Lang
>  Issue Type: Test
>  Components: lang.*
>Reporter: Benedikt Ritter
> Fix For: Patch Needed, 3.6
>
> Attachments: HashSetvBitSetTest.java
>
>
> HashSetvBitSetTest currently writes to Std out, which is bad practice for 
> unit tests. Since HashSetvBitSetTest is really a performance test and not a 
> unit test, we should reimplement it using OpenJMH. The results could then be 
> published on our website.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (LANG-1110) Implement HashSetvBitSetTest using JMH

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/LANG-1110?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15989021#comment-15989021
 ] 

ASF GitHub Bot commented on LANG-1110:
--

Github user asfgit closed the pull request at:

https://github.com/apache/commons-lang/pull/182


> Implement HashSetvBitSetTest using JMH
> --
>
> Key: LANG-1110
> URL: https://issues.apache.org/jira/browse/LANG-1110
> Project: Commons Lang
>  Issue Type: Test
>  Components: lang.*
>Reporter: Benedikt Ritter
> Fix For: Patch Needed
>
> Attachments: HashSetvBitSetTest.java
>
>
> HashSetvBitSetTest currently writes to Std out, which is bad practice for 
> unit tests. Since HashSetvBitSetTest is really a performance test and not a 
> unit test, we should reimplement it using OpenJMH. The results could then be 
> published on our website.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (LANG-1110) Implement HashSetvBitSetTest using JMH

[Pascal Schumacher (JIRA)]

[ 
https://issues.apache.org/jira/browse/LANG-1110?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15989025#comment-15989025
 ] 

Pascal Schumacher commented on LANG-1110:
-

Thanks everybody!

> Implement HashSetvBitSetTest using JMH
> --
>
> Key: LANG-1110
> URL: https://issues.apache.org/jira/browse/LANG-1110
> Project: Commons Lang
>  Issue Type: Test
>  Components: lang.*
>Reporter: Benedikt Ritter
>Assignee: Pascal Schumacher
> Fix For: 3.6
>
> Attachments: HashSetvBitSetTest.java
>
>
> HashSetvBitSetTest currently writes to Std out, which is bad practice for 
> unit tests. Since HashSetvBitSetTest is really a performance test and not a 
> unit test, we should reimplement it using OpenJMH. The results could then be 
> published on our website.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (LANG-1110) Implement HashSetvBitSetTest using JMH

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/LANG-1110?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15419731#comment-15419731
 ] 

ASF GitHub Bot commented on LANG-1110:
--

GitHub user C0rWin opened a pull request:

https://github.com/apache/commons-lang/pull/182

Add maven dependency for JMH framework.

In order to provide patch for LANG-1110, required dependcy on JMH lib.
Current commit add benchmark profile and ability to run JMH based benchmark 
by
executing "mvn test -P benchmark" command, moreover it's also possible to
specify exact benchmark name by running "mvn test -P benchmark
-Dbenchmark=benchmark.full.class.name".

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/C0rWin/commons-lang LANG-1256

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/commons-lang/pull/182.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #182


commit 9f4c0ec4197331687e410aca492d4071eded2462
Author: Artem Barger 
Date:   2016-08-13T00:33:08Z

Add maven dependency for JMH framework.

In order to provide patch for LANG-1110, required dependcy on JMH lib.
Current commit add benchmark profile and ability to run JMH based benchmark 
by
executing "mvn test -P benchmark" command, moreover it's also possible to
specify exact benchmark name by running "mvn test -P benchmark
-Dbenchmark=benchmark.full.class.name".




> Implement HashSetvBitSetTest using JMH
> --
>
> Key: LANG-1110
> URL: https://issues.apache.org/jira/browse/LANG-1110
> Project: Commons Lang
>  Issue Type: Test
>  Components: lang.*
>Reporter: Benedikt Ritter
> Fix For: Patch Needed, 3.5
>
>
> HashSetvBitSetTest currently writes to Std out, which is bad practice for 
> unit tests. Since HashSetvBitSetTest is really a performance test and not a 
> unit test, we should reimplement it using OpenJMH. The results could then be 
> published on our website.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (LANG-1110) Implement HashSetvBitSetTest using JMH

[Bruno P. Kinoshita (JIRA)]

[ 
https://issues.apache.org/jira/browse/LANG-1110?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15753237#comment-15753237
 ] 

Bruno P. Kinoshita commented on LANG-1110:
--

As I never heard of JMH before, decided to have fun checking out the pull 
request locally, and changing the HashSetvBitSetTest to use JMH and run it.

Will attach the HashSetvBitSetTest.java here. It took 54:09 minutes to run, 
without changing any settings in the pom.xml (Ubuntu 16.04, Macbook i5 2.7, 8 
cores, 16 GB, not virtualized). Here's the benchmark results that appear in the 
console. Not sure if we could add it to the project web site in some way. There 
may have some maven plug-in for that, but I couldn't find it by just googling 
JMH and maven plug-in.

{noformat}
...
Iteration  19: 183.914 ns/op
Iteration  20: 184.460 ns/op

# Run progress: 97.50% complete, ETA 00:00:40
# Fork: 10 of 10
# Warmup Iteration   1: 214.977 ns/op
# Warmup Iteration   2: 189.999 ns/op
# Warmup Iteration   3: 193.562 ns/op
# Warmup Iteration   4: 182.608 ns/op
# Warmup Iteration   5: 188.630 ns/op
# Warmup Iteration   6: 181.914 ns/op
# Warmup Iteration   7: 183.420 ns/op
# Warmup Iteration   8: 184.043 ns/op
# Warmup Iteration   9: 183.083 ns/op
# Warmup Iteration  10: 181.218 ns/op
# Warmup Iteration  11: 183.786 ns/op
# Warmup Iteration  12: 182.048 ns/op
# Warmup Iteration  13: 189.763 ns/op
# Warmup Iteration  14: 182.147 ns/op
# Warmup Iteration  15: 183.410 ns/op
# Warmup Iteration  16: 187.146 ns/op
# Warmup Iteration  17: 184.401 ns/op
# Warmup Iteration  18: 183.841 ns/op
# Warmup Iteration  19: 185.179 ns/op
# Warmup Iteration  20: 182.114 ns/op
Iteration   1: 183.059 ns/op
Iteration   2: 185.323 ns/op
Iteration   3: 184.455 ns/op
Iteration   4: 186.051 ns/op
Iteration   5: 187.268 ns/op
Iteration   6: 190.621 ns/op
Iteration   7: 184.044 ns/op
Iteration   8: 183.438 ns/op
Iteration   9: 183.282 ns/op
Iteration  10: 182.034 ns/op
Iteration  11: 183.265 ns/op
Iteration  12: 183.481 ns/op
Iteration  13: 183.454 ns/op
Iteration  14: 184.774 ns/op
Iteration  15: 189.690 ns/op
Iteration  16: 188.991 ns/op
Iteration  17: 183.585 ns/op
Iteration  18: 181.612 ns/op
Iteration  19: 182.598 ns/op
Iteration  20: 184.105 ns/op


Result "timeExtractRemoveAll":
  196.003 ±(99.9%) 9.039 ns/op [Average]
  (min, avg, max) = (164.015, 196.003, 432.640), stdev = 38.270
  CI (99.9%): [186.965, 205.042] (assumes normal distribution)


# Run complete. Total time: 00:26:55

BenchmarkMode  CntScore   Error  Units
HashSetvBitSetTest.testBitSetavgt  200   53.333 ± 2.263  ns/op
HashSetvBitSetTest.testHashSet   avgt  200  212.623 ± 4.806  ns/op
HashSetvBitSetTest.timeBitSetRemoveAll   avgt  200  133.441 ± 7.006  ns/op
HashSetvBitSetTest.timeExtractRemoveAll  avgt  200  196.003 ± 9.039  ns/op

Benchmark result is saved to 
target/jmh-result.org.apache.commons.lang3.HashSetvBitSetTest.json
[INFO] 
[INFO] --- maven-jar-plugin:2.6:jar (default-jar) @ commons-lang3 ---
[INFO] Building jar: 
/home/kinow/Development/java/workspace/commons-lang/target/commons-lang3-3.5-SNAPSHOT.jar
[INFO] 
[INFO] --- maven-site-plugin:3.4:attach-descriptor (attach-descriptor) @ 
commons-lang3 ---
[INFO] 
[INFO] --- maven-jar-plugin:2.6:test-jar (default) @ commons-lang3 ---
[INFO] Building jar: 
/home/kinow/Development/java/workspace/commons-lang/target/commons-lang3-3.5-SNAPSHOT-tests.jar
[INFO] 
[INFO] --- maven-install-plugin:2.5.2:install (default-install) @ commons-lang3 
---
[INFO] Installing 
/home/kinow/Development/java/workspace/commons-lang/target/commons-lang3-3.5-SNAPSHOT.jar
 to 
/home/kinow/.m2/repository/org/apache/commons/commons-lang3/3.5-SNAPSHOT/commons-lang3-3.5-SNAPSHOT.jar
[INFO] Installing /home/kinow/Development/java/workspace/commons-lang/pom.xml 
to 
/home/kinow/.m2/repository/org/apache/commons/commons-lang3/3.5-SNAPSHOT/commons-lang3-3.5-SNAPSHOT.pom
[INFO] Installing 
/home/kinow/Development/java/workspace/commons-lang/target/commons-lang3-3.5-SNAPSHOT-tests.jar
 to 
/home/kinow/.m2/repository/org/apache/commons/commons-lang3/3.5-SNAPSHOT/commons-lang3-3.5-SNAPSHOT-tests.jar
[INFO] 
[INFO] BUILD SUCCESS
[INFO] 
[INFO] Total time: 54:09 min
[INFO] Finished at: 2016-12-16T15:14:27+13:00
[INFO] Final Memory: 44M/720M
[INFO] 
{noformat}

I added the JMH annotations that I found in the [project 
examples|http://hg.openjdk.java.net/code-tools/jmh/file/tip/jmh-samples/src/main/java/org/openjdk/jmh/samples/]
 and that seemed to make sense to me. Someone with experience on JMH tests can 
probably suggest a better set up. 

Hope that helps
Bruno

> Implement HashSetvBitSetTest using JMH
> -

[jira] [Commented] (LANG-1110) Implement HashSetvBitSetTest using JMH

[Emmanuel Bourg (JIRA)]

[ 
https://issues.apache.org/jira/browse/LANG-1110?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15754476#comment-15754476
 ] 

Emmanuel Bourg commented on LANG-1110:
--

JMH is already used with commons-csv and commons-rng, I suggest reusing the 
same set up.

If we have more components adding JMH benchmarks we should probably put the 
common configuration in the parent pom.

> Implement HashSetvBitSetTest using JMH
> --
>
> Key: LANG-1110
> URL: https://issues.apache.org/jira/browse/LANG-1110
> Project: Commons Lang
>  Issue Type: Test
>  Components: lang.*
>Reporter: Benedikt Ritter
> Fix For: Patch Needed, 3.6
>
> Attachments: HashSetvBitSetTest.java
>
>
> HashSetvBitSetTest currently writes to Std out, which is bad practice for 
> unit tests. Since HashSetvBitSetTest is really a performance test and not a 
> unit test, we should reimplement it using OpenJMH. The results could then be 
> published on our website.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (LANG-1110) Implement HashSetvBitSetTest using JMH

[Bruno P. Kinoshita (JIRA)]

[ 
https://issues.apache.org/jira/browse/LANG-1110?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=1574#comment-1574
 ] 

Bruno P. Kinoshita commented on LANG-1110:
--

Oh, did not know we were already using it in commons. +1 for re-using what we 
have in those components and applying it in [lang]. Might have another go at it 
and try to update the pull request or submit a new one in some weeks. Thanks

> Implement HashSetvBitSetTest using JMH
> --
>
> Key: LANG-1110
> URL: https://issues.apache.org/jira/browse/LANG-1110
> Project: Commons Lang
>  Issue Type: Test
>  Components: lang.*
>Reporter: Benedikt Ritter
> Fix For: Patch Needed, 3.6
>
> Attachments: HashSetvBitSetTest.java
>
>
> HashSetvBitSetTest currently writes to Std out, which is bad practice for 
> unit tests. Since HashSetvBitSetTest is really a performance test and not a 
> unit test, we should reimplement it using OpenJMH. The results could then be 
> published on our website.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)