[VOTE] Release Apache Commons BCEL 6.3.1 based on RC1

[Gary Gregory]

We have fixed quite a few bugs and added some significant enhancements
since Apache Commons BCEL 6.3 was released, so I would like to release
Apache Commons BCEL 6.3.1.

Apache Commons BCEL 6.3.1 RC1 is available for review here:
https://dist.apache.org/repos/dist/dev/commons/bcel/6.3.1-RC1 (svn
revision 33105)

The Git tag commons-bcel-6.3.1-RC1 commit for this RC is
9174edf0d530540c9f6df76b4d786c5a6ad78a5d which you can browse here:

https://gitbox.apache.org/repos/asf?p=commons-bcel.git;a=commit;h=9174edf0d530540c9f6df76b4d786c5a6ad78a5d
You may checkout this tag using:
git clone https://gitbox.apache.org/repos/asf/commons-bcel.git --branch
commons-bcel-6.3.1-RC1

Maven artifacts are here:

https://repository.apache.org/content/repositories/orgapachecommons-1431/org/apache/bcel/bcel/6.3.1/

These are the Maven artifacts and their hashes in Nexus:

#Release SHA-512s
#Wed Mar 20 22:06:36 EDT 2019
bcel-6.3.1-test-sources-jar.asc=9d4193586ae639324235ad395fa9823e4a447ffae5fc5eea1daba183a1a5d357879803f6cad1279559c127e36a2bf9d55dd69145c6504906629c6d1214e7552b
bcel-6.3.1-bin-zip=46d40f897391cee2f1166a195e512d700524ce25ef3af2c1ccff7f540047000848873c9536c5feb2743aaf34826824e1a80bccf2e386f9c0de4af5fc66214511
bcel-6.3.1-tests-test-jar=4d230f77af84feb78467ab607363efd7de5ef70afc0e8d66c3c77ed2018c7925520d68cce2b2adaff96794d68a97292a565c2681b984a8de3629af7605827ddf
bcel-6.3.1-javadoc-javadoc=36f8246b07bb6b13bcdf715ca29998624a6faf78dff270cbbf09dfa2214793c35202ae0ea213eaeb38dcd1f8c297896f00ea5c03e967f721f1fbba8a3829c630
bcel-6.3.1-bin-zip.asc=ff67061e5050a100b30d70a46eb870b0684acf4e8ed163eea51593b55dd6fc9646272d2efb806c2d19fd8b9cb5100c36ef002c18d1de91d3255644ca4f5f59f0
bcel-6.3.1-src-tar.gz=4e601b054b003c2077fa00152020e36dc41e216172c8f3075e8ac63e2c6f4b0334a8ab5b075b53fa0b7e64c0c8fd9cd14a28617790f556467bd38182c6c6cbfd
bcel-6.3.1-bin-tar.gz.asc=694a8b63dbe06460eb18429a2a3cc60b67a3841e92341770ed239b757b925ab23b4d429ebca45979e629a102b9614e9c774644d82a63430f8ecdd305bb600a91
bcel-6.3.1-pom.asc=7c8215ff0b054ee445795fb05800fdb534b93fdb6565b3b99bb8913b8359e7746b6ed2195861dc899c41699d4362b28f6dd22c1e3046080db612d53876da89cd
bcel-6.3.1-tests-jar.asc=2cb1a700668d2a587d533cc824b836acff6152ab7f47d74b0b406468f95f1684c0cd1162dfdc9649d9b1f609ca31af76959aa43caffd6f84ad67fa103d001110
bcel-6.3.1-sources-jar.asc=2a654a56f184be2ed06ba2909715ae50bdc69e7fe19153d0df98d68aa39a73f6560fab9e60f3e6b076240853b0f6b2fcf60e375aa0db328c3889dc921fa67674
bcel-6.3.1-bin-tar.gz=eef969730f189d2e43ce77bee89123fad84f203145e76fe20a38b5748dfc6c878bb98a7365466408258ebca922913dcbb9802e4dc59db8c928f2b18a68bf0584
bcel-6.3.1-src-tar.gz.asc=cd99a46410311577869242dc78eb55cbbf43701e1dda88b80e838e95e4a769eb93c123dc9083a4dc2c3db08822add420b97c3c4586448db3e357aa8a3aebefdd
bcel-6.3.1-test-sources-java-source=74598336729ef45bce7c2cd04c25f3a30a898702c86308f0bf0557ff39f0368f3b821b86f0df5bb4df56083389c033aed3dc78402c81ea0a837ecc9c9b505846
bcel-6.3.1-src-zip.asc=bf45f2a52dfe27875de726095e769ebda68be37408ff2a92410fd0314f023637f222b7febeccf04d92024ccf8fed177ebc3aafd2b4579536c679593bdf665abd
bcel-6.3.1-javadoc-jar.asc=8646dfbe7dc86bad659f89a9772f862df5026f9338a0a03291abc6ed7a79758b91f89dcae310c3d81d8cc935838b9c29d1b89f1d72b8556ca40e9dbccbd837cd
bcel-6.3.1-sources-java-source=75a548736f9a9ece49e994bdd6b2e26588ddc6ca4d45a04dc2ebe2058080aea95b15c2eadb88caf743d35aaf2025a49a085bd391b7099dc4b832fd479ffd816d
bcel-6.3.1-src-zip=d0e98d097437c7d40b863403a8a05c6c5eaa2afea121a312f072ecf5775b8b3e12441c1d7db6ce91eb45fbb583643c5189bc8eace38e610f26dc074fa9247776
bcel-6.3.1-jar.asc=be630ac3f294b0a3eb77ad8f5ed3408bc5d8985c21785654e5f1f43137ed86b5febc3bf45fd986d8f2d35e3c2d9bbedfc33003b408a583fa18d583d15b86a871


(no need for .asc hashes!)

I have tested this with 'mvn clean install site -Ddoclint=none' using:
Apache Maven 3.6.0 (97c98ec64a1fdfee7767ce5ffb20918da4f719f3;
2018-10-24T14:41:47-04:00)
Maven home: C:\Java\apache-maven-3.6.0\bin\..
Java version: 1.8.0_202, vendor: Oracle Corporation, runtime: C:\Program
Files\Java\jdk1.8.0_202\jre
Default locale: en_US, platform encoding: Cp1252
OS name: "windows 10", version: "10.0", arch: "amd64", family: "windows"
Microsoft Windows [Version 10.0.16299.904]

Details of changes since 6.3 are in the release notes:

https://dist.apache.org/repos/dist/dev/commons/bcel/6.3.1-RC1/RELEASE-NOTES.txt

https://dist.apache.org/repos/dist/dev/commons/bcel/6.3.1-RC1/site/changes-report.html

Site:
https://dist.apache.org/repos/dist/dev/commons/bcel/6.3.1-RC1/site
(note some *relative* links are broken and the 6.3.1 directories are
not yet created - these will be OK once the site is deployed.)

CLIRR Report (compared to 6.3):

https://dist.apache.org/repos/dist/dev/commons/bcel/6.3.1-RC1/site/clirr-report.html

JApiCmp Report (compared to 6.3):

https://dist.apache.org/repos/dist/dev/commons/bcel/6.3.1-RC1/site/japicmp.html
is empty, see the clirr report, this will be addressed in a future version
of the commons-parent POM.

RAT Report:

https://dist.apac

Re: [BCEL] Release

[Bruno P. Kinoshita]

 +1 Gary. Will make sure to review and vote on the RC as soon as I see it 
coming. Or if I don't vote within the 72hours, feel free to remind me and I 
will quickly do so.
Cheers

On Thursday, 21 March 2019, 10:33:47 am NZDT, Gary Gregory 
 wrote:  
 
 Hi All:

I need the fix for BCEL-315 ASAP. I plan on creating on RC later today I
hope.

Based on changes.xml I think the next version should be 6.3.1 instead of
6.4.

Gary
  

Re: [Rng] New XoShiRo generators

[Gilles Sadowski]

undefined
Le mer. 20 mars 2019 à 21:39, Alex Herbert  a écrit :
>
>
>
> > On 20 Mar 2019, at 18:22, Gilles Sadowski  wrote:
> >
> >>> [...]
> >>> Perhaps the "RandomStressTester" should be made more flexible
> >>> and, instead of a hard-coded "GeneratorsList", accept a list of "enum"
> >>> identifiers. e.g. a list (ASCII file) like:
> >>> MT 3
> >>> MT_64 3
> >>> SPLIT_MIX_64 3
> >>> [etc.]
> >>> where the second columns indicates how many runs to perform for
> >>> this type of RNG.
> >>> [A (mandatory) flag specifying this file would replace the first 2
> >>> command-line arguments.]

Correcting: The second argument (number of threads) is still needed.

> >>
> >> That would be nice.
> >>
> >> But:
> >>
> >> 1. The GeneratorsList class allows a user to use another list that is on
> >> the Java classpath and test their own generators provided by that list.
> >> That is if I understand how Java can build things using reflection. The
> >> likelihood of ever needing to do this is ... ? (very small)
> >
> > And they could still do that if they add to the "RandomSource" list…
>
> Yes. So no need to worry about what others will do and just support named 
> enums in RandomSource.
>
> >
> >>
> >> 2. The GeneratorsList class can be auto generated from the enum
> >
> > There could be a provided input file, updated whenever "RandomSource"
> > is.
> > One advantage is they we could slightly expand the format to allow e.g.:
> >
> > TWO_CMRES 3
> > TWO_CMRES() 3
> > TWO_CMRES(1,2) 3
>
> I had wondered about supporting additional arguments to the 
> RandomSource.create method. Currently we only need to support Integer but 
> support for Integer, Long, Float, Double can come from using the canonical 
> form, e.g. 0, 0L, 0f, 0.0. Perhaps just jump hurdles when needed. An easier 
> approach would be to hard code the handling of TWO_CMRES_SELECT to know the 
> values inside the parentheses are two ints.
>
> >
> >> I suppose it could be updated to require an input text list (as per your
> >> example) but if called with no arguments it can print out a default text
> >> list using the order of the RandomSource enum.
> >>
> >> Thus it should not need to be modified if more generators are added.
> >
> > I'm not sure I understand here.
>
> If the program only needs to support RandomSource then it can easily 
> enumerate RandomSource and print out an example input file. Then there is no 
> need to provide an input file for the program since it can write a default 
> one.

If the user forgets to supply one, the program outputs one, and stops;
then the user reissues the command?

>
> It would have to know to handle TWO_CMRES_SELECT with arguments. But since it 
> is only one case this is easy to support.
>
>
> Q. Since this is in examples can it depend on another project, e.g. Commons 
> CLI?

Or this one: https://picocli.info/

> It would make the parsing of arguments easier for this and the data dumper 
> program.
>
> I’m not sure how recent Commons CLI is. The homepage states "a freshly 
> designed and not backwards compatible CLI 2 was created in 2004, but never 
> finished or released."
>
> But it looks like it has all the functionality to parse the type of arguments 
> we are discussing for these programs.
>
> Alex

-
To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
For additional commands, e-mail: dev-h...@commons.apache.org

[BCEL] Release

[Gary Gregory]

Hi All:

I need the fix for BCEL-315 ASAP. I plan on creating on RC later today I
hope.

Based on changes.xml I think the next version should be 6.3.1 instead of
6.4.

Gary

Re: [Rng] New XoShiRo generators

[Alex Herbert]

> On 20 Mar 2019, at 18:22, Gilles Sadowski  wrote:
> 
>>> [...]
>>> Perhaps the "RandomStressTester" should be made more flexible
>>> and, instead of a hard-coded "GeneratorsList", accept a list of "enum"
>>> identifiers. e.g. a list (ASCII file) like:
>>> MT 3
>>> MT_64 3
>>> SPLIT_MIX_64 3
>>> [etc.]
>>> where the second columns indicates how many runs to perform for
>>> this type of RNG.
>>> [A (mandatory) flag specifying this file would replace the first 2
>>> command-line arguments.]
>> 
>> That would be nice.
>> 
>> But:
>> 
>> 1. The GeneratorsList class allows a user to use another list that is on
>> the Java classpath and test their own generators provided by that list.
>> That is if I understand how Java can build things using reflection. The
>> likelihood of ever needing to do this is ... ? (very small)
> 
> And they could still do that if they add to the "RandomSource" list…

Yes. So no need to worry about what others will do and just support named enums 
in RandomSource.

> 
>> 
>> 2. The GeneratorsList class can be auto generated from the enum
> 
> There could be a provided input file, updated whenever "RandomSource"
> is.
> One advantage is they we could slightly expand the format to allow e.g.:
> 
> TWO_CMRES 3
> TWO_CMRES() 3
> TWO_CMRES(1,2) 3

I had wondered about supporting additional arguments to the RandomSource.create 
method. Currently we only need to support Integer but support for Integer, 
Long, Float, Double can come from using the canonical form, e.g. 0, 0L, 0f, 
0.0. Perhaps just jump hurdles when needed. An easier approach would be to hard 
code the handling of TWO_CMRES_SELECT to know the values inside the parentheses 
are two ints.

> 
>> I suppose it could be updated to require an input text list (as per your
>> example) but if called with no arguments it can print out a default text
>> list using the order of the RandomSource enum.
>> 
>> Thus it should not need to be modified if more generators are added.
> 
> I'm not sure I understand here.

If the program only needs to support RandomSource then it can easily enumerate 
RandomSource and print out an example input file. Then there is no need to 
provide an input file for the program since it can write a default one.

It would have to know to handle TWO_CMRES_SELECT with arguments. But since it 
is only one case this is easy to support.


Q. Since this is in examples can it depend on another project, e.g. Commons 
CLI? It would make the parsing of arguments easier for this and the data dumper 
program.

I’m not sure how recent Commons CLI is. The homepage states "a freshly designed 
and not backwards compatible CLI 2 was created in 2004, but never finished or 
released."

But it looks like it has all the functionality to parse the type of arguments 
we are discussing for these programs.

Alex


> 
> Gilles
> 
>>> [...]
> 
> -
> To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
> For additional commands, e-mail: dev-h...@commons.apache.org
> 

Re: [Rng] New XoShiRo generators

[Gilles Sadowski]

>> [...]
> > Perhaps the "RandomStressTester" should be made more flexible
> > and, instead of a hard-coded "GeneratorsList", accept a list of "enum"
> > identifiers. e.g. a list (ASCII file) like:
> > MT 3
> > MT_64 3
> > SPLIT_MIX_64 3
> > [etc.]
> > where the second columns indicates how many runs to perform for
> > this type of RNG.
> > [A (mandatory) flag specifying this file would replace the first 2
> > command-line arguments.]
>
> That would be nice.
>
> But:
>
> 1. The GeneratorsList class allows a user to use another list that is on
> the Java classpath and test their own generators provided by that list.
> That is if I understand how Java can build things using reflection. The
> likelihood of ever needing to do this is ... ? (very small)

And they could still do that if they add to the "RandomSource" list...

>
> 2. The GeneratorsList class can be auto generated from the enum

There could be a provided input file, updated whenever "RandomSource"
is.
One advantage is they we could slightly expand the format to allow e.g.:

TWO_CMRES 3
TWO_CMRES() 3
TWO_CMRES(1,2) 3

> I suppose it could be updated to require an input text list (as per your
> example) but if called with no arguments it can print out a default text
> list using the order of the RandomSource enum.
>
> Thus it should not need to be modified if more generators are added.

I'm not sure I understand here.

Gilles

>> [...]

-
To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
For additional commands, e-mail: dev-h...@commons.apache.org

Re: [Rng] New XoShiRo generators

[Alex Herbert]

On 20/03/2019 16:13, Gilles Sadowski wrote:

Le mer. 20 mars 2019 à 15:04, Alex Herbert  a écrit :


On 20/03/2019 13:08, Gilles Sadowski wrote:

- Perhaps there is another place for them?

I am happy to not clutter the source repo and keep them somewhere for
devs. But where? You tell me. Maybe in the examples-stress module they
are OK as that one is not officially part of the library.

Unless a script is robust and readily usable by a newbie (i.e. anyone
but the author), there is the risk that it becomes cruft.
What you did is great work but I doubt that many would be interested
in detailed tables of the failures, once one can easily compare the number
of failures.
If more is needed, there is no shortcut to reading the doc of the test suites
themselves and other resources on the web...

So, to summarize, what I think is interesting is to make it easy to rerun
the test suites and update the site (e.g. for the "current" JDK):
   * a document that mentions the external tools requirement
   * a standalone application for running "RandomStressTester"
   * a script that collects results from the above run, formats them into the
 "quality" table ready to be pasted into the "rng.apt" file, and copies the
 output files to their appropriate place (*not* assuming a git repository)

It would also be great to be able to easily start the many benchmarks
and similarly collects the results into the user guide tables.

I will write a script that will collect the results from the benchmark,
build the apt table and copy the output files to a directory.

Note that the benchmark files only contain the RNG class name:

# RNG: org.apache.commons.rng.core.source32.JDKRandom

This means the script has to be kept up to date with the corresponding
enum name, and the desired output order for the apt table. I can add
these to a simple array as the top which any newbie should be able to
extend.

It would be desirable for the script to search for results files in a
set of directories, identify them as dieharder (dh) or TestU01 (tu)
results and then output the files for each run N of a unique generator M
to the existing directory structure, e.g:

  > collate-benchmark-results.pl output dir1 dir2 ...

output/(dh|tu)/run_N/(dh|tu)_M

This should be done robustly such that the script can be pointed at the
git source tree for the site and it will figure out that the files are
already in the correct place. I think this would just involve sorting
the paths to all the results numerically not alphabetically (e.g. dh_1 <
dh_2 < dh_10) before processing them to output.

Perhaps the "RandomStressTester" should be made more flexible
and, instead of a hard-coded "GeneratorsList", accept a list of "enum"
identifiers. e.g. a list (ASCII file) like:
MT 3
MT_64 3
SPLIT_MIX_64 3
[etc.]
where the second columns indicates how many runs to perform for
this type of RNG.
[A (mandatory) flag specifying this file would replace the first 2
command-line arguments.]


That would be nice.

But:

1. The GeneratorsList class allows a user to use another list that is on 
the Java classpath and test their own generators provided by that list. 
That is if I understand how Java can build things using reflection. The 
likelihood of ever needing to do this is ... ? (very small)


2. The GeneratorsList class can be auto generated from the enum

I suppose it could be updated to require an input text list (as per your 
example) but if called with no arguments it can print out a default text 
list using the order of the RandomSource enum.


Thus it should not need to be modified if more generators are added.




For the JMH benchmarks I tend to run e.g.

  > mvn test -P benchmark -Dbenchmark=NextDoublePerformance

I then have a script to parse the JSON in
target/jmh-result.NextDoublePerformance.json to a Jira format or CSV
format table. Usually I add some relative score columns in a spreadsheet
then paste into Jira (which nicely accepts the tables).

It should not be too hard to generate the existing apt tables for
performance. I can look into this. I am assuming that the tables are
based on the highest 'number of samples' from each generator.

I also have a (Perl) script, that uses a JSON library, to parse the
JMH output; but it's quite ugly and not robust (IIRC, I need to make a
change in the code depending on what should serve as the "reference"
to be mapped to "1", in the table).

Regards,
Gilles

So this should be standardised somehow. One to think about.



[...]
You then have a utility for dumping output of any random source to file
in a variety of formats.

Although long output is not needed for the test suites it is useful for
native long generators.

WDYT?

Looks good!

OK. I will work on the raw data dumper as a Jira ticket. It is
encapsulated work that does not really effect anything else.


DieHarder has finished!

I think my stupidity is what caused previous crashes. I was running the
stress test within the source tree and possibly git checkout onto
an

Re: [Rng] New XoShiRo generators

[Gilles Sadowski]

Le mer. 20 mars 2019 à 15:04, Alex Herbert  a écrit :
>
>
> On 20/03/2019 13:08, Gilles Sadowski wrote:
> >
> >> - Perhaps there is another place for them?
> >>
> >> I am happy to not clutter the source repo and keep them somewhere for
> >> devs. But where? You tell me. Maybe in the examples-stress module they
> >> are OK as that one is not officially part of the library.
> > Unless a script is robust and readily usable by a newbie (i.e. anyone
> > but the author), there is the risk that it becomes cruft.
> > What you did is great work but I doubt that many would be interested
> > in detailed tables of the failures, once one can easily compare the number
> > of failures.
> > If more is needed, there is no shortcut to reading the doc of the test 
> > suites
> > themselves and other resources on the web...
> >
> > So, to summarize, what I think is interesting is to make it easy to rerun
> > the test suites and update the site (e.g. for the "current" JDK):
> >   * a document that mentions the external tools requirement
> >   * a standalone application for running "RandomStressTester"
> >   * a script that collects results from the above run, formats them into the
> > "quality" table ready to be pasted into the "rng.apt" file, and copies 
> > the
> > output files to their appropriate place (*not* assuming a git 
> > repository)
> >
> > It would also be great to be able to easily start the many benchmarks
> > and similarly collects the results into the user guide tables.
>
> I will write a script that will collect the results from the benchmark,
> build the apt table and copy the output files to a directory.
>
> Note that the benchmark files only contain the RNG class name:
>
> # RNG: org.apache.commons.rng.core.source32.JDKRandom
>
> This means the script has to be kept up to date with the corresponding
> enum name, and the desired output order for the apt table. I can add
> these to a simple array as the top which any newbie should be able to
> extend.
>
> It would be desirable for the script to search for results files in a
> set of directories, identify them as dieharder (dh) or TestU01 (tu)
> results and then output the files for each run N of a unique generator M
> to the existing directory structure, e.g:
>
>  > collate-benchmark-results.pl output dir1 dir2 ...
>
> output/(dh|tu)/run_N/(dh|tu)_M
>
> This should be done robustly such that the script can be pointed at the
> git source tree for the site and it will figure out that the files are
> already in the correct place. I think this would just involve sorting
> the paths to all the results numerically not alphabetically (e.g. dh_1 <
> dh_2 < dh_10) before processing them to output.

Perhaps the "RandomStressTester" should be made more flexible
and, instead of a hard-coded "GeneratorsList", accept a list of "enum"
identifiers. e.g. a list (ASCII file) like:
MT 3
MT_64 3
SPLIT_MIX_64 3
[etc.]
where the second columns indicates how many runs to perform for
this type of RNG.
[A (mandatory) flag specifying this file would replace the first 2
command-line arguments.]

>
> For the JMH benchmarks I tend to run e.g.
>
>  > mvn test -P benchmark -Dbenchmark=NextDoublePerformance
>
> I then have a script to parse the JSON in
> target/jmh-result.NextDoublePerformance.json to a Jira format or CSV
> format table. Usually I add some relative score columns in a spreadsheet
> then paste into Jira (which nicely accepts the tables).
>
> It should not be too hard to generate the existing apt tables for
> performance. I can look into this. I am assuming that the tables are
> based on the highest 'number of samples' from each generator.

I also have a (Perl) script, that uses a JSON library, to parse the
JMH output; but it's quite ugly and not robust (IIRC, I need to make a
change in the code depending on what should serve as the "reference"
to be mapped to "1", in the table).

Regards,
Gilles

>  [...]
>  You then have a utility for dumping output of any random source to file
>  in a variety of formats.
> 
>  Although long output is not needed for the test suites it is useful for
>  native long generators.
> 
>  WDYT?
> >>> Looks good!
> >> OK. I will work on the raw data dumper as a Jira ticket. It is
> >> encapsulated work that does not really effect anything else.
> >>
> >>
> >> DieHarder has finished!
> >>
> >> I think my stupidity is what caused previous crashes. I was running the
> >> stress test within the source tree and possibly git checkout onto
> >> another branch makes some of the directory paths stale killing any
> >> processes linked to those paths. I'll not do that again.
> > Hence, the "standalone" application is the right choice it seems.
> >
> >> FYI: Here are the old results with incorrect byte order:
> >>
> >> XorShiftSerialComposite : 24, 25, 23 : 134.1 +/- 16.1
> >> XorShiftXorComposite : 88, 105, 89 : 396.2 +/- 9.9
> >> SplitXorComposite : 0, 0, 0 : 90.8 +/- 21.9
> >>
> >> Here are the new results with correct

Re: [Rng] New XoShiRo generators

[Alex Herbert]

On 20/03/2019 11:14, Gilles Sadowski wrote:


I have updated the PR for the XorShift1024 variant to deprecate the old 
XOR_SHIFT_1024_S enum.

Please take a look to see if the wording is OK:

https://github.com/apache/commons-rng/pull/30/files#diff-630495e26d73fa22ada841dabde0ab47
 


How about:

/**
  * @deprecated Since 1.3, where it is recommended to use {@code
XOR_SHIFT_1024_S_PHI},
  * instead, due to its slightly better (more uniform) output.
  * {@code XOR_SHIFT_1024_S} is still quite usable but both versions
share the exact same
  * internal state, so that their outputs are correlated (and thus
should not be used together when
  * independent sequences are assumed).
  */

?


That is better as it states that the existing version is not broken and 
can still be used.


I've rearranged the text about sharing internal state as it could be 
misinterpreted since instances do not share state, only the method to 
maintain it. So I've changed it to variants of the same algorithm:


 * @deprecated Since 1.3, where it is recommended to use {@code 
XOR_SHIFT_1024_S_PHI}
 * instead due to its slightly better (more uniform) output. {@code 
XOR_SHIFT_1024_S}
 * is still quite usable but both are variants of the same 
algorithm and maintain their
 * internal state identically. Their outputs are correlated and the 
two should not be

 * used together when independent sequences are assumed.

I have missed off stating that they are correlated when identically 
seeded. When not identically seeded I presume it correct to say they are 
correlated given some defined lag? It's probably best to not mention it 
so a user does not think that using them together is OK with different 
seeds.


So I will remove the 'identically seeded' text from the class definition 
too and change it to:


 * Note: This supersedes {@link XorShift1024Star}. The sequences 
emitted by both

 * generators are correlated.
 *
 * This generator differs only in the final multiplier (a 
fixed-point representation
 * of the golden ratio), which eliminates linear dependencies from one 
of the lowest

 * bits.

I find the two paragraphs make it clearer they are correlated than if 
all in one paragraph.


I'll also add a note to XorShift1024Star that it has been superseded by:

 * Note: This has been superseded by {@link XorShift1024StarPhi}. 
The sequences emitted

 * by both generators are correlated.

Thus the recommendation is clear. Do not use them together to be assured 
of independence.


Alex




-
To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
For additional commands, e-mail: dev-h...@commons.apache.org

[LAZY][VOTE] Release Apache Commons Parent 48 based on RC1

[Rob Tompkins]

We have fixed quite a few bugs and added some significant enhancements since 
Apache Commons Parent 47 was released, so I would like to release Apache 
Commons Parent 48.

Apache Commons Parent 48 RC1 is available for review here:
https://dist.apache.org/repos/dist/dev/commons/commons-parent/48-RC1 (svn 
revision 33098)


The Git tag commons-parent-48-RC1 commit for this RC is 
f613ac722733b5e10a7028a635b7717d30fbb162 which you can browse here:

https://gitbox.apache.org/repos/asf?p=commons-parent.git;a=commit;h=f613ac722733b5e10a7028a635b7717d30fbb162

You may checkout this tag using:
git clone https://gitbox.apache.org/repos/asf/commons-parent.git --branch 
commons-parent-48-RC1

Maven artifacts are here:

https://repository.apache.org/content/repositories/orgapachecommons-${commons.nexus.repo.id}/org/apache/commons/commons-parent/48/

These are the Maven artifacts:

#Release SHA-512s
#Wed Mar 20 11:07:15 EDT 2019
commons-parent-48-src-zip=037e8e118e3d52f41b5a6682bef83dc716770e39dd7cc677762ff777c4e42f67eb5a1722684a35b68ed205cd82a5b1e9eb49927d759490d50817deef1a566caf
commons-parent-48-javadoc-javadoc=541be566d3e7a80a8ee6c90bb2973d346ad125932ac3f42e8b901f76a6a2ac5a918cc4bc1725b9c17ad5ab8c60b7e348f33f0a4f154659aef7c64636e04eaca5
commons-parent-48-site-xml=8dcd3a5fbef28277caa8a9f201d786e61d39fb7a00f4a5dfe3103a3a6fc403648327af83295ff2ab5d51c27b966156a499554860535770aaaeb4271ea20a2e69
commons-parent-48-pom=6873a43261b38dcb241a30f4e947f425f10c3de2cfbdf6b684e813ff0d7feddd56fcc6f96a357e430fb98cbf05cba64fd9b516f79d5bc47311f2e60a6e334ed6
commons-parent-48-src-tar.gz=b52eb5416ad28356ac24b54fb6ee831919584606f71f99cd7cfc7bd00e92fd40ae1e838bb9d7a6f2914d777313b0c7682fbbe6a158845be3461f75d9c120cfd4



I have tested this with ***'mvn clean install site'*** using: 
Apache Maven 3.6.0 (97c98ec64a1fdfee7767ce5ffb20918da4f719f3; 
2018-10-24T14:41:47-04:00)
Maven home: /usr/local/Cellar/maven/3.6.0/libexec
Java version: 1.8.0_191, vendor: Oracle Corporation, runtime: 
/Library/Java/JavaVirtualMachines/jdk1.8.0_191.jdk/Contents/Home/jre
Default locale: en_US, platform encoding: UTF-8
OS name: "mac os x", version: "10.14.3", arch: "x86_64", family: "mac"


Details of changes since 47 are in the release notes:

https://dist.apache.org/repos/dist/dev/commons/commons-parent/48-RC1/RELEASE-NOTES.txt

https://dist.apache.org/repos/dist/dev/commons/commons-parent/48-RC1/site/changes-report.html

Site:
https://dist.apache.org/repos/dist/dev/commons/commons-parent/48-RC1/site
(note some *relative* links are broken and the 48 directories are not yet 
created - these will be OK once the site is deployed.)

RAT Report:

https://dist.apache.org/repos/dist/dev/commons/commons-parent/48-RC1/site/rat-report.html

KEYS:
  https://www.apache.org/dist/commons/KEYS

Please review the release candidate and vote.
This vote will close no sooner that 72 hours from now.

  [ ] +1 Release these artifacts
  [ ] +0 OK, but...
  [ ] -0 OK, but really should fix...
  [ ] -1 I oppose this release because...

Thank you,

Rob Tompkins, 
Release Manager (using key B6E73D84EA4FCC47166087253FAAD2CD5ECBB314)

For following is intended as a helper and refresher for reviewers.

Validating a release candidate
==

These guidelines are NOT complete.

Requirements: Git, Java, Maven.

You can validate a release from a release candidate (RC) tag as follows.

1) Clone and checkout the RC:

git clone https://gitbox.apache.org/repos/asf/commons-parent.git -b 
commons-parent-48-RC1
cd commons-parent-48-RC1

2) Check Apache licenses:

mvn apache-rat:check

3) Build the package:

mvn -V clean package

You can record the Maven and Java version produced by -V in your VOTE reply.

4) Build the site for a single module project:

mvn site
Check the site reports in:
target\site\index.html

4) Build the site for a multi-module project:

mvn site
mvn site:stage
Check the site reports in:
target\site\index.html
-
To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
For additional commands, e-mail: dev-h...@commons.apache.org

Re: [Rng] New XoShiRo generators

[Alex Herbert]

On 20/03/2019 13:08, Gilles Sadowski wrote:



- Perhaps there is another place for them?

I am happy to not clutter the source repo and keep them somewhere for
devs. But where? You tell me. Maybe in the examples-stress module they
are OK as that one is not officially part of the library.

Unless a script is robust and readily usable by a newbie (i.e. anyone
but the author), there is the risk that it becomes cruft.
What you did is great work but I doubt that many would be interested
in detailed tables of the failures, once one can easily compare the number
of failures.
If more is needed, there is no shortcut to reading the doc of the test suites
themselves and other resources on the web...

So, to summarize, what I think is interesting is to make it easy to rerun
the test suites and update the site (e.g. for the "current" JDK):
  * a document that mentions the external tools requirement
  * a standalone application for running "RandomStressTester"
  * a script that collects results from the above run, formats them into the
"quality" table ready to be pasted into the "rng.apt" file, and copies the
output files to their appropriate place (*not* assuming a git repository)

It would also be great to be able to easily start the many benchmarks
and similarly collects the results into the user guide tables.


I will write a script that will collect the results from the benchmark, 
build the apt table and copy the output files to a directory.


Note that the benchmark files only contain the RNG class name:

# RNG: org.apache.commons.rng.core.source32.JDKRandom

This means the script has to be kept up to date with the corresponding 
enum name, and the desired output order for the apt table. I can add 
these to a simple array as the top which any newbie should be able to 
extend.


It would be desirable for the script to search for results files in a 
set of directories, identify them as dieharder (dh) or TestU01 (tu) 
results and then output the files for each run N of a unique generator M 
to the existing directory structure, e.g:


> collate-benchmark-results.pl output dir1 dir2 ...

output/(dh|tu)/run_N/(dh|tu)_M

This should be done robustly such that the script can be pointed at the 
git source tree for the site and it will figure out that the files are 
already in the correct place. I think this would just involve sorting 
the paths to all the results numerically not alphabetically (e.g. dh_1 < 
dh_2 < dh_10) before processing them to output.



For the JMH benchmarks I tend to run e.g.

> mvn test -P benchmark -Dbenchmark=NextDoublePerformance

I then have a script to parse the JSON in 
target/jmh-result.NextDoublePerformance.json to a Jira format or CSV 
format table. Usually I add some relative score columns in a spreadsheet 
then paste into Jira (which nicely accepts the tables).


It should not be too hard to generate the existing apt tables for 
performance. I can look into this. I am assuming that the tables are 
based on the highest 'number of samples' from each generator.






[...]
You then have a utility for dumping output of any random source to file
in a variety of formats.

Although long output is not needed for the test suites it is useful for
native long generators.

WDYT?

Looks good!

OK. I will work on the raw data dumper as a Jira ticket. It is
encapsulated work that does not really effect anything else.


DieHarder has finished!

I think my stupidity is what caused previous crashes. I was running the
stress test within the source tree and possibly git checkout onto
another branch makes some of the directory paths stale killing any
processes linked to those paths. I'll not do that again.

Hence, the "standalone" application is the right choice it seems.


FYI: Here are the old results with incorrect byte order:

XorShiftSerialComposite : 24, 25, 23 : 134.1 +/- 16.1
XorShiftXorComposite : 88, 105, 89 : 396.2 +/- 9.9
SplitXorComposite : 0, 0, 0 : 90.8 +/- 21.9

Here are the new results with correct byte order:

XorShiftSerialComposite : 13, 15, 10 : 105.5 +/- 1.8
XorShiftXorComposite : 57, 57, 57 : 102.9 +/- 1.5
SplitXorComposite : 0, 0, 0 : 99.9 +/- 3.2

So interestingly passing the correct byte order lowers the number of
failures. There are still lots.

And BigCrush (with the fix for passing the correct byte order):

XorShiftSerialComposite : 40, 39, 39 : 608.2 +/- 3.9
XorShiftXorComposite : 54, 53, 53 : 646.8 +/- 10.9
SplitXorComposite : 0, 0, 0 : 625.8 +/- 0.2

Curious to know whether it is also affected by the byte ordering.


I'll re-run BigCrush with the wrong byte ordering when I have updated 
the stress test code. I finished it yesterday but will look it over with 
fresh eyes before committing it.


Alex



-
To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
For additional commands, e-mail: dev-h...@commons.apache.org

Re: Google Summer of Code 2019 Mentor Registration

[Gilles Sadowski]

Hi Eric.

Le mar. 12 mars 2019 à 20:02, Eric Barnhill  a écrit :
>
> On Tue, Mar 12, 2019 at 11:48 AM Gilles Sadowski 
> wrote:
>
> >
> > There are also a couple of CM packages that would be worth porting
> > to [Numbers] or their own component:
> >   * o.a.c.math4.analysis.integration
> >   * o.a.c.math4.analysis.interpolation
> >   * o.a.c.math4.analysis.solvers
> > (with adaptation to the interfaces of Java 8 "function" package).
> >
> > As for the "o.a.c.math4.ml" package, it should be fairly easy to
> > port it to its own component, as there are no dependencies towards
> > other CM packages.
> > It could be worth having a small component focused on classification.
> >
> > WDYT?
> >
> >
>  Interpolation I am well familiar with and have used the commons library
> before, and would be happy to mentor.

Thanks.
Could you perhaps start a thread about it (for reference), and open
a JIRA report (in project NUMBERS for the meantime, even if it
is probably going to end up in its own new component)?

> The other analysis libraries are pretty far outside of my expertise, and I
> am not qualified to mentor alone anyway, but would be happy to be involved
> and learn how they work.

Several classes in "solvers" can be easily moved to [Numbers] but I'm
not sure all should: It was my intention to create a "rootfinder" module
and move "BrentSolver" which is probably the most generally useful
implementation (in real applications).

> I guess I am not too interested in putting time into ML components if Weka
> does it better.

Fair enough if the intention were to develop a complete library for
machine learning. ;-)

Gilles

P.S. Do you know that a potential GSoC candidate is waiting for your feedback?
https://issues.apache.org/jira/browse/STATISTICS-5

-
To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
For additional commands, e-mail: dev-h...@commons.apache.org

Re: [Rng] New XoShiRo generators

[Gilles Sadowski]

>> [...]
> > I was thinking of a utility for taking the output (of several runs) of
> > the test
> > suites and generate the table in "apt" format (with correct links).
>
> I previously did a lot of this when testing the cache of the int and
> boolean values. I could not find the scripts on local machines until I
> realised I committed them to that topic branch.
>
> Q. How is your Perl? I use it because I know it but I find the syntax
> ugly. I should switch to Python but Perl is so good for parsing text files.

I also many times contemplated learning Python, but everything
was more readily available in Perl. :-}

> This one renames results files to the same names as the order in master.
> I used it to rename results files from a partial run since they all
> start at 1.
>
> https://github.com/aherbert/commons-rng/blob/improvement-RNG-57-stress/commons-rng-examples/examples-stress/rename.pl
>
> This sort of idea may be useful when the number of generators gets large
> and a rerun of only a few of them is needed (or even just new ones).
>
> This one tabulates failures and systematic failures in the results for
> Jira posts:
>
> https://github.com/aherbert/commons-rng/blob/improvement-RNG-57-stress/commons-rng-examples/examples-stress/tabulate_results.pl
>
> There is some weird tabs vs spaces problem in that version but it works.
> Here's what it does on the current master. Note the first table matches
> the userguide and the script could easily be adapted to create the
> rng.apt table. The second table identifies unique tests for dieharder
> using the name and ntup parameter (n-tuples), and for BigCrush it is the
> test ID and name. It reports tests that always fail.
>
> > rng-stress-tabulate-results.pl
> src/site/resources/txt/userguide/stress/*/*/*
>
> || RNG identifier || Dieharder   || TestU01 (BigCrush) ||
> | JDK | 11 12 13 |74 72 75 |
> | MT | 0 0 0 |3 2 2 |
> | WELL_512_A | 0 0 0 |7 6 6 |
> | WELL_1024_A | 0 0 0 |4 4 5 |
> | WELL_19937_A | 0 0 0 |3 2 2 |
> | WELL_19937_C | 0 0 0 |2 2 3 |
> | WELL_44497_A | 0 0 0 |2 3 3 |
> | WELL_44497_B | 0 0 0 |2 2 2 |
> | ISAAC | 0 0 0 |0 1 0 |
> | MT_64 | 0 0 1 |3 2 3 |
> | SPLIT_MIX_64 | 0 0 0 |2 0 0 |
> | XOR_SHIFT_1024_S | 0 0 0 |2 0 0 |
> | TWO_CMRES | 1 1 1 |0 0 1 |
> | MWC_256 | 0 0 0 |0 0 0 |
> | KISS | 0 0 0 |1 2 0 |
>
> || RNG identifier || Test suite || Systematic failures || Fails ||
> | JDK | Dieharder | diehard_oqso:0 | 3/3 |
> | JDK | Dieharder | diehard_dna:0 | 3/3 |
> | JDK | Dieharder | rgb_minimum_distance:3 | 3/3 |
> | JDK | Dieharder | rgb_minimum_distance:4 | 3/3 |
> | JDK | Dieharder | rgb_minimum_distance:5 | 3/3 |
> | JDK | Dieharder | rgb_lagged_sum:15 | 3/3 |
> | JDK | Dieharder | rgb_lagged_sum:31 | 3/3 |
> | JDK | Dieharder | dab_bytedistrib:0 | 3/3 |
> | JDK | Dieharder | dab_filltree:32 | 3/3 |
> | JDK | TestU01 (BigCrush) | 1:SerialOver | 3/3 |
> | JDK | TestU01 (BigCrush) | 3:CollisionOver | 3/3 |
> | JDK | TestU01 (BigCrush) | 5:CollisionOver | 3/3 |
> | JDK | TestU01 (BigCrush) | 7:CollisionOver | 3/3 |
> | JDK | TestU01 (BigCrush) | 9:CollisionOver | 3/3 |
> | JDK | TestU01 (BigCrush) | 11:CollisionOver | 3/3 |
> | JDK | TestU01 (BigCrush) | 13:BirthdaySpacings | 3/3 |
> | JDK | TestU01 (BigCrush) | 14:BirthdaySpacings | 3/3 |
> | JDK | TestU01 (BigCrush) | 15:BirthdaySpacings | 3/3 |
> | JDK | TestU01 (BigCrush) | 16:BirthdaySpacings | 3/3 |
> | JDK | TestU01 (BigCrush) | 17:BirthdaySpacings | 3/3 |
> | JDK | TestU01 (BigCrush) | 18:BirthdaySpacings | 3/3 |
> | JDK | TestU01 (BigCrush) | 19:BirthdaySpacings | 3/3 |
> | JDK | TestU01 (BigCrush) | 20:BirthdaySpacings | 3/3 |
> | JDK | TestU01 (BigCrush) | 21:BirthdaySpacings | 3/3 |
> | JDK | TestU01 (BigCrush) | 22:ClosePairs | 3/3 |
> | JDK | TestU01 (BigCrush) | 23:ClosePairs | 3/3 |
> | JDK | TestU01 (BigCrush) | 24:ClosePairs | 3/3 |
> | JDK | TestU01 (BigCrush) | 25:ClosePairs | 3/3 |
> | JDK | TestU01 (BigCrush) | 26:SimpPoker | 3/3 |
> | JDK | TestU01 (BigCrush) | 28:SimpPoker | 3/3 |
> | JDK | TestU01 (BigCrush) | 30:CouponCollector | 3/3 |
> | JDK | TestU01 (BigCrush) | 31:CouponCollector | 3/3 |
> | JDK | TestU01 (BigCrush) | 34:Gap | 3/3 |
> | JDK | TestU01 (BigCrush) | 36:Gap | 3/3 |
> | JDK | TestU01 (BigCrush) | 40:Permutation | 3/3 |
> | JDK | TestU01 (BigCrush) | 41:Permutation | 3/3 |
> | JDK | TestU01 (BigCrush) | 42:Permutation | 3/3 |
> | JDK | TestU01 (BigCrush) | 43:Permutation | 3/3 |
> | JDK | TestU01 (BigCrush) | 44:CollisionPermut | 3/3 |
> | JDK | TestU01 (BigCrush) | 45:CollisionPermut | 3/3 |
> | JDK | TestU01 (BigCrush) | 46:MaxOft | 3/3 |
> | JDK | TestU01 (BigCrush) | 47:MaxOft | 3/3 |
> | JDK | TestU01 (BigCrush) | 48:MaxOft | 3/3 |
> | JDK | TestU01 (BigCrush) | 49:MaxOft | 3/3 |
> | JDK | TestU01 (BigCrush) | 50:SampleProd | 3/3 |
> | JDK | TestU01 (BigCrush) | 51:SampleProd | 3/3 |
> | JDK | TestU01 (BigCrush) | 52:SampleProd | 3/3 |
> | JDK | TestU01 (BigCrush) | 57:AppearanceSpacings | 3/3 |
> | JDK | TestU01 (BigCrush) | 59:WeightDistrib | 

Re: [Rng] New XoShiRo generators

[Gilles Sadowski]

Hi Alex.

Le mer. 20 mars 2019 à 01:32, Alex Herbert  a écrit :
>
>
>
> > On 19 Mar 2019, at 15:24, Gilles Sadowski  wrote:
> >>
> >> I am currently rerunning the dieharder test for the XorShift1024Star
> >> composites since that requires a little endian format on my machine. So
> >> far there are not as many failures when the byte order is reversed.
> >>
> >> Once that is done I think we can wrap this up by:
> >>
> >> - updating the stress test to support little/big endian format as input
> >> for the test suite
> >>
> >> - updating the stress test GeneratorsList to match the RandomSource enum
> >> order
> >>
> >> - merging the modified XorShift1024StarPhi generator
> >>
> >> - deprecating the XOR_SHIFT_1024_S enum in favour of XOR_SHIFT_1024_S_PHI
>
> I have updated the PR for the XorShift1024 variant to deprecate the old 
> XOR_SHIFT_1024_S enum.
>
> Please take a look to see if the wording is OK:
>
> https://github.com/apache/commons-rng/pull/30/files#diff-630495e26d73fa22ada841dabde0ab47
>  
> 

How about:

/**
 * @deprecated Since 1.3, where it is recommended to use {@code
XOR_SHIFT_1024_S_PHI},
 * instead, due to its slightly better (more uniform) output.
 * {@code XOR_SHIFT_1024_S} is still quite usable but both versions
share the exact same
 * internal state, so that their outputs are correlated (and thus
should not be used together when
 * independent sequences are assumed).
 */

?

>
> I have not deprecated the XorShift1024Star generator. That still works and is 
> the base for the new one. Just deprecating the enum value serves as a warning 
> for users to update, or use that generator with wisely.

+1
That's what the above text tries to convey.

Gilles

>
> Alex
>
>

-
To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
For additional commands, e-mail: dev-h...@commons.apache.org

Release Announcement: General Availability of Java 12 / JDK 12

[Rory O'Donnell]

Hi Benedikt,

*1) Release Announcement: General Availability of Java 12 / JDK 12 [1] *

 * JDK 12, the reference implementation of Java 12, is now Generally
   Available.
 * GPL-licensed OpenJDK builds from Oracle are available here:
   https://jdk.java.net/12

This release includes the following  eight features:

 * JEP 189: Shenandoah: A Low-Pause-Time Garbage Collector (Experimental)
 * JEP 230: Microbenchmark Suite
 * JEP 334: JVM Constants API
 * JEP 340: One AArch64 Port, Not Two
 * JEP 341: Default CDS Archives
 * JEP 344: Abortable Mixed Collections for G1
 * JEP 346: Promptly Return Unused Committed Memory from G1
 * JEP 325: Switch Expressions (Preview)
   

Thanks to everyone who contributed JDK 12, whether by creating features 
or enhancements, logging  bugs, or downloading and testing the 
early-access builds.


*2) JDK 13 EA build 12, under both the GPL and Oracle EA licenses, is 
now available at **http://jdk.java.net/13**.*


 * Proposed - Schedule for JDK 13 [2]
 o 2019/06/13 Rampdown Phase One
 o 2019/07/18 Rampdown Phase Two
 o 2019/08/08 Initial Release Candidate
 o 2019/08/22 Final Release Candidate
 o 2019/09/17 General Availability
 * Recent Bug fixes of Interest
 o Build 9:
 + 8214719: Deprecate -Xverify:none option
 + 8216360: Deprecate -XX:CompilationPolicyChoice
 o Build 10:
 + 8218995: Deprecate the -XX:FailOverToOldVerifier option
 o Build 12 : 8160247: Mark deprecated javax.security.cert APIs
   with forRemoval=true
 + 8220050: Deprecate -XX:-ThreadLocalHandshakes
 + Apache Lucene Reported - 8219448: split-if update_uses
   accesses stale idom data
 * Changes in this build [3]

Rgds,Rory

[1] https://mail.openjdk.java.net/pipermail/jdk-dev/2019-March/002718.html
[2] https://mail.openjdk.java.net/pipermail/jdk-dev/2019-March/002716.html
[3] Changes 
 
in this build



--
Rgds,Rory O'Donnell
Quality Engineering Manager
Oracle EMEA , Dublin, Ireland