Re: [numbers][gsoc] GSoC 2022 - NUMBERS-186 Proposal
Hello. Le lun. 28 mars 2022 à 00:32, Sumanth Rajkumar a écrit : > > Thanks Alex and Gilles for your feedback > > So currently Commons Math transform depends on Common complex numbers, and > the API involves usage of Complex Object Arrays instead of primitive array > data structures > > I also briefly looked into other library implementations besides Jtransform > and EJML that are not pure java but have java bindings such as JBLAS[1] and > NAG[2] > All of the implementation use single array data structures to represent > Complex Lists and higher dimensional matrices > > Since these involve parallel data pipelines I looked into libraries that > use SIMD [3] operations that use GP GPU (jcuda [4][5] /aparapi [6]) and CPU > (Java 17 Vector API [7]) Thanks for the investigation! > > Given all the alternative implementations, I agree it does not make sense > to re implement transforms here. Some transforms are (already) implemented here. Of course, it makes sense to wonder whether to keep maintaining those codes, or rely on external dependencies. The decision would depend on performance comparisons and whether users are able (or allowed) to interface with native libraries. [I know of one project where "pure Java" was a requirement...] > > So instead would it be useful to provide users with > 1) a complex numbers Linear Algebra and transforms API to compile against > and run with any of existing providers (apache commons, jtransform, EJML, > jblas) > AND > 2) a service provider interface to allow adapter implementations to > integrate existing and future providers such as jcudas/aparapi/vector APIs > > Do the commons library modularization dependency requirements apply to > compile time dependencies only or runtime also? Which "dependency requirements" are you referring to? > To minimize bloat, the runtime dependencies could be made optional and need > not be transitively included by default Flexibility would be ideal, indeed. > > Providing a Complex linear algebra and transforms API that can run with > different runtime providers would allow users to take advantage of hardware > capabilities and gracefully fallback to reference implementations > It could allow users to take advantage of Java 17 Vector APIs when > available without refactoring their existing libraries That looks great. > > Also do the Apache projects/ license allow for integration with non Apache > software (jtransform /jblas do not use Apache license, jcuda uses MIT > license) ? Licence issues are detailed here: https://www.apache.org/legal/resolved.html Best regards, Gilles > > Thanks > Sumanth > > [1] http://jblas.org/ > > [2] https://www.nag.com/numeric/nl/nagdoc_latest/clhtml/c06/c06conts.html > > [3] https://blogs.oracle.com/javamagazine/post/programming-the-gpu-in-java > > [4] http://jcuda.org/jcuda/jcufft/JCufft.html > > [5] > https://github.com/jcuda/jcuda-samples/tree/master/JCudaSamples/src/main/java/jcuda/jcufft/samples > > [6] http://aparapi.github.io/ > > [7] https://openjdk.java.net/jeps/414 > > > On Tue, 22 Mar 2022 at 10:07, Gilles Sadowski wrote: > > > Hello. > > > > > [...] > > > > > > > > Are we expecting complex-numbers to be an efficient pure java library > > that > > > > could be used by other java libraries such as commons-imaging for data > > > > compression (DCT /JPEG lossy compression)? > > > > > > > > > > Numbers should be seen as a toolbox to be used by other Java > > applications. > > > The best location for routines is something to discuss on the mailing > > list. > > > In the example of DCT, I am not aware if imaging currently has an encoder > > > implementation for this. There is a decoder: > > > org/apache/commons/imaging/formats/jpeg/decoder/Dct.jav > > > > Also: > > > > https://gitbox.apache.org/repos/asf?p=commons-math.git;a=blob;f=commons-math-transform/src/main/java/org/apache/commons/math4/transform/FastCosineTransform.java > > > > It would be a maintenance boon if "Commons" could come up with > > a consensus about which components must be dependency-free and > > which could depend on other (lower-level) "Commons" components. > > > > [Imaging] is clearly higher-level than [Math] and that such non-obvious > > algorithms should be maintained in a single place. Through the process > > of modularizing [Math], we have "commons-math-transform" module, > > with zero dependency, so it would bring zero bloat to [Imaging] users if > > we consolidate usage. > > > > Of course, that would imply testing and benchmarking all current > > implementations, and retain the best (taking various axes into account: > > performance, robustness, flexibility). > > > > Regards, > > Gilles > > > > > [...] - To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org For additional commands, e-mail: dev-h...@commons.apache.org
Re: [Math] Review of "genetic algorithm" module
Hello. Le lun. 28 mars 2022 à 10:15, Avijit Basak a écrit : > > [...] > > >The various "Standalone" classes also look quite similar; consolidating the > >"examples-ga" module (including full Javadoc) is necessary. > -- Could you please elaborate it more. IMHO as StandAlone classes are > dedicated to the specific module only, it would remain separate. Since we > have used a single domain to show utility of the different > types(adaptive/simple) of GA some classes have become similar. > > >I still don't > >understand why there are "...-legacy" modules in module "examples-ga". > >If you want to offer the option of running the "old" implementation, you > >could add a "legacy" flag (as "@Option" in the "Standalone" application). > -- There was a discussion on this some time back. The sole purpose of > keeping the legacy example module is for comparison with the new > implementation. It will be easier for anyone to visualize the quality > improvement we achieved here. I don't want to mix(by legacy flag) this > anyway with the new implementation. > Just quickly commenting on this point. IIUC, your purpose is for users to be able to run (an example application of) the old implementation. This can be achieved by having all the "legacy" codes within module commons-math-examples/examples-ga/examples-ga-math-functions (note: No "legacy" in the module's name), within a dedicated o.a.c.m.examples.ga.mathfunctions.legacy package. This code is then called by the exact same code/application as for the new implementation (with the corresponding command line switch): $ java -jar examples-ga-app.jar --legacy ... rest of the args ... Users can thus perform 2 runs; once with "--legacy" and one without it, and reach some conclusions. The duplicate codes only bring maintenance burden (to ensure that the "legacy" and non-"legacy" modules do indeed aim at solving the same problem). Whenever we then decide that the new code has been thoroughly tested, removal of the o.a.c.m.examples.ga.mathfunctions.legacy package will be a minimal change (as compared to the removal of a module). Regards, Gilles - To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org For additional commands, e-mail: dev-h...@commons.apache.org
JDK 18 General Availability, and oracle-actions/setup-java
Greetings! JDK 18 has been released (General Availability) on March 22nd as planned, the release cadence is working like clockwork! As a small token of gratitude, some of you have been specifically acknowledged in the "The Arrival of Java 18" announcement [1]. On behalf of the entire team, let me extend our thanks to all of you. With JDK 18 released, the focus should now be on making sure your project(s) compile and work on JDK 19. As always, if you face any issue with early-access builds of JDK 19 please let us know. To help you in this task, we have just released a GitHub action to install the OpenJDK Early-Access builds. For more information, please check the heads-up below. I'll conclude with a short teaser, i.e. JavaOne is Back! [2] Stay tuned for more details. [1] https://inside.java/2022/03/22/the-arrival-of-java18/ [2] https://www.oracle.com/cloudworld/javaone/ ## Heads-Up: oracle-actions/setup-java To help you test your project(s), we have released a GitHub Action [3] to download and install various JDK builds produced by Oracle. In addition to the latest OpenJDK GA builds (GPL v2 W/CPE) and the Oracle JDK builds (NFTC license), this action can also download and install OpenJDK early-access builds, and early-access builds of OpenJDK projects (ex. Project Loom, Project Valhalla, etc.). When doing tests using EA builds, it is key to always use the upstream EA builds from jdk.java.net as issues should be logged against those upstream builds, and ideally against a specific build version. This GitHub action is actively following the OpenJDK EA builds releases. Please make sure to check the announcement [4] for more details, and short FAQ. To help you isolate regression between different EA builds, we are working to add support for archived builds. If you have feedback, please either issue the Issue tracker [5] or just send me a mail. [3] https://github.com/marketplace/actions/setup-java-development-kits-built-by-oracle [4] https://inside.java/2022/03/11/setup-java/ [5] https://github.com/oracle-actions/setup-java/issues ## General Availability of Java 18 / JDK 18 JDK 18 is now Generally Available [6]. The OpenJDK builds which are provided under the GNU General Public License v2, with the Classpath Exception are available [7], the JDK 18 Release Notes are also available [8]. [6] https://mail.openjdk.java.net/pipermail/jdk-dev/2022-March/006458.html [7] https://jdk.java.net/18/ [8] https://jdk.java.net/18/release-notes Along with hundreds of smaller enhancements and over a thousand bug fixes, JDK 18 includes following JEPs: - JEP 400: UTF-8 by Default - JEP 408: Simple Web Server - JEP 413: Code Snippets in Java API Documentation - JEP 416: Reimplement Core Reflection with Method Handles - JEP 417: Vector API (Third Incubator) - JEP 418: Internet-Address Resolution SPI - JEP 419: Foreign Function & Memory API (Second Incubator) - JEP 420: Pattern Matching for switch (Second Preview) - JEP 421: Deprecate Finalization for Removal Thanks to everyone who contributed to JDK 18, whether by designing and implementing features or enhancements, by fixing bugs, or by downloading and testing the early-access builds. ## JDK 19 Early-Access builds JDK 19 Early-Access builds 15 are now available [9], and are provided under the GNU General Public License v2, with the Classpath Exception. The Release Notes are also available [10]. [9] https://jdk.java.net/19/ [10] https://jdk.java.net/19/release-notes ### JEPs targeted to JDK 19, so far: - JEP 422: Linux/RISC-V Port https://openjdk.java.net/jeps/422 ### Recent changes that maybe of interest: - JDK-8283415: Update java.lang.ref to use sealed classes - JDK-8280494: (D)TLS signature schemes - JDK-8282081: java.time.DateTimeFormatter: wrong definition of symbol F - JDK-8281181: Do not use CPU Shares to compute active processor count - JDK-7192189: Support endpoint identification algorithm in RFC 6125 - JDK-8277474: jarsigner does not check if algorithm parameters are disabled - JDK-8280357: If the users home directory is invalid, system property user.home is set to $HOME - JDK-8277204: Implement PAC-RET branch protection on Linux/AArch64 - JDK-8282411: Add useful predicates to ElementKind - JDK-8282131: java.time.ZoneId should be a sealed abstract class - JDK-8281375: Accelerate bitCount operation for AVX2 and AVX512 target ## Topics of Interest: - “Java 18 is Here!” - Inside Java Podcast https://inside.java/2022/03/22/podcast-023/ - “The Simple Web Server” - Inside Java Podcast https://inside.java/2022/03/04/podcast-022/ - “Finalization Deprecation” - Inside Java Podcast https://inside.java/2022/01/12/podcast-021/ - All About JDK 18 - Inside Java Newscast https://inside.java/2022/03/10/insidejava-newscast-021/ - JDK 18 - Security Enhancements https://seanjmullan.org/blog/2022/03/23/jdk18 - JDK 18 - Programmer's Guide to Snippets https://docs.oracle.com/en/java/javase/18/code-snippet/index.html - JDK
Re: [Math] Review of "genetic algorithm" module
Hi All
Please find my comments.
[...]
>I don't think that it's the right way to go; instantiating an
"ExecutorService"
>belongs to the GA application, not the GA library (whose relevant classes
>need "only" be thread-safe).
>There is some misunderstanding to be clarified in a dedicated discussion
>(please file a new JIRA ticket).
-- I have created a subtask under the same Jira(MATH-1563). Please share
your thoughts.
https://issues.apache.org/jira/browse/MATH-1643
>Side note: Conflicts and duplicate commits have accumulated in the
>dedicated "feature__MATH-1563__genetic_algorithm" branch.
>I did not know how to proceed in order to avoid ending up with a messy
>history in "master"; so I created a new branch (with the same name) with
>all the new GA-related files added as a single commit.
>Currently, this branch (based on your PR #205) fails the default goal,
>because of a CheckStyle issue. You shoudl always check locally that
>running "mvn" without arguments does not generate any errors.
>I also noticed that classes in "examples-ga" use "forbidden" library
>classes: "GeneticIllegalArgumentException" is an "internal" class; we
>must not advertize such classes in the example applications.
-- I have replaced the "GeneticIllegalArgumentException" by
"IllegalArgumentException".
>In general, it seems that "examples-ga" contains several classes and
>methods that do not need to be "public". This is especially true for
>classes like "MathFunction" and "Coordinate". [Having those "private"
>helps users to tell what is part of the library's functionality from what
is
>just "dummy" placeholder code.]
-- I have replaced the MathFunction and CoordinateDecoder with lambda.
However, the Coordinate class is a domain object (phenotype). So this needs
to remain public. This can be used in more than one place for the entire
application.
>Finally (for now), I've just noticed that there exist several classes named
>"MathFunction", with same implementation!
>Code duplication must be avoided, especially where we purport to display
>best practices.
-- As mentioned above this has been removed.
>The various "Standalone" classes also look quite similar; consolidating the
>"examples-ga" module (including full Javadoc) is necessary.
-- Could you please elaborate it more. IMHO as StandAlone classes are
dedicated to the specific module only, it would remain separate. Since we
have used a single domain to show utility of the different
types(adaptive/simple) of GA some classes have become similar.
>I still don't
>understand why there are "...-legacy" modules in module "examples-ga".
>If you want to offer the option of running the "old" implementation, you
>could add a "legacy" flag (as "@Option" in the "Standalone" application).
-- There was a discussion on this some time back. The sole purpose of
keeping the legacy example module is for comparison with the new
implementation. It will be easier for anyone to visualize the quality
improvement we achieved here. I don't want to mix(by legacy flag) this
anyway with the new implementation.
>Please use the new branch for all these ("cleanup") changes, as the basis
>a PR (with a *single* commit).
-- I have taken the changes and will create a new PR soon with all my
changes.
Thanks & Regards
--Avijit Basak
On Sun, 13 Mar 2022 at 06:39, Gilles Sadowski wrote:
> Hello.
>
> Le lun. 28 févr. 2022 à 07:11, Avijit Basak a
> écrit :
> >
> > Hi All
> >
> > Please see my comments below.
> >
> > > [...]
> > >I just had a very quick look.
> > >IIUC, you always provide "convenience" methods (e.g. the various
> > >signatures for the "evolve" functionality).
> > >Prior to merging into "master", we should simplify and limit the
> > >discussion to the core functionality, i.e. not try and make decisions
> > >for the user (like default values, ...). Please keep the API as simple
> > >as possible
> > -- I have removed the mentioned evolve method.
> > However, I had to catch two checked exceptions (InterruptedException,
> > ExecutionException) and rethrow them. As of now I have handled them using
> > the GeneticIllegalArgumentException. I think we need to introduce another
> > exception class to handle this. Please share your thought regarding this.
>
> I don't think that it's the right way to go; instantiating an
> "ExecutorService"
> belongs to the GA application, not the GA library (whose relevant classes
> need "only" be thread-safe).
> There is some misunderstanding to be clarified in a dedicated discussion
> (please file a new JIRA ticket).
>
> Side note: Conflicts and duplicate commits have accumulated in the
> dedicated "feature__MATH-1563__genetic_algorithm" branch.
> I did not know how to proceed in order to avoid ending up with a messy
> history in "master"; so I created a new branch (with the same name) with
> all the new GA-related files added as a single commit.
>
> Currently, this branch (based on your PR #205) fails the default goal,
> because of a CheckStyle issue. Yo
