HI Luc,
On 09/23/2015 03:02 AM, luc wrote:
Hi,
Le 2015-09-22 02:55, Ole Ersoy a écrit :
Hola,
On 09/21/2015 04:15 PM, Gilles wrote:
Hi.
On Sun, 20 Sep 2015 15:04:08 -0500, Ole Ersoy wrote:
On 09/20/2015 05:51 AM, Gilles wrote:
On Sun, 20 Sep 2015 01:12:49 -0500, Ole Ersoy wrote:
Wanted to float some ideas for the LeastSquaresOptimizer (Possibly
General Optimizer) design. For example with the
LevenbergMarquardtOptimizer we would do:
`LevenbergMarquardtOptimizer.optimize(OptimizationContext c);`
Rough optimize() outline:
public static void optimise() {
//perform the optimization
//If successful
c.notify(LevenberMarquardtResultsEnum.SUCCESS, solution);
//If not successful
c.notify(LevenberMarquardtResultsEnum.TOO_SMALL_COST_RELATIVE_TOLERANCE,
diagnostic);
//or
c.notify(LevenberMarquardtResultsEnum.TOO_SMALL_PARAMETERS_RELATIVE_TOLERANCE,
diagnostic)
//etc
}
The diagnostic, when turned on, will contain a trace of the last N
iterations leading up to the failure. When turned off, the Diagnostic
instance only contains the parameters used to detect failure. The
diagnostic could be viewed as an indirect way to log optimizer
iterations.
WDYT?
I'm wary of having several different ways to convey information to the
caller.
It would just be one way.
One way for optimizer, one way for solvers, one way for ...
Yes I see what you mean, but I think on a whole it will be worth it to
add additional sugar code that removes the need for exceptions.
But the caller may not be the receiver
(It could be). The receiver would be an observer attached to the
OptimizationContext that implements an interface allowing it to observe
the optimization.
I'm afraid that it will add to the questions of what to put in the
code and how. [We already had sometimes heated discussions just for
the IMHO obvious (e.g. code formatting, documentation, exception...).]
Hehe. Yes I remember some of these discussions. I wonder how much
time was spent debating the exceptions alone? Surely everyone must
have had this feeling in pit of their stomach that there's got to be a
better way. On the exception topic, these are some of the issues:
I18N
===================
If you are new to commons math and thinking about designing a commons
math compatible exception you should probably understand the I18N
stuff that's bound to exception (and wonder why it's bound the the
exception).
Not really true.
Well a lot of things are gray. Personally if I'm dealing with an API, I like
to understand it, so that there are no surprises. And I understand that the
I18N coupling might not force me to use it, but if I want to be smart about my
architecture, and simplify my design, then I should look at it. Maybe it is a
good idea. Maybe I should just gloss over it? Am I being sloppy if I just
gloss over it?
Or is there an alternative that provides the same functionality, or maybe
something better, that does not come with any of these side effects?
The I18N was really simple at start.
Yup I reviewed it and thought - it's probably no big deal - but as I started
looking into reusing the CM exceptions, I decided that it was not worth the
complexity. I think that if I throw and exception it is my responsibility to
make sure that whomever receives it has a very simple 0 minute or minimal
resolution time for dealing with it, so that when the code is handed over the
client feels confident. It should work like a fridge. We leave the door open
too long, a bell goes off, we close the door.
See the one from
the Orekit project <https://www.orekit.org/forge/projects/orekit/> which is
still in this state, and you will see adding a new message can be done really
simply without a lot of stuff.
Simple is a relative term. But really simple is different from simple. Really
simple would be the LevenbergMarquardtOptimizer in it's own module, separated
from everything else, with a minimal set of dependencies. If it's not this
simple, then it quickly grows more complex as we scale.
I18N here is basically one method (getLocalizedString)
that is never changed in one enumerate class (OrekitMessages), and adding a
message
is only adding one entry to the enumerate, that's all.
Sure and when a developer sees that one method, and they know how sharp the
Apache people are (Sincerely), they start looking into reusing this design. So
if we are going to send them down that path, then that should be the best path.
The huge pile we have now is only partly related to I18N.
That's what I mean by scaling. It's like the princess with the pea below the
bottom mattress. She keeps throwing mattresses on top, and the pea is still
bothering her. So after the 10th mattress, she decides they all have to come
down until she can find out what's going on. All 10 mattresses have to come
back off.
The context stuff
was introduced as an attempt to solve some programmatic retrieval of information
at catch time, not related to 18N.
A callback would:
- Provide the same context through a diagnostic
- Decouple from I18N, but still provide the means to retrieve a message.
- Be more tightly coupled to the root cause of the exception / error
The huge hierarchy was introduced to
go in a direction were one exception = one type.
That's what I was hoping would lead to `one type` equalling the precise root
cause of the exception, but this is not true. An exception can be reused in
multiple places with multiple different root causes.
The ArgUtils was introduced
due to some Serialization issues. None of this is 18N.
So if we look at this through the lens of:
- Core developer productivity
- API user productivity
Are these helping?
Yes our exception is crap. No I18N is not the only responsible.
Honestly until I started reading NodeJS code (About six months ago) I was super
happy with it, and thought that it was this way, because it's the only way, so
that's what we get. But now I think we can do much better.
Grab a coffee and spend a few hours, unless you are
obviously fairly new to Java like some ofthe people posting for help.
In this case when the exception occurs, there is going to be a lot of
tutoring going on on the users list.
Number of Exceptions
===================
Before you do actually design a new exception, you should probably see
if there is an exception that already fits the category of what you
I don't agree.
So you don't think someone should look to reuse an existing exception...because
that would lead to a lot more new exceptions?
The large list in the enumerate is only a list of
messages for user display.
The exceptions are one thing and user display is another, and should we be
mixing these? On the one hand we are saying the CM is a low level library and
that the developer should catch the exceptions, and instruct the client on how
to handle them, and now we are saying that there are message for user display...
And this is sort of a gray area, so I don't mean to nit pick it so much, but
there's is a cleaner way to do this, that does not mix the user display concept
with the exception / reporting of something different than a solution.
There would really be nothing wrong to add
even more messages,even if they are close to existing ones.
What if they are exactly the same as the existing ones...because we were too
lazy to scan? And there's a simple programmatic solution to that, but do we
really want this workflow?
In fact, I
even think we should avoid trying to reuse messages by merging them all
in something not meaningful to users (remember, messages are for display
only). Currently we too often get a message like:
Number is too large, 3 > 2
Nobody understands this at user level.
If instead there's an Enum, tied to the corresponding class, that represents
this condition at the line where it happens, then we can leave it up to the
developer to craft a message that will serve the client best.
And I'm not saying that there should not be messages. Just that they should be
isolated to a specific context, maintained within the parameters of that
context, and looked up by a single unique key only.
The reused message has lost its
signification. I would really prefer different messages for different cases
where the fixed part of the format would at least provide some hint to the
user.
Yes I think we are saying the same thing here.
are doing. So you start reading. Exception1...nop
Exception2...nop...Exception3...Exception999..But I think I'm getting
warmer. OK - Did not find it ... but I'm fairly certain that there is
a elegant place for it somewhere in the exception hierarchy...
I agree. The exception hierarchy is a mess.
And in all fairness it seemed really elegant at first, and we all went with it.
Handling of Exceptions
===================
If our app uses several of the commons math classes (That throw
exceptions of the same type), and one of those classes throws an
exception,what is the app supposed to do?
It depends on the application.
But it's possible to architect CM so that the case is that the developer is led
down one road only. And it's like a Hyperloop. They get in, and 7 minutes
later, they are sipping a Mojito in Cancun :).
Apache Commons Math is a low level
library it is used in many different contexts and there is no single
answer.
Unless there is.
I think most developers would find that question somewhat challenging.
There are numerous strategies. Catch all exceptions and log what
happened, etc. But what if the requirement is that if an exception is
thrown, the organization that receives it has 0 seconds to get to the
root cause of it and understand the dynamics. Is this doable? (Yes
obviously, but how hard is it...?).
It is not Apache Commons Math level of decision.
Not right now, but it could be.
We provide the exception
and users can catch it fast. What users do with it is up to them.
Sure. That's how it works right now, but this is causing both more CM core
developer overhead and API user overhead than there would be if exceptions are
replaced with Enums and throwing exceptions is replaced with a callback.
It seems that the reporting interfaces could quickly overwhelm
the "actual" code (one type of context per algorithm).
There would one type of Observer interface per algorithm. It would
act on the solution and what are currently exceptions, although these
would be translated into enums.
Unless I'm mistaken, the most common use-case for codes implemented
in a library such as CM is to provide a correct answer or bail out
in a non-equivocal way.
Most java developers are used to synchronous coding...call the method
get the response...catch the exception if needed. This is changing
with JDK8, and as we evolve and start using lambdas, we become more
accustomed to the functional callback style of programming.
Personally I want to be able to use an API that gives me what I need
when everything works as expected, allows me to resolve unexpected
issues with minimal effort, and is as simple, fluid, and lightweight
as possible.
It would make the code more involved to handle a minority of
(undefined) cases. [Actual examples would be welcome in order to
focus the discussion.]
Rough Outline (I've evolved the concept and moved away from the
OptimizationContext in the process of writing):
interface LevenbergMarquardtObserver {
public void hola(Solution s);
public void sugarHoneyIceTea(ResultType rt, Dianostics d);
}
public class LMObserver implements LevenbergMarquardtObserver {
private Application application;
public LMObserver(Application application) {
this.application = application;
}
public void hola(ResultType rt, Solution s) {
application.next(solution);
}
public void sugarHoneyIceTea(ResultType rt, Diagnostic s)
if (rt == ResultType.I_GOT_THIS_ONE) {
//I looked at the commons unit tests for this algorithm evaluating
//the diagnostics that shows how this failure can occur
//I'm totally fixing this! Steps aside!
}
else if (rt == ResultType.REALLY_COMPLICATED_STUFF)
{
//We need our best engineers...call India.
}
)
public class Application {
//Note nothing is returned.
LevenberMarquardtOptimizer.setOberver(new
LMObserver(this)).setLeastSquaresProblem(new
ClassThatImplementsTheProblem())).start();
public void next(Solution solution) {
//Do cool stuff.
}
}
Or an asynchronous variation:
public class Application {
//This call will not block because async is true
LevenberMarquardtOptimizer.setAsync(true).setOberver(new
LMObserver()).setLeastSquaresProblem(new
ClassThatImplementsTheProblem())).start();
//Do more stuff right away.
public void next(Solution solution) {
//When the thread running the optimization is done, this
method is called back.
//Do whatever comes next
}
}
The above would start the optimization in a separate thread that does
not / SHOULD NOT share data with the main thread.
The current reporting is based on exceptions, and assumes that if no
exception was thrown, then the user's request completed successfully.
Sure - personally I'd much rather deal with something similar to an
HTTP status code in a callback, than an exception . I think the code
is cleaner and the calback makes it more elegant to apply an adaptive
approach to handling the response, like slightly relaxing constraints,
convergence parameters, etc. Also by getting rid of the exceptions,
we no longer depend on the I18N layer that they are tied to and now
the messages can be more informative, since they target the root
cause. The observer can also run in the 'main' thread' while the
optimization can run asynchronously. Also WRT JDK9 and modules,
loosing the exceptions would mean one less dependency when the library
is up into JDK9 modules...which would be more in line with this
philosophy:
https://github.com/substack/browserify-handbook#module-philosophy
I'm not sure I fully understood the philosophy from the text in this
short paragraph.
But I do not agree with the idea that the possibility to quickly find
some code is more important than standards and best practices.
If you go to npmjs.org and type in Neural Network you will get 56
results all linked to github repositories.
In addition there's meta data indicating number of downloads in the
last day, last month, etc. Try typing in cosine. Odds are you will
find a package that does just want you want and nothing else. This is
very underwhelming and refreshing in terms of cloning off of github
and getting familar with tests etc. Also eye opening. How many of us
knew that we could do that much stuff with cosine! :).
I totally agree that in some circumstances, more information on the
inner working of an algorithm would be quite useful.
... Algorithm iterations become unit testable.
But I don't see the point in devoting resources to reinvent the wheel:
You mean pimping the wheel? Big pimpin.
I think that logging statements are easy to add, not disruptive at all,
and come in handy to understand a code's unexpected behaviour.
Assuming that a "logging" feature is useful, it can be added *now* using
a dependency towards a weight-less (!) framework such as "slf4j".
IMO, it would be a waste of time to implement a new communication layer
that can do that, and more, if it would be used for logging only in 99%
of the cases.
SLF4J is used by almost every other framework, so why not use it?
Logging and the diagnostic could be used together. The primary
purpose of the diagnostic though is to collect data that will be
useful in `sugarHoneyIceTea`.
I longed several times for the use of a logging library.
The only show-stopper has been the informal "no-dependency" policy...
JDK9 Jigsaw should solve dependency hell, so the less coupling
between commons math classes the better.
I wouldn't call "coupling" the dependency towards exception classes:
they are little utilities that can make sense in various parts of the
library.
If for example the Simplex solver is broken off into it's own module,
then it has to be coupled to the exceptions, unless it is exception
free.
[Unless one wants to embark on yet another discussion about exceptions;
whether there should be one class for each of the "messages" that exist
in "LocalizedFormats"; whether localization should be done in CM;
etc.]
I think it would be best to just eliminate the exceptions.
NO! A big no!
Before the NO is that big, how about examining just a few simple cases where
it's done? You might come around to liking it. Also it does not need to be a
all or nothing. Some modules could be exception free, whereas others, could
throw exceptions because it might be the simplest thing to do and what people
are used to.
Apache Commons Math is a very low level library. There are use cases
where you have huge code that relies on math almost everywhere.
True dat. But it's quite possible that some of the math that is being used does
not actually need to throw the exception. There are multiple exception
categories:
- Exceptions thrown by invalid input parameters
- Exceptions that signal that either something completely unexpected happened
(Even we are like WTF?)
- Exceptions that could happen in certain edge conditions that we understand
and know how to deal with
The first one we can eliminate by supplying a self validating object to the
routine. The next two are the ones that it might really be helpful they are
communicated as Enums via a callback.
It could be reviewed on a case by case basis, in the a process that considers
moving to JDK9 modules.
Look at
the Orekit library for example. We use Vector3D everywhere, we use Rotation
everywhere, we use ode in many places, we use some linear algebra, we use
optimizers, we use a few statistics, we use root solvers, we use derivatives,
we use BSP
No problem just get rid of the 3D rotation, kill the Vector3D, forget
statistics (Way too complicated!), and it's all good :). Kidding obviously, but
this does highlight what I'm talking about. If there are tons of CM classes
being utilities within a single method, and all of them can throw generic
exceptions that cut across multiple classes, the developer has to decode the
exception, causing her to waste time.
... Some of these uses looks as large scale call/return pattern
where you could ask users to check the result afterwards, but many, many of
the calls are much lower levels and you have literally several thousands of
calls to math.
That's all fine. CM will still have the exact same workings - it will just be
more efficient.
Just think about forcing user to check a vector can be normalized
(i.e. has not 0 norm) everywhere a vector is used. We would end up with
something like:
Rotation r = new Rotation(a, alpha);
if (!r.isValid()) {
// a vector was null
return error;
}
In this case the client developer could have made sure the vector was not null
and valid prior to passing it to a routine. It's possible that CM will be more
lightweight, efficient, and friendly if API users are told that it's their
responsibility to validate parameters. So:
RotationContext r = new RotationContext(a, alpha);
if (!r.isValid()) {
// Dag Nab It!!!
}
else {
//Rotate away
Rotation rotation = new Rotation(rotationContext);
}
Or if we know that the parameters are valid (Mass validation):
Rotation rotation = new Rotation(a, alpha);
Repeat the above 4325 times in your code ... welcome back to the 80's.
We only have to repeat the validation check 4325 times if there is actually a
possibility that the vector is invalid.
Exceptions are meant for, well, exceptional situations.
These are exceptional times :)
They avoid people
to cripple the calling code with if(error) statements.
We need if(error) at some point. I agree that if(error) should be minimized.
They ensure (and this
is the most important part) that as soon as the error is detected at very
low level it will be identified and reported back to the upper level where
it is caught and displayed, without forcing *all* the intermediate levels
to handle it.
Yes but this causes the "Root cause analysis deciphering effect" I've been
talking about.
When you have complex code with complex algorithms and several
nested levels of calls in different libraries, maintained by different teams
and usings tens of thousands calls to math, you don't want a call/return/check
error
type of programming like we used to do in the 80's.
Ahh the Commodore 64 days...I miss those days...Anyways - no one wants to be
hung up on some clown from the 80s. Right now we are hung up on a clown from
the 90s though! We need a Kim Kardashian API. :)
Cheers,
- Ole
best regards,
Luc
Anyways I'm obviously
interested in playing with this stuff, so when I get something up into
a repository I'll to do a callback :).
If you are interested in big overhauls, there is one that gathered
relative consensus: rewrite the algorithms in a "multithread-friendly"
way.
I think that's a tall order that will take us into JDK88 :). But
using callbacks and making potentially long running computations
asynchronous could be a middle ground that would allow simple multi
threaded use without fiddling around under the hood...
Some ideas were floated (cf. ML archive) but no implementation or
experiment... Perhaps with a well-defined goal such as performance
improvement, your design suggestions will become clearer to more people.
AFAIK, only the classes in the "o.a.c.m.neuralnet" package are currently
ready to be used with the "java.util.concurrent" framework.
FWIU Neural Nets are a great fit for concurrency. I think for the
others we will end up having discussions around how users would
control the number of threads, etc. again that makes some of us
nervous. An asynchronous operation that runs in one separate thread
is easier to reason about. If we want to test 10 neural net
configurations, and we have 10 cores, then we can start each by itself
by doing something like:
Nework.setAsync(true).addNeurons().connectNeurons().addObserver(observer).start().
//Now do 10 more
//If the observer is shared then notifications should be thread safe.
Cheers,
- Ole
P.S. Dang that was a long email. If I write one more of these, ban me :)
Best regards,
Gilles
Cheers,
Ole
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
For additional commands, e-mail: dev-h...@commons.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
For additional commands, e-mail: dev-h...@commons.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
For additional commands, e-mail: dev-h...@commons.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
For additional commands, e-mail: dev-h...@commons.apache.org
