Re: [Math] LeastSquaresOptimizer Design

[luc]

Le 2015-09-24 04:16, Ole Ersoy a écrit :

On 09/23/2015 03:09 PM, Luc Maisonobe wrote:
CM is not intended to be a design pattern people should mimic. We are 
so bad at this it would be a shame. No one in its right mind would 
copy or reuse this stuff. It is for internal use only and we don't 
even have the resources to manage it by ourselves so we can't consider 
it as a path people should follow as we are leading them. Here we 
would be leading them directly against the wall.


Hehe - I think that's like Michael Jordan saying - "Guys, don't try to
be like me.  I just play a little ball.  Dunk from the free throw
line.  Six world championships, but THATs it!".  In any case, I really
appreciate you and Gilles taking the time to talk.  Luc (And possibly
Gilles) - I can actually see why you are getting a bit annoyed,
because I'm ignoring something important.

I've been doing 90% NodeJS stuff lately (Which is event loop based and
relies callbacks) so I forgot one very important thing that I think
you have both tried to tell me.  The exception undoes the current
callstack / breaks the current program flow, bubbling up to the
handler.  Thts a good point.

OK - So scratch the callback thinking for synchronous code.  The
Lombok stuff should still be good though and hopefully some of the
callback discussion around and asynchronous option - I hope!  Geez.

What do you think about having one exception per class with an Enum
that encodes the various types of exceptional conditions that the
class can find itself in?  So in the case of
LevenbergMarquardtOptimizer there would be a:
- LevenbergMarquardtOptimizerException:
- LevenbergMarquardtOptimizerExceptionEnum

When the exception is thrown it sets the Enum indicating the root
cause.  The enum can then be used as a key to lookup the corresponding
message.

Any better?


Sure. I would suggest adding some parameters to help the upper level 
formatting
a meaningful message (say the number of iterations performed if you hit 
a max
iteration, so users become aware they should have set the limit higher). 
Nothing
over-engineered, a simple Object[] that can be used as last argument to 
something

like String.format() would be enough.

best regards,
Luc



Cheers,
- Ole


-
To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
For additional commands, e-mail: dev-h...@commons.apache.org


-
To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
For additional commands, e-mail: dev-h...@commons.apache.org

Re: [Math] LeastSquaresOptimizer Design

[luc]

Hi Gilles,

Le 2015-09-23 23:00, Gilles a écrit :

[...]

CM is not intended to be a design pattern people should mimic.
We are so bad at this


The crux is that the project's team is in effect not _interested_
in this.  [And I admit that I had not understood it for a long
time (hence the temptation to convince that it was important for
*some* people).]


it would be a shame. No one in its right mind would copy
or reuse this stuff. It is for internal use only


Then why is it so difficult to change (cf. all the nit-picking
about backward-compatibility)?
As was (relatively) recently discussed, we could "mark" some code
"for internal use" and be free to break compatibility at any time,
for the sake of (an attempt at) a better design.


I think it would be nice. IMHO, we are overzealous on compatibility.
Sometimes, I try to introduce some changes that may break compatibility
early for some non user-implementable interfaces but have to withdraw
my proposal (tried it for ode, tried ot for BSP trees, think I tried it
for optimizers).




and we don't even have
the resources to manage it by ourselves


There are (maybe) other people (like Ole?) who would like to
experiment with new design ideas (not new math algorithms!)
but are repelled by the (overly) conservative development process
which is mainly feature-driven (like in a commercial project,
shall I dare to say).


Surely. But it is not specifically commercial vs open source I think.
I know commercial projects that break compatibility all the time 
(perhaps

to force users buying a new licence) and some that are stable. I know
open-source projects that break compatibility all the time (perhaps
because the team is highly dynamic or doesn't care about users) and
some that are stable.




so we can't consider it as a
path people should follow as we are leading them. Here we would be
leading them directly against the wall.


True, unfortunately.
There is really no long-term design. Even short term (quasi-)decisions
when they concern the library as a whole, are not followed by action
(cf. "fluent API")...


I still have fluent API in my TODO list and still really wants to make
it appear. The major blocking factor is available time (and sometimes
also despair about probable endless forthcoming discussions but it is
only second in the list).

best regards,
Luc




[...]


Best,
Gilles


-
To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
For additional commands, e-mail: dev-h...@commons.apache.org


-
To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
For additional commands, e-mail: dev-h...@commons.apache.org

Re: [Math] LeastSquaresOptimizer Design

[Ole Ersoy]

On 09/24/2015 06:31 AM, luc wrote:

Le 2015-09-24 04:16, Ole Ersoy a écrit :

On 09/23/2015 03:09 PM, Luc Maisonobe wrote:

CM is not intended to be a design pattern people should mimic. We are so bad at 
this it would be a shame. No one in its right mind would copy or reuse this 
stuff. It is for internal use only and we don't even have the resources to 
manage it by ourselves so we can't consider it as a path people should follow 
as we are leading them. Here we would be leading them directly against the wall.


Hehe - I think that's like Michael Jordan saying - "Guys, don't try to
be like me.  I just play a little ball.  Dunk from the free throw
line.  Six world championships, but THATs it!".  In any case, I really
appreciate you and Gilles taking the time to talk.  Luc (And possibly
Gilles) - I can actually see why you are getting a bit annoyed,
because I'm ignoring something important.

I've been doing 90% NodeJS stuff lately (Which is event loop based and
relies callbacks) so I forgot one very important thing that I think
you have both tried to tell me.  The exception undoes the current
callstack / breaks the current program flow, bubbling up to the
handler.  Thts a good point.

OK - So scratch the callback thinking for synchronous code.  The
Lombok stuff should still be good though and hopefully some of the
callback discussion around and asynchronous option - I hope! Geez.

What do you think about having one exception per class with an Enum
that encodes the various types of exceptional conditions that the
class can find itself in?  So in the case of
LevenbergMarquardtOptimizer there would be a:
- LevenbergMarquardtOptimizerException:
- LevenbergMarquardtOptimizerExceptionEnum

When the exception is thrown it sets the Enum indicating the root
cause.  The enum can then be used as a key to lookup the corresponding
message.

Any better?


Sure. I would suggest adding some parameters to help the upper level formatting
a meaningful message (say the number of iterations performed if you hit a max
iteration, so users become aware they should have set the limit higher). Nothing
over-engineered, a simple Object[] that can be used as last argument to 
something
like String.format() would be enough.

Brilliant - I'll setup a repository and start experimenting.  Thanks again,
- Ole


-
To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
For additional commands, e-mail: dev-h...@commons.apache.org

Contributor names in commit logs? (was: [jira] [Commented] (VFS-567) Timeout in vsFTPd causes exception when executing another command)

[Bernd]

Hello,

do we have this rule to include the name of a patch contributor into
the commit message? I havent seen that beeing used so far in
commons-vfs, and the commit does contain Antonio's name in the
changes.xml due-to= credits. I think a while back we had the
discussion that not even this is required for smaller contributions.
(but I try to do it for VFS on all changes).

Gruss
Bernd



-- Forwarded message --
From: Antonio Petrelli (JIRA) 
Date: 2015-09-24 10:50 GMT+02:00
Subject: [jira] [Commented] (VFS-567) Timeout in vsFTPd causes
exception when executing another command
To: iss...@commons.apache.org



[ 
https://issues.apache.org/jira/browse/VFS-567?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14906040#comment-14906040
]

Antonio Petrelli commented on VFS-567:
--

Thanks Bernd, however, as I am an emeritus PMC member of other Apache
project, I remember that the name of the contributor should be
inserted in the commit log.

> Timeout in vsFTPd causes exception when executing another command
> -
>
> Key: VFS-567
> URL: https://issues.apache.org/jira/browse/VFS-567
> Project: Commons VFS
>  Issue Type: Bug
>Affects Versions: 2.1
> Environment: vsFTPd 3.0.2 on Kubuntu 14.10
>Reporter: Antonio Petrelli
>Assignee: Bernd Eckenfels
> Fix For: 2.1
>
> Attachments: commons-vfs-disconnect.diff, vfs-quit-problem.zip
>
>
> After a timeout in vsFTPd, a QUIT command is sent and a 421 Timeout response 
> is sent back to the Java client. After that, a SocketException (broken pipe) 
> is raised and not correctly managed.
> I am attaching a test case, this is the stack trace:
> Exception in thread "main" org.apache.commons.vfs2.FileSystemException: Could 
> not determine the type of file "ftps://localhost/javadev/vfs/input".
>   at 
> org.apache.commons.vfs2.provider.AbstractFileObject.getType(AbstractFileObject.java:1526)
>   at QuitProblemMain.main(QuitProblemMain.java:41)
> Caused by: java.net.SocketException: Pipe interrotta
>   at java.net.SocketOutputStream.socketWrite0(Native Method)
>   at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:109)
>   at java.net.SocketOutputStream.write(SocketOutputStream.java:153)
>   at sun.security.ssl.OutputRecord.writeBuffer(OutputRecord.java:431)
>   at sun.security.ssl.OutputRecord.write(OutputRecord.java:417)
>   at 
> sun.security.ssl.SSLSocketImpl.writeRecordInternal(SSLSocketImpl.java:864)
>   at sun.security.ssl.SSLSocketImpl.writeRecord(SSLSocketImpl.java:835)
>   at sun.security.ssl.AppOutputStream.write(AppOutputStream.java:123)
>   at sun.nio.cs.StreamEncoder.writeBytes(StreamEncoder.java:221)
>   at sun.nio.cs.StreamEncoder.implFlushBuffer(StreamEncoder.java:291)
>   at sun.nio.cs.StreamEncoder.implFlush(StreamEncoder.java:295)
>   at sun.nio.cs.StreamEncoder.flush(StreamEncoder.java:141)
>   at java.io.OutputStreamWriter.flush(OutputStreamWriter.java:229)
>   at java.io.BufferedWriter.flush(BufferedWriter.java:254)
>   at org.apache.commons.net.ftp.FTP.__send(FTP.java:505)
>   at org.apache.commons.net.ftp.FTP.sendCommand(FTP.java:479)
>   at 
> org.apache.commons.net.ftp.FTPSClient.sendCommand(FTPSClient.java:541)
>   at org.apache.commons.net.ftp.FTP.sendCommand(FTP.java:608)
>   at org.apache.commons.net.ftp.FTP.sendCommand(FTP.java:582)
>   at org.apache.commons.net.ftp.FTP.quit(FTP.java:864)
>   at 
> org.apache.commons.vfs2.provider.ftp.FTPClientWrapper.disconnect(FTPClientWrapper.java:118)
>   at 
> org.apache.commons.vfs2.provider.ftp.FTPClientWrapper.listFiles(FTPClientWrapper.java:152)
>   at 
> org.apache.commons.vfs2.provider.ftp.FtpFileObject.doGetChildren(FtpFileObject.java:136)
>   at 
> org.apache.commons.vfs2.provider.ftp.FtpFileObject.getChildFile(FtpFileObject.java:106)
>   at 
> org.apache.commons.vfs2.provider.ftp.FtpFileObject.getInfo(FtpFileObject.java:192)
>   at 
> org.apache.commons.vfs2.provider.ftp.FtpFileObject.doGetType(FtpFileObject.java:320)
>   at 
> org.apache.commons.vfs2.provider.AbstractFileObject.getType(AbstractFileObject.java:1517)
>   ... 1 more



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
For additional commands, e-mail: dev-h...@commons.apache.org

RE: Contributor names in commit logs? (was: [jira] [Commented] (VFS-567) Timeout in vsFTPd causes exception when executing another command)

[Gary Gregory]

There is such requirement for attribution that I know of. The changes.xml is 
what I see used most consistently and that's not a requirement either from what 
I can tell.
With Git, there is a formal way to trace contributions, but I'm not sure if 
that's used at the ASF.

Gary 

 Original message 
From: Bernd  
Date: 09/24/2015  08:59  (GMT-08:00) 
To: Commons Developers List  
Subject: Contributor names in commit logs? (was: [jira] [Commented] (VFS-567)
  Timeout in vsFTPd causes exception when executing another command) 

Hello,

do we have this rule to include the name of a patch contributor into
the commit message? I havent seen that beeing used so far in
commons-vfs, and the commit does contain Antonio's name in the
changes.xml due-to= credits. I think a while back we had the
discussion that not even this is required for smaller contributions.
(but I try to do it for VFS on all changes).

Gruss
Bernd



-- Forwarded message --
From: Antonio Petrelli (JIRA) 
Date: 2015-09-24 10:50 GMT+02:00
Subject: [jira] [Commented] (VFS-567) Timeout in vsFTPd causes
exception when executing another command
To: iss...@commons.apache.org



    [ 
https://issues.apache.org/jira/browse/VFS-567?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14906040#comment-14906040
]

Antonio Petrelli commented on VFS-567:
--

Thanks Bernd, however, as I am an emeritus PMC member of other Apache
project, I remember that the name of the contributor should be
inserted in the commit log.

> Timeout in vsFTPd causes exception when executing another command
> -
>
> Key: VFS-567
> URL: https://issues.apache.org/jira/browse/VFS-567
> Project: Commons VFS
>  Issue Type: Bug
>    Affects Versions: 2.1
> Environment: vsFTPd 3.0.2 on Kubuntu 14.10
>    Reporter: Antonio Petrelli
>    Assignee: Bernd Eckenfels
> Fix For: 2.1
>
> Attachments: commons-vfs-disconnect.diff, vfs-quit-problem.zip
>
>
> After a timeout in vsFTPd, a QUIT command is sent and a 421 Timeout response 
> is sent back to the Java client. After that, a SocketException (broken pipe) 
> is raised and not correctly managed.
> I am attaching a test case, this is the stack trace:
> Exception in thread "main" org.apache.commons.vfs2.FileSystemException: Could 
> not determine the type of file "ftps://localhost/javadev/vfs/input".
>   at 
>org.apache.commons.vfs2.provider.AbstractFileObject.getType(AbstractFileObject.java:1526)
>   at QuitProblemMain.main(QuitProblemMain.java:41)
> Caused by: java.net.SocketException: Pipe interrotta
>   at java.net.SocketOutputStream.socketWrite0(Native Method)
>   at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:109)
>   at java.net.SocketOutputStream.write(SocketOutputStream.java:153)
>   at sun.security.ssl.OutputRecord.writeBuffer(OutputRecord.java:431)
>   at sun.security.ssl.OutputRecord.write(OutputRecord.java:417)
>   at 
>sun.security.ssl.SSLSocketImpl.writeRecordInternal(SSLSocketImpl.java:864)
>   at sun.security.ssl.SSLSocketImpl.writeRecord(SSLSocketImpl.java:835)
>   at sun.security.ssl.AppOutputStream.write(AppOutputStream.java:123)
>   at sun.nio.cs.StreamEncoder.writeBytes(StreamEncoder.java:221)
>   at sun.nio.cs.StreamEncoder.implFlushBuffer(StreamEncoder.java:291)
>   at sun.nio.cs.StreamEncoder.implFlush(StreamEncoder.java:295)
>   at sun.nio.cs.StreamEncoder.flush(StreamEncoder.java:141)
>   at java.io.OutputStreamWriter.flush(OutputStreamWriter.java:229)
>   at java.io.BufferedWriter.flush(BufferedWriter.java:254)
>   at org.apache.commons.net.ftp.FTP.__send(FTP.java:505)
>   at org.apache.commons.net.ftp.FTP.sendCommand(FTP.java:479)
>   at 
>org.apache.commons.net.ftp.FTPSClient.sendCommand(FTPSClient.java:541)
>   at org.apache.commons.net.ftp.FTP.sendCommand(FTP.java:608)
>   at org.apache.commons.net.ftp.FTP.sendCommand(FTP.java:582)
>   at org.apache.commons.net.ftp.FTP.quit(FTP.java:864)
>   at 
>org.apache.commons.vfs2.provider.ftp.FTPClientWrapper.disconnect(FTPClientWrapper.java:118)
>   at 
>org.apache.commons.vfs2.provider.ftp.FTPClientWrapper.listFiles(FTPClientWrapper.java:152)
>   at 
>org.apache.commons.vfs2.provider.ftp.FtpFileObject.doGetChildren(FtpFileObject.java:136)
>   at 
>org.apache.commons.vfs2.provider.ftp.FtpFileObject.getChildFile(FtpFileObject.java:106)
>   at 
>org.apache.commons.vfs2.provider.ftp.FtpFileObject.getInfo(FtpFileObject.java:192)
>   at 
>org.apache.commons.vfs2.provider.ftp.FtpFileObject.doGetType(FtpFileObject.java:320)
>   at 
>org.apache.commons.vfs2.provider.AbstractFileObject.getType(AbstractFileObject.java:1517)
>   ... 1 more



--

Re: Contributor names in commit logs? (was: [jira] [Commented] (VFS-567) Timeout in vsFTPd causes exception when executing another command)

[Phil Steitz]

On 9/24/15 8:59 AM, Bernd wrote:
> Hello,
>
> do we have this rule to include the name of a patch contributor into
> the commit message? I havent seen that beeing used so far in
> commons-vfs, and the commit does contain Antonio's name in the
> changes.xml due-to= credits. I think a while back we had the
> discussion that not even this is required for smaller contributions.
> (but I try to do it for VFS on all changes).

I try to do this (though I sometimes forget and have to go back and
fix things later):

Always mention contributor in the commit (basically tells where the
code came from, no attribution means I did it myself).
Always include due-to in changes.xml.
Add contributor to contributors section of the pom if the
contribution is significant.

Very small changes may not be represented in changes.xml, but are
still attributed in the commit message.  To me, the commit message
describes the commit; the changes.xml and contributors references
acknowledge the contribution.

Different components do this differently.  I don't think we have or
need to have rigid rules, other than that really large contributions
should be granted under CLA.

Phil
>
> Gruss
> Bernd
>
>
>
> -- Forwarded message --
> From: Antonio Petrelli (JIRA) 
> Date: 2015-09-24 10:50 GMT+02:00
> Subject: [jira] [Commented] (VFS-567) Timeout in vsFTPd causes
> exception when executing another command
> To: iss...@commons.apache.org
>
>
>
> [ 
> https://issues.apache.org/jira/browse/VFS-567?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14906040#comment-14906040
> ]
>
> Antonio Petrelli commented on VFS-567:
> --
>
> Thanks Bernd, however, as I am an emeritus PMC member of other Apache
> project, I remember that the name of the contributor should be
> inserted in the commit log.
>
>> Timeout in vsFTPd causes exception when executing another command
>> -
>>
>> Key: VFS-567
>> URL: https://issues.apache.org/jira/browse/VFS-567
>> Project: Commons VFS
>>  Issue Type: Bug
>>Affects Versions: 2.1
>> Environment: vsFTPd 3.0.2 on Kubuntu 14.10
>>Reporter: Antonio Petrelli
>>Assignee: Bernd Eckenfels
>> Fix For: 2.1
>>
>> Attachments: commons-vfs-disconnect.diff, vfs-quit-problem.zip
>>
>>
>> After a timeout in vsFTPd, a QUIT command is sent and a 421 Timeout response 
>> is sent back to the Java client. After that, a SocketException (broken pipe) 
>> is raised and not correctly managed.
>> I am attaching a test case, this is the stack trace:
>> Exception in thread "main" org.apache.commons.vfs2.FileSystemException: 
>> Could not determine the type of file "ftps://localhost/javadev/vfs/input".
>>   at 
>> org.apache.commons.vfs2.provider.AbstractFileObject.getType(AbstractFileObject.java:1526)
>>   at QuitProblemMain.main(QuitProblemMain.java:41)
>> Caused by: java.net.SocketException: Pipe interrotta
>>   at java.net.SocketOutputStream.socketWrite0(Native Method)
>>   at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:109)
>>   at java.net.SocketOutputStream.write(SocketOutputStream.java:153)
>>   at sun.security.ssl.OutputRecord.writeBuffer(OutputRecord.java:431)
>>   at sun.security.ssl.OutputRecord.write(OutputRecord.java:417)
>>   at 
>> sun.security.ssl.SSLSocketImpl.writeRecordInternal(SSLSocketImpl.java:864)
>>   at sun.security.ssl.SSLSocketImpl.writeRecord(SSLSocketImpl.java:835)
>>   at sun.security.ssl.AppOutputStream.write(AppOutputStream.java:123)
>>   at sun.nio.cs.StreamEncoder.writeBytes(StreamEncoder.java:221)
>>   at sun.nio.cs.StreamEncoder.implFlushBuffer(StreamEncoder.java:291)
>>   at sun.nio.cs.StreamEncoder.implFlush(StreamEncoder.java:295)
>>   at sun.nio.cs.StreamEncoder.flush(StreamEncoder.java:141)
>>   at java.io.OutputStreamWriter.flush(OutputStreamWriter.java:229)
>>   at java.io.BufferedWriter.flush(BufferedWriter.java:254)
>>   at org.apache.commons.net.ftp.FTP.__send(FTP.java:505)
>>   at org.apache.commons.net.ftp.FTP.sendCommand(FTP.java:479)
>>   at 
>> org.apache.commons.net.ftp.FTPSClient.sendCommand(FTPSClient.java:541)
>>   at org.apache.commons.net.ftp.FTP.sendCommand(FTP.java:608)
>>   at org.apache.commons.net.ftp.FTP.sendCommand(FTP.java:582)
>>   at org.apache.commons.net.ftp.FTP.quit(FTP.java:864)
>>   at 
>> org.apache.commons.vfs2.provider.ftp.FTPClientWrapper.disconnect(FTPClientWrapper.java:118)
>>   at 
>> org.apache.commons.vfs2.provider.ftp.FTPClientWrapper.listFiles(FTPClientWrapper.java:152)
>>   at 
>> org.apache.commons.vfs2.provider.ftp.FtpFileObject.doGetChildren(FtpFileObject.java:136)
>>   at 
>> org.apache.commons.vfs2.provider.ftp.FtpFileObject.getChildFile(FtpFileObject.java:106)
>>   at 
>> 

Re: [Math] LeastSquaresOptimizer Design

[Ole Ersoy]

Hi Luc,

I gave this some more thought, and I think I may have tapped out to soon, even 
though you are absolutely right about what an exception does in terms bubbling 
execution to a point where it stops or we handle it.

Suppose we have an Optimizer and an Optimizer observer.  The optimizer will 
emit three different events given in the process of stepping through to the max 
number of iterations it is allotted:
- SOLUTION_FOUND
- COULD_NOT_CONVERGE_FOR_REASON_1
- COULD_NOT_CONVERGE_FOR_REASON_2
- END (Max iterations reached)

So we have the observer interface:

interface OptimizerObserver {

success(Solution solution)
update(Enum enum, Optimizer optimizer)
end(Optimizer optimizer)
}

So if the Optimizer notifies the observer of `success`, then the observer does 
what it needs to with the results and moves on.  If the observer gets an 
`update` notification, that means that given the current [constraints, numbers 
of iterations, data] the optimizer cannot finish.  But the update method 
receives the optimizer, so it can adapt it, and tell it to continue or just 
trash it and try something completely different.  If the `END` event is reached 
then the Optimizer could not finish given the number of allotted iterations.  
The Optimizer is passed back via the callback interface so the observer could 
allow more iterations if it wants to...perhaps based on some metric indicating 
how close the optimizer is to finding a solution.

What this could do is allow the implementation of the observer to throw the 
exception if 'All is lost!', in which case the Optimizer does not need an 
exception.  Totally understand that this may not work everywhere, but it seems 
like it could work in this case.

WDYT?

Cheers,
- Ole



On 09/23/2015 03:09 PM, Luc Maisonobe wrote:

Le 23/09/2015 19:20, Ole Ersoy a écrit :

HI Luc,

Hi Ole,


On 09/23/2015 03:02 AM, luc wrote:

Hi,

Le 2015-09-22 02:55, Ole Ersoy a écrit :

Hola,

On 09/21/2015 04:15 PM, Gilles wrote:

Hi.

On Sun, 20 Sep 2015 15:04:08 -0500, Ole Ersoy wrote:

On 09/20/2015 05:51 AM, Gilles wrote:

On Sun, 20 Sep 2015 01:12:49 -0500, Ole Ersoy wrote:

Wanted to float some ideas for the LeastSquaresOptimizer (Possibly
General Optimizer) design.  For example with the
LevenbergMarquardtOptimizer we would do:
`LevenbergMarquardtOptimizer.optimize(OptimizationContext c);`

Rough optimize() outline:
public static void optimise() {
//perform the optimization
//If successful
 c.notify(LevenberMarquardtResultsEnum.SUCCESS, solution);
//If not successful


c.notify(LevenberMarquardtResultsEnum.TOO_SMALL_COST_RELATIVE_TOLERANCE,

diagnostic);
//or


c.notify(LevenberMarquardtResultsEnum.TOO_SMALL_PARAMETERS_RELATIVE_TOLERANCE,

diagnostic)
//etc
}

The diagnostic, when turned on, will contain a trace of the last N
iterations leading up to the failure.  When turned off, the
Diagnostic
instance only contains the parameters used to detect failure. The
diagnostic could be viewed as an indirect way to log optimizer
iterations.

WDYT?

I'm wary of having several different ways to convey information to
the
caller.

It would just be one way.

One way for optimizer, one way for solvers, one way for ...

Yes I see what you mean, but I think on a whole it will be worth it to
add additional sugar code that removes the need for exceptions.


But the caller may not be the receiver
(It could be).  The receiver would be an observer attached to the
OptimizationContext that implements an interface allowing it to
observe
the optimization.

I'm afraid that it will add to the questions of what to put in the
code and how.  [We already had sometimes heated discussions just for
the IMHO obvious (e.g. code formatting, documentation, exception...).]

Hehe.  Yes I remember some of these discussions.  I wonder how much
time was spent debating the exceptions alone?  Surely everyone must
have had this feeling in pit of their stomach that there's got to be a
better way.  On the exception topic, these are some of the issues:

I18N
===
If you are new to commons math and thinking about designing a commons
math compatible exception you should probably understand the I18N
stuff that's bound to exception (and wonder why it's bound the the
exception).

Not really true.

Well a lot of things are gray.  Personally if I'm dealing with an API, I
like to understand it, so that there are no surprises.  And I understand
that the I18N coupling might not force me to use it, but if I want to be
smart about my architecture, and simplify my design, then I should look
at it.  Maybe it is a good idea.  Maybe I should just gloss over it?  Am
I being sloppy if I just gloss over it?

Or is there an alternative that provides the same functionality, or
maybe something better, that does not come with any of these side effects?


The I18N was really simple at start.

Yup I reviewed it and thought - it's probably no big deal - but as I
started looking into reusing the CM exceptions, I decided that it 

Re: Contributor names in commit logs? (was: [jira] [Commented] (VFS-567) Timeout in vsFTPd causes exception when executing another command)

[Phil Steitz]

On 9/24/15 9:50 AM, Bernd wrote:
> Phil, Is this somewhere codified?

As I said at the end, I don't think this needs to be codified.  The
only actual requirement is CLA / software grant coverage for large
contributions.  We have agreed several times (should be in the list
archives) not to standardize commit message formats / templates.  I
rant now and then about the content of them, which I think is the
important part, but even there I don't think that "rules" are the
answer.  That a commit comes from a contributed patch can be useful
information, so I include it.  No need to over-standardize.  I just
responded with how I do it.

Phil
>
> 2015-09-24 18:47 GMT+02:00 Phil Steitz :
>> On 9/24/15 8:59 AM, Bernd wrote:
>>> Hello,
>>>
>>> do we have this rule to include the name of a patch contributor into
>>> the commit message? I havent seen that beeing used so far in
>>> commons-vfs, and the commit does contain Antonio's name in the
>>> changes.xml due-to= credits. I think a while back we had the
>>> discussion that not even this is required for smaller contributions.
>>> (but I try to do it for VFS on all changes).
>> I try to do this (though I sometimes forget and have to go back and
>> fix things later):
>>
>> Always mention contributor in the commit (basically tells where the
>> code came from, no attribution means I did it myself).
>> Always include due-to in changes.xml.
>> Add contributor to contributors section of the pom if the
>> contribution is significant.
>>
>> Very small changes may not be represented in changes.xml, but are
>> still attributed in the commit message.  To me, the commit message
>> describes the commit; the changes.xml and contributors references
>> acknowledge the contribution.
>>
>> Different components do this differently.  I don't think we have or
>> need to have rigid rules, other than that really large contributions
>> should be granted under CLA.
>>
>> Phil
>>> Gruss
>>> Bernd
>>>
>>>
>>>
>>> -- Forwarded message --
>>> From: Antonio Petrelli (JIRA) 
>>> Date: 2015-09-24 10:50 GMT+02:00
>>> Subject: [jira] [Commented] (VFS-567) Timeout in vsFTPd causes
>>> exception when executing another command
>>> To: iss...@commons.apache.org
>>>
>>>
>>>
>>> [ 
>>> https://issues.apache.org/jira/browse/VFS-567?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14906040#comment-14906040
>>> ]
>>>
>>> Antonio Petrelli commented on VFS-567:
>>> --
>>>
>>> Thanks Bernd, however, as I am an emeritus PMC member of other Apache
>>> project, I remember that the name of the contributor should be
>>> inserted in the commit log.
>>>
 Timeout in vsFTPd causes exception when executing another command
 -

 Key: VFS-567
 URL: https://issues.apache.org/jira/browse/VFS-567
 Project: Commons VFS
  Issue Type: Bug
Affects Versions: 2.1
 Environment: vsFTPd 3.0.2 on Kubuntu 14.10
Reporter: Antonio Petrelli
Assignee: Bernd Eckenfels
 Fix For: 2.1

 Attachments: commons-vfs-disconnect.diff, vfs-quit-problem.zip


 After a timeout in vsFTPd, a QUIT command is sent and a 421 Timeout 
 response is sent back to the Java client. After that, a SocketException 
 (broken pipe) is raised and not correctly managed.
 I am attaching a test case, this is the stack trace:
 Exception in thread "main" org.apache.commons.vfs2.FileSystemException: 
 Could not determine the type of file "ftps://localhost/javadev/vfs/input".
   at 
 org.apache.commons.vfs2.provider.AbstractFileObject.getType(AbstractFileObject.java:1526)
   at QuitProblemMain.main(QuitProblemMain.java:41)
 Caused by: java.net.SocketException: Pipe interrotta
   at java.net.SocketOutputStream.socketWrite0(Native Method)
   at 
 java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:109)
   at java.net.SocketOutputStream.write(SocketOutputStream.java:153)
   at sun.security.ssl.OutputRecord.writeBuffer(OutputRecord.java:431)
   at sun.security.ssl.OutputRecord.write(OutputRecord.java:417)
   at 
 sun.security.ssl.SSLSocketImpl.writeRecordInternal(SSLSocketImpl.java:864)
   at sun.security.ssl.SSLSocketImpl.writeRecord(SSLSocketImpl.java:835)
   at sun.security.ssl.AppOutputStream.write(AppOutputStream.java:123)
   at sun.nio.cs.StreamEncoder.writeBytes(StreamEncoder.java:221)
   at sun.nio.cs.StreamEncoder.implFlushBuffer(StreamEncoder.java:291)
   at sun.nio.cs.StreamEncoder.implFlush(StreamEncoder.java:295)
   at sun.nio.cs.StreamEncoder.flush(StreamEncoder.java:141)
   at java.io.OutputStreamWriter.flush(OutputStreamWriter.java:229)
   at 

Re: [Math] LeastSquaresOptimizer Design

[Gilles]

On Thu, 24 Sep 2015 08:43:38 -0500, Ole Ersoy wrote:

On 09/24/2015 06:31 AM, luc wrote:

Le 2015-09-24 04:16, Ole Ersoy a écrit :

On 09/23/2015 03:09 PM, Luc Maisonobe wrote:
CM is not intended to be a design pattern people should mimic. We 
are so bad at this it would be a shame. No one in its right mind 
would copy or reuse this stuff. It is for internal use only and we 
don't even have the resources to manage it by ourselves so we can't 
consider it as a path people should follow as we are leading them. 
Here we would be leading them directly against the wall.


Hehe - I think that's like Michael Jordan saying - "Guys, don't try 
to

be like me.  I just play a little ball.  Dunk from the free throw
line.  Six world championships, but THATs it!".  In any case, I 
really
appreciate you and Gilles taking the time to talk.  Luc (And 
possibly

Gilles) - I can actually see why you are getting a bit annoyed,
because I'm ignoring something important.

I've been doing 90% NodeJS stuff lately (Which is event loop based 
and

relies callbacks) so I forgot one very important thing that I think
you have both tried to tell me.  The exception undoes the current
callstack / breaks the current program flow, bubbling up to the
handler.  Thts a good point.

OK - So scratch the callback thinking for synchronous code.  The
Lombok stuff should still be good though and hopefully some of the
callback discussion around and asynchronous option - I hope! Geez.

What do you think about having one exception per class with an Enum
that encodes the various types of exceptional conditions that the
class can find itself in?  So in the case of
LevenbergMarquardtOptimizer there would be a:
- LevenbergMarquardtOptimizerException:
- LevenbergMarquardtOptimizerExceptionEnum

When the exception is thrown it sets the Enum indicating the root
cause.  The enum can then be used as a key to lookup the 
corresponding

message.

Any better?


Sure. I would suggest adding some parameters to help the upper level 
formatting
a meaningful message (say the number of iterations performed if you 
hit a max
iteration, so users become aware they should have set the limit 
higher). Nothing
over-engineered, a simple Object[] that can be used as last argument 
to something

like String.format() would be enough.
Brilliant - I'll setup a repository and start experimenting.  Thanks 
again,

- Ole


I don't understand what Luc proposed.
But just having "Object[]" mentioned makes me shiver... :-{

Thanks to the "ExceptionContext" it is readily possible to add as many 
"messages"
as we want to be displayed. [There is no need to ask the caller to use 
"format()"

as it is done in CM.]
And there are also a methods for setting and getting an "Object".

I'd be for using more (possibly "local") exceptions if we want to 
convey
more, and more specific, information.  This should be done with getters 
that

return typed information, not "Object"s.

Gilles


-
To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
For additional commands, e-mail: dev-h...@commons.apache.org

Re: [Math] LeastSquaresOptimizer Design

[Ole Ersoy]

On 09/24/2015 03:23 PM, Luc Maisonobe wrote:

Le 24/09/2015 21:40, Ole Ersoy a écrit :

Hi Luc,

I gave this some more thought, and I think I may have tapped out to
soon, even though you are absolutely right about what an exception does
in terms bubbling execution to a point where it stops or we handle it.

Suppose we have an Optimizer and an Optimizer observer.  The optimizer
will emit three different events given in the process of stepping
through to the max number of iterations it is allotted:
- SOLUTION_FOUND
- COULD_NOT_CONVERGE_FOR_REASON_1
- COULD_NOT_CONVERGE_FOR_REASON_2
- END (Max iterations reached)

So we have the observer interface:

interface OptimizerObserver {

 success(Solution solution)
 update(Enum enum, Optimizer optimizer)
 end(Optimizer optimizer)
}

So if the Optimizer notifies the observer of `success`, then the
observer does what it needs to with the results and moves on.  If the
observer gets an `update` notification, that means that given the
current [constraints, numbers of iterations, data] the optimizer cannot
finish.  But the update method receives the optimizer, so it can adapt
it, and tell it to continue or just trash it and try something
completely different.  If the `END` event is reached then the Optimizer
could not finish given the number of allotted iterations.  The Optimizer
is passed back via the callback interface so the observer could allow
more iterations if it wants to...perhaps based on some metric indicating
how close the optimizer is to finding a solution.

What this could do is allow the implementation of the observer to throw
the exception if 'All is lost!', in which case the Optimizer does not
need an exception.  Totally understand that this may not work
everywhere, but it seems like it could work in this case.

WDYT?

With this version, you should also pass the optimizer in case of
success. In most cases, the observer will just ignore it, but in some
cases it may try to solve another problem, or to solve again with
stricter constraints, using the previous solution as the start point
for the more stringent problem. Another case would be to go from a
simple problem to a more difficult problem using some kind of
homotopy.

Great - whoooh - glad you like this version a little better - for a sec I 
thought I had complete lost it :).  Note to seeelf ... cancel therapy with Dr. 
Phil.  BTW - Gilles - this could also be used as a light weight logger.  The 
Optimizer could publish information deemed interesting on each ITERATION event. 
 The observer could then be wired with SLF4J and perform the same type of 
logging that the Optimizer would perform.  So CM could declare SLF4J as a test 
dependency, and unit tests could log iterations using it.

Lombok also has a @SLF4J annotation that's pretty sweet.  Saves the SLF4J 
boilerplate.

Cheers,
- Ole



best regards,
Luc


Cheers,
- Ole



On 09/23/2015 03:09 PM, Luc Maisonobe wrote:

Le 23/09/2015 19:20, Ole Ersoy a écrit :

HI Luc,

Hi Ole,


On 09/23/2015 03:02 AM, luc wrote:

Hi,

Le 2015-09-22 02:55, Ole Ersoy a écrit :

Hola,

On 09/21/2015 04:15 PM, Gilles wrote:

Hi.

On Sun, 20 Sep 2015 15:04:08 -0500, Ole Ersoy wrote:

On 09/20/2015 05:51 AM, Gilles wrote:

On Sun, 20 Sep 2015 01:12:49 -0500, Ole Ersoy wrote:

Wanted to float some ideas for the LeastSquaresOptimizer (Possibly
General Optimizer) design.  For example with the
LevenbergMarquardtOptimizer we would do:
`LevenbergMarquardtOptimizer.optimize(OptimizationContext c);`

Rough optimize() outline:
public static void optimise() {
//perform the optimization
//If successful
  c.notify(LevenberMarquardtResultsEnum.SUCCESS, solution);
//If not successful


c.notify(LevenberMarquardtResultsEnum.TOO_SMALL_COST_RELATIVE_TOLERANCE,


diagnostic);
//or


c.notify(LevenberMarquardtResultsEnum.TOO_SMALL_PARAMETERS_RELATIVE_TOLERANCE,


diagnostic)
//etc
}

The diagnostic, when turned on, will contain a trace of the last N
iterations leading up to the failure.  When turned off, the
Diagnostic
instance only contains the parameters used to detect failure. The
diagnostic could be viewed as an indirect way to log optimizer
iterations.

WDYT?

I'm wary of having several different ways to convey information to
the
caller.

It would just be one way.

One way for optimizer, one way for solvers, one way for ...

Yes I see what you mean, but I think on a whole it will be worth it to
add additional sugar code that removes the need for exceptions.


But the caller may not be the receiver
(It could be).  The receiver would be an observer attached to the
OptimizationContext that implements an interface allowing it to
observe
the optimization.

I'm afraid that it will add to the questions of what to put in the
code and how.  [We already had sometimes heated discussions just for
the IMHO obvious (e.g. code formatting, documentation,
exception...).]

Hehe.  Yes I remember some of these discussions.  I wonder how much
time was spent debating the exceptions alone?  

Re: [Math] LeastSquaresOptimizer Design

[Ole Ersoy]

On 09/24/2015 04:05 PM, Gilles wrote:

On Thu, 24 Sep 2015 08:43:38 -0500, Ole Ersoy wrote:

On 09/24/2015 06:31 AM, luc wrote:

Le 2015-09-24 04:16, Ole Ersoy a écrit :

On 09/23/2015 03:09 PM, Luc Maisonobe wrote:

CM is not intended to be a design pattern people should mimic. We are so bad at 
this it would be a shame. No one in its right mind would copy or reuse this 
stuff. It is for internal use only and we don't even have the resources to 
manage it by ourselves so we can't consider it as a path people should follow 
as we are leading them. Here we would be leading them directly against the wall.


Hehe - I think that's like Michael Jordan saying - "Guys, don't try to
be like me.  I just play a little ball.  Dunk from the free throw
line.  Six world championships, but THATs it!".  In any case, I really
appreciate you and Gilles taking the time to talk.  Luc (And possibly
Gilles) - I can actually see why you are getting a bit annoyed,
because I'm ignoring something important.

I've been doing 90% NodeJS stuff lately (Which is event loop based and
relies callbacks) so I forgot one very important thing that I think
you have both tried to tell me.  The exception undoes the current
callstack / breaks the current program flow, bubbling up to the
handler.  Thts a good point.

OK - So scratch the callback thinking for synchronous code. The
Lombok stuff should still be good though and hopefully some of the
callback discussion around and asynchronous option - I hope! Geez.

What do you think about having one exception per class with an Enum
that encodes the various types of exceptional conditions that the
class can find itself in?  So in the case of
LevenbergMarquardtOptimizer there would be a:
- LevenbergMarquardtOptimizerException:
- LevenbergMarquardtOptimizerExceptionEnum

When the exception is thrown it sets the Enum indicating the root
cause.  The enum can then be used as a key to lookup the corresponding
message.

Any better?


Sure. I would suggest adding some parameters to help the upper level formatting
a meaningful message (say the number of iterations performed if you hit a max
iteration, so users become aware they should have set the limit higher). Nothing
over-engineered, a simple Object[] that can be used as last argument to 
something
like String.format() would be enough.

Brilliant - I'll setup a repository and start experimenting. Thanks again,
- Ole


I don't understand what Luc proposed.
But just having "Object[]" mentioned makes me shiver... :-{

Thanks to the "ExceptionContext" it is readily possible to add as many 
"messages"
as we want to be displayed. [There is no need to ask the caller to use 
"format()"
as it is done in CM.]
And there are also a methods for setting and getting an "Object".

I'd be for using more (possibly "local") exceptions if we want to convey
more, and more specific, information.  This should be done with getters that
return typed information, not "Object"s.


Javascripters do what Luc is advocating all the time, so I'm used to it.

If the exception is specific to the class throwing the exception then we could 
attach a reference to the instance throwing the exception and use Lombok to 
generate binary getters.

Cheers,
- Ole



Gilles


-
To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
For additional commands, e-mail: dev-h...@commons.apache.org





-
To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
For additional commands, e-mail: dev-h...@commons.apache.org

Re: [Math] LeastSquaresOptimizer Design

[Gilles]

On Thu, 24 Sep 2015 17:02:15 -0500, Ole Ersoy wrote:

On 09/24/2015 03:23 PM, Luc Maisonobe wrote:

Le 24/09/2015 21:40, Ole Ersoy a écrit :

Hi Luc,

I gave this some more thought, and I think I may have tapped out to
soon, even though you are absolutely right about what an exception 
does
in terms bubbling execution to a point where it stops or we handle 
it.


Suppose we have an Optimizer and an Optimizer observer.  The 
optimizer

will emit three different events given in the process of stepping
through to the max number of iterations it is allotted:
- SOLUTION_FOUND
- COULD_NOT_CONVERGE_FOR_REASON_1
- COULD_NOT_CONVERGE_FOR_REASON_2
- END (Max iterations reached)

So we have the observer interface:

interface OptimizerObserver {

 success(Solution solution)
 update(Enum enum, Optimizer optimizer)
 end(Optimizer optimizer)
}

So if the Optimizer notifies the observer of `success`, then the
observer does what it needs to with the results and moves on.  If 
the

observer gets an `update` notification, that means that given the
current [constraints, numbers of iterations, data] the optimizer 
cannot
finish.  But the update method receives the optimizer, so it can 
adapt

it, and tell it to continue or just trash it and try something
completely different.  If the `END` event is reached then the 
Optimizer
could not finish given the number of allotted iterations.  The 
Optimizer
is passed back via the callback interface so the observer could 
allow
more iterations if it wants to...perhaps based on some metric 
indicating

how close the optimizer is to finding a solution.

What this could do is allow the implementation of the observer to 
throw
the exception if 'All is lost!', in which case the Optimizer does 
not

need an exception.  Totally understand that this may not work
everywhere, but it seems like it could work in this case.

WDYT?

With this version, you should also pass the optimizer in case of
success. In most cases, the observer will just ignore it, but in 
some

cases it may try to solve another problem, or to solve again with
stricter constraints, using the previous solution as the start point
for the more stringent problem. Another case would be to go from a
simple problem to a more difficult problem using some kind of
homotopy.

Great - whoooh - glad you like this version a little better - for a
sec I thought I had complete lost it :).


IIUC, I don't like it: it looks like "GOTO"...


Note to seeelf ... cancel
therapy with Dr. Phil.  BTW - Gilles - this could also be used as a
light weight logger.


I don't like this either (reinventing the wheel).


The Optimizer could publish information deemed
interesting on each ITERATION event.


If we'd go for an "OptimizerObserver" that gets called at every
iteration, there shouldn't be any overlap between it and "Optimizer":
iteration limit should be dealt with by the observer, the iterative
algorithm would just run "forever" until the observer is satisfied
with the current state (solution is good enough or the allotted
resources - be they time, iterations, evaluations, ... - are
exhausted).


The observer could then be wired
with SLF4J and perform the same type of logging that the Optimizer
would perform.  So CM could declare SLF4J as a test dependency, and
unit tests could log iterations using it.


As a "user", I'm interested in how the algorithms behave on my problem,
not in the CM unit tests.

The question remains unanswered: why not use slf4j directly?


Lombok also has a @SLF4J annotation that's pretty sweet.  Saves the
SLF4J boilerplate.


I understand that using annotations can be a time-saver, but IMO not
so much for a library like CM; so in this case, the risk of depending
on another library must be weighed against the advantages.

Regards,
Gilles


Cheers,
- Ole


[...]



-
To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
For additional commands, e-mail: dev-h...@commons.apache.org

Re: [Math] LeastSquaresOptimizer Design

[Gilles]

On Thu, 24 Sep 2015 18:01:13 -0500, Ole Ersoy wrote:

On 09/24/2015 04:05 PM, Gilles wrote:

On Thu, 24 Sep 2015 08:43:38 -0500, Ole Ersoy wrote:

On 09/24/2015 06:31 AM, luc wrote:

Le 2015-09-24 04:16, Ole Ersoy a écrit :

On 09/23/2015 03:09 PM, Luc Maisonobe wrote:
CM is not intended to be a design pattern people should mimic. 
We are so bad at this it would be a shame. No one in its right 
mind would copy or reuse this stuff. It is for internal use only 
and we don't even have the resources to manage it by ourselves so 
we can't consider it as a path people should follow as we are 
leading them. Here we would be leading them directly against the 
wall.


Hehe - I think that's like Michael Jordan saying - "Guys, don't 
try to

be like me.  I just play a little ball.  Dunk from the free throw
line.  Six world championships, but THATs it!".  In any case, I 
really
appreciate you and Gilles taking the time to talk.  Luc (And 
possibly

Gilles) - I can actually see why you are getting a bit annoyed,
because I'm ignoring something important.

I've been doing 90% NodeJS stuff lately (Which is event loop 
based and
relies callbacks) so I forgot one very important thing that I 
think

you have both tried to tell me.  The exception undoes the current
callstack / breaks the current program flow, bubbling up to the
handler.  Thts a good point.

OK - So scratch the callback thinking for synchronous code. The
Lombok stuff should still be good though and hopefully some of 
the
callback discussion around and asynchronous option - I hope! 
Geez.


What do you think about having one exception per class with an 
Enum

that encodes the various types of exceptional conditions that the
class can find itself in?  So in the case of
LevenbergMarquardtOptimizer there would be a:
- LevenbergMarquardtOptimizerException:
- LevenbergMarquardtOptimizerExceptionEnum

When the exception is thrown it sets the Enum indicating the root
cause.  The enum can then be used as a key to lookup the 
corresponding

message.

Any better?


Sure. I would suggest adding some parameters to help the upper 
level formatting
a meaningful message (say the number of iterations performed if 
you hit a max
iteration, so users become aware they should have set the limit 
higher). Nothing
over-engineered, a simple Object[] that can be used as last 
argument to something

like String.format() would be enough.
Brilliant - I'll setup a repository and start experimenting. Thanks 
again,

- Ole


I don't understand what Luc proposed.
But just having "Object[]" mentioned makes me shiver... :-{

Thanks to the "ExceptionContext" it is readily possible to add as 
many "messages"
as we want to be displayed. [There is no need to ask the caller to 
use "format()"

as it is done in CM.]
And there are also a methods for setting and getting an "Object".

I'd be for using more (possibly "local") exceptions if we want to 
convey
more, and more specific, information.  This should be done with 
getters that

return typed information, not "Object"s.


Javascripters do what Luc is advocating all the time, so I'm used to 
it.


If the exception is specific to the class throwing the exception then
we could attach a reference to the instance throwing the exception 
and

use Lombok to generate binary getters.



Why should the instance throwing the exception hold a field with the
information? Separation of concerns: optimizer does the computation,
then the exception holds what's needed for a full report of the 
failure.


Or the report is done at the observer's level, based on complete
information routinely returned by the optimizer at every step (cf.
previous mail).

Gilles


Cheers,
- Ole





-
To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
For additional commands, e-mail: dev-h...@commons.apache.org

Re: [Math] LeastSquaresOptimizer Design

[Luc Maisonobe]

Le 24/09/2015 21:40, Ole Ersoy a écrit :
> Hi Luc,
> 
> I gave this some more thought, and I think I may have tapped out to
> soon, even though you are absolutely right about what an exception does
> in terms bubbling execution to a point where it stops or we handle it.
> 
> Suppose we have an Optimizer and an Optimizer observer.  The optimizer
> will emit three different events given in the process of stepping
> through to the max number of iterations it is allotted:
> - SOLUTION_FOUND
> - COULD_NOT_CONVERGE_FOR_REASON_1
> - COULD_NOT_CONVERGE_FOR_REASON_2
> - END (Max iterations reached)
> 
> So we have the observer interface:
> 
> interface OptimizerObserver {
> 
> success(Solution solution)
> update(Enum enum, Optimizer optimizer)
> end(Optimizer optimizer)
> }
> 
> So if the Optimizer notifies the observer of `success`, then the
> observer does what it needs to with the results and moves on.  If the
> observer gets an `update` notification, that means that given the
> current [constraints, numbers of iterations, data] the optimizer cannot
> finish.  But the update method receives the optimizer, so it can adapt
> it, and tell it to continue or just trash it and try something
> completely different.  If the `END` event is reached then the Optimizer
> could not finish given the number of allotted iterations.  The Optimizer
> is passed back via the callback interface so the observer could allow
> more iterations if it wants to...perhaps based on some metric indicating
> how close the optimizer is to finding a solution.
> 
> What this could do is allow the implementation of the observer to throw
> the exception if 'All is lost!', in which case the Optimizer does not
> need an exception.  Totally understand that this may not work
> everywhere, but it seems like it could work in this case.
> 
> WDYT?

With this version, you should also pass the optimizer in case of
success. In most cases, the observer will just ignore it, but in some
cases it may try to solve another problem, or to solve again with
stricter constraints, using the previous solution as the start point
for the more stringent problem. Another case would be to go from a
simple problem to a more difficult problem using some kind of
homotopy.

best regards,
Luc

> 
> Cheers,
> - Ole
> 
> 
> 
> On 09/23/2015 03:09 PM, Luc Maisonobe wrote:
>> Le 23/09/2015 19:20, Ole Ersoy a écrit :
>>> HI Luc,
>> Hi Ole,
>>
>>> On 09/23/2015 03:02 AM, luc wrote:
 Hi,

 Le 2015-09-22 02:55, Ole Ersoy a écrit :
> Hola,
>
> On 09/21/2015 04:15 PM, Gilles wrote:
>> Hi.
>>
>> On Sun, 20 Sep 2015 15:04:08 -0500, Ole Ersoy wrote:
>>> On 09/20/2015 05:51 AM, Gilles wrote:
 On Sun, 20 Sep 2015 01:12:49 -0500, Ole Ersoy wrote:
> Wanted to float some ideas for the LeastSquaresOptimizer (Possibly
> General Optimizer) design.  For example with the
> LevenbergMarquardtOptimizer we would do:
> `LevenbergMarquardtOptimizer.optimize(OptimizationContext c);`
>
> Rough optimize() outline:
> public static void optimise() {
> //perform the optimization
> //If successful
>  c.notify(LevenberMarquardtResultsEnum.SUCCESS, solution);
> //If not successful
>
>
> c.notify(LevenberMarquardtResultsEnum.TOO_SMALL_COST_RELATIVE_TOLERANCE,
>
>
> diagnostic);
> //or
>
>
> c.notify(LevenberMarquardtResultsEnum.TOO_SMALL_PARAMETERS_RELATIVE_TOLERANCE,
>
>
> diagnostic)
> //etc
> }
>
> The diagnostic, when turned on, will contain a trace of the last N
> iterations leading up to the failure.  When turned off, the
> Diagnostic
> instance only contains the parameters used to detect failure. The
> diagnostic could be viewed as an indirect way to log optimizer
> iterations.
>
> WDYT?
 I'm wary of having several different ways to convey information to
 the
 caller.
>>> It would just be one way.
>> One way for optimizer, one way for solvers, one way for ...
> Yes I see what you mean, but I think on a whole it will be worth it to
> add additional sugar code that removes the need for exceptions.
>
>>> But the caller may not be the receiver
>>> (It could be).  The receiver would be an observer attached to the
>>> OptimizationContext that implements an interface allowing it to
>>> observe
>>> the optimization.
>> I'm afraid that it will add to the questions of what to put in the
>> code and how.  [We already had sometimes heated discussions just for
>> the IMHO obvious (e.g. code formatting, documentation,
>> exception...).]
> Hehe.  Yes I remember some of these discussions.  I wonder how much
> time was spent debating the exceptions alone?  Surely everyone must
> have had this 

Re: [Math] LeastSquaresOptimizer Design

[Ole Ersoy]

Why should the instance throwing the exception hold a field with the
information?
Separation of concerns: optimizer does the computation,
then the exception holds what's needed for a full report of the failure.

I would see what makes sense on on case by case basis.  For example if the 
Observer, which is implemented by the client / person using CM, realizes that 
it can't continue it can throw an application specific exception using a set of 
Enums that are coded for the application.

if ( optimizer.yourNeverGonnaGetItEver() ) {

   throw new ApplicationSpecificException(ApplicationErrorCodes.PARTYS_OVER, 
optimizer);
}

The error code should be specific enough for the application to understand the 
the optimizer argument is the optimizer (Object type), and then it can 
construct the message from there.



Or the report is done at the observer's level, based on complete
information routinely returned by the optimizer at every step (cf.
previous mail).

I would stick with the same process in this case.

Cheers,
- Ole

-
To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
For additional commands, e-mail: dev-h...@commons.apache.org

Re: Contributor names in commit logs? (was: [jira] [Commented] (VFS-567) Timeout in vsFTPd causes exception when executing another command)

[Bernd]

Phil, Is this somewhere codified?

2015-09-24 18:47 GMT+02:00 Phil Steitz :
> On 9/24/15 8:59 AM, Bernd wrote:
>> Hello,
>>
>> do we have this rule to include the name of a patch contributor into
>> the commit message? I havent seen that beeing used so far in
>> commons-vfs, and the commit does contain Antonio's name in the
>> changes.xml due-to= credits. I think a while back we had the
>> discussion that not even this is required for smaller contributions.
>> (but I try to do it for VFS on all changes).
>
> I try to do this (though I sometimes forget and have to go back and
> fix things later):
>
> Always mention contributor in the commit (basically tells where the
> code came from, no attribution means I did it myself).
> Always include due-to in changes.xml.
> Add contributor to contributors section of the pom if the
> contribution is significant.
>
> Very small changes may not be represented in changes.xml, but are
> still attributed in the commit message.  To me, the commit message
> describes the commit; the changes.xml and contributors references
> acknowledge the contribution.
>
> Different components do this differently.  I don't think we have or
> need to have rigid rules, other than that really large contributions
> should be granted under CLA.
>
> Phil
>>
>> Gruss
>> Bernd
>>
>>
>>
>> -- Forwarded message --
>> From: Antonio Petrelli (JIRA) 
>> Date: 2015-09-24 10:50 GMT+02:00
>> Subject: [jira] [Commented] (VFS-567) Timeout in vsFTPd causes
>> exception when executing another command
>> To: iss...@commons.apache.org
>>
>>
>>
>> [ 
>> https://issues.apache.org/jira/browse/VFS-567?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14906040#comment-14906040
>> ]
>>
>> Antonio Petrelli commented on VFS-567:
>> --
>>
>> Thanks Bernd, however, as I am an emeritus PMC member of other Apache
>> project, I remember that the name of the contributor should be
>> inserted in the commit log.
>>
>>> Timeout in vsFTPd causes exception when executing another command
>>> -
>>>
>>> Key: VFS-567
>>> URL: https://issues.apache.org/jira/browse/VFS-567
>>> Project: Commons VFS
>>>  Issue Type: Bug
>>>Affects Versions: 2.1
>>> Environment: vsFTPd 3.0.2 on Kubuntu 14.10
>>>Reporter: Antonio Petrelli
>>>Assignee: Bernd Eckenfels
>>> Fix For: 2.1
>>>
>>> Attachments: commons-vfs-disconnect.diff, vfs-quit-problem.zip
>>>
>>>
>>> After a timeout in vsFTPd, a QUIT command is sent and a 421 Timeout 
>>> response is sent back to the Java client. After that, a SocketException 
>>> (broken pipe) is raised and not correctly managed.
>>> I am attaching a test case, this is the stack trace:
>>> Exception in thread "main" org.apache.commons.vfs2.FileSystemException: 
>>> Could not determine the type of file "ftps://localhost/javadev/vfs/input".
>>>   at 
>>> org.apache.commons.vfs2.provider.AbstractFileObject.getType(AbstractFileObject.java:1526)
>>>   at QuitProblemMain.main(QuitProblemMain.java:41)
>>> Caused by: java.net.SocketException: Pipe interrotta
>>>   at java.net.SocketOutputStream.socketWrite0(Native Method)
>>>   at 
>>> java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:109)
>>>   at java.net.SocketOutputStream.write(SocketOutputStream.java:153)
>>>   at sun.security.ssl.OutputRecord.writeBuffer(OutputRecord.java:431)
>>>   at sun.security.ssl.OutputRecord.write(OutputRecord.java:417)
>>>   at 
>>> sun.security.ssl.SSLSocketImpl.writeRecordInternal(SSLSocketImpl.java:864)
>>>   at sun.security.ssl.SSLSocketImpl.writeRecord(SSLSocketImpl.java:835)
>>>   at sun.security.ssl.AppOutputStream.write(AppOutputStream.java:123)
>>>   at sun.nio.cs.StreamEncoder.writeBytes(StreamEncoder.java:221)
>>>   at sun.nio.cs.StreamEncoder.implFlushBuffer(StreamEncoder.java:291)
>>>   at sun.nio.cs.StreamEncoder.implFlush(StreamEncoder.java:295)
>>>   at sun.nio.cs.StreamEncoder.flush(StreamEncoder.java:141)
>>>   at java.io.OutputStreamWriter.flush(OutputStreamWriter.java:229)
>>>   at java.io.BufferedWriter.flush(BufferedWriter.java:254)
>>>   at org.apache.commons.net.ftp.FTP.__send(FTP.java:505)
>>>   at org.apache.commons.net.ftp.FTP.sendCommand(FTP.java:479)
>>>   at 
>>> org.apache.commons.net.ftp.FTPSClient.sendCommand(FTPSClient.java:541)
>>>   at org.apache.commons.net.ftp.FTP.sendCommand(FTP.java:608)
>>>   at org.apache.commons.net.ftp.FTP.sendCommand(FTP.java:582)
>>>   at org.apache.commons.net.ftp.FTP.quit(FTP.java:864)
>>>   at 
>>> org.apache.commons.vfs2.provider.ftp.FTPClientWrapper.disconnect(FTPClientWrapper.java:118)
>>>   at 
>>> org.apache.commons.vfs2.provider.ftp.FTPClientWrapper.listFiles(FTPClientWrapper.java:152)
>>>  

Re: [Math] LeastSquaresOptimizer Design

[Ole Ersoy]

On 09/24/2015 06:01 PM, Gilles wrote:

On Thu, 24 Sep 2015 17:02:15 -0500, Ole Ersoy wrote:

On 09/24/2015 03:23 PM, Luc Maisonobe wrote:

Le 24/09/2015 21:40, Ole Ersoy a écrit :

Hi Luc,

I gave this some more thought, and I think I may have tapped out to
soon, even though you are absolutely right about what an exception does
in terms bubbling execution to a point where it stops or we handle it.

Suppose we have an Optimizer and an Optimizer observer.  The optimizer
will emit three different events given in the process of stepping
through to the max number of iterations it is allotted:
- SOLUTION_FOUND
- COULD_NOT_CONVERGE_FOR_REASON_1
- COULD_NOT_CONVERGE_FOR_REASON_2
- END (Max iterations reached)

So we have the observer interface:

interface OptimizerObserver {

 success(Solution solution)
 update(Enum enum, Optimizer optimizer)
 end(Optimizer optimizer)
}

So if the Optimizer notifies the observer of `success`, then the
observer does what it needs to with the results and moves on.  If the
observer gets an `update` notification, that means that given the
current [constraints, numbers of iterations, data] the optimizer cannot
finish.  But the update method receives the optimizer, so it can adapt
it, and tell it to continue or just trash it and try something
completely different.  If the `END` event is reached then the Optimizer
could not finish given the number of allotted iterations. The Optimizer
is passed back via the callback interface so the observer could allow
more iterations if it wants to...perhaps based on some metric indicating
how close the optimizer is to finding a solution.

What this could do is allow the implementation of the observer to throw
the exception if 'All is lost!', in which case the Optimizer does not
need an exception.  Totally understand that this may not work
everywhere, but it seems like it could work in this case.

WDYT?

With this version, you should also pass the optimizer in case of
success. In most cases, the observer will just ignore it, but in some
cases it may try to solve another problem, or to solve again with
stricter constraints, using the previous solution as the start point
for the more stringent problem. Another case would be to go from a
simple problem to a more difficult problem using some kind of
homotopy.

Great - whoooh - glad you like this version a little better - for a
sec I thought I had complete lost it :).


IIUC, I don't like it: it looks like "GOTO"...


Inside the optimizer it would work like this:

while (!done) {
   if (can't converge) {
   observer.update(Enum.CANT_CONVERGE, this);
   }
}

Then in the update method either modify the optimizer's parameters or throw an 
exception.





Note to seeelf ... cancel
therapy with Dr. Phil.  BTW - Gilles - this could also be used as a
light weight logger.


I don't like this either (reinventing the wheel).


You still want me to go and see Dr. Phil? :)





The Optimizer could publish information deemed
interesting on each ITERATION event.


If we'd go for an "OptimizerObserver" that gets called at every
iteration,
there shouldn't be any overlap between it and "Optimizer":

So inside the Optimizer we could have:

while (!done) {
...
if (observer.notifyOnIncrement())
{
observer.increment(this);
}
}

Which would give us an opportunity to cancel the run if, for example, it's not 
converging fast enough.  In that case we set done to true in the observer, and 
then allow the Optimizer to get to the point where it checks if it's done, 
calls the END notification on the observer, and then the observer takes it from 
there.



iteration limit should be dealt with by the observer, the iterative
algorithm would just run "forever" until the observer is satisfied
with the current state (solution is good enough or the allotted
resources - be they time, iterations, evaluations, ... - are
exhausted).


It's possible to do it that way, although I think it's better if that code 
stays on the algorithm such that the Observer interface (The client / person 
using CM implements the Observer) is as simple as possible to implement.


The observer could then be wired
with SLF4J and perform the same type of logging that the Optimizer
would perform.  So CM could declare SLF4J as a test dependency, and
unit tests could log iterations using it.


As a "user", I'm interested in how the algorithms behave on my problem,
not in the CM unit tests.

You could still do that.  I usually take my problem, simplify it down to a data 
set that I think covers all corner cases, and then run it through my unit tests 
while looking at the logging output to get an idea of how my algorithm is 
behaving.



The question remains unanswered: why not use slf4j directly?


FWIU class path dependency conflicts for SLF4J are easily solved by excluding 
logging dependencies that other libraries bring in and then directly depending 
on the logging implementation that you want to use.  So people do run into