Re: [Math] Mismatch: doc/release/release.howto.txt

[Evan Ward]

Done.

On 04/25/2016 09:05 AM, Gilles wrote:
> On Mon, 25 Apr 2016 08:54:53 -0400, Evan Ward wrote:
>> 61c53a4 contains my updates after releasing 3.6.1
>
> Could you please make sure that the most up-to-date procedure
> is in the "develop" branch?
>
> Thanks,
> Gilles
>
>> Best Regards,
>> Evan
>>
>> On 04/22/2016 08:30 PM, Gilles wrote:
>>> Hi.
>>>
>>> Contents of
>>>   doc/release/release.howto.txt
>>> is different in branches "develop" and "MATH_3_X".
>>>
>>> Which one is correct?
>>>
>>> Regards,
>>> Gilles
>
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
> For additional commands, e-mail: dev-h...@commons.apache.org
>


-
To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
For additional commands, e-mail: dev-h...@commons.apache.org

Re: [Math] Mismatch: doc/release/release.howto.txt

[Evan Ward]

61c53a4 contains my updates after releasing 3.6.1

Best Regards,
Evan

On 04/22/2016 08:30 PM, Gilles wrote:
> Hi.
>
> Contents of
>   doc/release/release.howto.txt
> is different in branches "develop" and "MATH_3_X".
>
> Which one is correct?
>
> Regards,
> Gilles
>
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
> For additional commands, e-mail: dev-h...@commons.apache.org
>


-
To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
For additional commands, e-mail: dev-h...@commons.apache.org

Re: [Math] RNG refactoring

[Evan Ward]

I'm not sure if this is the problem, but a good rule of thumb is that if
you have pushed a commit you should merge it instead of rebasing it. It
looks to me like 6ddf476 and ce8c82f are the same, so I think when you
ran rebase it put it on top of the bug fix I pushed up recently,
duplicating the commit.

Best Regards,
Evan

On 03/25/2016 11:34 AM, Gilles wrote:
> Hi.
>
> Last week, and just now, I've pushed local branches that handle
> the following issues (and others, either related, or set to
> "Won't fix [in current code]" in JIRA[1]):
>
>  https://issues.apache.org/jira/browse/MATH-1335
>  https://issues.apache.org/jira/browse/MATH-1336
>  https://issues.apache.org/jira/browse/MATH-1337
>  https://issues.apache.org/jira/browse/MATH-1339
>  https://issues.apache.org/jira/browse/MATH-1158
>  https://issues.apache.org/jira/browse/MATH-1338
>
> [I've just seen that for branch "feature-MATH-1158" which is
> "git rebase"d on "feature-MATH-1335", the push is recreating
> all the MATH-1335 commits (as guessed from the flood of emails).
> Something I was not expecting: sorry I misunderstood how this
> is supposed to work...]
>
>
> Regards,
> Gilles
>
> [1] See the "links" in the relevant JIRA reports.
>
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
> For additional commands, e-mail: dev-h...@commons.apache.org
>


-
To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
For additional commands, e-mail: dev-h...@commons.apache.org

[Announce] [math] Apache Commons Math 3.6.1 Released

[Evan Ward]

The Apache Commons Team is pleased to announce the availability of:

Apache Commons Math 3.6.1

Apache Commons Math is a library of lightweight, self-contained
mathematics and statistics components addressing the most common
problems not available in the Java programming language or Commons Lang.

The release notes can be reviewed at:
http://www.apache.org/dist/commons/math/RELEASE-NOTES.txt

Distribution packages can be downloaded from:
https://commons.apache.org/proper/commons-math/download_math.cgi

When downloading, please verify signatures using the KEYS file available at:
http://www.apache.org/dist/commons

Maven artifacts are also available in the central Maven repository:

http://repo1.maven.org/maven2/org/apache/commons/

The Apache Commons Team

-
To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
For additional commands, e-mail: dev-h...@commons.apache.org

Re: [math] Staging site

[Evan Ward]

Thanks Thomas for the help. I think I have updated the site correctly now.

Gilles, I've been updating the "howto" as I go through the process and
I'll commit the updated version after I send out the announce emails. Up
until the site it has been mostly small changes.

Thanks,
Evan

On 03/22/2016 09:59 AM, Gilles wrote:
> On Mon, 21 Mar 2016 20:14:41 +0100, Thomas Neidhart wrote:
>> On 03/21/2016 07:32 PM, Evan Ward wrote:
>>> Hi,
>>>
>>> I'm on step 19 in Release.howto.txt, I've committed the updated site
>>> using svn, but I get a 404 when I try to view it at
>>> http://commons.staging.apache.org/proper/commons-math
>>>
>>> Any ideas what I'm doing wrong? I've noticed that the staging URL for
>>> other components such as [lang] or [io] also result in a 404. Has the
>>> URL changed?
>>
>> you do not need to copy to the staging site, the howto is wrong there.
>>
>> Once the commit to the site-content repository is done, the site is
>> immediately synced and visible at http://commons.apache.org/math. You
>> might need to refresh your browser though.
>>
>> btw. running mvn site-deploy should do the same as described in the
>> howto.
>
> Evan,
>
> Please update the "howto" to reflect what you did that actually worked.
>
> Thanks,
> Gilles
>
>>
>> Thomas
>>
>
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
> For additional commands, e-mail: dev-h...@commons.apache.org
>


-
To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
For additional commands, e-mail: dev-h...@commons.apache.org

[math] Staging site

[Evan Ward]

Hi,

I'm on step 19 in Release.howto.txt, I've committed the updated site
using svn, but I get a 404 when I try to view it at
http://commons.staging.apache.org/proper/commons-math

Any ideas what I'm doing wrong? I've noticed that the staging URL for
other components such as [lang] or [io] also result in a 404. Has the
URL changed?

Best Regards,
Evan




-
To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
For additional commands, e-mail: dev-h...@commons.apache.org

Re: [math] Fwd: Please add your release data for 'commons'

[Evan Ward]

Thanks Luc!

On 03/21/2016 11:48 AM, Luc Maisonobe wrote:
> Le 21/03/2016 13:54, Evan Ward a écrit :
>> Hi,
>>
>> Can a Math PMC member add our release data to the database? I don't have
>> the necessary permissions.
> Now that we have 3 PMC, I have updated the database.
> You can continue with the process.
>
> best regards,
> Luc
>
>> Thanks,
>> Evan
>>
>>
>>  Forwarded Message 
>> Subject: Please add your release data for 'commons'
>> Date:Mon, 21 Mar 2016 12:48:51 +
>> From:Apache Reporter Service <no-re...@reporter.apache.org>
>> Reply-To:d...@community.apache.org
>> To:  evanward <evanw...@apache.org>
>>
>>
>>
>> Hi,
>> This is an automated email from reporter.apache.org.
>> I see that you just pushed something to our release repository for the 
>> 'commons' project
>> in the following commit:
>>
>> r12800 at 2016-03-21 12:48:37 + (Mon, 21 Mar 2016)
>> Publish commons-math 3.6.1 Release
>>
>> If you are a PMC member of this project, we ask that you log on to:
>> https://reporter.apache.org/addrelease.html?commons
>> and add your release data (version and date) to the database.
>>
>> If you are not a PMC member, please have a PMC member add this information.
>>
>> While this is not a requirement, we ask that you still add this data to the
>> reporter database, so that people using the Apache Reporter Service will be
>> able to see the latest release data for this project.
>>
>> With regards,
>> The Apache Reporter Service.
>> 
>>
>>
>>
>>
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
> For additional commands, e-mail: dev-h...@commons.apache.org
>


-
To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
For additional commands, e-mail: dev-h...@commons.apache.org

Re: [VOTE][RRESULT][RC1] Release Commons Math 3.6.1

[Evan Ward]

On 03/21/2016 09:18 AM, Luc Maisonobe wrote:
> Le 21/03/2016 13:44, Evan Ward a écrit :
>> The vote passes with three votes in favor and none against.
> Well, in fact I am not sure it passes yet ...
>
> As far as I know, there are only 2 PMC members who voted: Oliver and myself.

Oh, I didn't realize there had to be 3 PMC members voting. I had already
released the maven artifacts by the time I receive your message. I'll
wait to publish the site until we have a vote from a third PMC member.
Hopefully it is a +1 so I don't have to figure out how to un-release
something. I'll update the release.howto.txt document to include a note
that a vote must have 3 PMC +1's to pass.

Best Regards,
Evan

>
> Could a third PMC member look at it?
>
> best regards,
> Luc
>
>> Thanks for taking time to review.
>>
>> Best Regards,
>> Evan
>>
>> On 03/19/2016 11:06 PM, Matt Sicker wrote:
>>> If you add the -Xdoclint:none command line argument, that will ignore the
>>> javadoc errors. I love how the JDK itself doesn't compile without that
>>> option.
>>>
>>> On 19 March 2016 at 15:38, Oliver Heger <oliver.he...@oliver-heger.de>
>>> wrote:
>>>
>>>> Build worked fine with Java 1.6 on Windows 10. Artifacts and site look
>>>> good. I could not build the site with Java 8 because of Javadoc errors,
>>>> but this is not blocking.
>>>>
>>>> So +1
>>>>
>>>> Oliver
>>>>
>>>> Am 17.03.2016 um 20:06 schrieb Evan Ward:
>>>>> This is a [VOTE] for releasing Apache Commons Math 3.6.1 from release
>>>>> candidate 1.
>>>>>
>>>>> Tag name:
>>>>>   MATH_3_6_1_RC1 (signature can be checked from git using 'git tag -v')
>>>>>
>>>>> Tag URL:
>>>>>
>>>>>
>>>> https://git-wip-us.apache.org/repos/asf?p=commons-math.git;a=tag;h=e40b144832c0e138b185c92326588a087e00909c
>>>>> Commit ID the tag points at:
>>>>>   16abfe5de688cc52fb0396e0609cb33044b15653
>>>>>
>>>>> Site:
>>>>>   http://home.apache.org/~evanward/commons-math-3.6.1-RC1-site
>>>>>
>>>>> Distribution files:
>>>>>   https://dist.apache.org/repos/dist/dev/commons/math/
>>>>>
>>>>> Distribution files hashes (SHA1):
>>>>>   4d4bb6741f9b5d095bcab24f5232a871ba579df0
>>>> commons-math3-3.6.1-bin.tar.gz
>>>>>   e088b160c19af7ca2596d91430b8a71aaa5ea5da  commons-math3-3.6.1-bin.zip
>>>>>   77ffe792e4bf0d4bcd7b4e2a8ce3b07df40a92b9
>>>> commons-math3-3.6.1-src.tar.gz
>>>>>   cadad1cbb7757b2616a96b20d357e3a6acb1b4c9  commons-math3-3.6.1-src.zip
>>>>>
>>>>> KEYS file to check signatures:
>>>>>   http://www.apache.org/dist/commons/KEYS
>>>>>
>>>>> Maven artifacts:
>>>>>
>>>>>
>>>> https://repository.apache.org/content/repositories/orgapachecommons-1142/org/apache/commons/commons-math3/3.6.1/
>>>>> [ ] +1 Release it.
>>>>> [ ] +0 Go ahead; I don't care.
>>>>> [ ] -0 There are a few minor glitches: ...
>>>>> [ ] -1 No, do not release it because ...
>>>>>
>>>>> This vote will close in 72 hours, at 2016-03-20T12:00:00Z (this is UTC
>>>>> time).
>>>>>
>>>>> Best Regards,
>>>>> Evan
>>>>>
>>>> -
>>>> To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
>>>> For additional commands, e-mail: dev-h...@commons.apache.org
>>>>
>>>>
>>
>> -
>> To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
>> For additional commands, e-mail: dev-h...@commons.apache.org
>>
>>
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
> For additional commands, e-mail: dev-h...@commons.apache.org
>


-
To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
For additional commands, e-mail: dev-h...@commons.apache.org

[math] Fwd: Please add your release data for 'commons'

[Evan Ward]

Hi,

Can a Math PMC member add our release data to the database? I don't have
the necessary permissions.

Thanks,
Evan


 Forwarded Message 
Subject:Please add your release data for 'commons'
Date:   Mon, 21 Mar 2016 12:48:51 +
From:   Apache Reporter Service 
Reply-To:   d...@community.apache.org
To: evanward 



Hi,
This is an automated email from reporter.apache.org.
I see that you just pushed something to our release repository for the 
'commons' project
in the following commit:

r12800 at 2016-03-21 12:48:37 + (Mon, 21 Mar 2016)
Publish commons-math 3.6.1 Release

If you are a PMC member of this project, we ask that you log on to:
https://reporter.apache.org/addrelease.html?commons
and add your release data (version and date) to the database.

If you are not a PMC member, please have a PMC member add this information.

While this is not a requirement, we ask that you still add this data to the
reporter database, so that people using the Apache Reporter Service will be
able to see the latest release data for this project.

With regards,
The Apache Reporter Service.

[VOTE][RRESULT][RC1] Release Commons Math 3.6.1

[Evan Ward]

The vote passes with three votes in favor and none against.

Thanks for taking time to review.

Best Regards,
Evan

On 03/19/2016 11:06 PM, Matt Sicker wrote:
> If you add the -Xdoclint:none command line argument, that will ignore the
> javadoc errors. I love how the JDK itself doesn't compile without that
> option.
>
> On 19 March 2016 at 15:38, Oliver Heger <oliver.he...@oliver-heger.de>
> wrote:
>
>> Build worked fine with Java 1.6 on Windows 10. Artifacts and site look
>> good. I could not build the site with Java 8 because of Javadoc errors,
>> but this is not blocking.
>>
>> So +1
>>
>> Oliver
>>
>> Am 17.03.2016 um 20:06 schrieb Evan Ward:
>>> This is a [VOTE] for releasing Apache Commons Math 3.6.1 from release
>>> candidate 1.
>>>
>>> Tag name:
>>>   MATH_3_6_1_RC1 (signature can be checked from git using 'git tag -v')
>>>
>>> Tag URL:
>>>
>>>
>> https://git-wip-us.apache.org/repos/asf?p=commons-math.git;a=tag;h=e40b144832c0e138b185c92326588a087e00909c
>>> Commit ID the tag points at:
>>>   16abfe5de688cc52fb0396e0609cb33044b15653
>>>
>>> Site:
>>>   http://home.apache.org/~evanward/commons-math-3.6.1-RC1-site
>>>
>>> Distribution files:
>>>   https://dist.apache.org/repos/dist/dev/commons/math/
>>>
>>> Distribution files hashes (SHA1):
>>>   4d4bb6741f9b5d095bcab24f5232a871ba579df0
>> commons-math3-3.6.1-bin.tar.gz
>>>   e088b160c19af7ca2596d91430b8a71aaa5ea5da  commons-math3-3.6.1-bin.zip
>>>   77ffe792e4bf0d4bcd7b4e2a8ce3b07df40a92b9
>> commons-math3-3.6.1-src.tar.gz
>>>   cadad1cbb7757b2616a96b20d357e3a6acb1b4c9  commons-math3-3.6.1-src.zip
>>>
>>> KEYS file to check signatures:
>>>   http://www.apache.org/dist/commons/KEYS
>>>
>>> Maven artifacts:
>>>
>>>
>> https://repository.apache.org/content/repositories/orgapachecommons-1142/org/apache/commons/commons-math3/3.6.1/
>>> [ ] +1 Release it.
>>> [ ] +0 Go ahead; I don't care.
>>> [ ] -0 There are a few minor glitches: ...
>>> [ ] -1 No, do not release it because ...
>>>
>>> This vote will close in 72 hours, at 2016-03-20T12:00:00Z (this is UTC
>>> time).
>>>
>>> Best Regards,
>>> Evan
>>>
>> -
>> To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
>> For additional commands, e-mail: dev-h...@commons.apache.org
>>
>>
>


-
To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
For additional commands, e-mail: dev-h...@commons.apache.org

Re: [VOTE][RC1] Release Commons Math 3.6.1

[Evan Ward]

On 03/17/2016 03:06 PM, Evan Ward wrote:
>
>
>
>
>
>
>
> This is a [VOTE] for releasing Apache Commons Math 3.6.1 from
> release candidate 1.
>
>
>
> Tag name:
>
>   MATH_3_6_1_RC1 (signature can be checked from git using 'git tag
> -v')
>
>
>
> Tag URL:
>
>  
> https://git-wip-us.apache.org/repos/asf?p=commons-math.git;a=tag;h=e40b144832c0e138b185c92326588a087e00909c
>
>
>
> Commit ID the tag points at:
>
>   16abfe5de688cc52fb0396e0609cb33044b15653
>
>
>
> Site:
>
>   http://home.apache.org/~evanward/commons-math-3.6.1-RC1-site
>
>
>
> Distribution files:
>
>   https://dist.apache.org/repos/dist/dev/commons/math/
>
>
>
> Distribution files hashes (SHA1):
>
>   4d4bb6741f9b5d095bcab24f5232a871ba579df0 
> commons-math3-3.6.1-bin.tar.gz
>
>   e088b160c19af7ca2596d91430b8a71aaa5ea5da 
> commons-math3-3.6.1-bin.zip
>
>   77ffe792e4bf0d4bcd7b4e2a8ce3b07df40a92b9 
> commons-math3-3.6.1-src.tar.gz
>
>   cadad1cbb7757b2616a96b20d357e3a6acb1b4c9 
> commons-math3-3.6.1-src.zip
>
>
>
> KEYS file to check signatures:
>
>   http://www.apache.org/dist/commons/KEYS
>
>
>
> Maven artifacts:
>
>  
> https://repository.apache.org/content/repositories/orgapachecommons-1142/org/apache/commons/commons-math3/3.6.1/
>
>
>
> [ ] +1 Release it.

+1 from me.

Regards,
Evan

>
> [ ] +0 Go ahead; I don't care.
>
> [ ] -0 There are a few minor glitches: ...
>
> [ ] -1 No, do not release it because ...
>
>
>
> This vote will close in 72 hours, at 2016-03-20T12:00:00Z (this is
> UTC
>
> time).
>
>
>
> Best Regards,
>
> Evan
>
>
>
>
>
>

Re: [math] Fix for MATH-1342

[Evan Ward]

On 03/16/2016 05:18 PM, luc wrote:
> Le 2016-03-16 20:13, Evan Ward a écrit :
>> Hi,
>
> Hi Evan,
>
>>
>> First, I'm trying to use the new branching strategy so I would
>> appreciate your patience if I'm not doing correctly.
>>
>> I've commited a fix for MATH-1342 in commit 7a8dc00. This is a problem
>> with event detection in all ODE integrators. So if the list approves I
>> would like to merge this into the develop branch.
>
> I have reviewed your fix and agrees with it (you know how this part of
> the code is important to me!).
>
> However, you have put the fix on a branch related to master, not to
> MATH_3_X.
>
>>
>> I would also like to create a bug fix release on the math3 branch for
>> this bug.
>
> +1 from me, but the fix should be ported to MATH_3_X in this case.
> As it would be a relase fixing only 3 bugs (counting MATH-1316 and
> MATH-1317 fixed two months ago just after 3.6 release, it should
> probably be numbered 3.6.1.
>
>> I have a commit that I've ported to that branch that I can
>> push up, if that is the right way to do it. I can also volunteer to do
>> the release, but I've never done a release before so someone may have to
>> help me through it.
>
> I have performed the last releases. The main document is the one in
> the [math] project itself, in file doc/release/release.howto.txt. I
> think it has been updated for the last two releases, so it should be
> fairly accurate by now.
>
> Prior to the release, you should upload a GPG key in the general file
> holding all release managers keys for commons, see step 10 of the
> procedure, that in fact should rather be step 0.

Thanks for the pointers. I've started going through the procedure now
and I hope to have an RC out by 4pm EDT. Twenty four steps seems a bit
much, but I do appreciate the detail. Thanks again for helping while
you're at a conference.

Regards,
Evan

>
> best regards,
> Luc
>
>>
>> Best Regards,
>> Evan
>>
>>
>> -
>> To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
>> For additional commands, e-mail: dev-h...@commons.apache.org
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
> For additional commands, e-mail: dev-h...@commons.apache.org
>


-
To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
For additional commands, e-mail: dev-h...@commons.apache.org

[math] Fix for MATH-1342

[Evan Ward]

Hi,

First, I'm trying to use the new branching strategy so I would
appreciate your patience if I'm not doing correctly.

I've commited a fix for MATH-1342 in commit 7a8dc00. This is a problem
with event detection in all ODE integrators. So if the list approves I
would like to merge this into the develop branch.

I would also like to create a bug fix release on the math3 branch for
this bug. I have a commit that I've ported to that branch that I can
push up, if that is the right way to do it. I can also volunteer to do
the release, but I've never done a release before so someone may have to
help me through it.

Best Regards,
Evan


-
To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
For additional commands, e-mail: dev-h...@commons.apache.org

[VOTE][RC1] Release Commons Math 3.6.1

[Evan Ward]

This is a [VOTE] for releasing Apache Commons Math 3.6.1 from release
candidate 1.

Tag name:
  MATH_3_6_1_RC1 (signature can be checked from git using 'git tag -v')

Tag URL:
 
https://git-wip-us.apache.org/repos/asf?p=commons-math.git;a=tag;h=e40b144832c0e138b185c92326588a087e00909c

Commit ID the tag points at:
  16abfe5de688cc52fb0396e0609cb33044b15653

Site:
  http://home.apache.org/~evanward/commons-math-3.6.1-RC1-site

Distribution files:
  https://dist.apache.org/repos/dist/dev/commons/math/

Distribution files hashes (SHA1):
  4d4bb6741f9b5d095bcab24f5232a871ba579df0  commons-math3-3.6.1-bin.tar.gz
  e088b160c19af7ca2596d91430b8a71aaa5ea5da  commons-math3-3.6.1-bin.zip
  77ffe792e4bf0d4bcd7b4e2a8ce3b07df40a92b9  commons-math3-3.6.1-src.tar.gz
  cadad1cbb7757b2616a96b20d357e3a6acb1b4c9  commons-math3-3.6.1-src.zip

KEYS file to check signatures:
  http://www.apache.org/dist/commons/KEYS

Maven artifacts:
 
https://repository.apache.org/content/repositories/orgapachecommons-1142/org/apache/commons/commons-math3/3.6.1/

[ ] +1 Release it.
[ ] +0 Go ahead; I don't care.
[ ] -0 There are a few minor glitches: ...
[ ] -1 No, do not release it because ...

This vote will close in 72 hours, at 2016-03-20T12:00:00Z (this is UTC
time).

Best Regards,
Evan

Re: [math] ConvergenceChecker

[Evan Ward]

Hi Ole,

On 02/05/2016 06:40 PM, Ole Ersoy wrote:
>
>
> On 02/05/2016 04:42 PM, Evan Ward wrote:
>> Yes, I use it. In some cases it is useful to watch the RMS residuals
> So if it were modularized and supported logging then this might
> satisfy the same requirement?

I'm not sure if I understand what you mean by logging, but I'm not
trying to put the values in a file, I'm making application flow
decisions based the values.

>> , in
>> other cases to watch the change in the states. I think it is there from
>> an acknowledgement that we can't enumerate all possible convergence
>> criteria,
> Has there ever been a case where the 'standard' convergence approach
> has been insufficient?

I think this depends on what is included in the standard convergence
checker. I think 90% of uses could be handled by watching the change in
cost or state. I like the option of specifying my own condition, so I
can control exactly when the algorithm stops.

>
> Also could you please look at this:
>
> public static LeastSquaresProblem countEvaluations(final
> LeastSquaresProblem problem,
>final
> Incrementor counter) {
> return new LeastSquaresAdapter(problem) {
>
> /** {@inheritDoc} */
> @Override
> public Evaluation evaluate(final RealVector point) {
> counter.incrementCount();
> return super.evaluate(point);
> }
>
> // Delegate the rest.
> };
> }
>
> Should this exist?

Looks useful for counting evaluations, but I think all of the LS
optimizers already do this. Anyone have a use case for countEvaluations?

>
> Thanks,
> Ole
>
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
> For additional commands, e-mail: dev-h...@commons.apache.org
>




smime.p7s
Description: S/MIME Cryptographic Signature

Re: [math] ConvergenceChecker

[Evan Ward]

Yes, I use it. In some cases it is useful to watch the RMS residuals, in
other cases to watch the change in the states. I think it is there from
an acknowledgement that we can't enumerate all possible convergence
criteria, so we provide a way for the user to define their own.

Regards,
Evan

On 02/05/2016 05:33 PM, Ole Ersoy wrote:
> Hi,
>
> The LeastSquaresProblem supports dropping in a custom
> ConvergenceChecker checker.  Does anyone do this? Can you
> help me better understand the value of it?
>
> TIA,
> Ole
>
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
> For additional commands, e-mail: dev-h...@commons.apache.org
>



-
To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
For additional commands, e-mail: dev-h...@commons.apache.org

Re: [math] [POLL] new TLP name

[Evan Ward]

I would prefer Apache Epsilon or Apache Math.

-Evan

On 02/01/2016 12:06 PM, Phil Steitz wrote:
> Please select your top choice among the following suggested names
> for the new [math]-based TLP.  All are welcome and encouraged to
> respond.  This POLL will be open for 72 hours, at which time two
> tallies will be presented:  one among those who have volunteered for
> the new PMC and a second among all respondents.  Hopefully, one name
> will emerge as consensus winner.  If not, I will kick off another
> runoff poll among the top choices.   Please respond with your top
> choice for the name.
>
> AjaMa
> Epsilon
> Erdos
> Euclid
> Euler
> Gauss
> JAML
> Math
> MathBlocks
> MathComponents (or Math Components)
> Mathelactica
> MathModules
> Megginson
> modMath
> Nash
> Newton
> Pythagoras
>
>
>
>
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
> For additional commands, e-mail: dev-h...@commons.apache.org
>


-
To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
For additional commands, e-mail: dev-h...@commons.apache.org

Re: [Math] Java version

[Evan Ward]

From an API perspective we can design a functional programming API in
Java 7, it will just be more verbose than in Java 8. One unique feature
that Java 8 does bring is multiple inheritance. Now that interfaces can
have method implementations classes can inherit methods from multiple
super classes. At this point I'm not sure how we would use this feature
as API designers, but it is another tool in the tool box.

I think 7 or 8 would be a good choice.

Regards,
Evan

On 01/14/2015 11:20 PM, Silviu Burcea wrote:
 I think Rebel Labs or Plumbr have some metrics about JDK usage.

 On Wed, Jan 14, 2015 at 10:21 PM, Hank Grabowski h...@applieddefense.com
 wrote:

 Java 8 has only been out for less than a year.  There is still a sizable
 percentage of groups that have not converted up to Java 8 for myriad
 reasons.  While I was surprised that we are requiring backwards
 compatibility with the ten year old Java 5 I think jumping all the way to
 requiring Java 8 may be a bit too much of a stretch.  I would vote for a
 minimum required version of Java 7 with the ability to run in Java 8.  I
 wish I could find metrics to quantify the penetration of each of the JDKs,
 but my gut says Java 7 would a reasonable cutoff.

 On Tue, Jan 13, 2015 at 8:31 PM, Gilles gil...@harfang.homelinux.org
 wrote:

 Raising this issue once again.
 Are we going to upgrade the requirement for the next major release?

  [ ] Java 5
  [ ] Java 6
  [ ] Java 7
  [ ] Java 8
  [ ] Java 9

 Counts up to now:

 Java 7  - 2
 Java 7 or 8 - 2
 Java 8  - 2

 Any more opionions?

 Gilles


 -
 To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
 For additional commands, e-mail: dev-h...@commons.apache.org







-
To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
For additional commands, e-mail: dev-h...@commons.apache.org

[math] Least Squares Outlier Rejection

[Evan Ward]

Hi,

A while ago I had bought up the idea of adding residual editing (aka data
editing, outlier rejection, robust regression) to our non-linear least
squares implementations.[1] As the name suggests, the idea is to de-weight
observations that don't match the user's model. There are several ways to
this including choosing a fixed cutoff, a fixed standard deviation cutoff,
or reducing a residual's weight based on its magnitude.[2]

However we add the data editing feature I think it will cause backward
incompatibilities with the released API. I've outlined below the two
options I see. I'm open to other ideas as well.

1. Replace edited residuals with 0's in the residual vector and Jacobian
(i.e. apply a 0 weight). This has the advantage of being simple to
implement and that our existing optimizers are already able to handle it.
The downside is evident when the user tries to obtain the number of
residuals that were edited. It is hard to tell the difference between an
edited residual, an apriori zero weight, or a model evaluation where the
residual and gradient is, in fact, zero. We can provide easy access to the
number of edited residuals by adding a method to the Evaluation interface.
(This is what I implemented in the patch in the original thread.) Now that
the code has been released though, this would cause a backward
incompatibility for some advanced users. Most users will likely use the
included factory and builder methods to define their
LeastSquaresProblem(LSP) and these users would not be affected by the
change. Only the users that provide a custom implementation of
LSP.Evauation would be affected.

2. Remove edited residuals from the gradient and Jacobian, so that the
resulting vector and matrix have fewer rows. The advantage here is that the
user can compare the length of the residual vector in the Optimum to the
number of observations in the LSP to determine the number of edited
residuals. The problem is that returning vectors/matrices with different
sizes from LSP.evaluate() would violate the contract. Additionally we would
have to modify our existing optimizers to deal with the variable lengths.
For GaussNewton the modification would be small, but for LevenburgMarquardt
I would likely have to re-write it since I don't understand the code (not
for lack of trying :P ). Users that implement LeastSquaresOptimizers would
likely have to modify their code as well.

To summarize, in both cases users that only use the provided [math] classes
would not have to modify their code, while users that provide custom
implementations of [math] interfaces would have to modify their code.

I would like to get this feature wrapped in for the next release. Please
let me know if you have a preference for either implementation and if there
are any other issues I should consider.

Best Regards,
Evan

[1] http://markmail.org/message/e53nago3swvu3t52
 https://issues.apache.org/jira/browse/MATH-1105
[2] http://www.mathworks.com/help/curvefit/removing-outliers.html
 http://www.mathworks.com/help/curvefit/least-squares-fitting.html

Re: [RESULT] [VOTE] Migrate Commons Math to Git

[Evan Ward]

Thanks Luc for volunteering to oversee this. What is the status of the
switch? I have a couple documentation updates I'm waiting to commit
because I thought this switch was imminent.

Best Regards,
Evan

On 08/25/2014 05:23 AM, Luc Maisonobe wrote:
 Hi all,

 Le 21/08/2014 11:07, Luc Maisonobe a écrit :
 Hi all,

 The migration is scheduled to Sunday. Here is the comment made by Danile
 Gruno from infra about this (see
 https://issues.apache.org/jira/browse/INFRA-8180).
 The migration is postponed as Daniel is busy on another migration.
 I'll keep the list informed when he will notify me.

 best regards,
 Luc

  I will begin the transition towards Git on Sunday, as that is the
  least busy day for commits in general.
  The following steps will take place:
  1) The Subversion repository will be made read-only.
  2) The Git repository will be set up and also be read-only.
  3) You will verify that the contents of the Git repository matches
 the Subversion contents.
  4) Once acked, I will make the Git repo writeable and that should be
 it.
 best regards,
 Luc

 Le 12/08/2014 21:21, Luc Maisonobe a écrit :
 Hi Phil,

 Le 11/08/2014 15:39, Phil Steitz a écrit :

 On Aug 11, 2014, at 1:25 AM, Luc Maisonobe l...@spaceroots.org
 wrote:

 Hi all,

 Le 03/08/2014 16:35, Phil Steitz a écrit :
 On 8/3/14 2:35 AM, Benedikt Ritter wrote:
 Hi Phil,

 I think Gilles changed from -1 to +0 due to problems with the
 git-svn bridge.
 Thanks, Benedikt and my apologies, Gilles.

 Any news about the migration?
 I am sorry, Luc.  I have just started a new job and my time is very
 limited atm.  Hopefully I will have some time soon but if you or
 someone else wants to step in to move this along, please go ahead.
 Next step is an infra JIRA to get the repo.
 Its fine with me. I have created a JIRA issue for the migration:

   https://issues.apache.org/jira/browse/INFRA-8180

 best regards,
 Luc

 Phil
 best regards, Luc

 Phil
 Benedikt


 2014-08-03 5:45 GMT+02:00 Phil Steitz phil.ste...@gmail.com:

 Sorry for the delay in the tally on this.

 This VOTE has passed, with +1 votes from

 Gary Gregory Thomas Niedhart Evan Ward Gilles Sadowski
 (changed from -1 to +1) * Luc Maisonobe Venkatesha M

 +0 from Benedikt Ritter

 and no other votes.


 Thanks!

 Phil


 -


 To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
 For additional commands, e-mail: dev-h...@commons.apache.org

 -


 To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
 For additional commands, e-mail: dev-h...@commons.apache.org

 -


 To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
 For additional commands, e-mail: dev-h...@commons.apache.org

 -


 To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
 For additional commands, e-mail: dev-h...@commons.apache.org




 -
 To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
 For additional commands, e-mail: dev-h...@commons.apache.org




 -
 To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
 For additional commands, e-mail: dev-h...@commons.apache.org




 -
 To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
 For additional commands, e-mail: dev-h...@commons.apache.org




-
To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
For additional commands, e-mail: dev-h...@commons.apache.org

Re: [math] Side effect of LevenbergMarquardtOptimizer

[Evan Ward]

On 09/08/2014 04:50 AM, Olexiy Movchan wrote:
 Hi Evan,

 Possibly a small jitter of initial guess would solve this issue. But it is 
 hard to tell if this method guaranties convergence in all problematic cases.

 Normalization approach already works and allows to converge in those cases.

For non-linear least squares you already need an accurate initial guess,
though I could see the convergence region being different between the
two methods. Have you tried it?

Best Regards,
Evan


 Thanks,
 Olexiy

 -Original Message-
 From: Evan Ward [mailto:evan.w...@nrl.navy.mil] 
 Sent: Thursday, September 04, 2014 7:52 PM
 To: dev@commons.apache.org
 Subject: Re: [math] Side effect of LevenbergMarquardtOptimizer

 Hi Olexiy,

 In my field we often encounter a similar problem when estimating attitude 
 since a quaternion is only a valid rotation when it is normalized. We often 
 escape this issue by estimating a small
 adjustment to an apriori guess. (For the details see [1].)  For this 
 technique to work the cost function must be smooth and the apriori guess must 
 be close enough to the true value. Both of these assumptions are also 
 required to apply a non-linear least squares optimizer. Perhaps you can apply 
 a similar technique to your problem. (It seems that your 'A'
 parameter is orientation in 3D space.)

 If there is a need for an extra steps, I would prefer to make those explicit 
 rather than depending on side effects of cost function evaluation.

 Best Regards,
 Evan

 [1] Crassidis, John L., and John L. Junkins. /Optimal Estimation of Dynamic 
 Systems/. Boca Raton, FL: CRC, 2012.

 On 09/04/2014 05:37 AM, Olexiy Movchan wrote:
 Hello,

 I created the math issue https://issues.apache.org/jira/browse/MATH-1144.

 In version 2.0, LevenbergMarquardtOptimizer passed point to evaluator by 
 reference. So our software could modify it on every step of algorithm.
 In version 3.3, point is copied and then passed to evaluator, so it can't be 
 updated by evaluator.

 We use LevenbergMarquardtOptimizer for 3d surface fitting (cylinders, cones, 
 tori) by sampled points. And surface parameters should be renormalized on 
 every step of algorithm. Please see this article:
 http://nvlpubs.nist.gov/nistpubs/jres/103/6/j36sha.pdf

 Also please read the description of MATH-1144 jira issue.

 Can you modify optimizer or evaluator interface to allow in/out parameters 
 there?

 Thanks,
 Olexiy Movchan



 -
 To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
 For additional commands, e-mail: dev-h...@commons.apache.org



-
To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
For additional commands, e-mail: dev-h...@commons.apache.org

Re: [math] Side effect of LevenbergMarquardtOptimizer

[Evan Ward]

Gilles,

I like this approach. My only thought is that a separate interface for
the validate method would be nicer for our Java  8 users. Then the
implementation could be a lambda: (p) - p.unitVector()

Best Regards,
Evan

On 09/07/2014 08:11 PM, Gilles wrote:
 On Thu, 4 Sep 2014 12:52:24 -0400, Evan Ward wrote:
 Hi Olexiy,

 In my field we often encounter a similar problem when estimating
 attitude since a quaternion is only a valid rotation when it is
 normalized. We often escape this issue by estimating a small
 adjustment to an apriori guess. (For the details see [1].)  For this
 technique to work the cost function must be smooth and the apriori guess
 must be close enough to the true value. Both of these assumptions are
 also required to apply a non-linear least squares optimizer. Perhaps you
 can apply a similar technique to your problem. (It seems that your 'A'
 parameter is orientation in 3D space.)

 If there is a need for an extra steps, I would prefer to make those
 explicit rather than depending on side effects of cost function
 evaluation.

 IIUC, the feature could be made explicit by adding a method to the
 MultivariateJacobianFunction interface to allow the user to change
 the point about to be evaluated:

 interface MultivariateJacobianFunction {
   PairRealVector, RealMatrix value(RealVector point);

   /** @param point Point provided by the optimizer. */
   /** @return the point that will actually be evaluated. */
   RealVector validate(RealVector point);
 }

 Thus, in LeastSquaresFactory:

 private static class LocalLeastSquaresProblem
extends AbstractOptimizationProblemEvaluation
implements LeastSquaresProblem {

// ...

private final MultivariateJacobianFunction model;

// ...

public Evaluation evaluate(final RealVector point) {
  final RealVector p = model.validate(point).copy(); // --- Change
 here (at line 403).

  if (lazyEvaluation) {
return new LazyUnweightedEvaluation(model,
target,
p);
  } else {
final PairRealVector, RealMatrix value = model.value(p);
return new UnweightedEvaluation(value.getFirst(),
value.getSecond(),
target,
p);
  }
   }

   // ...
 }

 What do you think?

 Best,
 Gilles


 Best Regards,
 Evan

 [1] Crassidis, John L., and John L. Junkins. /Optimal Estimation of
 Dynamic Systems/. Boca Raton, FL: CRC, 2012.

 On 09/04/2014 05:37 AM, Olexiy Movchan wrote:
 Hello,

 I created the math issue
 https://issues.apache.org/jira/browse/MATH-1144.

 In version 2.0, LevenbergMarquardtOptimizer passed point to
 evaluator by reference. So our software could modify it on every
 step of algorithm.
 In version 3.3, point is copied and then passed to evaluator, so it
 can't be updated by evaluator.

 We use LevenbergMarquardtOptimizer for 3d surface fitting
 (cylinders, cones, tori) by sampled points. And surface parameters
 should be renormalized on every step of algorithm. Please see this
 article:
 http://nvlpubs.nist.gov/nistpubs/jres/103/6/j36sha.pdf

 Also please read the description of MATH-1144 jira issue.

 Can you modify optimizer or evaluator interface to allow in/out
 parameters there?

 Thanks,
 Olexiy Movchan




 -
 To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
 For additional commands, e-mail: dev-h...@commons.apache.org




-
To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
For additional commands, e-mail: dev-h...@commons.apache.org

Re: [math] Side effect of LevenbergMarquardtOptimizer

[Evan Ward]

I was suggesting (though not clearly :) keeping the
MultivariateJacobianFunction interface as is and adding a new interface
for any post processing of the evaluated point. Something like:

interface StepFinalizer {

  RealVector apply(RealVector point);

}

Then we would add another getter/setter to LeastSquaresProblem for an
instance StepFinalizer.

As I think more about it, I think this interface could also be used to
satisfy another feature request we've had in the past: recording the
trajectory that the optimizer takes. In this case the step finalizer
would have a side effect of adding the point to a list for later
retrieval. What do you think?

Best Regards,
Evan



On 09/08/2014 11:13 AM, Gilles wrote:
 Hello.



 On Mon, 8 Sep 2014 08:56:36 -0400, Evan Ward wrote:

 Gilles,



 I like this approach. My only thought is that a separate interface
 for

 the validate method would be nicer for our Java  8 users. Then the

 implementation could be a lambda: (p) - p.unitVector()



 Do you mean define an interface like



 interface ValidatingMultivariateJacobianFunction

   extends MultivariateJacobianFunction {



 /**

  * @param point Point provided by the optimizer.

  * @return the point that will actually be evaluated.

  */

 RealVector validate(RealVector point);

 }



 and then use instanceof to check whether validation should occur

 or not?



 Gilles



 Best Regards,

 Evan



 On 09/07/2014 08:11 PM, Gilles wrote:

 On Thu, 4 Sep 2014 12:52:24 -0400, Evan Ward wrote:

 Hi Olexiy,



 In my field we often encounter a similar problem when estimating

 attitude since a quaternion is only a valid rotation when it is

 normalized. We often escape this issue by estimating a small

 adjustment to an apriori guess. (For the details see [1].)  For
 this

 technique to work the cost function must be smooth and the apriori
 guess

 must be close enough to the true value. Both of these assumptions
 are

 also required to apply a non-linear least squares optimizer.
 Perhaps you

 can apply a similar technique to your problem. (It seems that your
 'A'

 parameter is orientation in 3D space.)



 If there is a need for an extra steps, I would prefer to make those

 explicit rather than depending on side effects of cost function

 evaluation.



 IIUC, the feature could be made explicit by adding a method to the

 MultivariateJacobianFunction interface to allow the user to change

 the point about to be evaluated:



 interface MultivariateJacobianFunction {

   PairRealVector, RealMatrix value(RealVector point);



   /** @param point Point provided by the optimizer. */

   /** @return the point that will actually be evaluated. */

   RealVector validate(RealVector point);

 }



 Thus, in LeastSquaresFactory:



 private static class LocalLeastSquaresProblem

extends AbstractOptimizationProblemEvaluation

implements LeastSquaresProblem {



// ...



private final MultivariateJacobianFunction model;



// ...



public Evaluation evaluate(final RealVector point) {

  final RealVector p = model.validate(point).copy(); // ---
 Change

 here (at line 403).



  if (lazyEvaluation) {

return new LazyUnweightedEvaluation(model,

target,

p);

  } else {

final PairRealVector, RealMatrix value = model.value(p);

return new UnweightedEvaluation(value.getFirst(),

value.getSecond(),

target,

p);

  }

   }



   // ...

 }



 What do you think?



 Best,

 Gilles





 Best Regards,

 Evan



 [1] Crassidis, John L., and John L. Junkins. /Optimal Estimation of

 Dynamic Systems/. Boca Raton, FL: CRC, 2012.



 On 09/04/2014 05:37 AM, Olexiy Movchan wrote:

 Hello,



 I created the math issue

 https://issues.apache.org/jira/browse/MATH-1144.



 In version 2.0, LevenbergMarquardtOptimizer passed point to

 evaluator by reference. So our software could modify it on every

 step of algorithm.

 In version 3.3, point is copied and then passed to evaluator, so
 it

 can't be updated by evaluator.



 We use LevenbergMarquardtOptimizer for 3d surface fitting

 (cylinders, cones, tori) by sampled points. And surface parameters

 should be renormalized on every step of algorithm. Please see this

 article:


 http://nvlpubs.nist.gov/nistpubs/jres/103/6/j36sha.pdf



 Also please read the description of MATH-1144 jira issue.



 Can you modify optimizer or evaluator interface to allow in/out

 parameters there?



 Thanks,

 Olexiy Movchan











 -

 To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org

 For additional commands, e-mail: dev-h...@commons.apache.org

Re: [math] Side effect of LevenbergMarquardtOptimizer

[Evan Ward]

On 09/08/2014 12:06 PM, Gilles wrote:
 On Mon, 8 Sep 2014 11:50:43 -0400, Evan Ward wrote:
 I was suggesting (though not clearly :) keeping the
 MultivariateJacobianFunction interface as is and adding a new interface
 for any post processing of the evaluated point.

 Isn't it rather pre-processing (the point is changed before evaluation)?

before evaluation and after computing the point. We're talking about the
same thing.


 Something like:

 interface StepFinalizer {

   RealVector apply(RealVector point);

 }

 Then we would add another getter/setter to LeastSquaresProblem for an
 instance StepFinalizer.

 As I think more about it, I think this interface could also be used to
 satisfy another feature request we've had in the past: recording the
 trajectory that the optimizer takes. In this case the step finalizer
 would have a side effect of adding the point to a list for later
 retrieval. What do you think?

 In that latter case, it would not be clear (IMHO) which point was used by
 the optimizer and which should recorded in the list...  I'd keep the
 issue
 separate and use an informative name for the new validating interface.
 [Even if it could indeed be diverted to allow tracking, but this was
 already the case with the ConvergenceChecker.]

Thats a good point. Sounds good to me.


 Best,
 Gilles

 [...]


 -
 To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
 For additional commands, e-mail: dev-h...@commons.apache.org



-
To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
For additional commands, e-mail: dev-h...@commons.apache.org

Re: [math] Side effect of LevenbergMarquardtOptimizer

[Evan Ward]

Hi Olexiy,

In my field we often encounter a similar problem when estimating
attitude since a quaternion is only a valid rotation when it is
normalized. We often escape this issue by estimating a small
adjustment to an apriori guess. (For the details see [1].)  For this
technique to work the cost function must be smooth and the apriori guess
must be close enough to the true value. Both of these assumptions are
also required to apply a non-linear least squares optimizer. Perhaps you
can apply a similar technique to your problem. (It seems that your 'A'
parameter is orientation in 3D space.)

If there is a need for an extra steps, I would prefer to make those
explicit rather than depending on side effects of cost function evaluation.

Best Regards,
Evan

[1] Crassidis, John L., and John L. Junkins. /Optimal Estimation of
Dynamic Systems/. Boca Raton, FL: CRC, 2012.

On 09/04/2014 05:37 AM, Olexiy Movchan wrote:
 Hello,

 I created the math issue https://issues.apache.org/jira/browse/MATH-1144.

 In version 2.0, LevenbergMarquardtOptimizer passed point to evaluator by 
 reference. So our software could modify it on every step of algorithm.
 In version 3.3, point is copied and then passed to evaluator, so it can't be 
 updated by evaluator.

 We use LevenbergMarquardtOptimizer for 3d surface fitting (cylinders, cones, 
 tori) by sampled points. And surface parameters should be renormalized on 
 every step of algorithm. Please see this article:
 http://nvlpubs.nist.gov/nistpubs/jres/103/6/j36sha.pdf

 Also please read the description of MATH-1144 jira issue.

 Can you modify optimizer or evaluator interface to allow in/out parameters 
 there?

 Thanks,
 Olexiy Movchan

Re: [VOTE] Migrate Commons Math to Git

[Evan Ward]

+1

Evan

On 07/22/2014 03:16 PM, Thomas Neidhart wrote:
 On 07/22/2014 07:01 PM, Phil Steitz wrote:
 Looks like other projects are running VOTEs to ensure there is
 consensus for this action and including references to VOTE threads
 in INFRA JIRAs.  Lets do this.  The action I am proposing is that we
 request a new ASF git repo, make the current svn repo read-only
 (adding a README to make it clear) and change github mirroring to
 use the git repo.   Votes, please.  This vote will close in 72
 hours.  Note that this VOTE applies only to [math], i.e.
 http://svn.apache.org/repos/asf/commons/proper/math. 

 [x] +1 go for it

 Thomas

 -
 To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
 For additional commands, e-mail: dev-h...@commons.apache.org





smime.p7s
Description: S/MIME Cryptographic Signature

Re: [math] git svn setup

[Evan Ward]

On 05/30/2014 05:56 AM, Luc Maisonobe wrote:
 Le 30/05/2014 00:39, Mark Thomas a écrit :
 On 29/05/2014 23:16, Evan Ward wrote:
 Hi,

 I'm trying to setup write access for the clone I have of the github
 [math] repository. I thought that since every commit in that repository
 already had a git-svn-id it would be relatively easy, so I ran git svn
 init and git svn fetch. But the fetch has been running for ~5 hours now
 and seems to be checking out every commit individually! Is this typical
 of what other people using git svn have seen?  How did other git svn
 users setup their repository?
 And fairly soon the ASF svn server is probably going to ban you for abuse.

 Everything you need to set this up without hammering the svn server is
 here (and associated links):
 http://git.apache.org/

 It should take less than a minute with a decent internet connection.

Thanks Mark for the pointer to the wiki page. Even after following the
instructions there git svn was still trying to download the entire
repository one commit at a time. I also had to:

1. echo `git rev-parse HEAD`  .git/refs/remotes/origin/trunk 
2. add myself to the authors.txt file

Then it worked. :)


 When I was searching the mail archives about this I saw a few emails
 about [math] switching to git, but didn't see a final decision. Is that
 still in progress? Maybe I'll just wait until the writable git
 repository gets set up...
 That might be a long wait.
 Well, I would really like to see it happen.

 I am still volunteering to help people new to git on this, and I still
 think [math] could be a good candidate as the the commons component to
 switch to git.

If it means we can make the switch I'll volunteer to answer questions
about git as well.

Best Regards,
Evan


 best regards,
 Luc

 Mark


 -
 To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
 For additional commands, e-mail: dev-h...@commons.apache.org



 -
 To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
 For additional commands, e-mail: dev-h...@commons.apache.org




-
To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
For additional commands, e-mail: dev-h...@commons.apache.org

[math] git svn setup

[Evan Ward]

Hi,

I'm trying to setup write access for the clone I have of the github
[math] repository. I thought that since every commit in that repository
already had a git-svn-id it would be relatively easy, so I ran git svn
init and git svn fetch. But the fetch has been running for ~5 hours now
and seems to be checking out every commit individually! Is this typical
of what other people using git svn have seen?  How did other git svn
users setup their repository?

When I was searching the mail archives about this I saw a few emails
about [math] switching to git, but didn't see a final decision. Is that
still in progress? Maybe I'll just wait until the writable git
repository gets set up...

Best Regards,
Evan


-
To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
For additional commands, e-mail: dev-h...@commons.apache.org

Re: [VOTE] Release Math 3.3 based on RC1

[Evan Ward]

Hi Thomas,

I noticed that the user guide is using deprecated classes in the
examples for least squares optimization. Do we want to update the user
guide to help new users find/understand the new API?

Best Regards,
Evan

On 04/29/2014 04:07 PM, Thomas Neidhart wrote:
 Hi all,

 I would like to call a vote to release Commons Math 3.3 based on RC1.

  Math 3.3 RC1 is available for review here:
 https://dist.apache.org/repos/dist/dev/commons/math/
 (svn revision 5199)

  Maven artifacts are here:

 https://repository.apache.org/content/repositories/orgapachecommons-1025/org/apache/commons/commons-math3/3.3/

  Details of changes since 3.2 are in the release notes:
 https://dist.apache.org/repos/dist/dev/commons/math/RELEASE-NOTES.txt
 http://people.apache.org/builds/commons/math/3.3/RC1/changes-report.html

  The tag is here:
 https://svn.apache.org/repos/asf/commons/proper/math/tags/MATH_3_3_RC1
 (svn revision 1591059)

  Site:
 http://people.apache.org/builds/commons/math/3.3/RC1/
   (note the apidocs for the 3.3 release will be added with the release)

  Clirr Report (compared to 3.2):
 http://people.apache.org/builds/commons/math/3.3/RC1/clirr-report.html
(note that there are 4 false positives where the signature of a
 method has changed such that the parameter type is now the superclass of
 the previous type)

  RAT Report:
 http://people.apache.org/builds/commons/math/3.3/RC1/rat-report.html

  Findbugs Report:
 http://people.apache.org/builds/commons/math/3.3/RC1/findbugs.html
   (the listed error is a false positive, the relevant line has not
 changed since the last release and the floating point comparison should
 be correct in this case).

   KEYS:
   http://www.apache.org/dist/commons/KEYS

 Please review the release candidate and vote.
 This vote will close no sooner that 72 hours from now, i.e. after 1000
 GMT 02-May 2014.

   [ ] +1 Release these artifacts
   [ ] +0 OK, but...
   [ ] -0 OK, but really should fix...
   [ ] -1 I oppose this release because...

 Best regards,

 Thomas

 -
 To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
 For additional commands, e-mail: dev-h...@commons.apache.org



-
To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
For additional commands, e-mail: dev-h...@commons.apache.org

Re: [math] fitting.leastsquares and stat.regression

[Evan Ward]

Phil suggested we discuss the relationship of the two least squares
packages in [math]

Current Status

Currently both fitting.leastsquares and stat.regression have least
squares implementations. The fitting.leastsquares package supports
non-linear least squares with weights, pluggable optimization
algorithms, and (soon) data editing. The package is written from an
optimization/engineer's perspective.

As far as I can tell the stat.regression implementations are for linear
least squares. Some of the implementations contain neat optimizations,
for example requiring O(1) space for n data points. This package seems
to be written more from a statistician/economist's perspective.

Options

1. Keep separate packages.
2. delegate the implementation from one package to the other
3. merge into a single package. (could lead to some interesting
algorithms. e.g. non-linear general least squares)

Phis, please add any important points I've missed.

Best Regards,
Evan


 Phil Steitz commented on MATH-1105:
 ---

 Might be better to take this discussion to the ML. We now have two
 least squares impls - one in fitting/leastsquares and another in
 stats.regression (actually this has been true for some time). The
 stats side of it (residual analysis, ANOVA, etc.) belongs more
 naturally in stats.regression. It might make more sense to add this
 functionality there. Or maybe we just refactor to have the
 stats.regression classes use the impl in leastsquares. In any case, we
 should discuss on the ML.


 Least squares statistical data editing
 --

 Key: MATH-1105
 URL: https://issues.apache.org/jira/browse/MATH-1105
 Project: Commons Math
 Issue Type: Improvement
 Reporter: Evan Ward
 Attachments: 0001-Add-statistical-editing-capability.patch,
 0002-Integrate-data-editing-with-the-LS-framework.patch






-
To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
For additional commands, e-mail: dev-h...@commons.apache.org

Re: [math] refactoring least squares

[Evan Ward]

On 02/24/2014 05:47 PM, Gilles wrote:
 On Mon, 24 Feb 2014 14:41:34 -0500, Evan Ward wrote:
 On 02/24/2014 01:23 PM, Gilles wrote:
 On Mon, 24 Feb 2014 11:49:26 -0500, Evan Ward wrote:
 One way to improve performance would be to provide pre-allocated space
 for the Jacobian and reuse it for each evaluation.

 Do you have actual data to back this statement?

 I did some tests with the CircleVectorial problem from the test cases.
 The Jacobian is 1000x2, and I ran it 1000 times until hotspot stopped
 making it run faster. The first column is the current state of the code.
 The second column is with one less matrix allocation each problem
 evaluation.

trunk-1 alloc  %change
 lu 0.90 s   0.74  -17%
 chol   0.90 0.74  -17%
 qr 0.87 0.70  -20%


 I also see similar reductions in runtime using 1e6 observations and 3
 observations. We could save 3-4 allocations per evaluation, which
 extrapolates to 60%-80% in run time.

 I would not have expected such a big impact.
 How did you set up the benchmark? Actually, it would be a sane starting
 point to create a performance unit test (see e.g.
 FastMathTestPerformance
 in package o.a.c.m.util).

The test I used: https://gist.github.com/wardev/3f183dfaaf297dc2462c

I also modified CircleVectorial to use the MultivariateJacobianFunction
to avoid the translation copies.



 The
 LeastSquaresProblem interface would then be:

 void evaluate(RealVector point, RealVector resultResiduals, RealVector
 resultJacobian);

 I'm interested in hearing your ideas on other approaches to solve this
 issue. Or even if this is an issue worth solving.

 Not before we can be sure that in-place modification (rather than
 reallocation) always provides a performance benefit.

 I would like to hear other ideas for improving the performance.

 Design-wise, it is quite ugly to modify input parameters. I think that it
 could also hurt in the long run, by preventing other ways to improve
 performance.

I agree, it smells like fortran. ;)


 Why couldn't the reuse vs reallocate be delegated to implementations of
 the MultivariateJacobianFunction interface?

That is the change I made to produce the performance numbers. It works
well as long as the application has a single thread. Once it is
multi-threaded we would have to create a new Function for each thread.
I'm reluctant to do that because I think not copying the whole problem
for each thread is a nice feature.

Really I think we need 1 matrix per optimization context. The question
is: How does the Function, which is several method calls below the
optimize() method, access the current optimization context? If we reuse
the matrix at the Function level then we are implicitly tying the
optimization context to the thread context. By passing the matrices as
parameters we make the context explicit, but we clutter up the API.
Perhaps there is a better solution we haven't thought of yet.

 Eventualy, doesn't it boil down to creating RealVector and RealMatrix
 implementations that modify and return this rather than create a new
 object?

Interesting idea. I think this will remove the allocation+copy when
applying weights. That just leaves where matrix decompositions create a
copy of the input matrix.

Best Regards,
Evan



 Best Regards,
 Gilles


 Evan


 On 02/24/2014 12:09 PM, Luc Maisonobe wrote:
 Hi Evan,

 Le 24/02/2014 17:49, Evan Ward a écrit :
 I've looked into improving performance further, but it seems any
 further
 improvements will need big API changes for memory management.

 Currently using Gauss-Newton with Cholesky (or LU) requires 4 matrix
 allocations _each_ evaluation. The objective function initially
 allocates the Jacobian matrix. Then the weights are applied through
 matrix multiplication, allocating a new matrix. Computing the normal
 equations allocates a new matrix to hold the result, and finally the
 decomposition allocates it's own matrix as a copy. With QR there are 3
 matrix allocations each model function evaluation, since there is no
 need to compute the normal equations, but the third allocation+copy is
 larger. Some empirical sampling data I've collected with the jvisualvm
 tool indicates that matrix allocation and copying takes 30% to 80% of
 the execution time, depending on the dimension of the Jacobian.

 One way to improve performance would be to provide pre-allocated space
 for the Jacobian and reuse it for each evaluation. The
 LeastSquaresProblem interface would then be:

 void evaluate(RealVector point, RealVector resultResiduals, RealVector
 resultJacobian);

 I'm interested in hearing your ideas on other approaches to solve this
 issue. Or even if this is an issue worth solving.
 Yes, I think this issue is worth solving, especially since we are going
 to ship 3.3 and need to fix as much as possible before the release,
 thus
 avoiding future problems. Everything spotted now is worth fixing now.

 Your approach seems reasonable, as long as the work arrays are really

Re: [math] refactoring least squares

[Evan Ward]

I've looked into improving performance further, but it seems any further
improvements will need big API changes for memory management.

Currently using Gauss-Newton with Cholesky (or LU) requires 4 matrix
allocations _each_ evaluation. The objective function initially
allocates the Jacobian matrix. Then the weights are applied through
matrix multiplication, allocating a new matrix. Computing the normal
equations allocates a new matrix to hold the result, and finally the
decomposition allocates it's own matrix as a copy. With QR there are 3
matrix allocations each model function evaluation, since there is no
need to compute the normal equations, but the third allocation+copy is
larger. Some empirical sampling data I've collected with the jvisualvm
tool indicates that matrix allocation and copying takes 30% to 80% of
the execution time, depending on the dimension of the Jacobian.

One way to improve performance would be to provide pre-allocated space
for the Jacobian and reuse it for each evaluation. The
LeastSquaresProblem interface would then be:

void evaluate(RealVector point, RealVector resultResiduals, RealVector
resultJacobian);

I'm interested in hearing your ideas on other approaches to solve this
issue. Or even if this is an issue worth solving.

Best Regards,
Evan


-
To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
For additional commands, e-mail: dev-h...@commons.apache.org

Re: [math] refactoring least squares

[Evan Ward]

On 02/24/2014 01:23 PM, Gilles wrote:
 On Mon, 24 Feb 2014 11:49:26 -0500, Evan Ward wrote:
 One way to improve performance would be to provide pre-allocated space
 for the Jacobian and reuse it for each evaluation.

 Do you have actual data to back this statement?

I did some tests with the CircleVectorial problem from the test cases.
The Jacobian is 1000x2, and I ran it 1000 times until hotspot stopped
making it run faster. The first column is the current state of the code.
The second column is with one less matrix allocation each problem
evaluation.

   trunk-1 alloc  %change
lu 0.90 s   0.74  -17%
chol   0.90 0.74  -17%
qr 0.87 0.70  -20%


I also see similar reductions in runtime using 1e6 observations and 3
observations. We could save 3-4 allocations per evaluation, which
extrapolates to 60%-80% in run time.

 The
 LeastSquaresProblem interface would then be:

 void evaluate(RealVector point, RealVector resultResiduals, RealVector
 resultJacobian);

 I'm interested in hearing your ideas on other approaches to solve this
 issue. Or even if this is an issue worth solving.

 Not before we can be sure that in-place modification (rather than
 reallocation) always provides a performance benefit.

I would like to hear other ideas for improving the performance.



 Best Regards,
 Gilles

 Evan


On 02/24/2014 12:09 PM, Luc Maisonobe wrote:
 Hi Evan,

 Le 24/02/2014 17:49, Evan Ward a écrit :
 I've looked into improving performance further, but it seems any further
 improvements will need big API changes for memory management.

 Currently using Gauss-Newton with Cholesky (or LU) requires 4 matrix
 allocations _each_ evaluation. The objective function initially
 allocates the Jacobian matrix. Then the weights are applied through
 matrix multiplication, allocating a new matrix. Computing the normal
 equations allocates a new matrix to hold the result, and finally the
 decomposition allocates it's own matrix as a copy. With QR there are 3
 matrix allocations each model function evaluation, since there is no
 need to compute the normal equations, but the third allocation+copy is
 larger. Some empirical sampling data I've collected with the jvisualvm
 tool indicates that matrix allocation and copying takes 30% to 80% of
 the execution time, depending on the dimension of the Jacobian.

 One way to improve performance would be to provide pre-allocated space
 for the Jacobian and reuse it for each evaluation. The
 LeastSquaresProblem interface would then be:

 void evaluate(RealVector point, RealVector resultResiduals, RealVector
 resultJacobian);

 I'm interested in hearing your ideas on other approaches to solve this
 issue. Or even if this is an issue worth solving.
 Yes, I think this issue is worth solving, especially since we are going
 to ship 3.3 and need to fix as much as possible before the release, thus
 avoiding future problems. Everything spotted now is worth fixing now.

 Your approach seems reasonable, as long as the work arrays are really
 allocated at the start of the optimization and shared only through a few
 documented methods like the one you propose. This would mean we can say
 in the javadoc that these area should be used only to fulfill the API
 requirements and not copied elsewhere, as they *will* be modified as the
 algorithm run, and are explicitly devoted to avoid reallocation. I guess
 this kind of problems is more important when lots of observations are
 performed, which correspond to very frequent use case (at least in the
 fields I know about).

 For the record, what you propose seems similar to what is done in the
 ODE package, as the state vector and its first derivatives are also kept
 in preallocated arrays which are reused throughout the integration and
 are used to exchange data between the Apache Commons Math algorithm and
 the user problem to be solved. So it is somehting we already do elsewhere.

OK. We could keep the Evaluation interface, which would just reference
the pre-allocated residuals and matrix. If the result parameters are
null the LSP could allocate a matrix of the correct size automatically.
So then the interface would look like:

Evaluation evaluate(RealVector point, RealVector resultResiduals,
RealVector resultJacobian);


 best regards,
 Luc

 Best Regards,
 Evan


 -
 To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
 For additional commands, e-mail: dev-h...@commons.apache.org



 -
 To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
 For additional commands, e-mail: dev-h...@commons.apache.org



On 02/24/2014 01:16 PM, Gilles wrote:
 On Mon, 24 Feb 2014 18:09:26 +0100, Luc Maisonobe wrote:
 Hi Evan,

 Le 24/02/2014 17:49, Evan Ward a écrit :
 I've looked into improving performance further, but it seems any
 further
 improvements will need big API changes for memory

Re: [math] refactoring least squares

[Evan Ward]

On 02/24/2014 02:18 PM, Ted Dunning wrote:

 On Mon, Feb 24, 2014 at 10:23 AM, Gilles gil...@harfang.homelinux.org
 mailto:gil...@harfang.homelinux.org wrote:

 One way to improve performance would be to provide
 pre-allocated space
 for the Jacobian and reuse it for each evaluation.


 Do you have actual data to back this statement?


 The
 LeastSquaresProblem interface would then be:

 void evaluate(RealVector point, RealVector resultResiduals,
 RealVector
 resultJacobian);

 I'm interested in hearing your ideas on other approaches to
 solve this
 issue. Or even if this is an issue worth solving.


 Not before we can be sure that in-place modification (rather than
 reallocation) always provides a performance benefit.


 Allocation is rarely the problem in these situations.  The implied
 copying of data is.

 And even the copying isn't always a problem.  For instance, it often
 pays off big to copy data to column (or row) major representation to
 improve cache coherency.  The result is that a large fraction of the
 time is spent copying, but without the copying, the remaining time
 would take 10x longer.  The net time taken is 3x faster with the copy.

Agreed. Perhaps by providing the result matrix the optimization
algorithm can choose the best data layout for itself.

Regards,
Evan

Re: [math] refactoring least squares

[Evan Ward]

On Thu 20 Feb 2014 08:34:19 AM EST, Gilles wrote:

 Hi.

 On Thu, 20 Feb 2014 10:05:58 +0100, luc wrote:

 Hi all,

 I am looking more precisely at the least squares package and have a
 few adjustments to propose.

 The Evaluation interface declares a computeValue and a
 computeJacobian method among others.
 However, it seems the implementations are really simply built with
 the precomputed values and
 just return them. The other compute methods (for covariances for
 instance) do really compute
 something based on the stored Jacobian or value.
 Wouldn't it be more clear to use getValue and getJacobian for
 these two methods and keep
 compute for the other ones?


 I recall that we did not want ...Impl classes. Is another
 implementation
 foreseen?

 Is the computation vs access to a precomputed value mandated by
 the interface?


+1 for renaming to getXxx(). We could even rename all the methods to
start with get. Since we have an interface (and when the computations
happen is purposefully not specified), I see the pre-compute and cache
vs compute on method call tradeoff as an implementation detail. If all
the methods start with 'get' then the whole interface will be consistent
with itself and with Java conventions.

As for the Impl classes, theses are package private implementations of
interfaces, so the users won't see the name. For the LeastSquareProblem
interface I could see a lazy evaluation implementation, or perhaps an
implementation that applies data editing by zeroing out certain residuals.



 Another point is binding value and Jacobian together in a single
 object. I personally think it
 is a good thing, but if I remember well, some users explicitly asked
 for separating them to save
 some computation. I'm not sure about this, though, so I would like to
 have some advices.


 Unless I'm mixing up things, it was the opposite: having them within the
 same structure could allow for not duplicating computations necessary to
 compute both.


I agree with Gilles, for all my use cases it is easiest to compute both
value and Jacobian at the same time.

Regards,
Evan




 Last point is OptimizationProblem. Should this interface been in
 fitting or in a more general
 package, and in this case which one (optim, util)?


 What's the usefulness of an interface? It's only boiler-plate code which
 every implementation should reuse. The abstract class could indeed be
 relocated in util to stress its internal status.

 Regards,
 Gilles


 -
 To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
 For additional commands, e-mail: dev-h...@commons.apache.org

-
To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
For additional commands, e-mail: dev-h...@commons.apache.org

Re: [math] refactoring least squares

[Evan Ward]

On 02/18/2014 09:51 AM, luc wrote:
 Hi all,

 As you have probably noticed, I have committed the changes proposed by
 Evan. This ends up to about 20 different commits,
 as I have tried to stick with the progressive changes he made in his
 Github repository. Some of the initial commits
 do not abide by our development rules (missing Apache headers, author
 tags or some checkstyle problems), but some have been
 fixed in intermediate commits by Evan himself months ago during the
 discussions with Gilles, and I have fixed the remaining
 ones in the final commits. So if you take the current status, it is
 compliant.

 I kept the single leastsquares package, so did not introduce an
 additional leastsquares2.

 I have also taken care to preserve all changes already done since last
 summer when Evan designed his code. In particular, this
 means the new curve fitters have been adapted so they both use the new
 package and fix MAX-1014 has Gilles already did that.

 I would like to ask Evan to check in case I have mixed something, and
 after this check we can look at polishing this and
 perhaps extending some parts to other optimizers or move some
 interfaces upward if necessary.

Thanks Luc! It looks good to me.

One thing I would like to change/add before the 3.3 release are the
decomposition algorithms for GaussNewton. Using a CholeskyDecomposition
on the normal equations should be faster than LU by a factor of 2.
Similarly, using QR on the Jacobian, before forming the normal equations
should be more stable than any method that forms the normal equations. I
can implement these changes as extra Decomposition options. What is the
best way to send the patches? Attachments on JIRA? Pull requests on github?

Out of curiosity, why is /** {@inheritDoc} */ required? Isn't that the
default?

Best Regards,
Evan

 best regards
 Luc

 -
 To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
 For additional commands, e-mail: dev-h...@commons.apache.org



-
To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
For additional commands, e-mail: dev-h...@commons.apache.org

Re: [math] leastsquares refactoring

[Evan Ward]

On 02/13/2014 12:19 PM, Gilles wrote:
 Hello Luc.

 On Thu, 13 Feb 2014 16:10:36 +0100, Luc Maisonobe wrote:
 Hi Gilles,

 Le 08/02/2014 22:16, Gilles a écrit :

 Other issues to be discussed:

  * [MATH-1008]

a new package has been created o.a.c.m.fitting.leastsquares which
contains some interface for the new fluent API but they are also
used elsewhere so I guess it would be more logical to put them in a
more general place, e.g. WithMaxEvaluations should be moved to
o.a.c.m.optim. WDYT?

 Strictly speaking, these have potential use outside the o.a.c.m.optim
 package.

 Also, further improvements were postponed since the alternate proposal
 looked quite promising:
   https://issues.apache.org/jira/browse/MATH-1026


 I had a look at Evan's poposal in MATH-1026. I really like it!

 Do you think we could replace our current lestsquares (which has never
 been released yet), with this new one and finish it?

 In principle yes (see my first comment on the JIRA page). In practice,
 I'm
 not sure.

 Concerning the last
 comment by Evan in the JIRA system, I think they could be answered
 quickly. Also the questions about caching and lazy evaluations are
 implementation details that could be changed later on, even after 3.3
 release.

 My memory about all the discussions fades away. IIRC, the main problem
 with the proposed code is that it provided supposed enhancements to the
 functionality before showing that it could faithfully reproduce the
 existing functionality (albeit with another API).
 Because I could not be sure of the equivalence and did not have time to
 make changes and checks, I did not want to take the responsibility for
 committing the code in that uncertain state.
 Maybe it's fine; I just don't know.

I would just like to add that I made a significant effort to ensure the
API update is in a separate commit from all the other improvements.
Commit 7e9a15e [1] contains the API switch. As with any API change it
involved updating the test cases, so I realize there is a lot of code to
read to check for correctness. I did factor common test code into
reusable methods, but IMHO this should make it easier to review since
there is only one opportunity for a typo instead of dozens.

All the following commits in the pull request contain the functionality
improvements and code quality improvements (as well as moving to a
different package and test style changes).

Please let me know if you have any questions about the code or about why
I made a particular change.

Best Regards,
Evan


[1]
https://github.com/wardev/commons-math/commit/7e9a15efb535b91afad9d8275eb2864ea2295ab4

 Also, please take into account that, additionally to the API change for
 the code now in fitting.leastsquares, there has been some little
 improvements in the code itself (e.g. compare the implementations of
 LevenbergMarquardtOptimizer in optim vs fitting.leastsquares).
 Before trashing this new code, it would be good to make sure that those
 (tested) improvements have been retained in Evan's code.
 To sum up, what is in fitting.leastsquares should be the reference
 code, in the sense that post-release 3.3, _nothing_ from the deprecated
 optimization and optim.nonlinear.vector should make its way to code
 for 4.0.
 Hence, I'd favour keeping the current fitting.leastsquares until we are
 positively sure that everything there was indeed ported to Evan's code
 (whenever applicable).
 We could put both in an experimental sub-package; that would not delay
 a 3.3 release, while leaving room for non-compatible changes in a 3.4
 release.


 Best regards,
 Gilles


 -
 To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
 For additional commands, e-mail: dev-h...@commons.apache.org




-
To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
For additional commands, e-mail: dev-h...@commons.apache.org

Re: [COLLECTIONS] CollectionUtils.adapterCollection(CollectionE, TransformerA)?

[Evan Ward]

Hi John,

I've attached my form.

Thanks,
Evan

On Wed 18 Sep 2013 09:23:10 PM EDT, Bruno P. Kinoshita wrote:
 True, maybe with EachElement and a Procedure [1], or maybe 
 IteratorToGeneratorAdapter, or something similar to the 
 ArrayListBackedAggregator and some custom function :) Or maybe with something 
 totally new.

 [1] https://gist.github.com/kinow/6617618
 
 From: Matt Benson gudnabr...@gmail.com
 To: Commons Developers List dev@commons.apache.org
 Sent: Wednesday, September 18, 2013 5:29 PM
 Subject: Re: [COLLECTIONS] CollectionUtils.adapterCollection(CollectionE, 
 TransformerA)?


 Looks like a job for [functor].

 Matt
 On Sep 18, 2013 2:59 PM, Benedikt Ritter benerit...@gmail.com wrote:

 Hi,

 At work, I needed to create an adapter for an existing class that has a
 getter for a collection of objects. Now in my adapter I want to create a
 getter for a collection of adapted objects.
 I don't want to create a new list an add adapters for the contained objects
 to it. I just want to create the adapters on the fly when they are
 requested.

 I thought this should be implemented somewhere in commons-collections, but
 I could not find anything like that. I wonder if it would make sense to
 create such functionality. I have created an example at github [1]. What do
 you think? Am I missing something here?

 Regards,
 Benedikt

 [1] https://gist.github.com/britter/6614535


 -
 To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
 For additional commands, e-mail: dev-h...@commons.apache.org



business_cards_final.pdf
Description: Adobe PDF document


smime.p7s
Description: S/MIME Cryptographic Signature

Re: [math] fluent API for the ODE integrators

[Evan Ward]

Hi Luc,

On Mon 16 Sep 2013 12:04:21 PM EDT, Luc Maisonobe wrote:

 Hello,

 I have started (at last) to work on the fluent API for ODE integrators.
 It seems to be well suited for the needs, and would allow a better
 separation of the integrator engine and the integrator state. The
 engine will hold all the coefficients for the linear combinations within
 the integration steps and some general settings like min and max step
 sizes, or absolute and relative error thresholds for adaptive stepsize
 integrators. These data don't change during one integration run, and
 user may want to share them among several successive or parallel
 integration runs. The state will hold more transient values for numerous
 internal variables, like step size, start of step, pending discrete
 events, number of evaluations ... These data are updated continuously
 during an integration run and can certainly not be shared.

 Users will only see and configure the engine. Each time they call the
 integrate method on an engine, it will create on the fly a state
 dedicated for this call and everything within the integrator will
 reference this state instance. Users may call the integrate method
 several time to run them in parallel in different threads. As each call
 will have its own state instance, and as the engine itself which is
 shared by all states is immutable, everything should work without problem.

 These were the good news.

 The bad news are that it is difficult to separate the state from the
 engine as I have mixed everything right from the top level interface,
 which in this case is ODEIntegrator. The interface defines methods like
 getCurrentStepStart (which should refer to state) but also
 addStepHandler/addEventHandler (which should refer to engine). The next
 level interface (FirstOrederIntegrator) adds an integrate method which
 should belong to engine. The AbstractIntegrator class implements many of
 these methods and adds new protected methods which can be used by
 specific integrators.

 My current design is to have an internal IntegratorState class defined
 within AbstractIntegrator and providing all the state related fields and
 methods. For now, I don't think there is no need here for interfaces or
 abstract classes, the State is a simple thing. I may introduce a small
 hierarchy later on when I delve into the specific integrators, though.
 The integrate method in AbstractIntegrator would simply be something like:

 public void integrate(ExpandableStatefulODE equations, double t) {
 integrate(new IntegratorState(this), equations, t);
 }

 protected abstract integrate(IntegratorState state,
 ExpandableStatefulODE equations,
 double t);

 The protected integrate method with the state argument would be the same
 as the current integrate methods, just storing and updating variables in
 the State instance rather than in the integrator instance as done today.

 This should handle the engine part as desired, with a state instance
 created on the fly for each integration run.

+1 I like it. I think it will make the integrators easier to use and
more useful. :)


 Most of the IntegratorState class would be copied from the current
 AbstractIntegrator class, where many state related stuff is managed now.
 The problem is that many of these fields are protected fields
 (stepStart, stepSize, isLastStep, ...) and many methods are protected or
 even public methods (initIntegration, computeDerivatives). Of course,
 there are also the public methods inherited from the top level interface
 like getCurrentStepStart. I cannot simply remove the public method from
 the interface, nor remove the protected fields and methods from the
 AbstractIntegrator class, this would break compatibility...

 Most of these methods were never intended to be in the public API of the
 library, but due to Java limitations with cross packages visibility,
 some were put public. In the same spirit, the protected methods are not
 expected to be in the component API as it is not expected that people
 will provide their own implementations of ODE integrators: all the
 implementations and all the callers of the methods are within [math].

 So how can I handle these methods and fields? At least, I will deprecate
 them. I will also change all our own implementations to use the new
 IntegratorState class to store and update all these transient variables,
 so from [math] point of view, the deprecated protected methods and
 fields will not be used anymore. What should I do with the remnants and
 unused fields? As long as we are in the 3.X series, they should remain
 here, but how should they behave? Lets take one example: there is one
 method, intended to be called only by the specific integrators
 implementations:

 protected double acceptStep(AbstractStepInterpolator,
 double[], double[], double);

 It is called by AdamsBashforthIntegrator, AdamsMoultonIntegrator,
 EmbeddedRungeKuttaIntegrator, GraggBulirschStoerIntegrator and
 RungeKuttaIntegrator with lines 

Re: [Math] Fluent API, inheritance and immutability

[Evan Ward]

On 08/27/2013 07:27 AM, Gilles wrote:
 Hi.

 Sorry but the _main_ question of my previous post was...

 On Mon, 26 Aug 2013 16:35:51 -0400, Evan Ward wrote:
 On 08/26/2013 03:33 PM, Gilles wrote:
 On Mon, 26 Aug 2013 13:59:32 -0400, Evan Ward wrote:
 Hi again,

 I rearranged the least squares package and I've posted the results.[1]
 I've also created a pull request[2] and an associated issue.[3]

 [1] https://github.com/wardev/commons-math/tree/sepOpt
 [2] https://github.com/apache/commons-math/pull/1
 [3] https://issues.apache.org/jira/browse/MATH-1026

 A summary of what I changed: (See the git log for more details.)

 Thanks for the effort!
 Could you attach a patch to the issue page?
 Hmm, actually, there would be so many changes that I don't think it's
 really useful to have a patch.
 Wouldn't it be clearer to create entirely new classes for everything,
 in a new package? [Suggestions for a name?]

 ... here.

 [Then we can do a manual diff for selected files to see how things
 have evolved.]

 It is a bit much to view all at once. :) I tried to provide detailed
 commit messages and the diffs between commits should be more meaningful
 after the first one.

 As far as I'm concerned, I already agreed to a design change, so I don't
 need to look at what it looks like, so to speak.
 However, before I commit anything I want to be able to ensure that indeed
 _everything_ was translated to the new API, without additions or
 removals.
 This I can do if I can run the tests in parallel (current and new
 design); if I apply the diff, it will obviously destroy the current
 code!


I misunderstood. I'll put the new code in a leastsquares2 package and
restore the previous implementation in the leastsquares package. I'm not
really sure how this is different from comparing two revisions side by
side...

 [...]

 Regards,
 Gilles


 -
 To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
 For additional commands, e-mail: dev-h...@commons.apache.org



-
To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
For additional commands, e-mail: dev-h...@commons.apache.org

Re: [Math] Fluent API, inheritance and immutability

[Evan Ward]

On 08/26/2013 03:11 PM, Konstantin Berlin wrote:
 I looked only at the GuassNewton class as a general guide to how
 things work. I like it a lot! I only wish all of the optimizations
 were rewritten in this way.

 Several comments

 1) I believe this code can now be simplified

 // build the linear problem
 final double[] b = new double[nC];
 final double[][] a = new double[nC][nC];
 for (int i = 0; i  nR; ++i) {

 final double[] grad = weightedJacobian.getRow(i);
 final double residual = currentResiduals.getEntry(i);

 // compute the normal equation
 //residual is already weighted
 for (int j = 0; j  nC; ++j) {
 b[j] += residual * grad[j];
 }

 // build the contribution matrix for measurement i
 for (int k = 0; k  nC; ++k) {
 double[] ak = a[k];
 //Jacobian/gradient is already weighted
 for (int l = 0; l  nC; ++l) {
 ak[l] += grad[k] * grad[l];
 }
 }
 }

 Should be just this (I haven't checked the math)

 jt = weightedJacobian.transpose();
 a = jt* weightedJacobian;
 b= jt*currentResiduals;

Yes. I think if we rewrite the implementation we should dig a little
deeper into the math.

Solving J^T J x = J^T b using Cholesky decomposition can use mn^2/2 +
n^3/6 operations with error proportional to [cond(J)]^2. Solving J x = b
using QR decomposition can use mn^2 - n^3/3 operations with error
proportional to cond(J) + norm(r)*[cond(J)]^2. Using SVD to solve J x =
b uses ~ 5mn^2 operations, but can handle nearly rank deficient problems.[1]

To sum up, the normal equations with Cholesky is the fastest, but only
works on well conditioned problems. SVD is very slow and robust. QR is a
good compromise between speed an numerical stability.

[1] Heath, Michael T. /Scientific computing : an introductory survey/.
Boston: McGraw-Hill, 2002. Chapters 3 and 6.



 2) Based on the reasons I previously describe, would it be possible to
 separate the newton step into a separate function (which you would
 also add to the interface)?
 The idea here is that solving for the newton step can be
 computationally intensive depending on the size of the problem. So
 there is no one universal approach. There are several somewhat
 different ways that his can be solved, with
 some methods requiring information from previous steps in order to
 work. This would also allow to plug in line search or trust region
 control of step size. In order to allow all of these approaches, I
 would suggest this function:

 //this function would solve for dX
 public NewtonStepResult solveStep(LeastSquaresProblem lsp, RealVector
 currentPoint, Object prevStepInfo)
 {
   NewtonStepResult result  = new NewtonStepResult();

   //code to solve for the next step dx followed by
   result.newtonStep = dx;

   //code to update the step information
   //leave blank for now
   result.stepInfo = null;
 }

 where

 public class NewtonStepResult
 {
   public class RealVector newtonStep;
   public class Object stepInfo;
 }

What do we gain by incorporate the Newton step into the problem? We
could do it, but I think computing the Newton step better belongs in the
optimizer because there are several ways to solve for the step (see
above), and different optimizers may solve different equations to obtain
the next step. I think LM modifies the Jacobian before solving the
system. [citation needed]

Regards,
Evan

Re: [Math] Fluent API, inheritance and immutability

[Evan Ward]

On 08/27/2013 08:44 AM, Evan Ward wrote:
 On 08/27/2013 07:27 AM, Gilles wrote:
 Hi.

 Sorry but the _main_ question of my previous post was...

 On Mon, 26 Aug 2013 16:35:51 -0400, Evan Ward wrote:
 On 08/26/2013 03:33 PM, Gilles wrote:
 On Mon, 26 Aug 2013 13:59:32 -0400, Evan Ward wrote:
 Hi again,

 I rearranged the least squares package and I've posted the results.[1]
 I've also created a pull request[2] and an associated issue.[3]

 [1] https://github.com/wardev/commons-math/tree/sepOpt
 [2] https://github.com/apache/commons-math/pull/1
 [3] https://issues.apache.org/jira/browse/MATH-1026

 A summary of what I changed: (See the git log for more details.)
 Thanks for the effort!
 Could you attach a patch to the issue page?
 Hmm, actually, there would be so many changes that I don't think it's
 really useful to have a patch.
 Wouldn't it be clearer to create entirely new classes for everything,
 in a new package? [Suggestions for a name?]
 ... here.

 [Then we can do a manual diff for selected files to see how things
 have evolved.]
 It is a bit much to view all at once. :) I tried to provide detailed
 commit messages and the diffs between commits should be more meaningful
 after the first one.
 As far as I'm concerned, I already agreed to a design change, so I don't
 need to look at what it looks like, so to speak.
 However, before I commit anything I want to be able to ensure that indeed
 _everything_ was translated to the new API, without additions or
 removals.
 This I can do if I can run the tests in parallel (current and new
 design); if I apply the diff, it will obviously destroy the current
 code!

 I misunderstood. I'll put the new code in a leastsquares2 package and
 restore the previous implementation in the leastsquares package. I'm not
 really sure how this is different from comparing two revisions side by
 side...

The two least squares packages are both in
https://github.com/apache/commons-math/pull/2

Regards,
Evan


-
To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
For additional commands, e-mail: dev-h...@commons.apache.org

Re: [MATH] Re: commons-math pull request: Two implementations of least squares in separeat...

[Evan Ward]

Using https://github.com/apache/commons-math/pull/2.diff and `patch -p1 
 2.diff` worked for me. (On top of svn r1517789)

I've also attached the modified files to 
https://issues.apache.org/jira/browse/MATH-1026

Regards,
Evan

On Tue 27 Aug 2013 12:35:48 PM EDT, Gilles wrote:
 On Tue, 27 Aug 2013 08:40:25 -0700, Ted Dunning wrote:
 On Tue, Aug 27, 2013 at 7:41 AM, Gilles
 gil...@harfang.homelinux.orgwrote:

 The patch does not apply cleanly (special options needed to handle
 output from git?).


 Try different prefix levels.  The -p0 option is commonly helpful.

 I did that too. It didn't work.

 Gilles


 -
 To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
 For additional commands, e-mail: dev-h...@commons.apache.org


-
To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
For additional commands, e-mail: dev-h...@commons.apache.org

Re: [Math] Fluent API, inheritance and immutability

[Evan Ward]

On 08/27/2013 12:35 PM, Konstantin Berlin wrote:
 Sorry, I think there was a misunderstanding. I was suggesting, like
 you are pointing out, that you make a newton step function explicit
 inside the optimizer. I think it is a good idea that this function is
 explicit in all optimizers, so it can later be overwritten.At the
 least, it should be propagated back into all the base abstract method,
 not sure about interfaces. If this is not done, it would fairly
 difficult to extends these optimizers to use bound constraints, etc.
 This is a small step that can make a large difference.

I guess so. :) I like the explicit function idea. There is already an
enum (GaussNewtonOptimizer.Decomposition) to select the decomposition
algorithm. How about you try switching the method

DecompositionSolver getSolver(final RealMatrix matrix)

to something like

RealVector newtonStep(RealMatrix jacobian, RealVector residuals)

if it works well there we can think about adding other methods, such as
Cholesky and SVD.


 Thanks,
 Konstantin

 P.S. I also think that the start point should be a parameter into the
 optimize function rather than a property of the least squares problem.
 Since depending on the specific optimizer it might not be needed (ex.
 some implementations of quadratic programming). It would also make the
 notation easier to do multiple starts.


My thoughts for multiple starts would be to require a builder:

ListOptimum multiStart(LSOptimizer optimizer, LSBuilder builder,
ListRealVector starts){
for(RealVector start : starts){
ret.add(optimizer.optimize(builder.start(start).build()));
}
}


or to use the composition pattern that is used already for applying
weights and counting evaluations in LSFactory.

Regards,
Evan


-
To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
For additional commands, e-mail: dev-h...@commons.apache.org

Re: [Math] Fluent API, inheritance and immutability

[Evan Ward]

Hi again,

I rearranged the least squares package and I've posted the results.[1]
I've also created a pull request[2] and an associated issue.[3]

[1] https://github.com/wardev/commons-math/tree/sepOpt
[2] https://github.com/apache/commons-math/pull/1
[3] https://issues.apache.org/jira/browse/MATH-1026

A summary of what I changed: (See the git log for more details.)

1. Separate LeastSquaresProblem from LeastSquaresOptimizer. Update tests
to use the new interfaces.
2. Immutable fluent builders for optimizers.
3. Remove weights from the LeastSquaresProblem interface. They are
pre-applied to the Jacobian and residuals.
4. Single function for computed values and Jacobian. I'm not 100% on the
name. Call it MultivariateVectorMatrixFunction? or
MultivariateJacobianFunction? or
MultivariateVectorAndMatrixFunctionWithANameLongerThan80Characters? :)
5. Make LevenbergMarquardt thread safe by using explicit temporary
parameters instead of instance fields.
6. In GaussNewton change useLU to an Enum
7. Change ConvergenceCheckerPointVectorValuePair to
CovergenceCheckerEvaluation. Would allow conditions like RMS changes
less than 1e-3.
8. Use [math] linear package in interfaces (RealMatrix and RealVector
instead of double[] and double[][])

I did not understand some things in the test code:

1. Why is AbstractLeastSquaresOptimizerTestValidation (renamed to
EvaluationTestValidation) disabled by default and how do I tell if it
passed?
2. Other smaller questions about the tests. Theses are in comments in
the test package and marked with TODO.
3. LU and QR use a default singularity threshold. Caused test failures
when GaussNewton is configured to use QR. (MATH-1024)
4. A single test case in LevenburgMarquardt.testControlParameters() now
converges where it used to diverge. It estimates a reasonable rms after
15 iterations. I'm not sure why it used to diverge...

There are still several items on the TODO list. Code and discussion is
appreciated. :)

Affecting the interface:

4. Immutable fluent builders for LSProblem?
6. maxIterations and ConvergenceChecker seem to have duplicate
functionality. (To stop the algorithm after a certain number of
iterations.) Remove the maxIterations parameter and let the
ConvergenceCheck handle that job.

Implementation only:

2. Efficient diagonal weights. (don't need to create a new matrix, just
multiply in place)
3. Add more convenience methods to Factory and Builder. E.g. for
diagonal weights, ...
4. Add conditional logic to Factory so it can use efficient diagonal
weights even if they are provided in a matrix.
3. Use [math] linear package in optimizer implementations. To match the
current code in performance I think this would mean adding views or
specialized multiplication methods to the linear package. Depends on
MATH-765 and MATH-928.

Thanks for looking over the proposal and reviewing it.

Regards,
Evan

-
To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
For additional commands, e-mail: dev-h...@commons.apache.org

Re: [Math] Fluent API, inheritance and immutability

[Evan Ward]

On 08/26/2013 03:33 PM, Gilles wrote:
 On Mon, 26 Aug 2013 13:59:32 -0400, Evan Ward wrote:
 Hi again,

 I rearranged the least squares package and I've posted the results.[1]
 I've also created a pull request[2] and an associated issue.[3]

 [1] https://github.com/wardev/commons-math/tree/sepOpt
 [2] https://github.com/apache/commons-math/pull/1
 [3] https://issues.apache.org/jira/browse/MATH-1026

 A summary of what I changed: (See the git log for more details.)

 Thanks for the effort!
 Could you attach a patch to the issue page?
 Hmm, actually, there would be so many changes that I don't think it's
 really useful to have a patch.
 Wouldn't it be clearer to create entirely new classes for everything,
 in a new package? [Suggestions for a name?]
 [Then we can do a manual diff for selected files to see how things
 have evolved.]

It is a bit much to view all at once. :) I tried to provide detailed
commit messages and the diffs between commits should be more meaningful
after the first one. In the first commit I added/deleted files to fit
the new interfaces and ensured that all the tests pass. I refactored the
tests so that they can be run on multiple configurations of the same
optimizer. Now GN with QR is actually tested.

If you are looking for a place to start reading I suggest
LeastSquareProblem. The rest of the package is focused on implementing
or using that interface. If have any comments about specific sections,
github has a neat ability to comment on a particular line of the source...


 Could you please accompany all the comments below with file names and
 line numbers references?

These comments are essentially a digest of the commit log.[1] More
explanation is provided there along with diffs, file names, and line
number - all in a pretty UI.

[1] https://github.com/apache/commons-math/pull/1/commits


 1. Separate LeastSquaresProblem from LeastSquaresOptimizer. Update tests
 to use the new interfaces.
 2. Immutable fluent builders for optimizers.
 3. Remove weights from the LeastSquaresProblem interface. They are
 pre-applied to the Jacobian and residuals.
 4. Single function for computed values and Jacobian. I'm not 100% on the
 name. Call it MultivariateVectorMatrixFunction? or
 MultivariateJacobianFunction? or
 MultivariateVectorAndMatrixFunctionWithANameLongerThan80Characters? :)
 5. Make LevenbergMarquardt thread safe by using explicit temporary
 parameters instead of instance fields.
 6. In GaussNewton change useLU to an Enum
 7. Change ConvergenceCheckerPointVectorValuePair to
 CovergenceCheckerEvaluation. Would allow conditions like RMS changes
 less than 1e-3.
 8. Use [math] linear package in interfaces (RealMatrix and RealVector
 instead of double[] and double[][])

 I did not understand some things in the test code:

 1. Why is AbstractLeastSquaresOptimizerTestValidation (renamed to
 EvaluationTestValidation) disabled by default and how do I tell if it
 passed?

 Not sure that the rename is fine here.
 The comments on the test methods explain the purpose; and the result
 (plotting the output) should be examined manually.

I renamed it to match the subject under test. It can be renamed back.
I'm just not sure what I'm looking for in the output.

 2. Other smaller questions about the tests. Theses are in comments in
 the test package and marked with TODO.

 Please ask them here, so that we can arrive to a common answer.

These smaller questions are more of a code review and make much more
sense next to the code. :)

AbstractLeastSquaresOptimizerAbstractTest:
testGetIterations: Use a more specific test? could pass with 'return 1;'
testMoreEstimatedParametersUnsorted: the first two elements of point
were not previously checked
testInconsistentEquations: what is this actually testing?
testInconsistentSizes1: why is this part here? hasn't 'Ix=b' been tested
already?

LevenbergMarquardtOptimizerTest:
testNonInvertible: check that it is a bad fit? Why the extra conditions?
testControlParameters: why should this fail? It uses 15 evaluations.
(mentioned already)
checkEstimate: check it got the right answer when no exception?
QuadraticProblem: why is this here? It is not used.


 3. LU and QR use a default singularity threshold. Caused test failures
 when GaussNewton is configured to use QR. (MATH-1024)

 You are looking for trouble.
 The simpler route would be to do the conversion firs, ensuring that all
 tests still pass.
 Afterwards, new issues can be raised and resolved on their own.

All tests still pass, and GN+QR is now tested as throughly as GN+LU.


 4. A single test case in LevenburgMarquardt.testControlParameters() now
 converges where it used to diverge. It estimates a reasonable rms after
 15 iterations. I'm not sure why it used to diverge...

 Same here. You might have uncovered a bug, or you may have introduced
 one.


 There are still several items on the TODO list. Code and discussion is
 appreciated. :)

 Affecting the interface:

 4. Immutable fluent builders for LSProblem

Re: [jira] [Created] (MATH-1024) LU and QR have different default singularity thresholds

[Evan Ward]

On 08/21/2013 06:00 PM, Gilles wrote:
 Hi.

 On Wed, 21 Aug 2013 18:23:52 + (UTC), Evan Ward (JIRA) wrote:
 Evan Ward created MATH-1024:
 ---

  Summary: LU and QR have different default singularity
 thresholds
  Key: MATH-1024
  URL: https://issues.apache.org/jira/browse/MATH-1024
  Project: Commons Math
   Issue Type: Bug
 Affects Versions: 3.2
 Reporter: Evan Ward
 Priority: Minor


 LU uses 1e-11 and QR uses 0. This means by default QR never throws a
 Singularity exception. This caused divergence to go unreported in
 GaussNewtonOptimizer when it uses QR. I think a consistent default
 should be chosen. I will make the threshold explicit in GaussNewton.

 I could readily specify a threshold value (e.g. 1e-11) in class
 GaussNewtonOptimizer (where the solver is instantiated), if it
 is deemed more consistent.
 Any objection to make the change (in o.a.c.m.fitting.leastsquares)?

That is what I did. :)



 Best regards,
 Gilles


 -
 To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
 For additional commands, e-mail: dev-h...@commons.apache.org



-
To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
For additional commands, e-mail: dev-h...@commons.apache.org

Re: Need for an alternatives to the main line of code.

[Evan Ward]

I agree with James that this is a problem that git has already solved. 
For the changes I'm working on I have already cloned from the git 
mirror and plan to upload the patch as a fork + pull request on github. 
I think the branchyness of distributed version control makes it easy 
to have several experimental branches without needing a central 
repository for them.

Evan




On Wed 21 Aug 2013 01:01:22 PM EDT, Ted Dunning wrote:
 +1

 Sent from my iPhone

 On Aug 21, 2013, at 9:42, Ajo Fod ajo@gmail.com wrote:


 I hope you'll agree that as it stands, this makes CM capable of only
 solving a subset the mathematical problems of what it can solve with a more
 open policy.

 The argument for alternative designs of the API is great too because it
 allows people to comment on the API design as they use it.

 -
 To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
 For additional commands, e-mail: dev-h...@commons.apache.org


-
To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
For additional commands, e-mail: dev-h...@commons.apache.org

Re: [Math] Fluent API, inheritance and immutability

[Evan Ward]

Hi all,

Sorry for the delay. I'll try switching all of the least squares package
to the new/seperable API and post the results to github. From there we
can review, discuss and then decide if it makes sense to switch the rest
of the optimizers.


On 08/15/2013 09:12 AM, Konstantin Berlin wrote:
 Hi,

 As a quick first pass, I will comment on Evans code only for now. Most
 of my comments have to do with numerical aspects and associated
 design, rather than threading.

 I like the idea of the optimize function taking in a Problem class. I
 think it can add a lot of flexibility going forward.

 First I think to make things clear, LeastSquares should be renamed to
 IterativeLeastSquares. Since for unconstrained linear least squares,
 this whole hierarchy is not correct. For one thing they don't need an
 initial guess, for another, linear least squares can factor the matrix
 before experimental data is supplied, so it can support rapid
 recomputation.

From my background the problem is more commonly called General Least
Squares, or Non-Linear Least Squares. I shortened it to avoid (yet
another) long name. :) I don't plan to include linear least squares
problem in the new API. It seems hard to be simpler than new
QRDecomposition(A).getSolver().solve(b). :)


 There are several issues that I have with current example

 1) While the least-squares problem now encapsulates the weights, which
 I like, for some reason the weights are explicitly used in the actual
 GaussNewtonOptimizer and probably all other least-squares optimizer. I
 think that is bad for several reasons:
 i) the actual optimizer algorithm has nothing to do with weights, so
 it logically inconsistent. It forces people who have no weights to
 still go through the actual numerical computation.
 ii) it makes it harder to maintain and update the optimizer code. Just
 think about how much cleaner it would be if weights are actually
 included in the Jacobian.
 iIi) The inclusion of weight can be really optimized inside the
 LeastSquaresProblem, where depending on the problem they might be able
 to take it more efficiently than the general approach.

 2) All computations inside the optimizers should be done using linear
 algebra library (or is the library so bad?). Not only does it make the
 code easier to follow, it can make it a lot faster. Imagine a
 not-so-hypothetical example where the Jacobian is actually sparse
 (I know you guys are removing sparse linear algebra, but lets say it
 gets put back at some point later). Forming the Hessian from the
 Jacobian is a very expensive operation (J^T*J), but if J is sparse the
 speedup could be automatically propagated through the algorithm.

I agree with 1 and 2. You seem to have more experience with optimizers
than me; would you like to update GaussNewton.doOptimize() to assume the
weights are already included?


 3) Speaking of forming the Hessian, the actual computation of the
 descent step should be logically and explicitly separated into two
 separate function. I think the first function could actually be moved
 to the LeastSquaresProblem (need more thought on this)
  i) The first function will compute the step size p, where H*p = -g, H
 is the Hessian and g is the gradient. The reason for this is there are
 several ways this problem can be solved, and that should be deferred
 to a specific implementation later. For example, in a very large
 least-squares problem (or any large problem) you cannot actually form
 J^T*J (or H), and instead you solve the problem using conjugate
 gradient iterative method (which you already have in the library),
 where you compute J^T*J*x as J^T*(J*x). Again, here having the
 Jacobian be a matrix would really help future proof things. In the
 case of general non-linear optimizers you might want to support in the
 future a very important quasi-Newton algorithms BFGS or L-BFGS, were
 this step is done very differently.

 ii) Once the step size is computed, a line search or trust region
 method is used to potentially modify the step. Here also bound
 constraints can be enforced, so an optimizer that supports it can
 overwrite this function. This part should also be exposed, at least in
 the abstract classes, from which the final least-squares optimizer
 should be built.

I'm not sure I follow. Are you suggesting that the LSP class solve the
normal equations? Solving the normal equations and computing the next
step seems like the job of the optimizer.


 Thanks,
 Konstantin


Ajo Fod wrote:

 I like the immutability idea. I wonder if there are any guidelines on when
 the performance hit because of immutability is high enough to want mutable
 objects and synchronization?

 -Ajo

For me the dividing line is copying large matrices and arrays. I try to
avoid copying the residuals and the Jacobian because they can be
megabytes in size.

Regards,
Evan

-
To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
For 

Re: [Math] Fluent API, inheritance and immutability

[Evan Ward]

On 08/14/2013 11:30 AM, Gilles wrote:
 On Tue, 13 Aug 2013 17:47:18 -0400, Evan Ward wrote:
 On 08/13/2013 08:17 AM, Gilles wrote:
 (I.e. GN would
 not need 5 superclasses and fields for storing upper and lower
 bounds.)

 Be careful: GaussNewtonOptimizer does _not_ support bounds!

 That is my point! Because of it's complex inheritance every instance
 includes fields for storing bounds even though the algorithm has no use
 for them. (Declared in BaseMultivariateOptimizer) I was looking at the
 released (3.2) version.

 That's because, I was lazy; I should have added yet another layer:
   BaseMultivariateOptimizerWithBounds

 :-D

 Looks like it has since been fixed. :)

 Yes and no. In the latest version, I simply not included what is not used
 (yet). At some point, bounds must be stored somewhere for those
 optimizers
 that do use them. We'll probably need to insert that additional layer...
 [Which you'll have to do too in your scheme when you apply it to the
 optimizers that use bounds (or not).]



 With the API re-design you have the opportunity to design for thread
 safety, which can be hard to add on to an existing API.

 True in general, but in this case we didn't make a thorough analysis
 of advantages and drawbacks of creating a new instance for each
 application of a withXxx method.
 I previously hinted that such a behaviour might not be desirable,
 independently of the issue of thread-safety.

 Thanks for the whole discussion; It has given me some new ideas. :)

 You are most welcome to experiment with the current revision of the
 code in o.a.c.m.fitting.leastsquares, in order to pinpoint problems
 for your use-cases.

 This point is really interesting for comparing the two approaches (yours
 and the current code the above package). [More below...]

 You could also provide the concrete changes (patch) needed to move the
 data to another object, as you suggested.
 If it's better design and leads to simpler code, there shouldn't be
 any problem in applying it for the next release.

 I separated the algorithm from the problem definition for the
 GaussNewton optimizer. See
 https://gist.github.com/wardev/dfd16ac0ad3ba62ede05 If we decide to go
 this route LM would follow the same pattern. It still needs some
 polishing and a few smaller API decisions.

 A summary of the changes:
   - The two superclasses of GN were turned in to
 AbstractOptimizationProblem and LeastSquaresProblemImpl. I tried to keep
 most of the code the same so that a diff would be meaningful.
   - Interfaces were extracted from LeastSquaresProblemImpl and
 AbstractOptimizationProblem. I kept the upper level of the hierarchy as
 an implementation aid.
   - GaussNewton implements LeastSquaresOptimizer and does not have a
 super class. Evaluation and iteration counts are now locals.
   - There is now one method to evaluate the model/problem, see
 LeastSquaresProblem.evaluate(double[]). This returns a container with
 the value and Jacobian.

 In effect, this is another way to see the problem.

 And if we were starting from scratch, I wouldn't mind following that
 design, but as it is now, I'm not quite convinced that it is actually
 simpler than what I got too in the above package. Personally, I find
 a few things fairly odd. E.g. the counters are in the problem spec
 whereas it is a property of the optimization algorithm; also, the
 evaluation is not encapsulated anymore (an optimization algorithm
 must explicitly call incrementCount instead of it being done as
 part of the model evaluation). [I agree that they are moot points,
 but good (IMO) ideas in the current code should not be dropped
 without careful examination, lest that we go in circles.]

Yes, I wasn't sure what to do with the counters. My thought was that the
evaluation counter is a divergence condition and so it belongs with the
convergence condition in the problem definition. I could also see it as
part of the optimizer with the notion that if the algorithm can't find
an optimum after X evaluations it is not worth it to keep trying.

Since the iteration counter was incremented explicitly I thought it made
sense to increment the evaluation counter explicitly. With the interface
based design it would be easy to create a wrapper to track evaluations
to keep the original semantics:

public static LeastSquareProblem countEvaluations(final
LeastSquaresProblem lsp, final Incrementor counter){
return new LeastSquaresProblem(){
/* delegate all other methods to lsp */
   
public Evaluation evaluate(double[] point){
counter.increment();
return lsp.evaluate(point);
}
};
}


 Advantages:
   - GaussNewton is now trivially easy to make immutable and fluent and
 thread-safe. This will allow the same algorithm to be applied to
 different problems.

 Unless I'm mistaken, this is only because you off-loaded all data
 to another class. This other class is not less problematic for the
 viewpoint of thread safety: You can share

Re: [Math] Fluent API, inheritance and immutability

[Evan Ward]

Hi,

On 08/14/2013 12:23 PM, Konstantin Berlin wrote:
 Since I have no idea what you guys are describing, could you guys please
 do something like this? Or at least put rough outlines of the interfaces
 and classes somewhere, so other people can evaluate?

The *rough* interfaces and classes of my proposal are in [1] which you
can clone from. Gilles work so far is in the [math] trunk: [2]

Regards,
Evan

[1] https://gist.github.com/wardev/dfd16ac0ad3ba62ede05
[2]
http://svn.apache.org/viewvc/commons/proper/math/trunk/src/main/java/org/apache/commons/math3/fitting/leastsquares/


-
To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
For additional commands, e-mail: dev-h...@commons.apache.org

Re: [Math] Fluent API, inheritance and immutability

[Evan Ward]

On 08/13/2013 08:17 AM, Gilles wrote:
 (I.e. GN would
 not need 5 superclasses and fields for storing upper and lower bounds.)

 Be careful: GaussNewtonOptimizer does _not_ support bounds!

That is my point! Because of it's complex inheritance every instance
includes fields for storing bounds even though the algorithm has no use
for them. (Declared in BaseMultivariateOptimizer) I was looking at the
released (3.2) version. Looks like it has since been fixed. :)



 With the API re-design you have the opportunity to design for thread
 safety, which can be hard to add on to an existing API.

 True in general, but in this case we didn't make a thorough analysis
 of advantages and drawbacks of creating a new instance for each
 application of a withXxx method.
 I previously hinted that such a behaviour might not be desirable,
 independently of the issue of thread-safety.

 Thanks for the whole discussion; It has given me some new ideas. :)

 You are most welcome to experiment with the current revision of the
 code in o.a.c.m.fitting.leastsquares, in order to pinpoint problems
 for your use-cases.

 You could also provide the concrete changes (patch) needed to move the
 data to another object, as you suggested.
 If it's better design and leads to simpler code, there shouldn't be
 any problem in applying it for the next release.

I separated the algorithm from the problem definition for the
GaussNewton optimizer. See
https://gist.github.com/wardev/dfd16ac0ad3ba62ede05 If we decide to go
this route LM would follow the same pattern. It still needs some
polishing and a few smaller API decisions.

A summary of the changes:
  - The two superclasses of GN were turned in to
AbstractOptimizationProblem and LeastSquaresProblemImpl. I tried to keep
most of the code the same so that a diff would be meaningful.
  - Interfaces were extracted from LeastSquaresProblemImpl and
AbstractOptimizationProblem. I kept the upper level of the hierarchy as
an implementation aid.
  - GaussNewton implements LeastSquaresOptimizer and does not have a
super class. Evaluation and iteration counts are now locals.
  - There is now one method to evaluate the model/problem, see
LeastSquaresProblem.evaluate(double[]). This returns a container with
the value and Jacobian.

Advantages:
  - GaussNewton is now trivially easy to make immutable and fluent and
thread-safe. This will allow the same algorithm to be applied to
different problems.
  - If the model function and convergence checker are thread safe, then
so is the LeastSquaresProblem (if you don't mutate it). This would allow
different algorithms to be applied to the same problem. These are the
two big bullet points that I think make it worthwhile. :)
  - Allows algorithms to be applied to problems they weren't designed
for by using views. I.e. view a least squares problem as a direct
problem by creating an implementation of DirectProblem that delegates to
a LeastSquaresProblem. A similar delegation would work to view a
DirectOptimzier as a LeastSquaresOptimizer.
  - Same code for the most part, just reorganized.

Further decisions:
  - It would be nice to have a model interface that could be evaluated
with one method call. E.g. Pairdouble[], double[][] value(double[]).
  - Leave getWeights() out of the LeastSquaresProblem API? This would
help encapsulation, but would require further modification of the GN
implementation.
  - Should the LeastSquaresProblem interface include mutators? I think
it makes sense to only include the methods that the optimization
algorithm needs to read. That would keep the API smaller and not tie
the users to a particular implementation.
  - method and class names.

I'm sure there are a few more issues. :) Thanks for considering the
separate API.

Regards,
Evan






-
To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
For additional commands, e-mail: dev-h...@commons.apache.org

Re: [Math] Fluent API, inheritance and immutability

[Evan Ward]

On 08/09/2013 06:55 PM, Gilles wrote:
 [...]

 This does make it important to decide on a well written and
 complete API before releasing it.

 When the scope of the software is well circumscribed, that would be
 possible. With the whole of [Math]ematics, much less so. :-}
 And state-of-the-art in Java is a moving target, aimed at by changing
 CM contributors with differing needs and tastes; this adds to the
 unstable mix.

 That's a good point. I still prefer the interface design (though I may
 be in the minority) for two reasons. First, if a concrete class only
 publicly exposes the methods defined in an interface it encourages
 polymorphism. User code that uses one implementation can be easily
 switched to another and new implementations are less constrained.
 Second, it encourages composition over inheritance. I agree with Josh
 Bloch that composition produces more maintainable code. Adding new
 methods to an existing interface/class breaks composition.

 The problem is to get the interface right. As it happens, at some point
 we discover something that was not foreseen; and to correct/improve the
 design, compatibility must be broken.
 [But refactoring is not a failure of development; it's part of it.]

 I think the interface/abstract class discussion is partially separable
 from the immutable/mutable discussion. I see the algorithm as the part
 that could really benefit from the polymorphism. Perhaps separating the
 problem definition (data) from the algorithm will improve the
 flexibility of the API. For example,

 PointVectorValuePair solveMyNLLSProblem(NLLSOptimizer opt){
 //define problem to solve in an independent object
 NLLSProblem p = new NLLSProblem(/*model functions, weights,
 convergence checker, ...*/);

 //provide algorithm with the data it needs
 //algorithm has no problem specific state
 return opt.optimize(p);
 }

 I may be missing something, but how much better is it to store
 everything the optimizer needs in yet another class?
 [Then, that's a possible approach, but it's not what we started
 from in Commons Math, and when trying to fix some inconsistency
 or removing duplicate code, I tried to retain what could be from
 the existing design.]

I've looked at the implementations of GN and LM and it seems that the
abstract classes are primarily concerned with evaluating the model, and
the concrete classes use the evaluation to compute the next step. I
think separating those two concerns could simplify the implementation of
the optimizers. The NLLSProblem class would provide methods to evaluate
the weighted Jacobian, residuals, etc. The concrete class would look
almost the same as they do today, except calls to the parent class would
be replaced with calls to the NLLSProblem class.

The benefit is that the optimizer implementation/hierarchy would be
simpler and other classes could use the methods for model evaluation. I
don't think keeping the optimization algorithm and data in the same
class gains much since all the scratch space is reallocated on each call
to optimize. (Performance will be about the same either way.)

 [...] Thread safety is a tricky beast. I think we agree that the only
 way to guarantee thread safety is to only depend on final concrete
 classes that are thread safe themselves.

 I don't think so. When objects are immutable, thread-safety follows

 It is somewhat off topic, but a counter example would be Vector3D. Since
 the class is not final, a user could extend it and override all the
 methods and add some set{X,Y,Z} methods to make it mutable. Even though
 Vector3D is immutable, there is no _guarantee_ that every instanceof
 Vector3D is immutable unless it is final. This is why String is final.

 I think I don't get your point: If someone extends a class that is safe
 in a way that the extension is unsafe, that's his problem. ;-)

 [...] copying any large matrices or arrays is prohibitively
 expensive. For the NLLS package we would be copying a pointer to a
 function that can generate a large matrix. I think adding some
 documentation that functions should be thread safe if you want to use
 them from multiple threads would be sufficient.

 I you pass a pointer (i.e. a reference in Java), all bets are off:
 the
 class is not inherently thread-safe. That's why I suggested to
 mandate a
 _deep_ copy method (with a stringent contract that should allow a
 caller
 to be sure that all objects owned by an instance are disconnected from
 any
 other objects).

 As someone who has designed a thread safe application based on deep
 copying I don't think this is route to follow. A deep copy means you
 have to be able to copy an arbitrary (possibly cyclical) reference
 graph. Without the graph copy there are many subtle bugs. (References to
 the same object are now references to different objects.) With the graph
 copy the implementation is very complex. This is the reason
 Serialization has a separate patch up step after object creation,
 which leads to 

Re: [Math] Fluent API, inheritance and immutability

[Evan Ward]

On 08/12/2013 12:19 PM, Phil Steitz wrote:
 On 8/12/13 9:04 AM, Evan Ward wrote:
 On 08/09/2013 06:55 PM, Gilles wrote:
 [...]

 This does make it important to decide on a well written and
 complete API before releasing it.
 When the scope of the software is well circumscribed, that would be
 possible. With the whole of [Math]ematics, much less so. :-}
 And state-of-the-art in Java is a moving target, aimed at by changing
 CM contributors with differing needs and tastes; this adds to the
 unstable mix.
 That's a good point. I still prefer the interface design (though I may
 be in the minority) for two reasons. First, if a concrete class only
 publicly exposes the methods defined in an interface it encourages
 polymorphism. User code that uses one implementation can be easily
 switched to another and new implementations are less constrained.
 Second, it encourages composition over inheritance. I agree with Josh
 Bloch that composition produces more maintainable code. Adding new
 methods to an existing interface/class breaks composition.
 The problem is to get the interface right. As it happens, at some point
 we discover something that was not foreseen; and to correct/improve the
 design, compatibility must be broken.
 [But refactoring is not a failure of development; it's part of it.]

 I think the interface/abstract class discussion is partially separable
 from the immutable/mutable discussion. I see the algorithm as the part
 that could really benefit from the polymorphism. Perhaps separating the
 problem definition (data) from the algorithm will improve the
 flexibility of the API. For example,

 PointVectorValuePair solveMyNLLSProblem(NLLSOptimizer opt){
 //define problem to solve in an independent object
 NLLSProblem p = new NLLSProblem(/*model functions, weights,
 convergence checker, ...*/);

 //provide algorithm with the data it needs
 //algorithm has no problem specific state
 return opt.optimize(p);
 }
 I may be missing something, but how much better is it to store
 everything the optimizer needs in yet another class?
 [Then, that's a possible approach, but it's not what we started
 from in Commons Math, and when trying to fix some inconsistency
 or removing duplicate code, I tried to retain what could be from
 the existing design.]
 I've looked at the implementations of GN and LM and it seems that the
 abstract classes are primarily concerned with evaluating the model, and
 the concrete classes use the evaluation to compute the next step. I
 think separating those two concerns could simplify the implementation of
 the optimizers. The NLLSProblem class would provide methods to evaluate
 the weighted Jacobian, residuals, etc. The concrete class would look
 almost the same as they do today, except calls to the parent class would
 be replaced with calls to the NLLSProblem class.

 The benefit is that the optimizer implementation/hierarchy would be
 simpler and other classes could use the methods for model evaluation. I
 don't think keeping the optimization algorithm and data in the same
 class gains much since all the scratch space is reallocated on each call
 to optimize. (Performance will be about the same either way.)

 [...] Thread safety is a tricky beast. I think we agree that the only
 way to guarantee thread safety is to only depend on final concrete
 classes that are thread safe themselves.
 I don't think so. When objects are immutable, thread-safety follows
 It is somewhat off topic, but a counter example would be Vector3D. Since
 the class is not final, a user could extend it and override all the
 methods and add some set{X,Y,Z} methods to make it mutable. Even though
 Vector3D is immutable, there is no _guarantee_ that every instanceof
 Vector3D is immutable unless it is final. This is why String is final.
 I think I don't get your point: If someone extends a class that is safe
 in a way that the extension is unsafe, that's his problem. ;-)

 [...] copying any large matrices or arrays is prohibitively
 expensive. For the NLLS package we would be copying a pointer to a
 function that can generate a large matrix. I think adding some
 documentation that functions should be thread safe if you want to use
 them from multiple threads would be sufficient.
 I you pass a pointer (i.e. a reference in Java), all bets are off:
 the
 class is not inherently thread-safe. That's why I suggested to
 mandate a
 _deep_ copy method (with a stringent contract that should allow a
 caller
 to be sure that all objects owned by an instance are disconnected from
 any
 other objects).
 As someone who has designed a thread safe application based on deep
 copying I don't think this is route to follow. A deep copy means you
 have to be able to copy an arbitrary (possibly cyclical) reference
 graph. Without the graph copy there are many subtle bugs. (References to
 the same object are now references to different objects.) With the graph
 copy the implementation is very complex

Re: [Math] Fluent API, inheritance and immutability

[Evan Ward]

Hi again,

Sorry for the confusion my code sample caused; it made sense in my mind.
:) I was speaking from the perspective that an abstract class is a
skeletal implementation of an interface, created so that implementing
the interface is easier. For the non-linear least squares (NLLS) package
I was thinking something like: https://gist.github.com/anonymous/6184665

To address a few concerns I saw,

Gilles wrote:
 In this way, it does not; what I inferred from the partial code above was
 that there would only be _one_ create method. But:
 1. All create methods must be overridden at the next level of the class
 hierarchy (this is the duplication I was referring to in the first
 post).

True, but concrete classes should not be extended. Adding another
abstract class to the hierarchy would mean implementing the create
method form the superclass and delegating to a new abstract create
method that includes the new parameters. The abstract class hierarchy
would have to parallel the interface hierarchy.

 2. When a parameter is added somewhere in the hierarchy, all the classes
 below must be touched too.

True, but since abstract classes are just skeletal implementations of
interfaces you can't add any methods without breaking compatibility
anyway. (You would have to add the same method to the public interface
too.) This does make it important to decide on a well written and
complete API before releasing it.

 And, we must note, that the duplication still does not ensure real
 immutability unless all the passed arguments are themselves immutable.
 But:
 1. We do not have control over them; e.g. in the case of the optimizers
 the ConvergenceChecker interface could possibly be implemented by a
 non-thread-safe class (I gave one example of such a thing a few weeks
 ago when a user wanted to track the optimizer's search path)

True. Thread safety is a tricky beast. I think we agree that the only
way to guarantee thread safety is to only depend on final concrete
classes that are thread safe themselves. This is directly at odds with
the inversion of control/dependency injection paradigm. I think a
reasonable compromise is to depend on interfaces and make sure all the
provided implementations are thread safe. A simple sequential user won't
need to care about thread safety. A concurrent user will need to
understand the implications of Java threading to begin with. Accurate
documentation of which interfaces and methods are assumed to be thread
safe goes a long way here.

 2. Some users _complained_ (among other things :-) that we should not
 force immutability of some input (model function and Jacobian IIRC)
 because in some use-cases, it would be unnecessarily costly.

I agree that copying any large matrices or arrays is prohibitively
expensive. For the NLLS package we would be copying a pointer to a
function that can generate a large matrix. I think adding some
documentation that functions should be thread safe if you want to use
them from multiple threads would be sufficient.

 Consequently, if (when creating a new instance) we assign a reference
 passed to the fluent method, we cannot warrant thread-safety anymore;
 which in turn poses the question of whether this false sense of security
 warrants the increased code duplication and complexity (compare your code
 below with the same but without the constructors and create methods:
 even a toy example is already cut by more than half).

 Agreed that thread safety can only be guaranteed with the help of the
user. The immutable+fluent combination does add an additional layer of
indirection. On the other hand it is much simpler to debug and analyze,
especially in a concurrent environment.

 Then if we really want to aim at thread-safety, I think that the approach
 of mandating a copy() interface to all participating classes, would be
 a serious contender to just making all fields final. Let's also recall
 that immutability is not a goal but a means (the goal being
 thread-safety).

 I still think that the three tools mentioned in the subject line do not
 play along very well, unfortunately.
 I was, and still am, a proponent of immutability but IMO this discussion
 indicates that it should not be enforced at all cost. In particular, in
 small and non-layered objects, it is easy to ensure thread-safety
 (through
 final) but the objects that most benefit from a fluent API are big
 (with
 many configuration combinations), and their primary usage does not
 necessarily benefit from transparent instantiation.

 Given all these considerations, in the first steps for moving some CM
 codes
 to fluent APIs, I would aim for simplicity and _less_ code (if just to be
 able to make adjustments more quickly).
 Then, from simple code, we can more easily experiment (and compare,
 with
 diff) the merits and drawbacks of various routes towards thread-safety.

 OK?

Agreed on the goal of thread safety. Is the copy a shallow copy? If it
is, then copy is a complete punt of thread-safety to the 

[Math] Fluent API, inheritance and immutability

[Evan Ward]

Gilles wrote:
 To address a few concerns I saw,

 Gilles wrote:
 In this way, it does not; what I inferred from the partial code
 above was
 that there would only be _one_ create method. But:
 1. All create methods must be overridden at the next level of the
 class
 hierarchy (this is the duplication I was referring to in the first
 post).

 True, but concrete classes should not be extended.

 Why not?
 [Anyways, the duplication also occurs in the intermediate (abstract)
 levels.]

 Adding another
 abstract class to the hierarchy would mean implementing the create
 method form the superclass and delegating to a new abstract create
 method that includes the new parameters. The abstract class hierarchy
 would have to parallel the interface hierarchy.

 With a mutable instance, you avoid this; that's the point.


 2. When a parameter is added somewhere in the hierarchy, all the
 classes
 below must be touched too.

 True, but since abstract classes are just skeletal implementations of
 interfaces you can't add any methods without breaking compatibility
 anyway.

 Adding a (concrete) method in an abstract class will not break
 compatibility.

 (You would have to add the same method to the public interface
 too.)

 There you break compatibility; that's why we removed a lot of the
 interfaces in CM, because the API is not stable, and abstract classes
 allow non-breaking changes.

 This does make it important to decide on a well written and
 complete API before releasing it.

 When the scope of the software is well circumscribed, that would be
 possible. With the whole of [Math]ematics, much less so. :-}
 And state-of-the-art in Java is a moving target, aimed at by changing
 CM contributors with differing needs and tastes; this adds to the
 unstable mix.

That's a good point. I still prefer the interface design (though I may
be in the minority) for two reasons. First, if a concrete class only
publicly exposes the methods defined in an interface it encourages
polymorphism. User code that uses one implementation can be easily
switched to another and new implementations are less constrained.
Second, it encourages composition over inheritance. I agree with Josh
Bloch that composition produces more maintainable code. Adding new
methods to an existing interface/class breaks composition.

I think the interface/abstract class discussion is partially separable
from the immutable/mutable discussion. I see the algorithm as the part
that could really benefit from the polymorphism. Perhaps separating the
problem definition (data) from the algorithm will improve the
flexibility of the API. For example,

PointVectorValuePair solveMyNLLSProblem(NLLSOptimizer opt){
//define problem to solve in an independent object
NLLSProblem p = new NLLSProblem(/*model functions, weights,
convergence checker, ...*/);

//provide algorithm with the data it needs
//algorithm has no problem specific state
return opt.optimize(p);
}


 And, we must note, that the duplication still does not ensure real
 immutability unless all the passed arguments are themselves immutable.
 But:
 1. We do not have control over them; e.g. in the case of the optimizers
 the ConvergenceChecker interface could possibly be implemented by a
 non-thread-safe class (I gave one example of such a thing a few weeks
 ago when a user wanted to track the optimizer's search path)

 True. Thread safety is a tricky beast. I think we agree that the only
 way to guarantee thread safety is to only depend on final concrete
 classes that are thread safe themselves.

 I don't think so. When objects are immutable, thread-safety follows

It is somewhat off topic, but a counter example would be Vector3D. Since
the class is not final, a user could extend it and override all the
methods and add some set{X,Y,Z} methods to make it mutable. Even though
Vector3D is immutable, there is no _guarantee_ that every instanceof
Vector3D is immutable unless it is final. This is why String is final.

 (but
 note that the current optimizers in CM were never thread-safe).
 But thread-safety can exist even with mutable objects; it's just that
 more
 care must be taken to ensure it.

 This is directly at odds with
 the inversion of control/dependency injection paradigm. I think a
 reasonable compromise is to depend on interfaces and make sure all the
 provided implementations are thread safe.

 Yes, that a way, but again easier said that done.

 A simple sequential user won't
 need to care about thread safety. A concurrent user will need to
 understand the implications of Java threading to begin with. Accurate
 documentation of which interfaces and methods are assumed to be thread
 safe goes a long way here.

 I don't think I'm wrong if I say that most concurrent bugs are found in
 production rather than in contrived test cases.
 [That's why I advocated for introducing real applications as use-cases
 for CM.]

 2. Some users _complained_ (among other things :-) that we should not
 force 

[Math] Fluent API, inheritance and immutability

[Evan Ward]

Hi,

As a grateful user of the library, I would prefer the Fluent API and
immutability. I have implemented this pattern in some of my own projects
and it makes the API very easy to use.

Most of the time, when the interface is small, inheritance isn't really
necessary and I just implement the withXxx methods directly. A one line
'return new ...' isn't hard to maintain with IDE support for renames and
add/remove parameters.

When inheritance is part of the API you can parametrize the base class
and add an abstract create method, like so:

abstract class ParentT extends Parent implements Interface{
protected abstract T create(...);
public T withXxx(x){
return create(x, ...);
}
}
class Child extends ParentChild {
protected T create(...){
return new Child(...);
}
}

The create method forces the implementing class to have an appropriate
constructor. The withXxx methods are implemented only once and the
inheriting class only has to implement one more method. The type
parameterization is invisible to the user if they use the Child class or
the Interface. The down side is that Child can not be subclassed
correctly, but this enforces the rule design for inheritance or prevent
it. This is similar to the pattern used in MarkupBuilder from jatl.[1] 
(jatl uses a mutable base class)

Regards,
Evan

[1]
http://code.google.com/p/jatl/source/browse/src/main/java/com/googlecode/jatl/MarkupBuilder.java

-
To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
For additional commands, e-mail: dev-h...@commons.apache.org