Re: [math] Proposal: move build to m2
+0 Al - Original Message From: Phil Steitz [EMAIL PROTECTED] To: Jakarta Commons Developers List commons-dev@jakarta.apache.org Sent: Wednesday, April 4, 2007 8:56:06 AM Subject: [math] Proposal: move build to m2 As we ramp up to a 1.2 release, I think its a good idea to move to Maven 2 as the standard build. That means deprecating the m1 build (leaving in the 1.2 release, if possible), moving the site and nightlies, etc. Lets view this as lazy consensus - i.e., scream now or forever hold your peace. Assuming no objections in the next couple of days, we can start battling - er, I mean working - with m2. Phil Sucker-punch spam with award-winning protection. Try the free Yahoo! Mail Beta. http://advision.webevents.yahoo.com/mailbeta/features_spam.html
[math] Sparse Grid Interpolation Toolbox
Never having heard of sparse grid interpolation, I found the following interesting. The terms of the license seem to be the same as Apache's, in case we wanted to port this toolbox to Java. Al The Sparse Grid Interpolation Toolbox is a MATLAB toolbox for recovering expensive, possibly high-dimensional multivariate functions. It includes hierarchical sparse grid interpolation algorithms based on both piecewise multilinear and polynomial basis functions. http://www.ians.uni-stuttgart.de/spinterp/ __ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: [RE-VOTE] Release Commons Math 1.1
+0 --- Phil Steitz [EMAIL PROTECTED] wrote: The problems reported with math 1.1 RC4 have been fixed. I would like to call for another release vote, based on the release candidate available here: http://people.apache.org/~psteitz/commons-math/1-1-rc5 Release notes are here: http://people.apache.org/~psteitz/commons-math/1-1-rc5/RELEASE-NOTES.txt and apidocs: http://people.apache.org/~psteitz/commons-math/1-1-rc5/apidocs/ The RC5 jar (commons-math-1.1-RC5.jar) has also been deployed to the apache internal maven repository at cvs.apache.org/repository Votes, please. The vote will close in 72 hours. Thanks in advance! Phil --- [ ] +1 I support this release and am willing to help [ ] +0 I support this release but am unable to help [ ] -0 I do not support this release [ ] -1 I do not support this release, and here are my reasons __ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: [VOTE] Release commons math 1.1
Thanks for carrying the ball on this release, Phil. Here's my +0 (sorry, too swamped with everything else in my life!). Al --- Phil Steitz [EMAIL PROTECTED] wrote: There have been no problems reported with math 1.1 RC4, other than some small javadoc fixes, which have been applied to the release branch. Assuming a positive vote, I will cut a new signed release, including these fixes. Release notes are here: http://people.apache.org/~psteitz/commons-math/1-1-rc4/RELEASE-NOTES.txt And apidocs: http://people.apache.org/~psteitz/commons-math/1-1-rc4/apidocs/ The RC4 jar (commons-math-1.1-RC4.jar) has also been deployed to the apache internal maven repository at cvs.apache.org/repository Votes, please. The vote will close in 72 hours. Thanks in advance! Phil --- [ ] +1 I support this release and am willing to help [X] +0 I support this release but am unable to help [ ] -0 I do not support this release [ ] -1 I do not support this release, and here are my reasons --- __ Yahoo! DSL Something to write home about. Just $16.99/mo. or less. dsl.yahoo.com - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: [Math] SingularValueDecomposition test case
Never having used or even learned anything in detail about SVD, my only suggestion is that equations 2.6.1 and 2.6.4 in http://library.lanl.gov/numerical/bookcpdf/c2-6.pdf are, as the text therein says, the only defining requirements for the SVD of any matrix. You could, as the text says, verify that the calculated SVD is correct by making up any matrix and checking whether the two above equations hold for the SVD you get out of your code. Al --- Kim van der Linde [EMAIL PROTECTED] wrote: Hi All, In the past, I have plugged the JAMA SVD class to the matrix class of commons.math. I do now want to run a test in it, so, the questio becomes, does someone has a known test case for me? Cheers, Kim -- http://www.kimvdlinde.com __ Yahoo! Mail - PC Magazine Editors' Choice 2005 http://mail.yahoo.com - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: [VOTE] Release Commons Math 1.1
--- Phil Steitz [EMAIL PROTECTED] wrote: --8- [ ] +1 Release Math 1.1 [X] +0 General support but not definitive [ ] -0 Unhappy about the release but not definitive [ ] -1 Do not release Math 1.1 I regret and apologize for not participating for quite some time, family and work have been unusually urgent concerns. Consider my vote supportive of (but largely uninformed) releasing 1.1, except for the impression I've gotten from skimming some of the list traffic and Bugzilla reports. Al __ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: [math] location for 1.0, 1.1 javadocs
--- Stephen Colebourne [EMAIL PROTECTED] wrote: Phil Steitz wrote: I am in process of rolling what I hope will be the final 1.1 RC. We need to settle the issue of where to locate the 1.0 and 1.1 javadocs. If there are no objections, I plan to rename the /api directory that contains the 1.0javadoc to api-1-0 and create a api-1-1 for the 1.1 javadoc, keeping /apidocs for current development. Any problems with this? I believe this is a good pattern, although using a - instead of a . is a little unusual. This is the JodaTime route, [collections] is slightly different: apidocs - cvs latest api-1.0 - version 1.0 api-1.1 - version 1.1 api-release - symbolic link to latest version (used on website) Stephen I prefer this latter naming convention using the .'s in the version numbers. Al Start your day with Yahoo! - make it your home page http://www.yahoo.com/r/hs - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: [math] JDK version
Right, many production installations are frozen at 1.3 for the foreseeable future, so we have to accommodate that. Al --- Brent Worden [EMAIL PROTECTED] wrote: We support 1.3 and higher. Brent Worden - Original Message - From: Zhang [EMAIL PROTECTED] To: Phil Steitz [EMAIL PROTECTED] Subject: [math] JDK version Date: Fri, 12 Aug 2005 18:12:15 -0700 (PDT) Do we have requirement or guidelines on JDK version? Functions such as Math.expm1() and Math.cosh() are available in jdk 1.5 but not in jdk 1.4.2. I'm now using jdk 1.4.2 for developing and testing. Xiaogang Zhang __ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: [math] SoC Commons-Math Status
Thanks, James. Looking forward to the code! Al --- James M Stephenson [EMAIL PROTECTED] wrote: I was worried about using any code from Numerical Recipies, so I have refrained from doing so and am using it as a reference in addition to Cantrell's book. The only thing I am using it for is the bits of theory here and there within it, not the algorithms. I am developing those myself. The NR reference is not being used for the compressed matrix class, I am not using anything for that apart from Cantrell's book and the Lapack reference manual. I am not reusing any code or algorithms from either book. I will post my JUnit tests and code this evening for review. James Al Chou wrote: Yes, good point, Martin. James, please see http://www.mail-archive.com/commons-dev@jakarta.apache.org/msg61147.html for an example of our exercising a decision we made early in Commons Math's history regarding code derived from _Numerical Recipes_ (in short, using _NR_ as a source for code without getting written permission violates their license, and their license is not compatible with the Apache License anyway). Also please see http://jakarta.apache.org/commons/math/developers.html for general information and guidelines for Commons Math contributors. I strongly recommend you post whatever JUnit tests and code you currently have, regardless of what shape it's in, either to the list or somewhere accessible via Internet, before continuing with any more coding. At this point it is going to be important to make sure you're not wasting your time; deriving code from _NR_ is not compatible with (probably most if not all) open source licenses, so it's not just an Apache Software Foundation issue. Al --- Henri Yandell [EMAIL PROTECTED] wrote: Especially Phil/Al as mentors. On 8/8/05, Martin Cooper [EMAIL PROTECTED] wrote: Hmm, I'm a little concerned at the reference to Numerical Recipes. The NR folks have strict limitations on how their work may be used. For example, see: http://www.numerical-recipes.com/infotop.html#distinfo Can someone more familiar with this chime in on the extent to which we may have issues with this, if any? -- Martin Cooper On 8/8/05, James M Stephenson [EMAIL PROTECTED] wrote: Compressed Matrix class and arithmetic components are coming along, I should have a good post soon. I've got a few more methods to write for the compressed matrix class. I hope to be finished with my project on 8/21 right now including getting ripped up by the community, I really should not have done the hardest part first (The compressed matrix class). It is really proving to be difficult to come up with a completely sound method of doing it, especially one that follows along the lines of commons development, I am using the Numerical Recipes set of books as a good reference along with Cantrell's Modern Mathematical Methods for Physicists and Engineers. I will definitely have a post and a patch to discuss this week, and if all goes well, I should have the decomposer done next week midweek. Sincerely, James __ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: [math] SoC Commons-Math Status
Yes, good point, Martin. James, please see http://www.mail-archive.com/commons-dev@jakarta.apache.org/msg61147.html for an example of our exercising a decision we made early in Commons Math's history regarding code derived from _Numerical Recipes_ (in short, using _NR_ as a source for code without getting written permission violates their license, and their license is not compatible with the Apache License anyway). Also please see http://jakarta.apache.org/commons/math/developers.html for general information and guidelines for Commons Math contributors. I strongly recommend you post whatever JUnit tests and code you currently have, regardless of what shape it's in, either to the list or somewhere accessible via Internet, before continuing with any more coding. At this point it is going to be important to make sure you're not wasting your time; deriving code from _NR_ is not compatible with (probably most if not all) open source licenses, so it's not just an Apache Software Foundation issue. Al --- Henri Yandell [EMAIL PROTECTED] wrote: Especially Phil/Al as mentors. On 8/8/05, Martin Cooper [EMAIL PROTECTED] wrote: Hmm, I'm a little concerned at the reference to Numerical Recipes. The NR folks have strict limitations on how their work may be used. For example, see: http://www.numerical-recipes.com/infotop.html#distinfo Can someone more familiar with this chime in on the extent to which we may have issues with this, if any? -- Martin Cooper On 8/8/05, James M Stephenson [EMAIL PROTECTED] wrote: Compressed Matrix class and arithmetic components are coming along, I should have a good post soon. I've got a few more methods to write for the compressed matrix class. I hope to be finished with my project on 8/21 right now including getting ripped up by the community, I really should not have done the hardest part first (The compressed matrix class). It is really proving to be difficult to come up with a completely sound method of doing it, especially one that follows along the lines of commons development, I am using the Numerical Recipes set of books as a good reference along with Cantrell's Modern Mathematical Methods for Physicists and Engineers. I will definitely have a post and a patch to discuss this week, and if all goes well, I should have the decomposer done next week midweek. Sincerely, James __ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
[math] FW: [interest in] SummerOfCode2005 commons-math project
Rostyslav, I'm sure we'd be happy to have you join the project. I've copied this message to the Jakarta commons-dev mailing list. Al -Original Message- From: Uzhgorod [mailto:[EMAIL PROTECTED] Sent: Friday, June 10, 2005 3:16 PM To: [EMAIL PROTECTED] Cc: [EMAIL PROTECTED] Subject: SummerOfCode2005 commons-math project Rostyslav Slipetskyy Dobrianskogo St, 10/21, Uzhgorod, Ukraine. Hi! I'm a 3rd year student of Computer Science Dept in Kyiv-Mogyla University (Ukraine, Kyiv). I'm interested in the commons-math project of SummerOfCode2005. I am one of three members of a university ACM programming team. ACM (Association of Computer Machinery) is a world-famous organization which guides World Programming Contests. In September our team will fight for a grant to the world semi-finals. During numerous contests, I could see the advantages of well-written code libraries. Moreover, our team felt huge unsufficience of : -- efficiently implemented -- fully documented libraries which cover a wide range of different algorithmic areas. That is why we started to implement Graph Library, which was our student course paper at our university. The experience we gained, I really hope, will help me to do well if I have the opportunity to take part in the commons-math project. We wrote our Graph Library using C/C++, but personally I know Java syntax, and some percent of the contest problems during ACM contests we write on Java. Really a huge amount of ACM contest problems have in general mathematical background. That is why I think I obtain some mathematical skills. Frankly speaking, I have rather poor knowledge of Statistics :( , but I think I will manage to bridge this gap :) I'm really very interested in this project. It will a great chance for me to try my programming skills in a real challenge. Please, give me to know if the commons-math project is still available, if my abilities and knowledge satisfy the needs to do this project, and if it is possible for me to apply for this project. Thank you! -- Rostyslav __ Discover Yahoo! Stay in touch with email, IM, photo sharing and more. Check it out! http://discover.yahoo.com/stayintouch.html - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: [math][all] Mentoring for Google's Summer of Code program
--- Phil Steitz [EMAIL PROTECTED] wrote: On 6/3/05, robert burrell donkin [EMAIL PROTECTED] wrote: On Thu, 2005-06-02 at 22:58 -0700, Al Chou wrote: [deletia] I think the modified scope specified later in this thread is in scope for j-c-m. I agree that the scope of the initial proposal goes beyond j-c-m, or j-c or even Jakarta (the C# port). I like the idea of using the more modest extensions here to build community, then maybe play a bit over the boundaries - including even C# - in the sandbox and then think about incubating apache math. +1 It is nice to see our original apache sponsor still willing to help move us along :-) Agreed! For the moment, however, I think it is best to propose this as an extension to j-c-m. +1 IMHO one of the first jobs of the mentors will need to do will be to study the proposal and create a realistic core tasks together with a more ambitious list of additional tasks. (this should ensure that ryan is not unfairly financially penalised for his ambition.) it's hard to estimate a priori what the quality and quantity of any code produced by ryan will be but i agree that the code will need to satisfy the current standards especially in the area of development best practice. the mentors will need to work with ryan to ensure that he knows what's expected particular when it comes to using code developed by others. Agreed. That is what I am signing up for. Fortunately, the current code base, the developer guide and various other apache docs give good guidance on code and IP requirements. I will follow up with the SoC organizers on the whole issue of managing scope and deliverables (in some ways antithetical the apache way) because I agree that this could be onerous and we want to ensure that 1. Ryan and any other students who participate have a clear understanding of what is expected and how much they are signing up for 2. The quality of what is committed meets the standards of j-c-m 3. We do not introduce any IP issues Great! Also, specifically what parts of the proposal are not already addressed by existing libraries such as Colt? I thought we were over the licensing hurdle for Colt, so anything it already does probably should not be duplicated by code newly written for the proposed project. This may not be a consensus or popular view, but even if/when we eventually become apache math, I will be opposed to annexing code without community. So, regarding Colt in particular, if the Colt community decides to join us at some point and wants to do a software grant, go through the incubator, etc., that will be great. Until such time, I think it is fine for us to duplicate functionality with j-c-m APIs and j-c-m volunteers for things that fit within our scope. What is essential to me is that we have developers actively supporting all code that comes in. Anything that we commit, we need to be able to support. Phil, that's an interesting point, which I hadn't considered. Here's an off-the-cuff counterproposal: let us annex the parts of Colt that we need, when we need them, if that seems the right thing to do for the particular need at the time. That way we will ensure community exists (our own) for the code that is imported. Al __ Do you Yahoo!? Yahoo! Mail - Find what you need with new enhanced search. http://info.mail.yahoo.com/mail_250 - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: [math][all] Mentoring for Google's Summer of Code program
--- robert burrell donkin [EMAIL PROTECTED] wrote: On Thu, 2005-06-02 at 22:58 -0700, Al Chou wrote: Questions of scope of Commons-Math aside gasp, what is the scope of the summer project? The scope of Ryan's proposal seems mighty ambitious, even if you remove the parts that aren't currently in scope for Commons-Math. i think that it would be useful to separate the question of scope from the question of mentoring and community. i definitely agree that the commons-math group is the right one for mentoring this code. i also agree (with simon) that jakarta commons is the best community for growing this code (whilst math remains here, of course). +1 in terms of scope, IMHO there are two separate questions: is the proposed codebase in scope for the jakarta-commons and is it in scope for the commons-math component. opinions on these questions welcomed :) if the codebase is not in scope for jakarta-commons then i'd be willing to sponsor this as an incubator project (when it is ready). it could be developed in the sandbox until then. if the codebase is in scope for jakarta-commons but not in scope for commons-math then one solution would be to factor the out-of-scope code into a separate component with it's own scope. so, i see no reason why the project couldn't be pushed forward whilst the questions of scope are argued about. I think for the project to succeed in any case, its scope will have to be constrained quite a lot. Ryan's post that preceded this one seems quite good in limiting the scope, and I argue that it's close to being in scope for Commons-Math now. If you intend to stay true to Apache's charter, the code will have to be developed firsthand, not copied from any other source, unless the license of the source is compatible or the author(s) are amenable to adding an Apache-compatible alternative license to their code or changing over to an Apache-compatible license. Coding that much from scratch and providing JUnit tests will be a heck of a lot of work (maybe a little onerous if you do TDD). And forgive my skepticism, but code developed for coursework is unlikely to be as bulletproof as the proposal aims for. it does sound ambitious (but there's nothing wrong with that) IMHO one of the first jobs of the mentors will need to do will be to study the proposal and create a realistic core tasks together with a more ambitious list of additional tasks. (this should ensure that ryan is not unfairly financially penalised for his ambition.) it's hard to estimate a priori what the quality and quantity of any code produced by ryan will be but i agree that the code will need to satisfy the current standards especially in the area of development best practice. the mentors will need to work with ryan to ensure that he knows what's expected particular when it comes to using code developed by others. Also, specifically what parts of the proposal are not already addressed by existing libraries such as Colt? I thought we were over the licensing hurdle for Colt, so anything it already does probably should not be duplicated by code newly written for the proposed project. All the above said, I'm not trying to be discouraging, and in fact I would be willing to participate in mentoring (I'll probably learn something new myself!). I just want reasonable expectations and goals to be set. And perhaps we can all ride the wave of youthful exuberance to grow a library that looks like the beginning of an Apache-Math-Java. +1 :) g back at ya. Al __ Discover Yahoo! Get on-the-go sports scores, stock quotes, news and more. Check it out! http://discover.yahoo.com/mobile.html - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: [math][all] Mentoring for Google's Summer of Code program
Ryan, Your list below seems much more achievable. I suggest cutting it even a bit further, say, dropping Gaussian quadrature as you already hint at (because implementing it would drag in some of the special functions that you note we don't have) and the ODE solver (unless all you mean is a simple initial value problem integrator, say 4th order Runge-Kutta). I'm not familiar with B-spline or downhill simplex; univariate minimization is very close to root finding, as I'm sure you know, so I don't have a problem with including that. And to actually expand the scope, maybe we could add rational function interpolation as well as polynomial. To keep a flavor of typical software development in this project, I would also be interested to see the code be evolved from simple/naive implementations to the more sophisticated ones, at least for some parts of the project. Implementing a sophisticated algorithm from a reference work may feel good, but I argue you'll learn more about both the mathematics and about software development by taking baby steps and experiencing for yourself the reasons the more sophisticated algorithms became necessary or at least desirable. In other words, it's OK to write a naive and slow but correct implementation and then modify it toward some more sophisticated version. Finally, I wanted to clarify that I mistyped in my last post: I meant to say that providing unit tests might be _less_ onerous if the code were done via TDD. Testing an existing algorithm was always a pain to me early in my graduate career, and I learned through hard experience that it's necessary. Since then TDD has sprung up, and doing it actually makes writing tests (or executable examples, to be more accurate) feel good. Al --- Ryan Li [EMAIL PROTECTED] wrote: Hi all, Many thanks for your comments so far (please keep them coming!). I do realise that the proposal will probably take a number of specialised numerical analysts a few months if not years to complete. I'm all for cutting things down a bit (a lot rather), with an initial focus on the more classical methods for which efficient algorithms are well documented and can be implemented relatively easily. Seeing commons-math as a generic and compact library based on which other more sophisticated code can be written, I think the following could be useful: - performance tuning for matrix computation and add more decomposition methods - univariate integration (Romberg is what I have in mind) - true polynomial interpolation, as opposed to spline, it will be usually be used by other higher level numerical algorithms e.g. Romberg integration or other Richardson extrapolation procedures - univariate minimization - robust derivative-free multidimensional optimization (downhill simplex) - B-spline - ODE solver - (maybe?) Gaussian quadratures and then possibly move to the more ambitious part. At some point I would also like to add more special functions, and optimise the performance of existing ones. Please let me know what you think about this plan. I am actually a happy user of Colt myself, to me it seems that it focuses on the data structure side of numerical computation (which it does an excellent job), what I find it lacks specifically are the algorithm side of things like root finding, interpolation, integration, optimization (and possibly) ODE solver, the performance of some of the special functions also left something to be desired after they replaced the old IMSL code. There are scattered efforts on all the things mentioned above, some seem to be quite mature but most are experimental, I think it would be nice to have them all in one library (maybe just adapt if license compatible and code reasonably stable). As for code quality, I'll make sure I do extensive research into the literatures and (probably more important) existing implementations in whatever language before getting my hands on the keyboard, part of the reason I wanted to start such a project is that when I was doing coursework, I referenced many books and papers only to realise that some of the ideas were clearly better than what I was suggested to do in the project! Best regards, Ryan On 6/3/05, Rory Winston [EMAIL PROTECTED] wrote: I agree, it is a hugely ambitious project. Which is not necessarily a bad thing. I think a good start would be to qualify the scope of the project sooner rather than later, and get a firm idea for exactly what will (hopefully) be achieved. Jakarta Commons Developers List commons-dev@jakarta.apache.org wrote: Questions of scope of Commons-Math aside gasp, what is the scope of the summer project? The scope of Ryan's proposal seems mighty ambitious, even if you remove the parts that aren't currently in scope for Commons-Math. If you intend to stay true to Apache's charter, the code will have to be developed
Re: [math][all] Mentoring for Google's Summer of Code program
Questions of scope of Commons-Math aside gasp, what is the scope of the summer project? The scope of Ryan's proposal seems mighty ambitious, even if you remove the parts that aren't currently in scope for Commons-Math. If you intend to stay true to Apache's charter, the code will have to be developed firsthand, not copied from any other source, unless the license of the source is compatible or the author(s) are amenable to adding an Apache-compatible alternative license to their code or changing over to an Apache-compatible license. Coding that much from scratch and providing JUnit tests will be a heck of a lot of work (maybe a little onerous if you do TDD). And forgive my skepticism, but code developed for coursework is unlikely to be as bulletproof as the proposal aims for. Also, specifically what parts of the proposal are not already addressed by existing libraries such as Colt? I thought we were over the licensing hurdle for Colt, so anything it already does probably should not be duplicated by code newly written for the proposed project. All the above said, I'm not trying to be discouraging, and in fact I would be willing to participate in mentoring (I'll probably learn something new myself!). I just want reasonable expectations and goals to be set. And perhaps we can all ride the wave of youthful exuberance to grow a library that looks like the beginning of an Apache-Math-Java. Al --- Phil Steitz [EMAIL PROTECTED] wrote: I have volunteered to help Ryan with the proposal below and to serve as a mentor if it is accepted. Pls see the links infra and the general Wiki page (http://code.google.com/summerofcode.html) for more info on Google's Summer of Code. With Ryan's permission, I am forwarding the proposal. The content as stated clearly goes beyond the current scope of [math], but several items are in scope. Comments, please. After a little discussion - including maybe the inevitable reopening of the what do we want to be when we grow up - assuming others are favorable, I will work with Ryan to get something suitable onto the Wiki for consideration as one of the apache projects. Comments from all commons community members are welcome. Thanks! Phil -- Forwarded message -- From: Phil Steitz [EMAIL PROTECTED] Date: Jun 2, 2005 4:50 AM Subject: [Fwd: Mentoring for Google's Summer of Code program] To: [EMAIL PROTECTED] Original Message Subject: Mentoring for Google's Summer of Code program Date: Wed, 1 Jun 2005 07:23:25 -0700 (PDT) From: Ryan Li [EMAIL PROTECTED] To: [EMAIL PROTECTED] Dear Phil, I'm just wondering if you or other developers of the The Jakarta Commons Math library would be interested in mentoring me for the Google Summer of Code program (more details at http://code.google.com/mentfaq.html and http://code.google.com/summerofcode.html). Just so you know that the Apache Software Foundation is already a mentoring organization (http://wiki.apache.org/general/SummerOfCode2005). Please see below for my draft project proposal (as you can see I've already mentioned the Jakarta Commons Math library in it and I've put down ASF as my mentor, so you or other team members will probably be contacted by Google soon). Thank you very much for your consideration! Ryan Du Li A college student specialising in optimization and numerical analysis -- Project description: Numerical Library for Financial Modelling / Scientific Computation in Java (with a C# port) The goal of this project is to develop a reusable toolkit for developing mathematical models for valuing financial products and risk management. Due to its numerical intensive nature, it can also be used effectively in other applications that make uses of highly efficient numerical algorithms. Specific language features and design patterns will be used aggressively to allow for high performance and reusability. The C# port will be made in parallel as the Java version is developed, a number of features of C# (up to version 2.0) will be used, in particular operator overloading will be used for all matrix operations. Highly efficient numerical algorithms will be implemented in the following areas: 1. a generic matrix and linear algebra package (including support for dense and sparse matrices and commonly used decomposition methods), an existing library might be used for this part of the toolkit. 2. an approximation package, with an emphasis on the support for B-splines, which is used to approximate curves in general, given a set of points on the curve. Very useful for any sort of data fitting, and heavily used in term structure modelling in finance 3. generic PDE/ODE solvers, with built-in support for Poisson/heat/wave equations. Useful in a large number of situations in modelling. 3. an optimization package, including linear programming (simplex method, later also interior
Re: [math] FFT
Hi, Bear, Thanks for your contributionary sentiment. Unfortunately, we cannot accept code based on _Numerical Recipes_, given their license. --- Bear Giles [EMAIL PROTECTED] wrote: Cleaning out some old files from a grad class. I have one- and two-dimensional FFT code in C, ported to Java. It's from the Numerical Recipes book so it's limited to 2^n. It needs unit tests and test cases. FFT begs the question of generalizing to Discrete Wavelet transformations (DWT). If I can find my notes I'll also be able to contribute multigrid code. This is a fairly efficient algorithm to solve PDQs. I'll leave it to others to add support for parallel- and clustered-processing. Bear While this stuff is cool, I personally find myself having to hold back when I think about adding this kind of material, given the project's charter of being a library of lightweight, self-contained mathematics and statistics components addressing the most common practical problems not immediately available in the Java programming language. I think FFT's are so widely used (where they're used at all) that they're a reasonable if not completely obvious candidate for inclusion in Commons-Math. I'm not convinced that PDE solution algorithms are, though. If the library were a full-fledged mathematical/scientific library, I would have no hesitation at all about this material, I assure you. Sorry to be a wet blanket! Al Albert Davidson Chou Get answers to Mac questions at http://www.Mac-Mgrs.org/ . __ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: [math] 1.0 changes to support JDK 1.3
--- Phil Steitz [EMAIL PROTECTED] wrote: I just discovered that we have some JDK 1.4 dependencies in the 1.0 release code. I would like to make the following changes to allow compilation on 1.3. Diffs showing the changes are at the bottom of this message: 1) Eliminate the use of JDK 1.4 exception nesting in MatrixIndexException, InvalidMatrixException (making these exceptions no longer support nesting). 2) Eliminate use of java.beans.Expression in CertifiedDataAbstractTest 3) Move BeanTransformer and BeanTransformerTest to /experimental (they are both in /test now). These changes will allow compilation on 1.3, but MathExceptionTest.testSerialization will fail because the stack traces are not serialized. To fix this, we need to implement custom serialization and I personally am OK with either deferring this to 1.1 or leaving as is. If there are no objections, I will commit these changes and push forward with the release. -Phil Sounds good, Phil. Way to go! Al - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: [vote][math] Release Math 1.0
I agree, move forward with the release. In case I forgot to vote before, here's my +0 (my company is releasing a major new version this month, and I'm selling a house and buying another, so I have no time!). Great job getting this release out! Al --- Mark R. Diggory [EMAIL PROTECTED] wrote: You Rock! My opinion is yes, we can move forward and continue the release. Finding and correcting bugs is part of the process. Expect to see minor releases next year, expect issues we can't see right now. This current phase of stability needs to be captured in a release. Release often, release early, -Mark Phil Steitz wrote: I was about to close this vote as passed when I got a private email indicating a small but functionally important bug, which I have now fixed (PR #32531, ChiSquareTestImpl was incorrectly rejecting tables containing 0's). Do we need to restart the vote, or can I close it as successful and cut the release? -Phil Phil Steitz wrote: There have been no bug reports against commons-math-1.0-RC2, which has been out for a couple of weeks now. I propose that we release commons-math-1.0, based on CVS HEAD, tagged and renamed for 1.0. There have been no changes to [math] since 1.0-RC2 was cut, which is available here: http://www.apache.org/~psteitz/commons-math-1.0-RC2/dist/ Votes, please. Voting will last at least 72 hours. Thanks. --- [ ] +1 I support this release and am willing to help [ ] +0 I support this release but am unable to help [ ] -0 I do not support this release [ ] -1 I do not support this release, and here are my reasons --- Here is my +1. -- Phil Steitz - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: [math][vote] Release 1.0-RC2
Oh, yeah. g +1 Thanks for all your work, Phil! Al --- Phil Steitz [EMAIL PROTECTED] wrote: nudge/ -Original Message- From: Phil Steitz Sent: Sun 11/7/2004 6:06 PM To: Jakarta Commons Developers List Cc: Subject: [math][vote] Release 1.0-RC2 This vote is to approve the public release of commons math 1.0-RC2. This release candidate incorporates community feedback on RC1. Like RC1, this will be a publicly announced RC to enable full feedback for a final 1.0 release in about two weeks if all is well. A complete list of changes since RC1 is available here: http://www.apache.org/~psteitz/commons-math-1.0-RC2/docs/changes-report.html The release candidate distribution files are here: http://www.apache.org/~psteitz/commons-math-1.0-RC2/dist/ The updated web site is available here: http://www.apache.org/~psteitz/commons-math-1.0-RC2/docs/ Voting will last at least 72 hours. As always, it would be useful if others could check the md5 checksums and asc signatures. Thanks, Phil - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: [math] R-based tests
--- Phil Steitz [EMAIL PROTECTED] wrote: I have used R to compute target values for statistics (and other things) in test cases where certified data tests are not available. To make these tests repeatable, I have been saving scripts that can be executed in R using the source() function. I would like to add these scripts to CVS somewhere. Any ideas as to where they should go? These are not really executable and don't need to be accessible to the build or JUnit tests. My first instinct is to place them just as you would if they were JUnit classes, just so they're easy to find by what they're related to. Al - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: [math] Questions regarding probability distributions
--- F Norin [EMAIL PROTECTED] wrote: Do you have any references for the quantum physics cases? I certainly didn't specialize in quantum physics (plasma physics typically uses almost everything _but_ quantum physics), but I did get as far as a EE graduate course in QED and never encountered such probability models. Maybe it's because I wasn't in a physics department or ever really encountered molecular models in what I was studying? I don't have any references handy (not my area of expertise), but it's often mentioned as an application area in articles dealing with these distributions. For instance, I guess physics models that uses Brownian motion models could use these distributions as they do show up in connection with Brownian motion theory. Thanks. My engineering physics curriculum was a little non-standard (when compared to physics departments' programs), and the statistical thermodynamics course didn't actually cover statistical mechanics (hence the peculiar course title, I suppose), which is where I presume one would ordinarily encounter the analysis of Brownian motion. Honestly, I doubt most of the users of Commons Math will be needing this kind of distribution, but I guess if we merge in (parts of) Colt, we might end up attracting that kind of user. As a matter of fact, many concepts in probability theory that at first sight may seem rather obscure can actually be used for practical applications. Stochastic modeling of the financial markets is a good example where very advanced mathematics is actually put to practical use. Ah, interesting point. I only just learned a teeny bit about integration of stochastic equations in portfolio value projection from a couple of C/C++ Users Journal articles earlier this year. A graduate complex analysis course I sat in on just the first week of started out with numerous examples of applications to reasonably real-world problems that I never would have thought of (not the ones I learned about in other courses that used complex analysis, for sure). Math certainly does have an uncanny way of connecting both with other areas of itself and the real world. I think Commons Math will be unusually hard pressed (compared to other math libraries) to choose what we feel are the most useful / commonly used features for inclusion in the library. If we were working on a more general purpose math library, the constraints obviously wouldn't be as tight. Al - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: [math] Questions regarding probability distributions
Hi, Frank,
--- F Norin [EMAIL PROTECTED] wrote:
Note: There are also distributions that are neither discrete, continuous
or a mixture of the two. For example, there are numerous distributions
based upon the Cantor ternary sets.
Practical counter-examples like what you have above are more compelling ;-)
Sure, a standard example of a Cantor ternary set distribution can be created
like this:
[deletia]
Distributions like this are actually used in practical applications as
probability models within fields such as biophysics, molecular biology and
quantum mechanics.
Do you have any references for the quantum physics cases? I certainly didn't
specialize in quantum physics (plasma physics typically uses almost everything
_but_ quantum physics), but I did get as far as a EE graduate course in QED and
never encountered such probability models. Maybe it's because I wasn't in a
physics department or ever really encountered molecular models in what I was
studying?
Honestly, I doubt most of the users of Commons Math will be needing this kind
of distribution, but I guess if we merge in (parts of) Colt, we might end up
attracting that kind of user. In any case, I learned something new today,
which is cool.
Al
[Proving that X is a probability distribution isn't exactly trivial- it
requires rather advanced concepts from measure theory, see Chung, A course
in Probability Theory, Academic Press (1974), p.12-13, for a rigorous
treatment of this.]
but if you want a completely generic and typesafe definition you should
go for something like
public interface ProbabilityDistribution {
public Probability distributionFunction(Number x);
}
I think we can make it work with doubles and don't see a big loss there. I
guess this is where I get off the bus ;-) -- though I see your point.
Ok, but I do think there is a strong case for having a separate Probability
class.
/Frank N
-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
RE: [math] Questions regarding probability distributions
--- Phil Steitz [EMAIL PROTECTED] wrote: Frank, After reading carefully again and thinking about some practical examples, I agree that the current framework has a fundamental and unecessary limitation. The point mass at 0, continuous beyond 0 example below does occur in practical applications (e.g. component lifetimes, 0 = defective). As I said in a previous post, the distributions package was designed to house commonly used parametric distributions like the ones that are implemented now; but there is no reason that the framework could not be used to support any kind of distribution. Therefore, since the change to add a base interface is small and does not really complicate the structure or client code, I am +0 for adding it. Any other opinions on this? +0 from me as well. Al - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: [math] Questions regarding probability distributions
--- Phil Steitz [EMAIL PROTECTED] wrote: Thanks for the feedback and the contribution! See comments interspersed below. [EMAIL PROTECTED] wrote: Hi I've been looking around for open source mathematical statistics software in Java in the last couple of weeks and have tested the commons-math package. After looking into it I have a few questions and comments. 1. The MathUtils.factorial(final int n) method throws an IllegalArgumentException for n=0 which is clearly incorrect since 0!=1 by definition. Some may disagree with that definition (really amounts to a convention on how to handle vacuuous products); but I think that I agree with you. Unless others object, I will consider this a bug and make the change. Thanks for pointing this oddity out. Would you mind opening a bugzilla ticket to track this? http://issues.apache.org/bugzilla/enter_bug.cgi?product=Commonscomponent=Math factorial( 0 ) === 1 by definition, as stated above. Any definition of factorial that differs from that is not mathematically correct and will surprise those who expect the mathematical definition. So, +1 to making the change. 3. Wouldn't it be nice to have convenience methods for calculating the moments for each distribution? Something along the lines of public double getMoment(int order) throws MomentDoesNotExistException; for calculating the moments EX^order (if they exist)? At least there could be methods for obtaining the mean and variance of a distribution. Yes!! We thought about adding these to the interfaces but were concerned that in some cases they would be hard to implement (or impossible -- if they don't exist ;-) There is nothing preventing us from adding them to the implementations, however, or adding another interface or utilities to compute them. Does anyone see a reason that we need to add them now, before releasing 1.0? Not being a statistician, I think I can only vote -0 (not -1) to adding them at this time, in good conscience. 4. Since the chi-squared and exponential distributions are just special cases of the gamma distribution, there is no need to have separate implementation classes for these. In my opinion, one should avoid having multiple implementations of the same distribution unless there is some strong reason for it. Well, the chi-square implementation wraps a gamma instance. The exponential implementation computes the exponential density explicitly. In any case, the distributions are different (though related) and sufficiently useful in their own rights that exposing them separately will be easier for users. I agree they should remain separate. Although we have yet to really see who our users are, it seems reasonable, given Commons Math's charter, to keep ease of comprehension/use as one of our goals. Mathematical sophistication should not be a prerequisite (or lack thereof present a barrier) to entry into using this library. Al - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: [math] RC2 Release Plan
--- Phil Steitz [EMAIL PROTECTED] wrote: Phil Steitz wrote: 4) Add the following new methods to both RealMatrix and BigMatrix interfaces: RealMatrix getSubMatrix (int startRow, int endRow, int startColumn, int endColumn) RealMatrix getSubMatrix (int[] rows, int[] columns) RealMatrix getRowMatrix(int row) RealMatrix getColumnMatrix(int row) RealMatrix createRowMatrix(double[] row) RealMatrix createColumnMatrix(double[] column) Any objections to putting the last two methods above in a MatrixUtils No objection, that makes sense to me. class and change the names to makeXxx? Despite being longer, I like the originally proposed method names better. Al - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: [math] Matrix subMatrix and mean methods
--- Mark R. Diggory [EMAIL PROTECTED] wrote: Wolfgang Hoschek wrote: Mark, I am not worried about fracturing. My understanding is that you wouldn't start doing colt releases build by apache, right? You'd rather take some parts or derivatives of the code and add them to commons-math CVS as you see fit (probably in a significantly refactored/modified form). I thought that was the overall idea, and it seems to me a good one. We have discussed a Larger project at Apache that would be more expansive than what is currently capable in the Jakarta Commons. Whether or not such a project takes off has been limited more by uncertainty of what its boundaries would be. If all goes well, Colt as an external library could fade into oblivion in a year or two, or whatever time it takes for math to really shape up. That's absolutely fine with me. Do you suspect that you or CERN will maintain historical archiving of the Colt project indefinitely if it did? This is one of the characteristics of ASF, we maintain historical CVS and distribution archives for all the projects that have existed in Apache. So, if Colt moved into Apache, it would have a long term, canonical archival home for its existence even if it's contents were integrated into Apache Math. If it was not your interest to maintain it any longer, then such an option would be beneficial for the both historical referencing and reusage. As far as myself as project member/committer: Thanks for the kind invitation, but I'm really busy with other things these days. However, I can be there for you in case you'd have questions about how/why things work the way they work in colt. To reach me, please make sure to cc me; commons-dev is a high volume mailing list. Of course, we would enjoy any involvement you can provide. I understand your career focus may be in different directions now. I have no particular opinion on (A), it's up to you. B) sounds confusing to me as it might indicate to users that they should use that library instead of commons-math. Why have a jakarta level project if all you want is to take some parts or derivatives of it and integrate them in math? Regards, Wolfgang. Codebases have been donated to Apache Jakarta in the past which maintain similar capabilities. For instance, Oro and Apache Regexp were projects from different development groups, the code donation was to, in my belief, provide a mechanism to merge these projects at a time in the future, taking the best of both worlds. http://jakarta.apache.org/oro/index.html http://jakarta.apache.org/regexp/index.html I do adimately believe we would not want to consider releasing a full version of Colt unless you were interested in migrating the entire project into Apache, this is what I mean by fracturing, we do not want to see two separate Colt communities competing for membership, this would not be in either of our interests, would confuse the userbase and not allow us to build a shared development community. I agree we should not be releasing Apache versions of the whole Colt library (which technically wouldn't even be possible, as the hep.aida.* packages are LGPL'd, not under the the new CERN license). In any case, the question of the scope of Commons-Math continually comes up, and the merging in of Colt is yet another impetus for discussing it. If we were to include large portions or all of the CERN-licensed code of Colt, we would be in a position to claim a much larger scope/charter for the project. But would we want to? Some project members seem very interested in doing so (and I am certainly guilty of such thoughts), but I don't think it necessarily makes sense. Commons-Math's charter is a sound one, and I would not want to alienate/inconvenience users whose needs are well met by its current scope and charter (assuming there are any such) by increasing it unnecessarily. A separate project would probably be more appropriate if we wanted a larger scope. Al - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: [math] Matrix subMatrix and mean methods
--- Mark R. Diggory [EMAIL PROTECTED] wrote: Al Chou wrote: I agree we should not be releasing Apache versions of the whole Colt library (which technically wouldn't even be possible, as the hep.aida.* packages are LGPL'd, not under the the new CERN license). In any case, the question of the scope of Commons-Math continually comes up, and the merging in of Colt is yet another impetus for discussing it. If we were to include large portions or all of the CERN-licensed code of Colt, we would be in a position to claim a much larger scope/charter for the project. But would we want to? Some project members seem very interested in doing so (and I am certainly guilty of such thoughts), but I don't think it necessarily makes sense. Commons-Math's charter is a sound one, and I would not want to alienate/inconvenience users whose needs are well met by its current scope and charter (assuming there are any such) by increasing it unnecessarily. A separate project would probably be more appropriate if we wanted a larger scope. Al Al, I really agree that merging or adding much of the code from Colt can result in a project outside the scope of Commons Math. (Not that this can't stop us from using portions of it within Commons Math initially, which I promote). I really perceive the need for a parent project that manages numerical and mathematical codebases that are considered to be outside the scope of Commons Math. I agree that there are concerns with the hep LGPL packages and suspect they would not be allowed into Apache due to these concerns. Yes, I agree that Colt has much that would be useful to us in Commons Math, e.g., templated multi-dimensional matrices, both dense and sparse. But we should draw a boundary somewhere that Commons Math should not cross as we incorporate Colt code. For instance, I suspect most Java server-side programmers will need special functions beyond what we already provide (largely just to support our statistics functionality). However, the function objects in that same package are intriguing and possibly quite useful to us and our users. I think a minority of classes from cern.jet.random would be valuable, the rest being too advanced for our scope. The templated lists and maps look neat, but I don't know whether we really need them, nor whether Commons Math is really the place for them anyway (somewhere else in Commons may make sense, say Lang?). And the concurrent programming stuff is really for high performance computing, which I'd say is out of scope for Commons Math. Surprisingly, I think I've just drawn the boundary, at a high level. Or at least a straw man proposal of one. Thoughts? Al - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: [math] Matrix subMatrix and mean methods
--- Kim van der Linde [EMAIL PROTECTED] wrote: Phil Steitz wrote: I agree here as well. Do you see use cases where you will want to start with a double[][] array, perform matrix operations on it (say some decomposition) and then run stats on the resulting matrix? Will it be too onerous to make copies of the double[][] at each stage? If so, we need to think about access to the double[][]. I think keeping getData() will do the job. I generally do not like to store the result of a calculation in the original matrix, it always confuses me. If the content is of a different type, the name should be different. Is there the intention to add an EigenvalueDecomposition class to the package? In that case, which algorithm is targeted to be used? Jacobi? We have not discussed this. Feel free to add to the wish list and propose an algorithm. We also need to discuss how best to implement the strategy pattern for decompositions in general, as there will often be multiple algorithms to choose from. I have plugged in the EigenvalueDecomposition class from JAMA, which works fine for the PCA and CPCA modules. They do have a whole set of decompositions, and they might be easy to implement in the package (if they are free to be used). Colt contains a port of much of JAMA, IIRC, and as Wolfgang posted not too long ago, Colt's new license is much more compatible with the Apache license. I'm not sure if it's sufficiently compatible for us to start merging in code, though. Al - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: [RESULT][VOTE] Release Math 1.0
--- Phil Steitz [EMAIL PROTECTED] wrote: There were not enough +1 votes to proceed with the release. Bug fixes were also applied during the vote. Therefore, we cannot proceed with the release at this time. Three issues were reported with the release package: 1) Extraneous files (forgot to do clean co before release build - my bad) 2) Failing tests for French locale (pr #31325 - fixed in CVS) 3) Integer values not handled correctly by Frequency class (fixed in CVS) The vote thread also included discussion of repackaging and/or API changes. At this point, we can a) Cut a clean release candidate including the fixes applied during the vote and restart the release vote b) Cut an RC2 based on current CVS, vote on releasing this as a test release, similar to RC1. Follow with a 1.0 vote two weeks later. c) Reopen discussion of API changes, agree on release plan, bundle changes into RC2, then do b). My preference would be a); but if others have reservations, I am open to the other options as well. I would appreciate it if committers and other interested parties would weigh in with their opinions. Thanks in advance. I synched to CVS and browsed the class hierarchy. Maybe I'm a naive non-statistician, but I don't see a major problem and thus vote for (a) as well. Al - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: [math] Maven changes plugin
--- Phil Steitz [EMAIL PROTECTED] wrote: I just added a changes.xml file to track changes between releases. Does anyone have a problem with using this? It will (hopefully) make preparing release notes easier. I like it; I'll just have to make sure I remember to update the file when I make changes. Not being a production Java programmer or Maven user, I don't know if that's common practice the way writing submission notes when checking code into a revision control system is. Al - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: [VOTE] Release Math 1.0
--- Phil Steitz [EMAIL PROTECTED] wrote: RC1 has been out for almost 2 weeks now and there has been just one bug reported, which has been fixed in CVS. I propose, therefore, that we move forward with the official 1.0 release. An updated release candidate is available here: http://jakarta.apache.org/~psteitz/commons-math-1.0/ Here is my +1. -Phil [ ] +1 I fully support this release [X ] +0 I give qualified support to this release [ ] -0 I have reservations about this release [ ] -1 This release should not be cut --- Once the jar file's name is changed as Mark pointed out, I'm willing to release 1.0 even if only to invite comment. I haven't been active at all with the code for some time, so I must trust that all the tests are passing; if so, then let's release and see what the world has to tell us about what has been built so far. Al - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: [Math] Colt
--- Wolfgang Hoschek [EMAIL PROTECTED] wrote: FYI, if some of you Math guys would like to refer to Colt and/or take ideas as you see fit: A lot of licensing issues with COLT have been cleaned up in the latest stable release (1.2.0). In particular, there are no GPL and no commercial clauses anymore. The software distribution has moved to a new site: http://dsd.lbl.gov/~hoschek/colt/ where you can check this out and/or download the release. Wolfgang, That's fantastic. I noticed the current licenses last week and thought that everything except the hep.aida.* packages were pretty compatible with the Apache license, though of course IANAL. Thanks very much, I hope we can benefit from your generosity! Al - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: [MATH] Matrix indices
--- Phil Steitz [EMAIL PROTECTED] wrote: Brent Worden wrote: Ok, I tally (pre-apologies if I misrepresent anyone): Four votes for 0-based indexing (Andrew, Kim, Mark, and Stephen). Three votes for 1-based indexing (Al, Phil, and myself). Should we go ahead with the 0-based change and put this to rest? Or do we still need some debate? I, for one, want to move on. +1 for moving on :-) Interesting that astonishment is in the eye of the beholder. Mathematicians may be astonished by the 0-based indexing; but as long as it is clearly documented, most will not care. I am assuming from the above that you, Brent, are OK with the change. That leaves only me and Al as holdouts. I can live with it, and Al seems ambivalent, so I will go ahead and make the change. Assuming you are OK with this, Al? Yes, go ahead. We'll let our users tell us whether we made the right decision here. Al - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: [Jakarta Commons Wiki] Updated: MathWishList
--- Phil Steitz [EMAIL PROTECTED] wrote: [EMAIL PROTECTED] wrote: Date: 2004-09-02T06:34:37 Editor: AlChou [EMAIL PROTECTED] Wiki: Jakarta Commons Wiki Page: MathWishList URL: http://wiki.apache.org/jakarta-commons/MathWishList no comment Change Log: -- @@ -6,6 +6,7 @@ * Add remedian statistic - [http://www.agoras.ua.ac.be/abstract/Remrob90.htm The Remedian: a Robust Averaging method for Large Data Sets] * Add population variance and standard deviation statistic. * Add 0-1 random number generators. -- Not sure what is being requested here. RandomData.nextInt(0, 1) will generate random 0-1 values. Are we talking about random binary strings? + * I assumed this item meant implement good random number generators, e.g., like those described in ''Numerical Recipes'' or Knuth. -- AlChou Interestingly, as of 1.2 at least, the JDK-supplied PRNG is Knuth's 3.2.1 (linear congruential). I am +1 for adding additional PRNGs, but I would also recommend that if we do that we also include benchmark tests (like the ones here: http://csrc.nist.gov/rng/SP800-22b.pdf), performance metrics and clear documentation indicating what the relative strengths / weaknesses of the additional PRNGs are. Huh. So why do I and others think so poorly of the JDK's default PRNG? Well, I don't know about others, but I was just ignorant of that fact. Al - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: [MATH] Matrix indices
--- Mark R. Diggory [EMAIL PROTECTED] wrote: Al Chou wrote: My personal preference would originally have been to use 1-based indexing (actually, I really prefer Fortran's ability to let the user define the lower bound index value in each array dimension if they so choose, even though that facility is not that often used), but that was based on my assumption that commons-math is a library for enabling people who know some math to more easily do math in Java. However, the few use cases we've heard about seem to be split between that kind of user and Java programmers who have some need for math capabilities. The former class of user would more likely want/expect 1-based indexing, the latter would expect 0-based indexing. Quoting from our own home page: Guiding principles: 1. Real-world application use cases will determine development priority. ... 4. In situations where multiple standard algorithms exist, a Strategy pattern will be used to support multiple implementations. Unfortunately, nowhere in our charter do I know of a statement that would help us decide the priority between the two classes of users I described above (I would be happy if someone pointed out some passage of our charter that I don't know about that would resolve this decision). I think ideally we would provide facilities to handle both 0- and 1-based indexing (and even arbitrary lower-bound-based indexing, given that I'm pipe-dreaming here). The _Numerical Recipes_ solution to this problem is to stick to 1-based indexing throughout, as all algorithms are described in standard mathematical notation where the first index in each dimension is 1 (probably also because it made for easier [possibly at least partially automated] translation from the original 1-based Fortran source code into other languages). For 0-based languages (viz., C and presumably C++) they provided array data types that automatically translated between the mathematical 1-based notation and the underlying 0-based array data structure, at the expense of having one extra element in each dimension of each array, I believe. Perhaps we could do something similar and even provide a user setting that would choose between 0-based and 1-based behavior -- probably a global setting, to prevent the kind of confusion that would result from mixing the two representations unintentionally. Al At first this sounds like a concession to both sides of the issue, But, my fear is that this will introduce more complexity and confuse the user. I would say one or the other, not both. I think theres too much ridiculous partisanship when it comes to subjects like whats the best language. Its amazing what people with fixate on and defend as important (myself included). Thats why I suggest indexing in that of the implementation language. If I were working with Fortran, I'd argue to use 1 = x n. I'm sure it doesn't matter very much either way... -Mark I think switching back and forth between 1-based mathematical notation and 0-based programming language notation would be horrible. My preference would be to stay in mathematical notation throughout my mathematical code. I wonder if it would work to return (when asked) larger-by-one double arrays that only contain actual data in elements (1,1) through (n,n). I guess there's a problem of what to put in the zeroth elements, given that these are double arrays (not object arrays), so that the values put in those elements would have to be valid double values. One could mistakenly access and use the zeroth elements without necessarily recognizing that their very existence should be ignored. In any case, I feel that whatever we choose to do should feel as natural as possible for the majority of users. I don't know who makes up the majority of this library's users, but I suspect it's Java programmers who just need a bit of math in their code. If so, that _may_ mean 0-based indexing is what's going to seem more natural to the majority of commons-math users (I do worry about such users getting confused when they read 1-based mathematical explanations of what our routines do, assuming they are not familiar with the mathematics and are having to reconcile a new concept in an unfamiliar notation with what they're familiar with as the programming language's natural notation). Al - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: [math] [vote] Release 1.0-RC1
--- Phil Steitz [EMAIL PROTECTED] wrote: This vote is to approve the public release of commons math 1.0-RC1. This will be a publicly announced RC to enable full feedback for a final release in about two weeks if all is well. The distribution files are here: http://www.apache.org/~psteitz/commons-math-1.0-RC1 The math website has been updated to reflect 1.0-RC1: http://jakarta.apache.org/commons/math/ Voting will last at least 72 hours. As always, it would be useful if someone could check the md5 and asc signatures. Thanks, Phil - [X] +1 Go ahead and release 1.0-RC1 [ ] +0 [ ] -0 [ ] -1 Don't release 1.0-RC1, because... - - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: [all] Math needs a user a dev email list.
--- Mark R. Diggory [EMAIL PROTECTED] wrote: This is the type of discussion I'm referring too. Here is a perfect example of why commons components would benefit from separate users lists. Users are not interested in the same discussion issues as developers, the commons users list is ill adapted to the cross pollination developer paradigm. Commons Users wish to discuss issues related specifically to the component they are using while Commons developers wish to maintain maximum interaction with other Commons components to optimize code reuse and experience. Lets be clear here. I do not want to see math development issues occur on a list separate from the other components. I do not want to see any Commons Component start is own dev list. Nor do I want to see math separate from the commons. I want to see math user issues on a separate list, in fact I think any Commons project should have the option to produce their own users list as a means to provide user support for the component independent of the other components. +1 to all of the above paragraph. What I'm talking about is User Support, which should be of great importance to the community, as important as developer interaction. After a great deal of discussion, maybe this subject should be voted on to bring the subject to completion. Otherwise, it will just get more heated. +1 if a vote will resolve the issue and let the commons-dev community get back to development discussions. I was going to reply to Phil's last message but then saw this and Mark's next reply, and it seems to me that commons-dev should try to remain a single community, whereas commons-*-user might well be separate communities per specific instance of the *. Speaking for myself, as a user of a piece of third-party software often I don't care about the discussions its developers are having about its internals, but rather only care about the user-level interface or API and how to use it, which isn't necessarily the same kind of developer's-point-of-view as that of the developers of the internals of that software, even if it may still be code-centric. For example, I use Java, but I wouldn't want to have to follow Sun's internal java-dev mailing list (or whatever the equivalent may be) in order to get answers to my questions about Java. Does that make me not a member of the Java community, or just not a member of the Java internals developers community? -Mark Kim van der Linde wrote: --- Martin Cooper [EMAIL PROTECTED] wrote: It seems that you are missing the single most important facet of Jakarta Commons here - community. Commons is one community, A community from whom? Whom are in it? Only hard-core developers? Or the developers and users at large? Apparently, I have a different idea about e-mail lists than most here. So be it. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: [all] Math needs a user email list.
--- J.Pietschmann [EMAIL PROTECTED] wrote: Stephen Colebourne wrote: For me, [math] goes beyond the role of a simple library of common code. IT would be interesting to hear what you (and others) think about the further evolution of [math]. I also really like to know what current users of [math] currently do with the library, and what they expect from further development. I too am very curious about what uses commons-math has been put to and what uses people would like or plan to put it to. I was delighted and tantalized by the user questions that came in from Deutsche Bank not too long ago. From my point of view, [math] provides currently mainly basic to moderatly advanced functionality to do descriptive and analytical statistics. The algebra and solver packages are more or less utilities for this purpose. Building on this, obvious development directions would include graphical data representation, like GnuPlot or JFreeChart (BTW there aren't many good plotting/charting packages, and various initiatives in ASF communities have stalled. Why???), higher level functionality for statistical tests, or going deeper into the simulation domain. What do you think? I would really like to know what real-world uses the library should support; not being anything like a statistician, I have been somewhat less than enthusiastic about the direction commons-math has taken, but I fully admit that the subdisciplines I'm stronger in are not ones that non-scientific/mathematical users are likely to use and that statistics is an area that such users are much more likely to find helpful. So, I am supportive of commons-math growing in whatever directions its actual users need (and at the same time pragmatically opposed to growth into too-esoteric or user-irrelevant regions). We have reiterated many times that the charter is _not_ to build a complete mathematical library. Al - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: [all] Math needs a user email list.
--- Mark R. Diggory [EMAIL PROTECTED] wrote: Hello all, After some thought, I've come to the conclusion that Commons should setup a separate user lists for the Math group. +1 Al - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: [math] Design review pre 1.0
--- Stephen Colebourne [EMAIL PROTECTED] wrote: Congratulations on the work put into [math]! Stephen - Original Message - From: Phil Steitz [EMAIL PROTECTED] To: Jakarta Commons Developers List [EMAIL PROTECTED] Sent: Sunday, July 25, 2004 7:13 PM Subject: Re: [math] Design review pre 1.0 Updated response interspersed. Pls correct / comment as necessary... Yes, many thanks to Phil for so much work done! I'm sorry I haven't had time to contribute more. Al = Albert Davidson Chou Get answers to Mac questions at http://www.Mac-Mgrs.org/ . - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
[math] where to cite references (was RE: cvs commit: jakarta-commons/math/src/java/org/apache/commons/math/stat/univariate/moment Kurtosis.java Skewness.java)
I rather prefer hyperlinks in the Javadoc as well. I don't see why it should be brittle to link from the Javadoc to the user guide -- it's not as if we're frequently changing the list of references. The fact that the user guide is online at the project site should help us keep it up to date (it's visible to the public, which always gives me more incentive to do the right thing g). One thing that occurs to me as I look at the [math] site: like other Apache project sites, it's not immediately obvious from the navigation menu (or any other part of the home page) how one should submit feedback about the project. Sure, there are several links that sound promising (e.g., About Us Contributors, General Information Contributing Patches, Jakarta Community Mailing Lists), but these typically point to Jakarta Commons in general rather than Commons-Math specifically (I realize that may be because we're using a Commons template to generate the page), but no obvious submit a bug report, submit feedback, request support, ask a question sort of feature. Users have to be rather motivated to discover a way to send us a message (most likely Project Documentation Issue Tracking), and I think the project is probably the poorer for it, as many users aren't nearly motivated enough to dig in that far (the company where I work develops a product that helps address that and many other related issues with Web sites/apps). On a different note, I'm not current on how the Commons-Math site is generated; I have a vague memory that Mark's been initiating that process manually. Is there a strong reason the nightly build shouldn't do it? Some of the content seems a little (a few weeks, as opposed to months) out of date, such as the TODO list, which Phil, Mark, and Brent have diligently been working through (apologies to any whom I forgot to mention). Al --- Phil Steitz [EMAIL PROTECTED] wrote: I would prefer to keep the references that we actually need in the javadoc itself. The ones that I removed were just references to (standard, straightforward) definitions, which I had added inline. In general, I want to include definitions in the javadoc unless this is not tractable (too many dependent concepts, formulas too complex, etc.), in which case a reference to a definitive source should be included. When we use algorithms that are not elementary or widely known, we should include references to docs or papers (e.g. the Chan et al references for the moment updating formulas). Unfortunately, the first sentence above is a PITA to maintain and leaves us open to broken links (last time I rank linkcheck, the p-value reference was broken :-( That makes suggestions like Mark's appealing, but I don't like losing the hyperlinks. Is it excessively brittle to include links from the javadoc back to the user guide? If so, is there any stable place (better than ~psteitz) where I can host some definitions? Phil -Original Message- From: Mark R. Diggory [mailto:[EMAIL PROTECTED] Sent: Fri 7/2/2004 8:51 AM To: Jakarta Commons Developers List Subject: Re: cvs commit: jakarta-commons/math/src/java/org/apache/commons/math/stat/univariate/moment Kurtosis.java Skewness.java What if we maintained Citation Numbers in the javadoc and then maintained references to external sources in the Math Site documentation? Al Chou wrote: --- [EMAIL PROTECTED] wrote: psteitz 2004/07/01 22:29:14 Modified: math/src/java/org/apache/commons/math/stat/univariate/moment Kurtosis.java Skewness.java Log: Removed link to external definition, as formula has been added to javadoc. Maybe it's my grad school experiences talking, but I would prefer to retain links to external references. That way if a user (or even one of us developers) ever needs to look up the original references, its easy to do, and there's an explicit statement of which source of information we used, rather than doing a Web search and wondering which (if any) of the resulting resources was the original reference. Maybe that doesn't matter to anyone but me? Al - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: [math] where to cite references (was RE: cvs commit: jakarta-commons/math/src/java/org/apache/commons/math/stat/univariate/moment Kurtosis.java Skewness.java)
--- Phil Steitz [EMAIL PROTECTED] wrote: All good points below. How about the following strategy: 1) Web site always reflects state of current development (essentially CVS head). We get better at keeping it up to date (I have been lazy about this), trying to update it after every significant change and at least once a month. -1 IIRC, Mark will argue against having the Web site reflect the current state of development, and I now would agree with him. The Web site should reflect the current official release version. Perhaps we could have an additional section of the site where the current development version is reflected nightly? 2) With each release (yes, there really will be releases ;-), we tar generated html with relative links including both javadocs and user guide. Struts and tomcat at least used to do this (haven't grabbed distros of either recently). That way users always have guide + javadoc matching what they have deployed. This should not be a problem using maven. +1 3) We try to limit dependencies on external html links as much as possible, favoring inline exposition in the javadoc or user guide when this is feasible and dead tree references when what is needed is a reference citation. The last sentence conflicts with what we have in the developer's guide, so I guess I am proposing that we change this policy. I am just afraid that if we include a lot of external links in the html that we distribute with releases, we will have no way of keeping them up to date. I am not proposing that we hold 1.0 until we get rid of them all --just try to eliminate the less stable ones and do not continue to add more. I would prefer a stable hyperlink over a paper reference any day. Providing both would be acceptable, so that if the hyperlink has gone stale, there's still a concrete reference to look up. I for one would feel more confident doing a Web search when I found a reference link unreachable if I knew ultimately there was a paper reference to consult. Al Phil Mark R. Diggory wrote: Nightly update tends to be frowned upon because usually the site documentation is tied to a specific release. I think the Maven folks are working on a means to have older versions of docs archived so that they can be accessed. The issue is one of referential integrity. Say we release javadoc for version 1.0 and have user doc on the math site that it links to. Then (lets say) we update the doc on the site such that a page is removed. Now there are versions of the javadoc out in the wild that point to a dead link. So, linking in such a way is very restrictive to our being able to update the user guide. Now, I think the user guide is not that large that it couldn't be distributed on combination with the javadoc, then the javadoc references could be based on relative linking to the local copy of the userdoc. Ultimately we are trying to resolve the fact that we can't use xdoc in such a way as to embed it into javadoc directly. Would we even want to?! I think its best if we maintain a parallel xdoc structure to the javadoc package structure and distribute them together as one package that is relatively linked. This really just means packaging up portions of the site docs such that they are expanded from the tar file and that they can still be navigated in the local file system in a browser -Mark - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: cvs commit: jakarta-commons/math/src/java/org/apache/commons/math/stat/univariate/moment Kurtosis.java Skewness.java
--- [EMAIL PROTECTED] wrote: psteitz 2004/07/01 22:29:14 Modified:math/src/java/org/apache/commons/math/stat/univariate/moment Kurtosis.java Skewness.java Log: Removed link to external definition, as formula has been added to javadoc. Maybe it's my grad school experiences talking, but I would prefer to retain links to external references. That way if a user (or even one of us developers) ever needs to look up the original references, its easy to do, and there's an explicit statement of which source of information we used, rather than doing a Web search and wondering which (if any) of the resulting resources was the original reference. Maybe that doesn't matter to anyone but me? Al = Albert Davidson Chou Get answers to Mac questions at http://www.Mac-Mgrs.org/ . - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: [math] Getting 1.0 out the door -- tasks remaining
--- Phil Steitz [EMAIL PROTECTED] wrote: J.Pietschmann wrote: Phil Steitz wrote: 1) Decide what to do about inverse cumulative probabilities where p = 1 (easy solution is to document and throw) Nearly +1 My own nearly +1 on this just turned to -1. After looking some more at the code and thinking some more, I think that both p=1 and p=0 should be handled correctly in all cases. The difficult cases are when the probability density function has unbounded support. Here is what I propose for the values of inverseCumulativeProbability() at p=0 and p=1 for current distributions. Unless otherwise noted, these values are intented to be independent of distribution parameters. Distribution p=0 p=1 -- Binomial 0 Integer.MAX_VALUE Chisquare 0 Double.POSITIVE_INFINITY Exponential0 Double.POSITIVE_INFINITY F 0 Double.POSITIVE_INFINITY Gamma 0 Double.POSITIVE_INFINITY HyperGeometric 0 finite, parameter-dependent Normal Double.NEGATIVE_INFINITY Double.POSITIVE_INFINITY TDouble.NEGATIVE_INFINITY Double.POSITIVE_INFINITY Other than the value for Chisquare with p=1 (which causes R to hang), these values are consistent with what R returns using the q* functions. It might be more convenient to return Double.MAX_VALUE, -Double.MAX_VALUE in place of the INFINITY's (since then we could just use getDomainLowerBound at 0 and 1) but this would not be correct mathematically. If there are no objections, I will find a way to get the values above returned. +1 to the values in the table above. As a user I would prefer to be returned an infinity rather than MAX_VALUE where possible (it's too bad the integer types don't provide infinity values), because even though I would often recognize 1e+308 or thereabouts as Double.POSITIVE_INFINITY, I would still have to do that conversion mentally, and I would always wonder whether the returned value was actually MAX_VALUE or just the implementation-dependent representation of POSITIVE_INFINITY. Also consider what would happen if the data type were changed to float. Then if MAX_VALUE were used, the numeric value returned for p = 1 would differ depending on the data type. With the infinity values, although there's a class difference between Double.POSITIVE_INFINITY and Float.POSITIVE_INFINITY, the concept is clearly identical. It's strange that BigDecimal doesn't provide infinity values, though. Maybe that's something Commons should address at some point. Al __ Do you Yahoo!? Friends. Fun. Try the all-new Yahoo! Messenger. http://messenger.yahoo.com/ - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: [lang] [math] org.apache.commons.lang.math.Fraction class
--- Phil Steitz [EMAIL PROTECTED] wrote: C. Scott Ananian wrote: On Thu, 3 Jun 2004, Stephen Colebourne wrote: This presumes that everyone wants a reduced fraction. I believe that there are use cases for holding an unreduced one. The main one that strikes me is education. The org.apache.commons.math.Fraction class is not targetted at education. It's intended to be useful for working programmers. The problem is that we do not know what the working programmers are going to use this class for. Your view seems to be that Fraction should really be RationalNumber -- so that two equivalent fractions are equal. The problem is that the class is not named or currently implemented that way (in terms of representation and identity). Representing the fractions themselves instead of collapsing immediately to the equivalence classes (rational numbers) is more flexible; though as you point out, more care has to be taken to prevent overflows and some efficiency in the arithmetic operations may be sacrificed as a result. If we just fix the computational bugs in the current implementation, the overflow situation should be OK with the arithmetic operations implemented as they are now, since they call reduce() before returning results. We may in fact want to add alternative methods that do not reduce returned fractions, or that otherwise restrict / control denominators. While I do not have specific use cases in mind, it could be that some number-theoretic applications for this sort of thing may exist. The fact that all current operations reduce returned values led me initially to agree that it was pointless to maintain the disctinction between equivalent fractions. Thinking about it some more, I am -0 to changing Fraction to RationalNumber. I think our best path is to start by fixing the computational bugs and then see what sorts of applications emerge. It seems to me that number-theoretic users would gravitate toward commons.math rather than commons.lang.math, and that such users (and no one else) might conceivably find non-reduced representations of fractions useful (it sure would be nice to have someone here with number theory experience to tell us; the only use I think I'd ever have is if I cared to know the exact history of the values in a calculation or derivation, but that would probably arise only in the context of computerized algebra, which I have never tried to program). That seems to argue for moving this class directly into commons.math as is, and leaving behind an always-reducing version of itself in commons.lang.math. But I am definitely +1 to first fixing the defects in the current class where it lives and deferring other decisions. Al __ Do you Yahoo!? Friends. Fun. Try the all-new Yahoo! Messenger. http://messenger.yahoo.com/ - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: [math] AbstractDescriptive Statistics
--- Mark R. Diggory [EMAIL PROTECTED] wrote: I'm curious about AbstractDescriptiveStatistics, currently our Type Hierarchy looks like this: Object -- DescriptiveStatistics (implements Statistical Summary) -- AbstractDescriptiveStatistics -- DescriptiveStatisticsImpl -- ListUnivariateImpl -- BeanListUnivariateImpl Why don't we consolidate AbstractDescriptiveStatistics into DescriptiveStatistics? Then we will have On the basis of the above diagram (I haven't looked at the code), it does seem strange that AbstractDescriptiveStatistics is a descendant of DescriptiveStatistics (unless the latter is an interface?). Object -- DescriptiveStatistics (implements StatisticalSummary) -- DescriptiveStatisticsImpl -- ListUnivariateImpl -- BeanListUnivaraiteImpl Which is much more similar to the SummaryStatistics Hierarchy Object -- SummaryStatistics (implements StatisticalSummary) -- SummaryStatisticsImpl Likewise, for consistency, we should probibly rename ListUnivariateImpl and BeanListUnivariateImpl to ListDescriptiveStatisticsImpl and BeanDescriptiveStatisticsImpl If no one has objections, I can make the changes. +0, depending on whether multivariates are in our near future (on which I currently have no opinion). Al = Albert Davidson Chou Get answers to Mac questions at http://www.Mac-Mgrs.org/ . __ Do you Yahoo!? Friends. Fun. Try the all-new Yahoo! Messenger. http://messenger.yahoo.com/ - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: [math] Design review pre 1.0
--- Mark R. Diggory [EMAIL PROTECTED] wrote: Al Chou wrote: Before we go too far down this path, it would be very helpful to know just how much performance penalty is incurred by specifying strictfp. That FAQ certainly suggests that the difference is large and undesirable, but like profiling, you never really know what it is until you actually measure it Suggestion: conduct an informal timing test of a few representative functions, say, some of the transcendental functions in java.lang.Math, with and without strictfp. A loop doing 100,000 of these method calls should be sufficient to have runtime lasting several seconds to several minutes depending on the operation. Run it at least three times to get an idea of the mean runtime and standard deviation. The tough part is that I think all the java.lang.Math functions already are strict in that they simply call their strict counterparts in java.lang.StrictMath. Yeah, and almost all that stuff seems to be JNI calls, IIRC. I downloaded the fdlibm from metalab.unc.edu just for kicks, as well as some interesting-sounding other libraries in the same directory. Looks like that C source code is GPL'ed, though, so we couldn't use it for commons-math (we may have discussed that before), though maybe it could serve as a model for a Java implementation just for benchmarking the penalty strictfp incurs. I saw a gamma function evaluator in there, and probably one or two others at least, that could be good tests. Al __ Do you Yahoo!? Friends. Fun. Try the all-new Yahoo! Messenger. http://messenger.yahoo.com/ - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: [math] Design review pre 1.0
--- Mark R. Diggory [EMAIL PROTECTED] wrote: Phil Steitz wrote: [Yoav] You probably want strictfp: http://www.jguru.com/faq/view.jsp?EID=17544. [Phil] I am not sure that we want this, but I am by no means a JVM expert. From what I understand, the decision comes down to strict consistency of results on different platforms (mostly involving NaN and other boundary conditions) vs. performance. In most practical applications, I would personally be more interested in performance. It would be a major PITA (given the way things have to be declared); but I suppose that in theory we could support both. I am open to discussion on this, but my vote at this point would be to release without strictfp support for 1.0. Its tough that its a modifier and not some sort of JVM option, seems it would make libraries alot more flexible if you controlled it in the behavior of the JVM and not something you have to compile into your code. To provide this functionality such that it could be enabled or disabled we'ed need to have twin libraries or some sort of wrapper methods, one with it in place and the other with it removed. Before we go too far down this path, it would be very helpful to know just how much performance penalty is incurred by specifying strictfp. That FAQ certainly suggests that the difference is large and undesirable, but like profiling, you never really know what it is until you actually measure it Suggestion: conduct an informal timing test of a few representative functions, say, some of the transcendental functions in java.lang.Math, with and without strictfp. A loop doing 100,000 of these method calls should be sufficient to have runtime lasting several seconds to several minutes depending on the operation. Run it at least three times to get an idea of the mean runtime and standard deviation. Al = Albert Davidson Chou Get answers to Mac questions at http://www.Mac-Mgrs.org/ . __ Do you Yahoo!? Friends. Fun. Try the all-new Yahoo! Messenger. http://messenger.yahoo.com/ - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: [bug report] Commons-Math binomial distribution method returns 1 when it should return 0
As Brent says in the latest comment on the bug report, the cumulative probability does seem to be behaving as its mathematical definition specifies. Perhaps what is wanted by the reporter is not just the cumulative probability at a point n but rather the difference between the cumulative probabilities at two points surrounding n, e.g, cumulativeProbability(n + 1) - cumulativeProbability(n - 1) for the discrete case (or x + dx and x - dx for the continuous case). Al DO NOT REPLY TO THIS EMAIL, BUT PLEASE POST YOUR BUG RELATED COMMENTS THROUGH THE WEB INTERFACE AVAILABLE AT http://issues.apache.org/bugzilla/show_bug.cgi?id=28829. ANY REPLY MADE TO THIS MESSAGE WILL NOT BE COLLECTED AND INSERTED IN THE BUG DATABASE. http://issues.apache.org/bugzilla/show_bug.cgi?id=28829 [bug report] Commons-Math binomial distribution method returns 1 when it should return 0 Summary: [bug report] Commons-Math binomial distribution method returns 1 when it should return 0 Product: Commons Version: unspecified Platform: All URL: http://http:// OS/Version: All Status: NEW Severity: Normal Priority: Other Component: Math AssignedTo: [EMAIL PROTECTED] ReportedBy: [EMAIL PROTECTED] I wanted to open a ticket for this issue so we can address it formally. -Mark. Original Message Dear Mark, I'm not sure if you're the right person to inform about this, but we believe there is an error in the results produced by the org.apache.commons.math.distribution.BinomialDistributionImpl.cumulativeProbability(int numberOfSuccesses) method. When the probability of success is zero, and numberOfSuccesses is greater than zero, we would expect the cumulative probability to be zero. The result returned is, however, one. org.apache.commons.math.distribution.BinomialDistributionImpl.probability(int numberOfSuccesses) behaves as expected. Obviously this is very easy to workaround, but I thought the development team should know. After significant testing on these two functions we are preparing to use the commons math package in a production application, and would like to thank you and the other members of the team for starting work on this project. We understand that the library is in development, and despite the extra integration work that will be required by us as the library develops, the software you have developed is already of great use. Best regards, David Morgan-Brown Deutsche Bank -- This e-mail may contain confidential and/or privileged information. If you are not the intended recipient (or have received this e-mail in error) please notify the sender immediately and destroy this e-mail. Any unauthorized copying, disclosure or distribution of the material in this e-mail is strictly forbidden. -- Mark Diggory Software Developer Harvard MIT Data Center http://www.hmdc.harvard.edu -- Albert Davidson Chou Get answers to Mac questions at the Mac-Mgrs home page: http://www.mac-mgrs.org/ - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: [math] Distributions tests
--- Phil Steitz [EMAIL PROTECTED] wrote: To close out BZ #28829, I need to add some degenerate test cases to BinomialDistributionTest. As I started to do this, I realized that adding more test points would be easier and we could eliminate duplication of code across test classes in the distributions package if we defined abstract test classes for discrete and continuous distributions that provided generic verification methods. The setup for the discrete case looks like this: Abstract base class for [EMAIL PROTECTED] DiscreteDistribution} tests. p To create a concrete test class for a discrete distribution implementation, implement makeDistribution() to return a distribution instance to use in tests and each of the test data generation methods below. In each case, the test points and test values arrays returned represent parallel arrays of inputs and expected values for the distribution returned by makeDistribution(). p makeDensityTestPoints() -- arguments used to test probability density calculation makeDensityTestValues() -- expected probability densities makeCumulativeTestPoints() -- arguments used to test cumulative probabilities makeCumulativeTestValues() -- expected cumulative probabilites makeInverseCumulativeTestPoints() -- arguments used to test inverse cdf evaluation makeInverseCumulativeTestValues() -- expected inverse cdf values Then one default method for each type of test runs through the test points / expected values and checks the test instance. To support tests using multiple instances, setXxx methods would also be provided for the instance and test data arrays. Similar setup for continuous. This will make it easier to add more test cases (including adding missing tests for discrete pdf calculations) and simplify development of new distribution test classes. Any objections? Phil Sounds very reasonable to me. Al __ Do you Yahoo!? Win a $20,000 Career Makeover at Yahoo! HotJobs http://hotjobs.sweepstakes.yahoo.com/careermakeover - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: [math] Numerical Python (NumPy)
Eric, I hadn't thought of verifying our code against NumPy, but it's not a bad idea. The Object-Oriented Numerics list has a Web presence at http://www.OONumerics.org/ . Python is a plenty powerful language, but I much prefer Ruby's sensibilities and have even learned to appreciate the power of Tcl. Al --- Eric MacAdie [EMAIL PROTECTED] wrote: Al Chou wrote: Numerical Python (NumPy) came up the past couple of days on the Object-Oriented Numerics list, and because I know it's been hosted at SourceForge for years, I started wondering whether it could be a source of ideas or even algorithms for us. The manual http://www.pfdubois.com/numpy/html2/numpy.html states: Permission to use, copy, modify, and distribute this software for any purpose without fee is hereby granted, provided that this entire notice is included in all copies of any software which is or includes a copy or modification of this software and in all copies of the supporting documentation for such software. so it seems like we'd be fine using whatever we needed of it, unless I misunderstand the Apache License. Al = Albert Davidson Chou Get answers to Mac questions at http://www.Mac-Mgrs.org/ . I did volunteer to verify the results of Commons Math against aother packages. I have looked at R and tried to find some data. It is more work than I thought, so I honestly have not gotten very far. But I will look at NumPy. I have heard Python is pretty easy. Is there a web page for the Object-Oriented Numerics list? EKMacAdie __ Do you Yahoo!? Yahoo! Photos: High-quality 4x6 digital prints for 25¢ http://photos.yahoo.com/ph/print_splash - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
[math] Numerical Python (NumPy)
Numerical Python (NumPy) came up the past couple of days on the Object-Oriented Numerics list, and because I know it's been hosted at SourceForge for years, I started wondering whether it could be a source of ideas or even algorithms for us. The manual http://www.pfdubois.com/numpy/html2/numpy.html states: Permission to use, copy, modify, and distribute this software for any purpose without fee is hereby granted, provided that this entire notice is included in all copies of any software which is or includes a copy or modification of this software and in all copies of the supporting documentation for such software. so it seems like we'd be fine using whatever we needed of it, unless I misunderstand the Apache License. Al = Albert Davidson Chou Get answers to Mac questions at http://www.Mac-Mgrs.org/ . __ Do you Yahoo!? Yahoo! Photos: High-quality 4x6 digital prints for 25¢ http://photos.yahoo.com/ph/print_splash - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: [math] Graph theory
--- Allen Lee [EMAIL PROTECTED] wrote: I've always thought something similar to the boost graph library (http://www.boost.org/libs/graph/doc/index.html) would be useful. Now that Java has generics as well it might be an easier first-cut translation... We wouldn't be able to ship a generics-dependent version, though, as we're aiming for compatibility at least as far back as JDK 1.3, if I remember correctly. If a source-to-source compilation of genericized code to non-genericized code is still possible in Java 1.5 and beyond (I know it was with gjc, which is Java generics' parent), then we could ship the the machine-translated source code and have both ease of human translation from BOOST and backward compatibility with the server JVM versions that are still widely prevalent in production environments. Al = Albert Davidson Chou Get answers to Mac questions at http://www.Mac-Mgrs.org/ . __ Do you Yahoo!? Yahoo! Photos: High-quality 4x6 digital prints for 25¢ http://photos.yahoo.com/ph/print_splash - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: [math] email addresses in project.xml
--- Mark R. Diggory [EMAIL PROTECTED] wrote: Maybe we can research in Maven and see if these links can be turned off? -1 on email addresses not being obfuscated. Yes, -1 for me, too. I'm fortunate to have been late coming onto spammers' radar screens, but I'd like to limit any further exposure as much as possible. +1 on removing them if the link can't be removed. +1 for me, too. -Mark On Sun, 2004-04-11 at 12:29, Phil Steitz wrote: I notice that we now have obfuscated email addresses in project.xml. Maven turns these into mailto's, which is not good. I suggest that we either remove the email addresses altogether or replace them with real addresses (my preference). Any objections to replacing with real addresses? - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] -- Mark R. Diggory Software Developer - VDC Project Harvard MIT Data Center http://www.hmdc.harvard.edu __ Do you Yahoo!? Yahoo! Tax Center - File online by April 15th http://taxes.yahoo.com/filing.html - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: [Math] - RealMatrix
--- Inger, Matthew [EMAIL PROTECTED] wrote: The basic reason i inquired is that by using doubles, you're limiting the precision available when doing certain operations. Take the following matrix: [ 4 6 ] [ 6 14 ] If you try to take the inverse of that matrix, the correct answer is: [ 0.7 -0.3 ] [ -0.3 0.2 ] however, by using double in java, we get something like: [ 0.7002 -0.3001 ] [ -0.3001 0.20007 ] using BigDecimal isntead, we might get a slightly more accurate result (though i admit, in most cases, people won't go to 16 digits) A valid point, though again, usage will dictate whether such levels of precision are necessary (also, most usage I've seen just lives with the fact that most base-10 numbers are not exactly represented in base-2; the inverse matrix above would probably be considered close enough by many if the difference were explained by representation inaccuracy). Essentially all numerical computing to date has been done with, at best(!), double precision. Some techniques that could in fact increase precision in principle are, to my knowledge, never used in practice (e.g., sorting a list of numbers before summing, so that they can be summed from smallest to largest -- and if there's a possibility of having both negative and positive signs, summing the like-signed elements and then finally the resulting two opposite-signed partial sums). I guess performance comes into play, as well as a mathematician's view (even though many who are not mathematicians do numerical computing) that _that_ level of nitpickiness is just too much g. But if we were to have use cases in which exactness was paramount, very high precision (or perhaps using a RationalNumber class) would of course be the right thing to provide. Al __ Do you Yahoo!? Yahoo! Small Business $15K Web Design Giveaway http://promotions.yahoo.com/design_giveaway/ - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: [math] ArrayUtils.isAscending, isDescending
--- Phil Steitz [EMAIL PROTECTED] wrote: I missed the beginning of this thread; my Yahoo mail has been strangely low in activity the past two days. Rather than espouse an opinion on where these methods should go, let me ask why a parallel sort would be so bad. Especially if you sorted by using an index table. I did a clean-room implementation of index table sorting in Ruby a few weeks ago for my wife, based on the text description of the algorithm in _NR_ (I was very careful not to read the code -- though I'm pretty sure I had to type it in once and use it years ago in grad school, I sure wouldn't remember even a line of it now). The use case that touched this off was validating the knot point array passed in to spline interpolation. If the x[] values are not strictly increasing, the activation is suspect, so I would prefer to throw rather than speculatively sort the y[] vector in parallel. Same problem applies to the constructor of the PolynomialSpline wrt x[] and polynomials[] (where parallel sorting would be even more dubious). Phil It just occurred to me that a use case that perhaps we have not considered is in graphics and scalable fonts, where relations that are not functions (i.e., they are multi-valued functions) can easily occur -- think of your favorite drawing program's ability to draw Bezier curves with loops in them. I know that font systems use cubic (Postscript) and quadratic (TrueType) curves to describe font outlines -- though that case doesn't necessarily argue for multivalued interpolating segments. Anyway, food for thought. Al __ Do you Yahoo!? Yahoo! Small Business $15K Web Design Giveaway http://promotions.yahoo.com/design_giveaway/ - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: [Math] - RealMatrix
--- Phil Steitz [EMAIL PROTECTED] wrote: Al Chou wrote: But if we were to have use cases in which exactness was paramount, very high precision (or perhaps using a RationalNumber class) would of course be the right thing to provide. And of course, you can quickly waste 50 digits of individual value representation precision with bad numerics ;-) Phil Ah-yup. Accuracy/precision, like security and so many other things (e.g., performance), is vulnerable to the weakest-link-in-the-chain effect. Al __ Do you Yahoo!? Yahoo! Small Business $15K Web Design Giveaway http://promotions.yahoo.com/design_giveaway/ - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: [Math] - RealMatrix
Yup. It's interesting that the two results straddle the correct value in the final nonzero digit. Al --- Inger, Matthew [EMAIL PROTECTED] wrote: Not only that, but as you mention, order matters. Doing the following operations produces different outputs: NumberFormat fmt = NumberFormat.getInstance(); fmt.setMaximumFractionDigits(48); fmt.setMinimumFractionDigits(48); double res = ((double)2.0 / (double)3.0) * (double)14.0; System.out.println(fmt.format(res)); // outputs: 9.3320 double res2 = ((double)14.0 * (double)2.0) / (double) 3.0; System.out.println(fmt.format(res2)); // outputs: 9.3340 conceptually, and mathematically, these two equations are identical and should produce the exact same result. However they do not (even using BigDecimal, they do not). -Original Message- From: Al Chou [mailto:[EMAIL PROTECTED] Sent: Friday, April 09, 2004 9:23 AM To: Jakarta Commons Developers List Subject: RE: [Math] - RealMatrix --- Inger, Matthew [EMAIL PROTECTED] wrote: The basic reason i inquired is that by using doubles, you're limiting the precision available when doing certain operations. Take the following matrix: [ 4 6 ] [ 6 14 ] If you try to take the inverse of that matrix, the correct answer is: [ 0.7 -0.3 ] [ -0.3 0.2 ] however, by using double in java, we get something like: [ 0.7002 -0.3001 ] [ -0.3001 0.20007 ] using BigDecimal isntead, we might get a slightly more accurate result (though i admit, in most cases, people won't go to 16 digits) A valid point, though again, usage will dictate whether such levels of precision are necessary (also, most usage I've seen just lives with the fact that most base-10 numbers are not exactly represented in base-2; the inverse matrix above would probably be considered close enough by many if the difference were explained by representation inaccuracy). Essentially all numerical computing to date has been done with, at best(!), double precision. Some techniques that could in fact increase precision in principle are, to my knowledge, never used in practice (e.g., sorting a list of numbers before summing, so that they can be summed from smallest to largest -- and if there's a possibility of having both negative and positive signs, summing the like-signed elements and then finally the resulting two opposite-signed partial sums). I guess performance comes into play, as well as a mathematician's view (even though many who are not mathematicians do numerical computing) that _that_ level of nitpickiness is just too much g. But if we were to have use cases in which exactness was paramount, very high precision (or perhaps using a RationalNumber class) would of course be the right thing to provide. Al __ Do you Yahoo!? Yahoo! Small Business $15K Web Design Giveaway http://promotions.yahoo.com/design_giveaway/ - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Bug (and fix) on Commons-Math SplineInterpolator
--- Phil Steitz [EMAIL PROTECTED] wrote: Should be fixed now in CVS. In addition to fixing the impl, I made the following changes: SplineInterpolator.interpolate(double[], double[]) now returns a PolynomialSplineInterpolator (new class), which has an array of PolynomialFunctions representing the spline segments. Both PolynomialSplineInterpolator and PolynomialFunction implement the new DifferentiableUnivariateRealFunction interface. PolynomialSplineInterpolator exposes its polynomials and knot point arrays as read-only properties (getters return copies). I added tests to SplineInterpolatorTest (replaces InterpolatorTest) to verify that the correct coefficients are being computed (in the degenerate cases, testing against analytical values, for the sin case, using R as a reference) and that the PolynomialSplineFunctions give consistent values at the knot points and the polynomials match up (agree throgh 2 derivatives) at the knot points. Phil Whew, that's a lot of work! Not to discount any of it, but as I started reading the Javadoc for PolynomialSplineFunction I noticed you explicitly say the first two derivatives of adjacent polynomials are constrained to agree at the knot points Isn't that a property more specifically of a cubic spline, or am I just ignorant of the definition of splines that use higher (or lower!) order polynomials? Al __ Do you Yahoo!? Yahoo! Small Business $15K Web Design Giveaway http://promotions.yahoo.com/design_giveaway/ - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: [math] Some changes to Polynomial
--- Phil Steitz [EMAIL PROTECTED] wrote: Al Chou wrote: --- Phil Steitz [EMAIL PROTECTED] wrote: 0. To help debug the SplineInterpolater (PR #28019 et al), I need to expose the coefficients in o.a.c.m.analysis.Polynomial as a read-only property (returning an array copy). Any objections to adding this? +1 if you do it by adding a package-level-accessible (i.e., no access modifier keyword; the JUnit test would be able to access it by being in the same package) getter method -- which sounds like what you're proposing. I actually intended to make this public, but read-only and using copy semantics. Any client should have (read-only) access to this basic property of the Polynomial, IMHO. Yes, you're right. +1 Al __ Do you Yahoo!? Yahoo! Finance Tax Center - File online. File on time. http://taxes.yahoo.com/filing.html - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: [math] sparse algebra
--- Phil Steitz [EMAIL PROTECTED] wrote: Pierre Auslander wrote: Gents, would it be of any interest to contribute sparse matrices (with iterative decomposition, solvers, etc.) to commons.math? Some time ago I've implemented such a library in C++, using iterative algorithms. It allowed us to handle very large sparse matrices (e.g. 100'000 by 100'000 correlation matrices used in risk management) in record space and time. I need now a Java version and think of translating the library and possibly contributing it to jakarta/commons. I think that this would make a good addition to commons-math (post 1.0). I completely agree. Al __ Do you Yahoo!? Yahoo! Finance Tax Center - File online. File on time. http://taxes.yahoo.com/filing.html - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: [math] Some changes to Polynomial
--- Phil Steitz [EMAIL PROTECTED] wrote:
Al Chou wrote:
Have we considered a design for the general derivative case (i.e. for
UnivariateRealFunction objects)? I was thinking about a
Differentiable interface that either extends from URF or is a base
interface. It would have a single derivative method with a URF return
value.
Specialized URF types, like PolynomialFunction, could implement and
general derivative method and could provide their own specialized
derivative methods. Much like java.util.List with iterator and
listIterator.
With this approach, the derivative impl for PolynomialFunction could
be:
public PolynomialFunction polynomialDerivative() {
return new PolynomialFunction(differentiate(coefficients));
}
public UnivariateRealFunction derivative() {
return polynomialDerivative();
}
I like this. If there are no objections, I will make these changes to
Polynomial and add DURF extending URF (unless others feel that a base
interface would be better??)
I have a couple more questions on this, actually for URF in general:
1. Should we throw MathException, IllegalArgumentException or some new
exception if value() is invoked with an argument outside of the domain of
the function? I would expect function implementations (including
derivatives) to be able to distinguish convergence / computation problems
from bad arguments and my inclination is to add something like
DomainException to throw when arguments are not in the domain of the
function.
DomainException (or some similar name) as a descendant of
IllegalArgumentException, maybe?
2. Should URF provide information about the domain of the function? Two
possibilites come to mind: (a) a boolean isTotal() property that just
determines whether the domain = R; and/or (b) a boolean isDefined(double)
method that determines whether or not the argument is in the domain of the
function. I think that it is reasonable to expect all implementations to
provide these; though (b) might be a challenge for some.
A great idea. Part of me wants to make this information live in an optionally
implemented interface, for those cases where it's too difficult to implement.
However, another part of me says that if you write a function evaluation
algorithm, you really ought to know what its domain of validity is. Chances
are you're having to handle different regions of that domain differently
anyway, to provide good accuracy or convergence rate throughout the domain, so
it's not a stretch to say you should know what the whole domain is.
That seems pretty reasonable. Should we at least briefly explore what
multivariable differentiation would look like and see if that makes it
clearer
what it should look like with a single variable?
For instance, a UnivariateRealFunction can be considered a special case of
a
ScalarRealFunction (I thought about writing MultivariateRealFunction there,
but
semantically it seems wrong to call Univariate... a special case of
Multivariate...). For a RealFunction of n variables, the quintessential
derivatives are the first partial derivatives with respect to each of the n
variables, from which the gradient and other del-operator-based vector
functions spring. For second derivatives we quickly generate the matrix of
all
mixed partial derivatives.
Where is this leading? I guess one direction might be an interface
containing
a differentiate() method that calculates a first derivative, which in n
dimensions would require the user to specify which of the n variables to
differentiate with respect to, but in one dimension would not, for obvious
reasons. So I guess the Brent's suggestion holds, and the generalization
to
more dimensions would be
public ScalarRealFunction derivative(Variable x) {
// implementation of first derivative with respect to x
}
I don't care to tackle vector functions of any number of variables at the
moment. g
I think that things change fundamentally when you consider the
multivariate case -- even for scalar functions. The derivative becomes
the Jacobian matrix.
I agree that it is a good idea to think about these things now; but I
guess my feeling is that things will get matrix / transformation oriented
when we start representing differentiable transformations and I don't
think it is a good idea to start by modeling univariate real functions as
1x1 transformations. Of course, we can always add a
DifferentiableTransformation interface that DURFs could implement by
returning 1x1 Jacobians ;-)
I think it will be best for us to follow the time-honored tradition of
treating R-R functions specially, exploiting the nice closure property
that the derivative of a real-valued real function is another R-R
function. Good idea to raise this now, though.
I did think about the Jacobian, though my personal experience in mathematical
physics didn't encounter it very
Re: [math] Some changes to Polynomial
--- Phil Steitz [EMAIL PROTECTED] wrote:
0. To help debug the SplineInterpolater (PR #28019 et al), I need to
expose the coefficients in o.a.c.m.analysis.Polynomial as a read-only
property (returning an array copy). Any objections to adding this?
+1 if you do it by adding a package-level-accessible (i.e., no access modifier
keyword; the JUnit test would be able to access it by being in the same
package) getter method -- which sounds like what you're proposing.
While reviewing the code, I also noticed that the current impl uses
naive evaluation (using Math.pow, etc.). I would like to change this to
use Horner's Method. Here is what I have ready to commit:
1. Add protected static double evaluate(double[] coefficients, double
argument) implementing Horner to get the function value; and change
value(double) to just call this.
2. Add protected static double[] differentiate(double[] coefficients) to
return the coefficients of the derivative of the polynomial with
coefficients equal to the actual parameter. Then change
firstDerivative(x) to just return
evaluate(differentiate(coefficients), x). Similar for secondDerivative.
I could adapt Horner for the derivatives, but that seems messy to me and
the slight memory cost to create the temp arrays seems worth it.
3. I would also like to add
public PolynomialFunction derivative() {
return new PolynomialFunction(differentiate(coefficients));
}
Any objections to this?
+1, these sound reasonable.
Interestingly, while Horner's method should give better numerics, it
actually fails to get within 1E-15 for one of the quintic test cases,
performing worse than the naive impl. The error is in the 16th
significant digit, which is not surprising. I would like to change the
tolerance to 1E-12 (current tests actually succeed at 1E-14).
Phil
I wonder why that is? Does Math.pow() use higher-than-double precision and
then cast down to double? I think we should consider carefully what is implied
by the fact that Horner's method has worse precision. Also, I would in
principle like to leave the tolerance as tight as possible, 1E-14 in this case.
Al
__
Do you Yahoo!?
Yahoo! Finance Tax Center - File online. File on time.
http://taxes.yahoo.com/filing.html
-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
Re: [math] Some changes to Polynomial
--- [EMAIL PROTECTED] wrote:
On Tue, 30 Mar 2004 21:29:42 -0700, Phil Steitz wrote:
0. To help debug the SplineInterpolater (PR #28019 et al), I need to
expose the coefficients in o.a.c.m.analysis.Polynomial as a
read-only
property (returning an array copy). Any objections to adding this?
1. Add protected static double evaluate(double[] coefficients,
double
argument) implementing Horner to get the function value; and change
value(double) to just call this.
+1
2. Add protected static double[] differentiate(double[] coefficients)
to
return the coefficients of the derivative of the polynomial with
coefficients equal to the actual parameter. Then change
firstDerivative(x) to just return
evaluate(differentiate(coefficients), x). Similar for
secondDerivative.
I could adapt Horner for the derivatives, but that seems messy to me
and
the slight memory cost to create the temp arrays seems worth it.
+1 on the differntiate method.
Are the firstDerivate and secondDerivative methods needed anymore with
the addition of your derivative method below? I would favor removing
the prior two methods if possible.
+1 to Brent's suggestion. These methods seem very redundant now, and I never
really liked naming these explicitly, as they seem to beg for derivatives of
all orders to then be named explicitly as well.
3. I would also like to add
public PolynomialFunction derivative() {
return new PolynomialFunction(differentiate(coefficients));
}
Have we considered a design for the general derivative case (i.e. for
UnivariateRealFunction objects)? I was thinking about a
Differentiable interface that either extends from URF or is a base
interface. It would have a single derivative method with a URF return
value.
Specialized URF types, like PolynomialFunction, could implement and
general derivative method and could provide their own specialized
derivative methods. Much like java.util.List with iterator and
listIterator.
With this approach, the derivative impl for PolynomialFunction could
be:
public PolynomialFunction polynomialDerivative() {
return new PolynomialFunction(differentiate(coefficients));
}
public UnivariateRealFunction derivative() {
return polynomialDerivative();
}
Counter thoughts?
Brent Worden
http://www.brent.worden.org/
That seems pretty reasonable. Should we at least briefly explore what
multivariable differentiation would look like and see if that makes it clearer
what it should look like with a single variable?
For instance, a UnivariateRealFunction can be considered a special case of a
ScalarRealFunction (I thought about writing MultivariateRealFunction there, but
semantically it seems wrong to call Univariate... a special case of
Multivariate...). For a RealFunction of n variables, the quintessential
derivatives are the first partial derivatives with respect to each of the n
variables, from which the gradient and other del-operator-based vector
functions spring. For second derivatives we quickly generate the matrix of all
mixed partial derivatives.
Where is this leading? I guess one direction might be an interface containing
a differentiate() method that calculates a first derivative, which in n
dimensions would require the user to specify which of the n variables to
differentiate with respect to, but in one dimension would not, for obvious
reasons. So I guess the Brent's suggestion holds, and the generalization to
more dimensions would be
public ScalarRealFunction derivative(Variable x) {
// implementation of first derivative with respect to x
}
I don't care to tackle vector functions of any number of variables at the
moment. g
Al
__
Do you Yahoo!?
Yahoo! Finance Tax Center - File online. File on time.
http://taxes.yahoo.com/filing.html
-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
Re: [math][proposal] stat package structure
--- Mark R. Diggory [EMAIL PROTECTED] wrote: On Sun, 2004-03-21 at 22:40, Al Chou wrote: --- Phil Steitz [EMAIL PROTECTED] wrote: [deletia] 4. Similarly, I would like to create an inference or test subpackage and put TestStatistic there. +1 I wonder if there's a better name than those two. I see Mark voted for test, but -- perhaps because I've never done much statistics -- that makes me think that I'm looking at a JUnit tests directory tree. A quick skim through _NR_ chapter 14 didn't turn up any better names, though. good point, although we avoid test package names for JUnit tests in favor of the original package name of the class being tested, something I think is very smart to do. I'm not too picky here, I just picked test because its shorter, inference is more descriptive too. Along the same lines distributions could be confused with the the generic idea of a distribution of packages directory, maybe probability is better there? Right, though given that Java package names generally map to directory names, there's room for confusion, especially given that we do have a standard test directory paralleling java under src. Maybe it should be organized more along the following lines? o.a.c.m.stat.probability... o.a.c.m.stat.univariate... o.a.c.m.stat.multivariate... o.a.c.m.stat.inference... I'm not very picky about these names, either, as I'm not a probability/statistics person, as I said before. If others like the above, that's fine by me. Al __ Do you Yahoo!? Yahoo! Finance Tax Center - File online. File on time. http://taxes.yahoo.com/filing.html - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
[math] Re: cvs commit math/src/test/org/apache/commons/math/analysis InterpolatorTest.java
Phil,
I noticed that in testInterpolateLinearDegenerateThreeSegment() you didn't
insert a TODO comment when you deleted most of the body of the method (quoted
below)
Al
--- [EMAIL PROTECTED] wrote:
psteitz 2004/02/15 22:30:21
Modified:math/src/test/org/apache/commons/math/analysis
InterpolatorTest.java RealSolverTest.java
math/src/test/org/apache/commons/math/stat
DescriptiveStatisticsTest.java
Log:
Commented out sysouts.
Revision ChangesPath
1.11 +19 -52
jakarta-commons/math/src/test/org/apache/commons/math/analysis/InterpolatorTest.java
Index: InterpolatorTest.java
===
RCS file:
/home/cvs/jakarta-commons/math/src/test/org/apache/commons/math/analysis/InterpolatorTest.java,v
retrieving revision 1.10
retrieving revision 1.11
diff -u -r1.10 -r1.11
--- InterpolatorTest.java 29 Jan 2004 16:48:49 - 1.10
+++ InterpolatorTest.java 16 Feb 2004 06:30:21 - 1.11
@@ -42,18 +42,19 @@
...
public void testInterpolateLinearDegenerateThreeSegment()
throws MathException {
-System.out.println( deg 3 seg);
+// System.out.println( deg 3 seg);
double xval[] = { 0.0, 0.5, 1.0, 1.5 };
double yval[] = { 0.0, 0.5, 1.0, 1.5 };
UnivariateRealInterpolator i = new SplineInterpolator();
UnivariateRealFunction f = i.interpolate(xval, yval);
-double x;
-x = 0.0;
-System.out.println(
-x=
-+ x
-+ y=
-+ f.value(x));
-
-x = 0.5 - 1E-6;
-System.out.println(
-x=
-+ x
-+ y=
-+ f.value(x));
-
-x = 0.5;
-System.out.println(
-x=
-+ x
-+ y=
-+ f.value(x));
-
-x = 1 - 1E-6;
-System.out.println(
-x=
-+ x
-+ y=
-+ f.value(x));
-
-x = 1;
-System.out.println(
-x=
-+ x
-+ y=
-+ f.value(x));
-
-x = 1.5 - 1E-6;
-System.out.println(
-x=
-+ x
-+ y=
-+ f.value(x));
}
__
Do you Yahoo!?
Yahoo! Finance: Get your refund fast by filing online.
http://taxes.yahoo.com/filing.html
-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
Re: [math] Re: cvs commit math/src/test/org/apache/commons/math/analysis InterpolatorTest.java
--- Phil Steitz [EMAIL PROTECTED] wrote: Al Chou wrote: Phil, I noticed that in testInterpolateLinearDegenerateThreeSegment() you didn't insert a TODO comment when you deleted most of the body of the method (quoted below) Al Oops! I should have commented that block out, as I did the others and insert a todo. I will fix this evening. Sorry. Phil Cool, thanks. Al __ Do you Yahoo!? Yahoo! Finance: Get your refund fast by filing online. http://taxes.yahoo.com/filing.html - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
[math] J2SE 1.5 static import of methods
Maybe you all know about this feature of the just-beta Java 1.5, but I was pleasantly surprised to learn on slide 46 of the presentation below http://www.javasig.com/Archive/lectures/JavaSIG-Tiger.pdf that the new static import feature allows you to write things like import static Math; x = cos(PI * theta); instead of x = Math.cos(Math.PI * theta); (Note that not only does PI not need to be qualified by Math., neither does the cos call.) It'll be nice when the day comes that we can actually use this feature as the default Al __ Do you Yahoo!? Yahoo! Finance: Get your refund fast by filing online. http://taxes.yahoo.com/filing.html - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: [math] Details on Cutting the release
--- Mark R. Diggory [EMAIL PROTECTED] wrote: [deletia] *jar release* (jar) Archive: commons-math-0.1-dev.jar [deletia] So the jar has no tests and no experimental code within it. Any Comments? -Mark Sorry to chime in so late, but my preference would be to include the tests no matter which release package it is. They serve as examples of using the code, even if one decided to suspect them as a validation of the correctness of the code (which is up to the individual user). I know that may seem to hybridize all release packages, but if it were standard to include test source with all our release packages, it maybe wouldn't seem like they were hybrids as much as it would be just an additional form of documentation. Al = Albert Davidson Chou Get answers to Mac questions at http://www.Mac-Mgrs.org/ . __ Do you Yahoo!? Yahoo! SiteBuilder - Free web site building tool. Try it! http://webhosting.yahoo.com/ps/sb/ - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: [math] site updated today
--- Mark R. Diggory [EMAIL PROTECTED] wrote: Hi, I updated the site a few minutes ago. All recent changes are now in javadoc etc. http://jakarta.apache.org/commons/math/ -Mark p.s. With the amount of crap starting to show up in my apache email account, I recommend we drop the email addresses or machine proof them. http://jakarta.apache.org/commons/math/team-list.html i.e. replace [EMAIL PROTECTED] with user at apache.org -- Mark Diggory I like the logo! I finally got my CLA faxed in over the holidays and received my membership info just over a week ago, so while we're making the above change I'd like to get my email address and userid updated as well (not that I've contributed anything lately g). Al = Albert Davidson Chou Get answers to Mac questions at http://www.Mac-Mgrs.org/ . __ Do you Yahoo!? Yahoo! SiteBuilder - Free web site building tool. Try it! http://webhosting.yahoo.com/ps/sb/ - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: [math] Bug fixes
--- Phil Steitz [EMAIL PROTECTED] wrote: I would like to fix PR #25972 and tidy up a few other things. Any objections to my adding myself to STATUS and committing, or shall I submit patches? Phil I don't understand why the bug report is filed against Math, but I'm fine with you going ahead, Phil. Al = Albert Davidson Chou Get answers to Mac questions at http://www.Mac-Mgrs.org/ . __ Do you Yahoo!? Yahoo! Hotjobs: Enter the Signing Bonus Sweepstakes http://hotjobs.sweepstakes.yahoo.com/signingbonus - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: [math] Clover Test Coverage
--- Phil Steitz [EMAIL PROTECTED] wrote: Al Chou wrote: --- Mark R. Diggory [EMAIL PROTECTED] wrote: http://jakarta.apache.org/commons/math/clover/index.html I have to say, I'm very impressed with the clover test coverage tool. This report is very cool and shows us exactly where coverage is low. However, I'm not convinced that test coverage = quality assurance. One could easily write tests that cause test coverage to score high while not being very informative in and of themselves. That's one of the overarching lessons in software testing. There are some pre-existing tests for commons.lang.StringUtils.split(*) that pass regardless of how I order two operations in code I submitted in a patch yesterday.So while the tests seem to exercise those methods and thus provide test coverage, only one out of the three relevant tests is sensitive to those changes, which makes me nervous about both the tests and what the lack of sensitivity to changes says about the code (both the pre-existing code and my patch), because it seems like the logic should be different depending on the task ordering. If you can improve the data coverage of the tests, please submit patches. I agree that path converage is necessary but not sufficient for good tests. As bugs are identified, test cases should be added to cover the conditions leading to the failures. I need to look at the code some more before I can figure out why the existing tests are insensitive to what I thought was a necessary task sequence reordering. I dearly hoped last night to be able to create a test that would catch this change; I really don't yet understand why the existing tests don't. Al = Albert Davidson Chou Get answers to Mac questions at http://www.Mac-Mgrs.org/ . __ Do you Yahoo!? Free Pop-Up Blocker - Get it now http://companion.yahoo.com/ - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: [lang] unexpected StringUtils.split behavior (was RE: suggestion for new StringUtils.method)
Hi, Phil, I thought no one would ever ask, and I was sitting here modifying my code to conform to the existing tests so that I could at least submit my two new methods. I'll open a ticket with a patch for those, then open a ticket for my proposed change in behavior, plus patches to split( String, String, int ) and my split( String, String, boolean, int ) to implement the change. Thanks, Al --- Phil Steitz [EMAIL PROTECTED] wrote: Al Chou wrote: While testing, I discovered that my expectations for the behavior of the split( *, ..., int max ) methods didn't match their actual behavior. I expected to get a maximum of max substrings, all of which were delimited in the parent string by the specified delimiters. Instead, what you get is max - 1 such substrings, plus the rest of the parent string as the final result substring. This behavior seems counter to what StringTokenizer would do, which is surprising, given the Javadoc comments about using the split methods as alternatives to StringTokenizer. Can you open a Bugzilla ticket and attach a test case that shows the problem and a patch that shows how you think it should be fixed? Phil = Albert Davidson Chou Get answers to Mac questions at http://www.Mac-Mgrs.org/ . __ Do you Yahoo!? Free Pop-Up Blocker - Get it now http://companion.yahoo.com/ - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
[lang] unexpected StringUtils.split behavior (was RE: suggestion for new StringUtils.method)
I guess my previous post got lost in the noise, so I'm reposting. I have two
new StringUtils.split methods that can split a string at occurrences of a
substring rather than splitting at the individual characters in the specified
delimiter string.
While testing, I discovered that my expectations for the behavior of the split(
*, ..., int max ) methods didn't match their actual behavior. I expected to
get a maximum of max substrings, all of which were delimited in the parent
string by the specified delimiters. Instead, what you get is max - 1 such
substrings, plus the rest of the parent string as the final result substring.
This behavior seems counter to what StringTokenizer would do, which is
surprising, given the Javadoc comments about using the split methods as
alternatives to StringTokenizer.
Currently, my tests reflect my expectations for the behavior, and I modified
the existing split( String, String, int ) method to match my expectations. I
didn't want to submit such a change as a proposed patch without first getting
feedback from the community about whether my expectations are wrong. I am
happy to submit only code that does not change the behavior of the existing
methods, if need be.
Al
--- Al Chou [EMAIL PROTECTED] wrote:
This thread is a good entree for my question. I was adding a new
StringUtils.split method that can split a string using a whole string as the
delimiter, rather than the characters within that string. In running my
JUnit tests, I discovered unexpected behavior in the existing method:
String stringToSplitOnNulls = ab de fg ;
String[] splitOnNullExpectedResults = { ab, de } ;
String[] splitOnNullResults = StringUtils.split( stringToSplitOnNulls, null,
2
) ;
assertEquals( splitOnNullExpectedResults.length, splitOnNullResults.length )
;
for ( int i = 0 ; i splitOnNullExpectedResults.length ; i+= 1 )
{
assertEquals( splitOnNullExpectedResults[i], splitOnNullResults[i] ) ;
}
The result of the split call is
ab, de fg
and it doesn't look to me like StringTokenizer's documentation implies this
behavior
Al
=
Albert Davidson Chou
Get answers to Mac questions at http://www.Mac-Mgrs.org/ .
=
Albert Davidson Chou
Get answers to Mac questions at http://www.Mac-Mgrs.org/ .
__
Do you Yahoo!?
Free Pop-Up Blocker - Get it now
http://companion.yahoo.com/
-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
Re: [Math] common-math and bloated 3rd party libraries
--- Mark R. Diggory [EMAIL PROTECTED] wrote: I'm trying organize this thread a little bit more than was accomplished in the discussion. Thanks, Mark. Good job. 1.) Argument exists concerning the dependency requirements of Commons Math. To in fact be modular and easily integrated some discrepancy arises concerning interdependency with other commons components. The main question is; Are all these dependencies really required? a.) logging: It sounds like a good idea to make logging a runtime/compile time dependency on only the test cases and not the main +1 b.) Some discussion arises concerning some of the higher level interfaces and their dependencies on commons such as Discovery. We should review and grade if this Discovery pattern will really of true value in the places we've applied it (As opposed to just providing simple method of instantiation on these object directly...) +1 c.) j2sdk1.3.1 vs. j2sdk1.4.1: we are a project in a group dedicated to providing tools that can operate in Java Servlet/EBJ environments. Many vendors are still operating on 1.3.1. We have concerns as to our stuff working there. Thus we need to make sure we use only mechanisms supported on 1.3.1 for the time being (and then operate on a longer timescale to determine how facilitate usage of 1.4.1 in the future). I +1 2.) Server vs Application schools, I tend to think this is a Red Herring, this arises primarily by some great comments Eric made I also think that if we do a good job addressing points 1a and 1b above, the issue of requiring or having dependencies on too many other jars will be ameliorated by simply having reduced the number upon which math depends. Al __ Do you Yahoo!? Protect your identity with Yahoo! Mail AddressGuard http://antispam.yahoo.com/whatsnewfree - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: suggestion for new StringUtils.method
This thread is a good entree for my question. I was adding a new
StringUtils.split method that can split a string using a whole string as the
delimiter, rather than the characters within that string. In running my JUnit
tests, I discovered unexpected behavior in the existing method:
String stringToSplitOnNulls = ab de fg ;
String[] splitOnNullExpectedResults = { ab, de } ;
String[] splitOnNullResults = StringUtils.split( stringToSplitOnNulls, null, 2
) ;
assertEquals( splitOnNullExpectedResults.length, splitOnNullResults.length ) ;
for ( int i = 0 ; i splitOnNullExpectedResults.length ; i+= 1 )
{
assertEquals( splitOnNullExpectedResults[i], splitOnNullResults[i] ) ;
}
The result of the split call is
ab, de fg
and it doesn't look to me like StringTokenizer's documentation implies this
behavior
Al
=
Albert Davidson Chou
Get answers to Mac questions at http://www.Mac-Mgrs.org/ .
__
Do you Yahoo!?
Protect your identity with Yahoo! Mail AddressGuard
http://antispam.yahoo.com/whatsnewfree
-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
Re: [math] Proposal for Package restructuring and Class renaming
--- Mark R. Diggory [EMAIL PROTECTED] wrote: Al Chou wrote: OK, I see. The one thing I notice is that the names are getting awfully long, especially for the non-default case. I guess that's a price we pay for having descriptive (no play on words intended) names like DescriptiveStatistics Maybe the Implementations could be abbreviated somewhat o.a.c.math.stat.DescriptiveStatistics o.a.c.math.stat.StorelessDscrStatsImpl o.a.c.math.stat.DscrStatsImpl We could also consider pushing the actual implementation off into its own packages o.a.c.math.stat.impl.StorelessDscrStatsImpl o.a.c.math.stat.impl.DscrStatsImpl This would even push all the univariate stat providers off into this hierarchy as well o.a.c.math.stat.impl.univar.StorelessUnivariateStatistic o.a.c.math.stat.impl.univar.UnivariateStatistic Too much renaming and reorganization. I didn't mean to complain too loudly, and if the result is to use abbreviations, I retract my comments. I probably should have given more than half a second's thought to what alternative names might be shorter, but in the absence of well-thought-out shorter names, I much prefer the current proposal of DescriptiveStatistics. Never use abbreviations unless everyone already knows them (e.g., sin for sine), I say. Al __ Do you Yahoo!? Protect your identity with Yahoo! Mail AddressGuard http://antispam.yahoo.com/whatsnewfree - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: [math] Proposal for Package restructuring and Class renaming
--- Mark R. Diggory [EMAIL PROTECTED] wrote: Al Chou wrote: Would you move the existing ones into org.apache.commons.math.distributions.statistical or something so that the probability distributions could be organized together under *.probability? Also, I noticed that the current package uses the singular distribution rather than distributions. I suspect its unclear where this boundary would be drawn, I think all the distributions would be both beneficial for both random number distributions and statistical usage. I guess if it became clear that there was a strong separation between the two then separate packages would be warranted, but I'm not convinced of a difference. Yourself and others may have more informed opinions. -Mark I don't have an informed opinion, so I'll fall back to the default opinion of lump everything together until/unless it's clear how to split it up. Al __ Do you Yahoo!? Protect your identity with Yahoo! Mail AddressGuard http://antispam.yahoo.com/whatsnewfree - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: [math] Proposal for Package restructuring and Class renaming
--- Mark R. Diggory [EMAIL PROTECTED] wrote: Al Chou wrote: --- Mark R. Diggory [EMAIL PROTECTED] wrote: I have several modifications I'm planning to make, but in the spirit of consensus I want to propose them and attempt to get some agreement. So math developer opinions on the subject would be good. 1.) o.a.c.math.stat.distributions -- o.a.c.math.distributions Gives this package a more generic position to hold more than just stat distributions. What other kinds of distributions did you have in mind? I'm asking out of complete ignorance. Probability Distributions (Gamma, Beta, Poisson, Exponential, Logarithmic, Hyperbolic ...) great examples of these are in Colt's cern.jet.stat and cern.jet.random packages. ... but are bound up as implementations of RandomNumberGeneration classes...not that that a bad thing. Eventually ours could be used in random number generation, I think they should be a more dominant package. -Mark Would you move the existing ones into org.apache.commons.math.distributions.statistical or something so that the probability distributions could be organized together under *.probability? Also, I noticed that the current package uses the singular distribution rather than distributions. Al = Albert Davidson Chou Get answers to Mac questions at http://www.Mac-Mgrs.org/ . __ Do you Yahoo!? Protect your identity with Yahoo! Mail AddressGuard http://antispam.yahoo.com/whatsnewfree - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: [math] Proposal for Package restructuring and Class renaming
--- Mark R. Diggory [EMAIL PROTECTED] wrote: Al Chou wrote: --- Mark R. Diggory [EMAIL PROTECTED] wrote: ... 2.) Like in my last emails concerning Univariate I would like to, (and have done so in my checkout successfully) Make the following Class changes: interface o.a.c.m.stat.StoreUnivariate -- abstract class o.a.c.m.stat.DescriptiveStatistics this actually becomes a factory class and uses Discovery to instantiate new instances of the following implementations *default implementation* o.a.c.m.stat.StoreUnivariateImpl -- o.a.c.m.stat.univariate.StatisticsImpl Forgive me for not refamiliarizing myself with the code first, but should the storeless version perhaps be the default implementation instead? What do we lose by going that way? I'm thinking it would be nice to keep memory usage lower if possible. The Storeless version (UnivariateImpl) doesn't support rank Statistics because of its storeless nature, the more fully featured implementation is StoreUnivariateImpl, it does everything, but has the limitation of requiring storage of the values. These are two different implementations with different internal storage configurations. I choose StoreUnivariateImpl because I think the default should have full capabilities. The storeless version is more of an Optimized solution, It probably wise to suggest that one use it only if one needs that functionality (ie trying to get moments across huge datasets or realtime value streams of sorts) That sounds reasonable. Thanks for the refresher (I looked at the current code based on your remarks, too). Before we go overboard, can you give a quick example of instantiating one of the implementations? Or perhaps, both the default and one alternative ... Yes, like that For the default Discovery configured implementation: DescriptiveStatistics stats = DescriptiveStatistics.newInstance(); stats.addValue(5.0); ... double mean = stats.getMean(); For any alternate Implementations: DescriptiveStatistics stats = DescriptiveStatistics.newInstance(StorelessDescriptiveStatisticsImpl.class); stats.addValue(5.0); ... double mean = stats.getMean(); and/or DescriptiveStatistics stats = DescriptiveStatistics.newInstance(o.a.c.math.stat.impl.StorelessDescriptiveStatisticsImpl); stats.addValue(5.0); ... double mean = stats.getMean(); depending n which people like more OK, I see. The one thing I notice is that the names are getting awfully long, especially for the non-default case. I guess that's a price we pay for having descriptive (no play on words intended) names like DescriptiveStatistics Al __ Do you Yahoo!? Protect your identity with Yahoo! Mail AddressGuard http://antispam.yahoo.com/whatsnewfree - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: [math] BeanUtils, BeanTransformers and BeanListUnivariate
--- Mark R. Diggory [EMAIL PROTECTED] wrote: I'm starting to consider that the implementations we have of higher-end Univariates (ListUnivariate/BeanListUnivariate) are a bit premature. In Repast they/we encountered that reflection costs tend to make wanting to work with Collections as the core of a mathematical evaluation a bit costly, In RePast the solution to this was to pickup the trove API (similar to BCEL) and actually generate bytecode optimizations of these types of method calls on the Collections of Objects. Are we at a point where something like BeanListUnivariate (While a nice example usage of the API) is not something we ideally want to get people using when we release as it may require significant improvement or re-evaluation. If so, one trick would be to move its implementation into the test package hierarchy, as such we would be making it an Example usage of the API and not a full fledged Implementation people would use in the Long run. +1 It would be nice to see how people are actually using Commons-Math before we provide something like this functionality. While I would err toward guessing that people are doing the more bare-bones numerics I used to do, others might not, and only real usage will tell us whether it's needed as a core feature. Also, I've been considering some naming and consolidation of the lower end Univariate API. I think this is poorly named (think we discussed this before). I'm considering that Abstract/StoreUnivariate/Impl should probably be named DescriptiveStatistics. and that Abstract/Univariate/Impl StorelessDescriptiveStatistics or LiteDescriptiveStatistics Any thoughts? This would also allow us to remove runtime dependencies on commons-logging and bean-utils. +1 I never liked the Univariate label, and I have a feeling many users would be confused by it. Also, it seems to beg the question of extension to more than one dimension -- is the next one Bivariate (and the one after that Trivariate, etc.), or do we jump immediately to Multivariate? Al = Albert Davidson Chou Get answers to Mac questions at http://www.Mac-Mgrs.org/ . __ Do you Yahoo!? Protect your identity with Yahoo! Mail AddressGuard http://antispam.yahoo.com/whatsnewfree - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: [math] Proposal for Package restructuring and Class renaming
--- Mark R. Diggory [EMAIL PROTECTED] wrote: I have several modifications I'm planning to make, but in the spirit of consensus I want to propose them and attempt to get some agreement. So math developer opinions on the subject would be good. 1.) o.a.c.math.stat.distributions -- o.a.c.math.distributions Gives this package a more generic position to hold more than just stat distributions. What other kinds of distributions did you have in mind? I'm asking out of complete ignorance. 2.) Like in my last emails concerning Univariate I would like to, (and have done so in my checkout successfully) Make the following Class changes: interface o.a.c.m.stat.StoreUnivariate -- abstract class o.a.c.m.stat.DescriptiveStatistics this actually becomes a factory class and uses Discovery to instantiate new instances of the following implementations *default implementation* o.a.c.m.stat.StoreUnivariateImpl -- o.a.c.m.stat.univariate.StatisticsImpl Forgive me for not refamiliarizing myself with the code first, but should the storeless version perhaps be the default implementation instead? What do we lose by going that way? I'm thinking it would be nice to keep memory usage lower if possible. *alternate implementations* o.a.c.m.stat.UnivariateImpl -- o.a.c.m.stat.univariate.StorelessStatisticsImpl o.a.c.m.stat.ListUnivariateImpl -- o.a.c.m.stat.univariate.ListStatisticsImpl o.a.c.m.stat.BeanListUnivariateImpl -- o.a.c.m.stat.univariate.BeanListStatisticsImpl The benefit of this is that the Alternate Implementations can all be instantiated from the o.a.c.m.stat.DescriptiveStatistics factories newInstance(...) methods. Thus alternate implementations of DescriptiveStatistics can be written as Service Providers and set in the environment/JVM configuration. We can now write SP's for other tools like Matlab, Mathematica, JLink, C++ libraries, R, Omegahat ... the list goes on and on... Someday, I'd like to see this design extended for Bivariate Statistics and Regression Classes. Eventually for Random Number generation as well. Before we go overboard, can you give a quick example of instantiating one of the implementations? Or perhaps, both the default and one alternative implementation? Is it: import org.apache.commons.math.stat.*; ... StoreUnivariateImpl defaultImplementation = DescriptiveStatistics.newInstance() ; StoreUnivariateImpl storagelessImplementation = DescriptiveStatistics.newInstance( StorelessStatisticsImpl ) ; Al = Albert Davidson Chou Get answers to Mac questions at http://www.Mac-Mgrs.org/ . __ Do you Yahoo!? Protect your identity with Yahoo! Mail AddressGuard http://antispam.yahoo.com/whatsnewfree - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Scripting in Java ?
Danny Angus, I deleted your message accidentally and thus don't have your email address (Jakarta's mail archive completely removes them from the messages rather than simply obscuring them), otherwise I might have asked this question privately. I asked the same question on the BSF list last weekend but have not gotten any reply (though to be fair, that list is pretty inactive). Is it possible to run a script via BSF whose lifetime is comparable to that of the parent Java application? My wish is to develop a substantial part of a Java application in a non-Java scripting language and have it participate as almost a first-class citizen within its parent. My thought was perhaps to spawn a thread to run the BSF script execution and communicate with the (scripting-language-created) objects living in that thread. Al = Albert Davidson Chou Get answers to Mac questions at http://www.Mac-Mgrs.org/ . __ Do you Yahoo!? The New Yahoo! Shopping - with improved product search http://shopping.yahoo.com - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: [math] Would code for generating IC50 curves fit here?
--- Eric Pugh [EMAIL PROTECTED] wrote: Hi all, I am faced with a needing to generate IC50 (also called EC50) curves for a project. A) Does anyone know of any packages that do this? B) If not, would this be something of interest for Math? I don't think we have any curve-fitting code yet. I think a set of more widely used curves (a brief search on Google suggests that IC50/EC50 curves are used primarily in pharmacology, is that right?) would be higher on the priority list, if we were to put curve fitting in scope for some release (it probably wouldn't be the initial release, given the situation to date). Al = Albert Davidson Chou Get answers to Mac questions at http://www.Mac-Mgrs.org/ . __ Do you Yahoo!? Yahoo! SiteBuilder - Free, easy-to-use web site design software http://sitebuilder.yahoo.com - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
[math] Re: cvs commit: apply(Functor x) strategy
--- [EMAIL PROTECTED] wrote: mdiggory2003/07/14 20:45:10 Modified:math/src/java/org/apache/commons/math/stat UnivariateImpl.java AbstractStoreUnivariate.java ListUnivariateImpl.java AbstractUnivariate.java StoreUnivariateImpl.java Log: Application of apply(Functor x) strategy (thank you Al Chou) for evaluating UnivariateStatistics against the internal storage collection without exposing the collection or its bounds. Mark, I assume all the existing unit tests passed. I don't think you mentioned performance testing in your last message. I'm curious what effect the applyable strategy has on efficiency. Al = Albert Davidson Chou Get answers to Mac questions at http://www.Mac-Mgrs.org/ . __ Do you Yahoo!? SBC Yahoo! DSL - Now only $29.95 per month! http://sbc.yahoo.com - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: [math] stat package design
--- Mark R. Diggory [EMAIL PROTECTED] wrote: Prior to my changes last night increment was actually calculating the entire statistic. getValue was only returning the value of a precalculated property. The problem is that the cpu cycles need to be spent no matter the approach of full calculation in increment or partial calc in increment and partial calc in getValue. I think addVAlue is going to get called alot more than getValue, so I've optimized to reduce the amount of calculation going on in increment as much as possible. (it would be possible to maintain a boolean state about if increment has been called, this way calling getValue repeatedly will not result in repeatedly calculating the same statistical value over and over. I'll look into adding this. Sorry to post-and-run, but I'm on a business trip this week. The above is memoization, isn't it? Sounds like a good idea, especially for values that take a lot of operations to compute. Al = Albert Davidson Chou Get answers to Mac questions at http://www.Mac-Mgrs.org/ . __ Do you Yahoo!? SBC Yahoo! DSL - Now only $29.95 per month! http://sbc.yahoo.com - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: [math] abstact nonsense was Re: [math][functor] More Design Concerns
--- Mark R. Diggory [EMAIL PROTECTED] wrote: Anton Tagunov wrote: 3) BTW, probably does the future introduction of Generics (Java 1.5) promise any opportunities to work with primitive values and yet have no code duplication (a bit like STL)? I've not spent much time looking at Generics yet. I have allot to learn in this area. I believe for our purposes it suffices to describe generics as the ability to write something like (I'm too lazy to look up the exact syntax): ArrayListDouble myArray = new ArrayList()Double ; Double d ; int position = 0 ; myArray.add( new Double( 1.0 ) ) ; // Here's the part where generics make life easier. Look, Ma, no cast: d = myArray.get( position ) ; Al = Albert Davidson Chou Get answers to Mac questions at http://www.Mac-Mgrs.org/ . __ Do you Yahoo!? SBC Yahoo! DSL - Now only $29.95 per month! http://sbc.yahoo.com - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: [math] abstact nonsense was Re: [math][functor] More Design Concerns
--- Craig R. McClanahan [EMAIL PROTECTED] wrote: My understanding is that this is exactly what you'll get from the auto-unboxing capability. The compiler will be able to see that the right hand side returns a Double, and generate the code to unbox it into a double primitive for you. This is separate from Generics because it also works in other scenarios: Double d1 = new Double(1.0); // A lowly scalar instance of the wrapper double d2 = d1; // But no cast here either! d1 = d2 + 0.5; // Or here ... it is bidirectional IMO, that's even more useful than generics (or limited C++-style templates, if you prefer to think of them that way). As a math-using person, I really never wanted to have to make much of a distinction between a double and a Double (although I somewhat understand the CS considerations that could make having the distinction desirable). Al = Albert Davidson Chou Get answers to Mac questions at http://www.Mac-Mgrs.org/ . __ Do you Yahoo!? SBC Yahoo! DSL - Now only $29.95 per month! http://sbc.yahoo.com - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
[math] Fwd: Re: Change utility class to singleton with interface?
The beginnings of a thread in the Refactoring Yahoo list that may be of interest for our current design discussions: Change utility class to singleton with interface? http://groups.yahoo.com/group/refactoring/message/3797 Al = Albert Davidson Chou Get answers to Mac questions at http://www.Mac-Mgrs.org/ . __ Do you Yahoo!? SBC Yahoo! DSL - Now only $29.95 per month! http://sbc.yahoo.com - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: [math][functor] More Design Concerns
--- J.Pietschmann [EMAIL PROTECTED] wrote: [EMAIL PROTECTED] wrote: I would steer away for primative as much as possible. Keep in mind that excessive object creation can and usually is a significant performance drain, both because it is slow in itself despite all kinds of optimization as well as causing more GC. I think we should consider what the typical use cases will be for this library. Of course we could turn out to be wrong about that, but given that it's not supposed to be a heavy-duty numerics library, perhaps a more purely-OO design is not necessarily a bad thing. For instance, if we find that the library is mostly used to calculate the minimum, maximum, and average of a set of numbers for a report, and that report is only run once a month, then the efficiency of primitives vs. objects isn't that important. Of course we should not paint ourselves into a corner regarding the design. If many users start using the library for intensive numerical tasks, it would be good if the library's design was flexible enough to accommodate any needed efficiency improvements. One thing that I am curious about is the design of COLT, which claims to have very good performance. Al = Albert Davidson Chou Get answers to Mac questions at http://www.Mac-Mgrs.org/ . __ Do you Yahoo!? SBC Yahoo! DSL - Now only $29.95 per month! http://sbc.yahoo.com - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: [math][functor] More Design Concerns
--- J.Pietschmann [EMAIL PROTECTED] wrote: Al Chou wrote: Does staticness preclude extensibility? ... I clearly have never studied for a Java certification. g Thanks for the clarification, Phil. No difference to C++ or any other language with class methods I ever met. I'm not much of a C/C++ programmer, either (even less so than I am a Java programmer, actually). In Ruby it's possible to rename an inherited class method and then define a new method with the original name, effectively allowing overriding of class methods. Maybe it's the dynamic vs. static (no pun intended) typing that accounts for the difference. Al = Albert Davidson Chou Get answers to Mac questions at http://www.Mac-Mgrs.org/ . __ Do you Yahoo!? SBC Yahoo! DSL - Now only $29.95 per month! http://sbc.yahoo.com - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: [math][functor] More Design Concerns
--- Phil Steitz [EMAIL PROTECTED] wrote: (2) Considerations a.) Is consistent library design important?Can all these models interace effectively? Are all these different design models required? Is there a single design model that can span the entire library? IMHO, the most important considerations are 1) how easy the library will be to navigate and use 2) how maintainable it will be and 3) how well it will perform (including resource requirements). For all of these, I think that it is best to look at practical application use cases -- e.g., if someone wants to solve a linear system or estimate a bivariate regression model, how easy will it be for them to do that using commons-math? How well will the solution perform and scale? How easy will it be for us to extend it? Since the library does different kinds of things to satisfy some fundamentally different use cases (e.g. generating a random sample vs modelling algebraic operations on a real matrix) it is natural that multiple design patterns and implementation strategies are used. Trying to force all commons-math components into a single abstract model is not necessary, IMHO, and would likely make the library harder to use and maintain. I agree with Phil. I started to reply to Mark's original message when he posted it, but I decided to wait, as I felt I was largely basing my opinion on a possibly inappropriate context (object oriented numerics in C++ discussions I've followed over the years that largely center on expression templates). I do feel pretty strongly that unless consolidating the design of the library makes it easier for end users to use, I wouldn't make it a priority. I do support examining the architecture and design so that we don't release nonsense or paint ourselves into a corner out of stupidity or laziness, of course. b.) Which design strategy is of more interest to the project? Small compact classes used to complete one specific task, where the programming is primarily OOD in nature? or Feature rich interfaces that encapsulate full ranges of functionality, where programming is primarily procedural in nature? Here again, my opinion is that this should be determined by the practical problem being addressed. Does the intent of commons-math not favor the small classes approach somewhat? d.) Should static method utilities be avoided at all costs in both cases? OR are there specific situations were static functions do not create garbage collection considerations and issues (like, when only returning primitives). I am starting to think that we should avoid static methods, and in fact change StatUtils to require instantiation, but this has nothing to do with garbage collection, since with a stateless collection of static methods, there is no garbage to collect -- just a class loaded once per JVM. As long as there is no state associated with the class, I don't see how classloader problems could really come into play (unless users were relying on classloader priority to load different versions, which is IMHO a bad idea and could apply to any of our classes). The real issue here is extensibility. As I think more about the use cases for StatUtils, I am less convinced than I was before that the convenience and efficiency of not having to create an instance is worth the anxiety about support for extensibility. Therefore, I would support changing the methods in StatUtils to be non-static. Does staticness preclude extensibility? I assume final-ity would, but we didn't declare any methods final, AFAIK. (3.) A couple proposals: (i.) Brent and Pietschmann can you make suggestions/recommendations as to how your function object model could be applied to StaticUtil situations? Are you familiar with the Functors project and is there a possibility that they should be considered as the basic design strategy or base of implementation for your Function Object design? if they are a Commons project as well is there a possible dependency here we could take advantage of? My opinion here is that a univariate real function is an object in its own right. I suppose that it could extend Functor, but I do not see the point in this and I would personally not like having to lose the typing and force use and casting to Double to implement Object evaluate(Object obj); After briefly looking at Functor's Javadocs, I feel that its interface goes in a somewhat different direction from that which a mathematical API should. While there are commonalities, it seems to me that some of what Functor does is more easily and naturally expressed using existing mathematical operators (reminiscent of our discussion of whether, for instance, an isPositive method would be worth providing in commons-math). If we were writing a library to do abstract algebra, we might need interfaces more like Functor's. It should also be noted that this
Re: [math] Recent commits in analysis package
--- Mark R. Diggory [EMAIL PROTECTED] wrote: I suspect its the same case with Quintic solver as well, If they are just for test examples, I'll move them back there. -Mark Phil Steitz wrote: Mark R. Diggory wrote: Yes, sorry about that, I had missed adding those in the last patch. Thanks. Works for me now. What do you think about SinFunction? Do we really need that in /java? It is only used in unit tests. Yes, I suspect the quintic originally came from my root finder JUnit test case. Al = Albert Davidson Chou Get answers to Mac questions at http://www.Mac-Mgrs.org/ . __ Do you Yahoo!? SBC Yahoo! DSL - Now only $29.95 per month! http://sbc.yahoo.com - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
[math] Re: DO NOT REPLY [Bug 21023] - [PATCH] [math] Natural spline interpolation
--- Additional Comments From [EMAIL PROTECTED] 2003-06-25 01:16 --- I'll apply this, but its really alot easier to apply cvs generated patches over tarballs or unix diff. I would prefer this format over all others. Not only does cvs diff generate patches directly against the cvs copies, assuring your diffing against whats presently in the cvs, but you can also add files using it. In otherwords, if you create a file and the run cvs diff on the directory, when I apply the patch, the file will be created. Mark, I would have used cvs diff, but the file I was patching wasn't in CVS yet, so I had to patch the patch attached to the bug report. I didn't want to wait until the original patch was committed and possibly forget to even propose my patch. Al = Albert Davidson Chou Get answers to Mac questions at http://www.Mac-Mgrs.org/ . __ Do you Yahoo!? SBC Yahoo! DSL - Now only $29.95 per month! http://sbc.yahoo.com - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: [math] Re: DO NOT REPLY [Bug 21023] - [PATCH] [math] Natural spline interpolation
--- Mark R. Diggory [EMAIL PROTECTED] wrote: Understandable, and thats sensible, as long as I can find a way to apply the patch, I'll accept it. There is a way that the cvs diff strategy would still work, it would have been to create a cvs diff that applied all the changes that the tgz provided plus your changes. But, someone out there probably thinks that uncouth. I don't know if this is something that can be altered with unix diff or not, the primary problem I always have applying your patches has to do with the diff header which usually looks something like this: --- SplineInterpolator.java~1~Mon Jun 23 07:41:18 2003 +++ SplineInterpolator.java Mon Jun 23 20:31:07 2003 @@ -55,11 +55,12 @@ Maybe its an option or something I don't know about diff, but when I try to apply it I always get this can't find SplineInterpolator.java~1~ file error. Which makes sense because there never really is one in the filesystem. Do you know where the ~1~ file comes from? Yeah, that's a jEdit backup file (I have it set to save up to 10 previous versions, hence the ~N~ notation rather than the single ~ you'd get from Emacs or Vim). My usual solution for this is to cut, paste and edit a cvs diff generated header into the top of your file like this: Index: SplineInterpolator.java === RCS file: /home/cvs/jakarta-commons-sandbox/math/src/java/org/apache/commons/math/SplineInterpolator.java,v retrieving revision 1.1 diff -u -r1.1 SplineInterpolator.java --- SplineInterpolator.java Mon Jun 23 07:41:18 2003 +++ SplineInterpolator.java Mon Jun 23 20:31:07 2003 then I can apply it the same way I do others. I'll make sure to do that sort of thing in the future when necessary. Al = Albert Davidson Chou Get answers to Mac questions at http://www.Mac-Mgrs.org/ . __ Do you Yahoo!? SBC Yahoo! DSL - Now only $29.95 per month! http://sbc.yahoo.com - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: [math] Tasks remaining for initial release
--- Phil Steitz [EMAIL PROTECTED] wrote: * RealMatrixImpl is missing one method implementation -- getRank(). The most accurate way to implement this would be to add Singular Value Decomposition and use this to compute effective numerical rank. If someone wants to volunteer to do this, we can leave this in; otherwise I suggest dropping the rank computation from the interface. +1 for dropping the rank computation. I'd guess most users won't miss it. * Rootfinding framework. We need to get this rectified or just decide to stay with the simple solution in place now. My vote would be to rectify J's framework. +1 for rectifying the Pietschmann framework. * Interpolation. Al is working on cubic spline interpolation. Right? Right. * Continued work on javadoc, checkstyle, and test coverage. We need to look at test data coverage as well as path coverage. In some cases, we have good path coverage, but we have not tested all of the data boundary conditions. +1 * Additional performance and accuracy testing. If anyone is interested in helping out here, what we could really use is a wider selection of test cases for the core numerical functions and validation against either other packages (e.g. R for the statistical stuff), verified datasets, or experiments comparing implementions using floats to doubles. * Additional code review. I am planning to review the current state of all of the code to verify that the code matches the documentation and to identify obvious inefficiencies or numerical problems. It would be a good idea for others to do the same. All feedback/suggestions for improvement are welcome -- especially if accompanied by patches :-) +1 for both the above. I suspect we may be in a test-and-fix phase for a while once we enter it. Al = Albert Davidson Chou Get answers to Mac questions at http://www.Mac-Mgrs.org/ . __ Do you Yahoo!? SBC Yahoo! DSL - Now only $29.95 per month! http://sbc.yahoo.com - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: [math] Tasks remaining for initial release
--- J.Pietschmann [EMAIL PROTECTED] wrote: Al Chou wrote: * Interpolation. Al is working on cubic spline interpolation. Right? Right. Sorry if I preempted your work, but I just posted cubic spline interpolation to bugzilla for general review. There are a few design questions, to be discussed separately, and the unit test still need work. That's OK, but it would have been nice if you had informed the rest of the gang that you were even working on it. In this case I don't mind, as it's made me refresh my knowledge and even learn things I never knew before, but to avoid duplicated work in the future, let's all try to keep each other up to date on what we're working on. Are you currently working on your root finding code? IIRC, it had a few problems that prevented it from being committed into CVS. Al = Albert Davidson Chou Get answers to Mac questions at http://www.Mac-Mgrs.org/ . __ Do you Yahoo!? SBC Yahoo! DSL - Now only $29.95 per month! http://sbc.yahoo.com - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
[math] Does LU decomposition assume matrix is square?
I finally decided that cubic spline would be my first attempt at implementing interpolation, partly because of the difficulty of finding an alternative reference to NR for the Stoer Bulirsch rational function method, partly because I started to have doubts about the desirability of interpolation using high-order polynomials/rationals, and partly because I think being forced to implement tridiagonal linear systems solution is probably good for the library. I could go the NR route and embed the tridiagonal solver into the cubic spline routine, but I think it's worthwhile to provide the tridiagonal solver separately for others to use, given the frequency with which tridiagonal systems appear (in physics anyway; am I dreaming too much to think that someone might use this library to solve second order differential equations by finite differences?). I happened to notice as I started to plan my work that RealMatrixImpl.solve, which uses LU decomposition, doesn't explicitly check whether the matrix is square before proceeding with the decomposition. I think (but am not sure) that LU decomposition assumes the matrix is square. Currently it already checks whether the vector of right-hand-sides of the equations is equal to the number of rows in the matrix, which I think implicitly assumes that the matrix is square (otherwise it would probably be more correct to check against the number of columns). Al = Albert Davidson Chou Get answers to Mac questions at http://www.Mac-Mgrs.org/ . __ Do you Yahoo!? SBC Yahoo! DSL - Now only $29.95 per month! http://sbc.yahoo.com - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: [math] Does LU decomposition assume matrix is square?
--- Phil Steitz [EMAIL PROTECTED] wrote: Al Chou wrote: I finally decided that cubic spline would be my first attempt at implementing interpolation, partly because of the difficulty of finding an alternative reference to NR for the Stoer Bulirsch rational function method, partly because I started to have doubts about the desirability of interpolation using high-order polynomials/rationals +1 This is also easier to document and understand for users. Yes, in reacquainting myself with the algorithms I rediscovered one of the subtleties of using polynomial or rational interpolation, which is that you probably should pick a small subset of your tabulated points near where you want to interpolate and use only those in the interpolation, thus keeping the degree of the interpolating function kind of low and hopefully limiting the arbitrary wiggliness of the interpolating function. Cubic spline allows you to do what seems like the easiest thing, which is provide a whole table of data points and ask for interpolation anywhere within the tabulated domain. Of course, you can do a similar thing with polynomial or rational interpolation, but either the library or the user would have to provide the wrapper that asks for how many points to use in the neighborhood of the interpolation point, or alternatively the degree of the interpolating function to use, both of which require more decisions on the user's part in the process of using interpolation. , and partly because I think being forced to implement tridiagonal linear systems solution is probably good for the library. I could go the NR route and embed the tridiagonal solver into the cubic spline routine, but I think it's worthwhile to provide the tridiagonal solver separately for others to use, given the frequency with which tridiagonal systems appear (in physics anyway; am I dreaming too much to think that someone might use this library to solve second order differential equations by finite differences?). I would either embed the tridiagonal solution or just use RealMatrixImpl.solve(). Personally, I would probably take the second approach, for initial release, since the implementation exists. If users complain about the speed (which will only happen if they are fitting splines using large numbers of points), we can modify the implementation to use the faster approach. I would not see a tridiagonal solver as something that we should add to intial release of commons-math. Good suggestion. I'll start from there, given that I discovered more design decisions to be made as I sketched converting RealMatrixImpl to a RealTridiagonalMatrix I happened to notice as I started to plan my work that RealMatrixImpl.solve, which uses LU decomposition, doesn't explicitly check whether the matrix is square before proceeding with the decomposition. I think (but am not sure) that LU decomposition assumes the matrix is square. No, but it does require that the number of rows = the number of columns. I will add this check. Good catch. Currently it already checks whether the vector of right-hand-sides of the equations is equal to the number of rows in the matrix, which I think implicitly assumes that the matrix is square (otherwise it would probably be more correct to check against the number of columns). The check against the constant matrix column dimension is to make sure that the linear system(s) represented by the matrix equation are well-defined. You are correct that the coefficient matrix also needs to be square. I will add this to solve(). Once again, good catch! Cool, glad I haven't lost (all) my marbles. g Al = Albert Davidson Chou Get answers to Mac questions at http://www.Mac-Mgrs.org/ . __ Do you Yahoo!? SBC Yahoo! DSL - Now only $29.95 per month! http://sbc.yahoo.com - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: [math] Limits on StatUtil content (was Re: [math] design patterns ...)
--- Mark R. Diggory [EMAIL PROTECTED] wrote: Phil Steitz wrote: I keep reminding myself, we are the developers, there is always room for refactoring in the future. If there becomes a clear hindrance with the use of static methods, then we can refactor them into a class that needs to be instantiated. This is not too difficult to accomplish. Not for us, maybe, but it could be a real pain for the users who have written code using the static methods directly. We also need to keep reminding ourselves that what we are developing is a library for others to use. Refactoring public interfaces post release is a slow and painful process. Given that MathUtils and StatUtils are going to be public, we need to be committed to supporting the static methods that they will expose. Now thats a very strong point, A cautious side note: Its very difficult to design a product for a hypothetical future user, I've been trying to do this for the last 3 years, its very very difficult, and I'm constantly pulling in the reigns, to stop overdevelopment that is based on bad assumptions about what this hypothetical user-base is going to want. The user is, infact, part of the Open Source development process. This is where philosphies like, Release early, Release often become benificial. The user input is really needed to help drive the development direction properly. I agree we need real user input to help guide the design, just so long as we don't let ourselves be completely constrained by what the initial small user base wants/likes (the way Unix make did when it had 10 users). I think if we warn alpha/beta users that public interfaces are subject to change, we may be OK. Al = Albert Davidson Chou Get answers to Mac questions at http://www.Mac-Mgrs.org/ . __ Do you Yahoo!? SBC Yahoo! DSL - Now only $29.95 per month! http://sbc.yahoo.com - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
