Re: [Compress] Modular Commons Compress and better NIO2 Path API support
> Oh, sorry, I don't have that idea. In the plan, I will maintain > this fork for a long time. Just from a users perspective: Such projects often tend to slowly dying over the time even if single developers are enthusiastic at first place. So you should really consider if it would be more profitable to have your changes integrated upstream as they sound exactly how commons-math has moved in the past (more modules, clean up old apis and so on..) so it seems valid for commons-compress as well. Am 12.02.22 um 15:26 schrieb Glavo: Oh, sorry, I don't have that idea. In the plan, I will maintain this fork for a long time. The reason for branching it is, on the one hand, to modularize, and on the other hand, to abandon some compatibility baggage, so that it is easier to use in the new version of JDK. For example, I deleted ZipEncoding and directly used java.nio.charset.Charset. In particular, some adjustments to the encoding and file APIs have caused a little incompatibility, so I don't think it can merge back to the main line. Gilles Sadowski 于2022年2月12日周六 21:52写道: Hello. Le sam. 12 févr. 2022 à 10:08, Glavo a écrit : I maintained a fork based on Common Compress, fully adapted to JPMS, and improved the support for NIO2 path API. All the archivers and compressors are split into separate modules. Its core module is less than 90KB, and you can choose to add optional components freely. Because some APIs are not compatible with changes, I renamed its package, so it can coexist with the original commons compress. Now I have released a beta version for trial use. Welcome to try it. Please give me feedback on any problems. Thank you very much. Here is the link: https://github.com/Glavo/kala-compress Are you indirectly suggesting that the two code bases be merged? If so, please post to the "dev" ML. Regards, Gilles - To unsubscribe, e-mail: user-unsubscr...@commons.apache.org For additional commands, e-mail: user-h...@commons.apache.org - To unsubscribe, e-mail: user-unsubscr...@commons.apache.org For additional commands, e-mail: user-h...@commons.apache.org
Re: [math] example for constrain parameters for Least squares
Hi Gilles and Randy, that sounds really interesting I'll try this out, both the sigmodial and the scaling stuff! I'm relative new to all this least squares things and I'm not a mathematician so I need to learn a lot, so don't hesitate to point out things that might seem obvious to the advanced user, that's really welcome :-) Am 08.06.21 um 15:23 schrieb Gilles Sadowski: Hello. Le mar. 8 juin 2021 à 08:14, Christoph Läubrich a écrit : Hi Gilles, I have used the the INFINITY approach for a while now and it works quite good. I just recently found a problem where I got very bad fits after a handful of iterations using the LevenbergMarquardtOptimizer. The problem arises whenever there is a relative small range of valid values for *one* parameter. Too keep up with the Gausian example assume that the mean is only valid in a small window, but norm and sigma are completely free. My guess is, if in the data there are outlier that indicates a strong maximum outside this range the optimizer try to go in that 'direction' because I reject this solution it 'gives up' as it seems evident that there is no better solution. This then can result in a gausian that is very thin and a really bad fit (cost e.g about 1E4). If I help the optimizer (e.g. by adjusting the initial guess of sigma) it finds a much better solution (cost about 1E-9). So what I would need to tell the Optimizer (not sure if this is possible at all!) that not the *whole* solution is bad, but only the choice of *one* variable so it could use larger increments for the other variables. If you want to restrict the range of, say, the mean: public class MyFunc implements ParametricUnivariateFunction { private final Sigmoid meanTransform; public MyFunc(double minMean, double maxMean) { meanTransform = new Sigmoid(minMean, maxMean); } public double value(double x, double ... param) { final double mu = meanTransform.value(param[1]); // param[1] is the mean. final double diff = x - mu; final double norm = param[0]; // param[0] is the height. final double s = param[2]; // param[2] is the standard deviation. final double i2s2 = 1 / (2 * s * s); return Gaussian.value(diff, norm, i2s2); } } // ... final MyFunc f = new MyFunc(min, max); final double[] best = fitter.fit(f); // Perform fit. final double bestMean = new Logit(min, max).value(best[1]); HTH, Gilles - To unsubscribe, e-mail: user-unsubscr...@commons.apache.org For additional commands, e-mail: user-h...@commons.apache.org
Re: [math] example for constrain parameters for Least squares
Hi Gilles, I have used the the INFINITY approach for a while now and it works quite good. I just recently found a problem where I got very bad fits after a handful of iterations using the LevenbergMarquardtOptimizer. The problem arises whenever there is a relative small range of valid values for *one* parameter. Too keep up with the Gausian example assume that the mean is only valid in a small window, but norm and sigma are completely free. My guess is, if in the data there are outlier that indicates a strong maximum outside this range the optimizer try to go in that 'direction' because I reject this solution it 'gives up' as it seems evident that there is no better solution. This then can result in a gausian that is very thin and a really bad fit (cost e.g about 1E4). If I help the optimizer (e.g. by adjusting the initial guess of sigma) it finds a much better solution (cost about 1E-9). So what I would need to tell the Optimizer (not sure if this is possible at all!) that not the *whole* solution is bad, but only the choice of *one* variable so it could use larger increments for the other variables. Am 11.08.20 um 16:11 schrieb Gilles Sadowski: Hello. Le mar. 11 août 2020 à 12:08, Christoph Läubrich a écrit : Thanks for your patience, maybe a better/simpler example would be [1], I want to find the best fit using LSB "LSB" ? under the constraint that the height of the curve never is above 0.7 (even though without constrains the optimizer would find a better solution around 8.5). It occurs to me that this would be more properly defined as a least-square problem by assigning an error bar (weight) to each data point. So my first idea way to copy/extend GausianCurveFitter to accept some kind of "maxHeight(...)", adjust the function to return INF for h>maxHeight, I was just unsure that return INF is allowed (per definition) as it is not mentioned anywhere in the userdocs. And if I want to limit others (like max position) it would work the same way. Hopefully "infinity" will not cause an issue; you could try and check. [Then if it does, just use any suitably large value.] In my final problem I need to limit the height, width and position of the bell-curve to fall into certain bounds, but there is no direct relation (e.g. width must be three times the height). Then, width is totally correlated (to height) and there are only 2 parameters to fit (height and mean); hence it's probably better (and more efficient) to use that fact instead of defining the correlation as a constraint i.e. define (untested): ---CUT--- public class MyFunc implements ParametricUnivariateFunction { public double value(double x, double ... param) { final double diff = x - param[1]; // param[1] is the mean. final double norm = param[0]; // param[0] is the height. final double s = 3 * norm; // "constraint". final double i2s2 = 1 / (2 * s * s); return Gaussian.value(diff, norm, i2s2); } // And similarly for the "gradient": // https://gitbox.apache.org/repos/asf?p=commons-math.git;a=blob;f=src/main/java/org/apache/commons/math4/analysis/function/Gaussian.java;h=08dcac0d4d37179bc85a7071c84a6cb289c09f02;hb=HEAD#l143 } ---CUT--- to be passed to "SimpleCurveFitter". Regards, Gilles [1] https://crlbucophysics101.files.wordpress.com/2015/02/gaussian.png?w=538&h=294 Am 11.08.20 um 11:18 schrieb Gilles Sadowski: Hi. 2020-08-11 8:51 UTC+02:00, Christoph Läubrich : Hi Gilles, Just to make clear I don't suspect any error with GausianCurveFitter, I just don't understand how the advice in the user-doc to restrict parameter (for general problems) could be applied to a concrete problem and thus chosen GausianCurvefitter as an example as it uses LeastSquaresBuilder. I also noticed that Gaussian Fitter has a restriction on parameters (norm can't be negative) that is handled in a third way (returning Double.POSITIVE_INFINITY instead of Parameter Validator) not mentioned in the userdoc at all, so I wonder if this is a general purpose solution for restricting parameters (seems the simplest approach). I'd indeed suggest to first try the same trick as in "GaussianCurveFitter" (i.e. return a "high" value for arguments outside a known range). That way, you only have to define a suitable "ParametricUnivariateFunction" and pass it to "SimpleCurveFitter". One case for the "ParameterValidator" is when some of the model parameters might be correlated to others. But using it makes it necessary that you handle yourself all the arguments to be passed to the "LeastSquaresProblem". To take the gausian example for my use case, consider an observed signal similar to [1], given I know (from other source as the plain data) for example that the result must be found in th
Re: [math] AffineTransformMatrix2D
Hi Matt, thanks for the feedback, I have commented on the bug. Regarding the "direct" method you are most probably right. I have arrays that are easily can extend 10k or even 100k points that means in extreme cases 200k Objects created and instantly destroyed as well as the corresponding method calls. Even if this is not noticeable in some scenarios it adds an unnecessary penalty. I have already thought about the util method (and actually using it currently) but its a bit bad to code against internal representations. In the end of course I do not need to use the Affine transform at all and can work on transform-matrix itself but thats not really the goal :-) What do you think about two methods, transformX and transformY that take a single double and return the transformed value? public double transformX(final double x, double y) { return LinearCombination.value(m00, x, m01, y) + m02; } public double transformY(final double x, double y) { return LinearCombination.value(m10, x, m11, y) + m12; } I also noticed that there is no dedicated "shear" method, if I open a PR could it be in-cooperated? Am 21.03.21 um 19:07 schrieb Matt Juntunen: Hello, It's entirely possible that the OSGi headers are incorrect. I've created a new issue (GEOMETRY-116 [1]) for it. We can continue that part of this discussion there. The "internal" package in commons-geometry-core is intended to be internal from the standpoint of the library. Classes under this package (and "internal" packages in other modules) are not considered part of the public API and are not guaranteed to be binary compatible with previous releases. From a practical standpoint, however, these packages do need to be exported and available for use by the other modules in the library. In regard to the new method you mentioned, I am hesitant to add that to the API. The main reason is that there are a large number of possible ways to represent 2D vectors as arrays. For example, the xy components could be in two parallel arrays (as in your example), a single array with interleaved xy components (eg, [x1,y1,x2,y2,...]), a multidimensional array (eg, [[x1,y1],[x2,y2]...]), etc. Adding this method could open up a can of worms as to which formats are accepted by the API. It would keep the API cleaner by leaving this functionality to consuming applications. You could, for example, use AffineTransformMatrix2D as-is for matrix creation and manipulation and create a utility method in your application that performs the required transformations using the target format. public static void applyTransform(AffineTransformMatrix2D m, double[] x, double[] y) { // input checking, etc double[] mArr = m.toArray(); for (int i = 0; i < x.length; ++i { x[i] = LinearCombination.value(mArr[0], x[i], mArr[1], y[i]) + mArr[2]; y[i] = LinearCombination.value(mArr[3], x[i], mArr[4], y[i]) + mArr[5]; } } I would also be interested in seeing if there is an actual performance improvement when skipping the Vector2D instantiations. In my experience, optimizations such as this sometimes don't really make a difference. Regards, Matt J [1] https://issues.apache.org/jira/projects/GEOMETRY/issues/GEOMETRY-116 From: Christoph Läubrich Sent: Sunday, March 21, 2021 8:40 AM To: user@commons.apache.org Subject: Re: [math] 2D Line getOffset and natural orientation Hi Matt, just some feedback on the AffineTransformMatrix2D in general I think it would be good to have methods that accept a plain array. for example I have an array of x and y values and like to transform them. Currently I need to first construct a Vector of each x/y pair, then apply the transform (what create another Vector), and copy the data back to the array. so a method apply [double[] x, double[] y) that simply performs the transform in-place would prevent a lot of instantiation overhead. Am 20.03.21 um 13:25 schrieb Matt Juntunen: It's available on maven central [1]. You can find the code on the Apache git repo [2] or on Github [3]. The user guide [4] gives an overview of the library along with code examples so you can get a feel for how it works. Don't hesitate to ask if you have any questions. -Matt [1] https://mvnrepository.com/artifact/org.apache.commons/commons-geometry-euclidean [2] https://gitbox.apache.org/repos/asf?p=commons-geometry.git [3] https://github.com/apache/commons-geometry [4] https://commons.apache.org/proper/commons-geometry/userguide/index.html From: Patrik Karlström Sent: Saturday, March 20, 2021 4:30 AM To: Commons Users List Subject: Re: [math] 2D Line getOffset and natural orientation Thanks for the clarification, I would be happy to try out beta1 of commons-geometry, is there a staging repo for it? /Patrik Den fre 19 mars 2021 kl 00:29 skrev Matt Juntunen :
Re: [math] 2D Line getOffset and natural orientation
Hi Matt, just some feedback on the AffineTransformMatrix2D in general I think it would be good to have methods that accept a plain array. for example I have an array of x and y values and like to transform them. Currently I need to first construct a Vector of each x/y pair, then apply the transform (what create another Vector), and copy the data back to the array. so a method apply [double[] x, double[] y) that simply performs the transform in-place would prevent a lot of instantiation overhead. Am 20.03.21 um 13:25 schrieb Matt Juntunen: It's available on maven central [1]. You can find the code on the Apache git repo [2] or on Github [3]. The user guide [4] gives an overview of the library along with code examples so you can get a feel for how it works. Don't hesitate to ask if you have any questions. -Matt [1] https://mvnrepository.com/artifact/org.apache.commons/commons-geometry-euclidean [2] https://gitbox.apache.org/repos/asf?p=commons-geometry.git [3] https://github.com/apache/commons-geometry [4] https://commons.apache.org/proper/commons-geometry/userguide/index.html From: Patrik Karlström Sent: Saturday, March 20, 2021 4:30 AM To: Commons Users List Subject: Re: [math] 2D Line getOffset and natural orientation Thanks for the clarification, I would be happy to try out beta1 of commons-geometry, is there a staging repo for it? /Patrik Den fre 19 mars 2021 kl 00:29 skrev Matt Juntunen : Hello. I agree that the docs are not really clear on the "natural orientation" part there. Basically, the possible results are as follows: * positive - The argument lies on the right side of the calling line, with the "right side" defined as the side on the right when looking along the calling line in its defined direction. (This is what I believe is meant by "natural orientation".) * negative - The argument lies on the left side of the calling line. * zero - The lines have points in common, meaning they intersect or are coincident. On a side note, if you're writing geometric code, I would suggest trying out the new(-ish) commons-geometry library [1]. It is a rewrite and extension of the commons-math geometry code and, IMHO, the API is more user-friendly. (Disclaimer: I wrote the majority of it so I am completely biased 🙂 Regards, Matt J [1] https://commons.apache.org/proper/commons-geometry/ [https://avatars.githubusercontent.com/u/47359?s=400&v=4]< https://github.com/apache/commons-geometry/blob/master/commons-geometry-euclidean/src/main/java/org/apache/commons/geometry/euclidean/twod/Line.java#L370 apache/commons-geometry< https://github.com/apache/commons-geometry/blob/master/commons-geometry-euclidean/src/main/java/org/apache/commons/geometry/euclidean/twod/Line.java#L370 Apache Commons Geometry. Contribute to apache/commons-geometry development by creating an account on GitHub. github.com From: Patrik Karlström Sent: Thursday, March 18, 2021 3:42 AM To: user@commons.apache.org Subject: Re: [math] 2D Line getOffset and natural orientation Oh, scratch that! I had three parallel lines and picked the wrong pair (and the correct values came from another calculation). Still interested in "natural orientation" though. Den tors 18 mars 2021 kl 07:19 skrev Patrik Karlström : Sometimes (50/50?) I'm getting an unexpected signum from org.apache.commons.math3.geometry.euclidean.twod.Line.getOffset(Line line). The absolute value itself is correct. From the javadoc: "The offset is 0 if both lines are the same, it is positive if the line is on the right side of the instance and negative if it is on the left side, according to its natural orientation." I suspect my problem relates to "according to its natural orientation." What is a natural orientation and what can I do to adapt to it, when to reverse my lines? If that's the case. https://commons.apache.org/proper/commons-math/javadocs/api-3.6.1/org/apache/commons/math3/geometry/euclidean/twod/Line.html#getOffset(org.apache.commons.math3.geometry.euclidean.twod.Line) - To unsubscribe, e-mail: user-unsubscr...@commons.apache.org For additional commands, e-mail: user-h...@commons.apache.org - To unsubscribe, e-mail: user-unsubscr...@commons.apache.org For additional commands, e-mail: user-h...@commons.apache.org
Re: [math] 2D Line getOffset and natural orientation
Taking a deeper look the whole package org.apache.commons.geometry.core.internal seems to be required elsewhere and should be public+exported. Is there any rationale to make this "internal"? Internal packages should really only be used in the module itself. If its proven to be usefull in other modules too it should be proper exported. Otherwise the module is not usable in an OSGi framework. Am 20.03.21 um 13:25 schrieb Matt Juntunen: It's available on maven central [1]. You can find the code on the Apache git repo [2] or on Github [3]. The user guide [4] gives an overview of the library along with code examples so you can get a feel for how it works. Don't hesitate to ask if you have any questions. -Matt [1] https://mvnrepository.com/artifact/org.apache.commons/commons-geometry-euclidean [2] https://gitbox.apache.org/repos/asf?p=commons-geometry.git [3] https://github.com/apache/commons-geometry [4] https://commons.apache.org/proper/commons-geometry/userguide/index.html From: Patrik Karlström Sent: Saturday, March 20, 2021 4:30 AM To: Commons Users List Subject: Re: [math] 2D Line getOffset and natural orientation Thanks for the clarification, I would be happy to try out beta1 of commons-geometry, is there a staging repo for it? /Patrik Den fre 19 mars 2021 kl 00:29 skrev Matt Juntunen : Hello. I agree that the docs are not really clear on the "natural orientation" part there. Basically, the possible results are as follows: * positive - The argument lies on the right side of the calling line, with the "right side" defined as the side on the right when looking along the calling line in its defined direction. (This is what I believe is meant by "natural orientation".) * negative - The argument lies on the left side of the calling line. * zero - The lines have points in common, meaning they intersect or are coincident. On a side note, if you're writing geometric code, I would suggest trying out the new(-ish) commons-geometry library [1]. It is a rewrite and extension of the commons-math geometry code and, IMHO, the API is more user-friendly. (Disclaimer: I wrote the majority of it so I am completely biased 🙂 Regards, Matt J [1] https://commons.apache.org/proper/commons-geometry/ [https://avatars.githubusercontent.com/u/47359?s=400&v=4]< https://github.com/apache/commons-geometry/blob/master/commons-geometry-euclidean/src/main/java/org/apache/commons/geometry/euclidean/twod/Line.java#L370 apache/commons-geometry< https://github.com/apache/commons-geometry/blob/master/commons-geometry-euclidean/src/main/java/org/apache/commons/geometry/euclidean/twod/Line.java#L370 Apache Commons Geometry. Contribute to apache/commons-geometry development by creating an account on GitHub. github.com From: Patrik Karlström Sent: Thursday, March 18, 2021 3:42 AM To: user@commons.apache.org Subject: Re: [math] 2D Line getOffset and natural orientation Oh, scratch that! I had three parallel lines and picked the wrong pair (and the correct values came from another calculation). Still interested in "natural orientation" though. Den tors 18 mars 2021 kl 07:19 skrev Patrik Karlström : Sometimes (50/50?) I'm getting an unexpected signum from org.apache.commons.math3.geometry.euclidean.twod.Line.getOffset(Line line). The absolute value itself is correct. From the javadoc: "The offset is 0 if both lines are the same, it is positive if the line is on the right side of the instance and negative if it is on the left side, according to its natural orientation." I suspect my problem relates to "according to its natural orientation." What is a natural orientation and what can I do to adapt to it, when to reverse my lines? If that's the case. https://commons.apache.org/proper/commons-math/javadocs/api-3.6.1/org/apache/commons/math3/geometry/euclidean/twod/Line.html#getOffset(org.apache.commons.math3.geometry.euclidean.twod.Line) - To unsubscribe, e-mail: user-unsubscr...@commons.apache.org For additional commands, e-mail: user-h...@commons.apache.org - To unsubscribe, e-mail: user-unsubscr...@commons.apache.org For additional commands, e-mail: user-h...@commons.apache.org
Re: [math] 2D Line getOffset and natural orientation
Another thing I noted: DoubleFunction2N is used but resides in the org.apache.commons.geometry.core.internal package and thus is also not exported/useable in an OSGi environment this would give CNF exceptions if actually run inside one. Am 20.03.21 um 13:25 schrieb Matt Juntunen: It's available on maven central [1]. You can find the code on the Apache git repo [2] or on Github [3]. The user guide [4] gives an overview of the library along with code examples so you can get a feel for how it works. Don't hesitate to ask if you have any questions. -Matt [1] https://mvnrepository.com/artifact/org.apache.commons/commons-geometry-euclidean [2] https://gitbox.apache.org/repos/asf?p=commons-geometry.git [3] https://github.com/apache/commons-geometry [4] https://commons.apache.org/proper/commons-geometry/userguide/index.html From: Patrik Karlström Sent: Saturday, March 20, 2021 4:30 AM To: Commons Users List Subject: Re: [math] 2D Line getOffset and natural orientation Thanks for the clarification, I would be happy to try out beta1 of commons-geometry, is there a staging repo for it? /Patrik Den fre 19 mars 2021 kl 00:29 skrev Matt Juntunen : Hello. I agree that the docs are not really clear on the "natural orientation" part there. Basically, the possible results are as follows: * positive - The argument lies on the right side of the calling line, with the "right side" defined as the side on the right when looking along the calling line in its defined direction. (This is what I believe is meant by "natural orientation".) * negative - The argument lies on the left side of the calling line. * zero - The lines have points in common, meaning they intersect or are coincident. On a side note, if you're writing geometric code, I would suggest trying out the new(-ish) commons-geometry library [1]. It is a rewrite and extension of the commons-math geometry code and, IMHO, the API is more user-friendly. (Disclaimer: I wrote the majority of it so I am completely biased 🙂 Regards, Matt J [1] https://commons.apache.org/proper/commons-geometry/ [https://avatars.githubusercontent.com/u/47359?s=400&v=4]< https://github.com/apache/commons-geometry/blob/master/commons-geometry-euclidean/src/main/java/org/apache/commons/geometry/euclidean/twod/Line.java#L370 apache/commons-geometry< https://github.com/apache/commons-geometry/blob/master/commons-geometry-euclidean/src/main/java/org/apache/commons/geometry/euclidean/twod/Line.java#L370 Apache Commons Geometry. Contribute to apache/commons-geometry development by creating an account on GitHub. github.com From: Patrik Karlström Sent: Thursday, March 18, 2021 3:42 AM To: user@commons.apache.org Subject: Re: [math] 2D Line getOffset and natural orientation Oh, scratch that! I had three parallel lines and picked the wrong pair (and the correct values came from another calculation). Still interested in "natural orientation" though. Den tors 18 mars 2021 kl 07:19 skrev Patrik Karlström : Sometimes (50/50?) I'm getting an unexpected signum from org.apache.commons.math3.geometry.euclidean.twod.Line.getOffset(Line line). The absolute value itself is correct. From the javadoc: "The offset is 0 if both lines are the same, it is positive if the line is on the right side of the instance and negative if it is on the left side, according to its natural orientation." I suspect my problem relates to "according to its natural orientation." What is a natural orientation and what can I do to adapt to it, when to reverse my lines? If that's the case. https://commons.apache.org/proper/commons-math/javadocs/api-3.6.1/org/apache/commons/math3/geometry/euclidean/twod/Line.html#getOffset(org.apache.commons.math3.geometry.euclidean.twod.Line) - To unsubscribe, e-mail: user-unsubscr...@commons.apache.org For additional commands, e-mail: user-h...@commons.apache.org - To unsubscribe, e-mail: user-unsubscr...@commons.apache.org For additional commands, e-mail: user-h...@commons.apache.org
Re: [math] 2D Line getOffset and natural orientation
Hi Matt, I try to use the new package in an OSGi application but only the org.apache.commons.geometry.euclidean package is exported. Is thsi intentional? I'd like to use AffineTransformMatrix2D from the org.apache.commons.geometry.euclidean.twod package for example but don't understand how one is created if the package is private... Am 20.03.21 um 13:25 schrieb Matt Juntunen: It's available on maven central [1]. You can find the code on the Apache git repo [2] or on Github [3]. The user guide [4] gives an overview of the library along with code examples so you can get a feel for how it works. Don't hesitate to ask if you have any questions. -Matt [1] https://mvnrepository.com/artifact/org.apache.commons/commons-geometry-euclidean [2] https://gitbox.apache.org/repos/asf?p=commons-geometry.git [3] https://github.com/apache/commons-geometry [4] https://commons.apache.org/proper/commons-geometry/userguide/index.html From: Patrik Karlström Sent: Saturday, March 20, 2021 4:30 AM To: Commons Users List Subject: Re: [math] 2D Line getOffset and natural orientation Thanks for the clarification, I would be happy to try out beta1 of commons-geometry, is there a staging repo for it? /Patrik Den fre 19 mars 2021 kl 00:29 skrev Matt Juntunen : Hello. I agree that the docs are not really clear on the "natural orientation" part there. Basically, the possible results are as follows: * positive - The argument lies on the right side of the calling line, with the "right side" defined as the side on the right when looking along the calling line in its defined direction. (This is what I believe is meant by "natural orientation".) * negative - The argument lies on the left side of the calling line. * zero - The lines have points in common, meaning they intersect or are coincident. On a side note, if you're writing geometric code, I would suggest trying out the new(-ish) commons-geometry library [1]. It is a rewrite and extension of the commons-math geometry code and, IMHO, the API is more user-friendly. (Disclaimer: I wrote the majority of it so I am completely biased 🙂 Regards, Matt J [1] https://commons.apache.org/proper/commons-geometry/ [https://avatars.githubusercontent.com/u/47359?s=400&v=4]< https://github.com/apache/commons-geometry/blob/master/commons-geometry-euclidean/src/main/java/org/apache/commons/geometry/euclidean/twod/Line.java#L370 apache/commons-geometry< https://github.com/apache/commons-geometry/blob/master/commons-geometry-euclidean/src/main/java/org/apache/commons/geometry/euclidean/twod/Line.java#L370 Apache Commons Geometry. Contribute to apache/commons-geometry development by creating an account on GitHub. github.com From: Patrik Karlström Sent: Thursday, March 18, 2021 3:42 AM To: user@commons.apache.org Subject: Re: [math] 2D Line getOffset and natural orientation Oh, scratch that! I had three parallel lines and picked the wrong pair (and the correct values came from another calculation). Still interested in "natural orientation" though. Den tors 18 mars 2021 kl 07:19 skrev Patrik Karlström : Sometimes (50/50?) I'm getting an unexpected signum from org.apache.commons.math3.geometry.euclidean.twod.Line.getOffset(Line line). The absolute value itself is correct. From the javadoc: "The offset is 0 if both lines are the same, it is positive if the line is on the right side of the instance and negative if it is on the left side, according to its natural orientation." I suspect my problem relates to "according to its natural orientation." What is a natural orientation and what can I do to adapt to it, when to reverse my lines? If that's the case. https://commons.apache.org/proper/commons-math/javadocs/api-3.6.1/org/apache/commons/math3/geometry/euclidean/twod/Line.html#getOffset(org.apache.commons.math3.geometry.euclidean.twod.Line) - To unsubscribe, e-mail: user-unsubscr...@commons.apache.org For additional commands, e-mail: user-h...@commons.apache.org - To unsubscribe, e-mail: user-unsubscr...@commons.apache.org For additional commands, e-mail: user-h...@commons.apache.org
Re: [math] DerivativeStructure and erf/erfx
Thanks Gilles, I'm currently trying to dive into all of this a bit and just wondering: Will there still be new releases of commons-math3, or is it postponed in favor of math4? I'm asking this because it might be an option to add erf(x) support to Derivative structure in math3 first (as it is not ported yet), but of course this only makes sense if it will be released some day. Am 19.08.20 um 23:51 schrieb Gilles Sadowski: Hi. Le mer. 19 août 2020 à 08:38, Christoph Läubrich a écrit : Im mostly using - math3.analysis In view of the next major version of CM, it would be nice to refactor codes in that package using the JDK java.util.function package, e.g. replace o.a.c.m.analysis.UnivariateFunction with java.util.function.DoubleUnaryOperator and drop redundant utilities in o.a.c.m.analysis.FuncitonUtils - math3.fitting - math3.linear - math3.fitting.leastsquares IMO, a way to modernize CM is to modularize it. A first step could consist in "fitting" and "linear" becoming (maven) modules. Regards, Gilles And some of the common util/exception things that drip in when using the above. Am 14.08.20 um 18:42 schrieb Gilles Sadowski: Le jeu. 13 août 2020 à 15:46, Christoph Läubrich a écrit : Hi, thnaks for the information, I'm currently using the "old" commons-math3 package, would it be possible to use erf with DerivativeStructure there? Technically, the issue could be that the tools in the "o.a.c.m.special" package were not designed as those in the "o.a.c.m.analysis.function" package. And IIUC, "DerivativeStructure" builds on the latter. But that doesn't mean it cannot be done (I've not used it personally). A lot of work has been done on the "master" branch; a 3.7 release would mean backporting everything; not a trivial task and (if at all possible) it would take more time (by a tall order) than anyone is willing to devote to that library. IMHO, it's about time that we release v4.0 (which has been delayed by more than 4 years, according to when the decision has been taken to start development on that next major version). > We are looking for volunteer maintainers for the math tools available for commons math in general or for "commons-numbers"? Both and more: * Commons Geometry (spun off the "o.a.c.m.geometry" package) * Commons Statistics (intended to replace the "o.a.c.m.stat" package) * Commons RNG (replacement of the "o.a.c.m.random" package, and pretty feature-complete as of v1.3). I must confess I have only recently noticed that there is development in commons-math-4 and might be out of date with my knowledge a bit as an "old" math3 user :-) Which classes/packages have you been using? Gilles Am 13.08.20 um 15:12 schrieb Gilles Sadowski: Hi. Le jeu. 13 août 2020 à 12:48, Christoph Läubrich a écrit : I'd like to use the org.apache.commons.math3.special.Erf with a DerivativeStructure but is seems there is no pre-defined function for this (like exp()...). So the question is, can Erf transformed inside a DerivativeStructure somehow or does the DerivativeStructure need to support that function directly? Please note that "Erf" has been moved to the new "Commons Numbers" component[1], along with the other Gamma-related functions. The "DerivativeStructure" is one among several utilities that still need to be ported (and refactored) there.[2] We are looking for volunteer maintainers for the math tools available from the Commons libraries. The move to more focused, smaller, components should hopefully simplify development... Best, Gilles [1] https://gitbox.apache.org/repos/asf?p=commons-numbers.git;a=tree;f=commons-numbers-gamma/src/main/java/org/apache/commons/numbers/gamma;h=cb95316a3be45ebfcf938eab63f362f59e590fca;hb=HEAD [2] https://issues.apache.org/jira/browse/NUMBERS-69 - To unsubscribe, e-mail: user-unsubscr...@commons.apache.org For additional commands, e-mail: user-h...@commons.apache.org - To unsubscribe, e-mail: user-unsubscr...@commons.apache.org For additional commands, e-mail: user-h...@commons.apache.org
Re: [math] DerivativeStructure and erf/erfx
Im mostly using - math3.analysis - math3.fitting - math3.linear - math3.fitting.leastsquares And some of the common util/exception things that drip in when using the above. Am 14.08.20 um 18:42 schrieb Gilles Sadowski: Le jeu. 13 août 2020 à 15:46, Christoph Läubrich a écrit : Hi, thnaks for the information, I'm currently using the "old" commons-math3 package, would it be possible to use erf with DerivativeStructure there? Technically, the issue could be that the tools in the "o.a.c.m.special" package were not designed as those in the "o.a.c.m.analysis.function" package. And IIUC, "DerivativeStructure" builds on the latter. But that doesn't mean it cannot be done (I've not used it personally). A lot of work has been done on the "master" branch; a 3.7 release would mean backporting everything; not a trivial task and (if at all possible) it would take more time (by a tall order) than anyone is willing to devote to that library. IMHO, it's about time that we release v4.0 (which has been delayed by more than 4 years, according to when the decision has been taken to start development on that next major version). > We are looking for volunteer maintainers for the math tools available for commons math in general or for "commons-numbers"? Both and more: * Commons Geometry (spun off the "o.a.c.m.geometry" package) * Commons Statistics (intended to replace the "o.a.c.m.stat" package) * Commons RNG (replacement of the "o.a.c.m.random" package, and pretty feature-complete as of v1.3). I must confess I have only recently noticed that there is development in commons-math-4 and might be out of date with my knowledge a bit as an "old" math3 user :-) Which classes/packages have you been using? Gilles Am 13.08.20 um 15:12 schrieb Gilles Sadowski: Hi. Le jeu. 13 août 2020 à 12:48, Christoph Läubrich a écrit : I'd like to use the org.apache.commons.math3.special.Erf with a DerivativeStructure but is seems there is no pre-defined function for this (like exp()...). So the question is, can Erf transformed inside a DerivativeStructure somehow or does the DerivativeStructure need to support that function directly? Please note that "Erf" has been moved to the new "Commons Numbers" component[1], along with the other Gamma-related functions. The "DerivativeStructure" is one among several utilities that still need to be ported (and refactored) there.[2] We are looking for volunteer maintainers for the math tools available from the Commons libraries. The move to more focused, smaller, components should hopefully simplify development... Best, Gilles [1] https://gitbox.apache.org/repos/asf?p=commons-numbers.git;a=tree;f=commons-numbers-gamma/src/main/java/org/apache/commons/numbers/gamma;h=cb95316a3be45ebfcf938eab63f362f59e590fca;hb=HEAD [2] https://issues.apache.org/jira/browse/NUMBERS-69 - To unsubscribe, e-mail: user-unsubscr...@commons.apache.org For additional commands, e-mail: user-h...@commons.apache.org - To unsubscribe, e-mail: user-unsubscr...@commons.apache.org For additional commands, e-mail: user-h...@commons.apache.org
Re: [math] DerivativeStructure and erf/erfx
Hi, thnaks for the information, I'm currently using the "old" commons-math3 package, would it be possible to use erf with DerivativeStructure there? > We are looking for volunteer maintainers for the math tools available for commons math in general or for "commons-numbers"? I must confess I have only recently noticed that there is development in commons-math-4 and might be out of date with my knowledge a bit as an "old" math3 user :-) Am 13.08.20 um 15:12 schrieb Gilles Sadowski: Hi. Le jeu. 13 août 2020 à 12:48, Christoph Läubrich a écrit : I'd like to use the org.apache.commons.math3.special.Erf with a DerivativeStructure but is seems there is no pre-defined function for this (like exp()...). So the question is, can Erf transformed inside a DerivativeStructure somehow or does the DerivativeStructure need to support that function directly? Please note that "Erf" has been moved to the new "Commons Numbers" component[1], along with the other Gamma-related functions. The "DerivativeStructure" is one among several utilities that still need to be ported (and refactored) there.[2] We are looking for volunteer maintainers for the math tools available from the Commons libraries. The move to more focused, smaller, components should hopefully simplify development... Best, Gilles [1] https://gitbox.apache.org/repos/asf?p=commons-numbers.git;a=tree;f=commons-numbers-gamma/src/main/java/org/apache/commons/numbers/gamma;h=cb95316a3be45ebfcf938eab63f362f59e590fca;hb=HEAD [2] https://issues.apache.org/jira/browse/NUMBERS-69 - To unsubscribe, e-mail: user-unsubscr...@commons.apache.org For additional commands, e-mail: user-h...@commons.apache.org - To unsubscribe, e-mail: user-unsubscr...@commons.apache.org For additional commands, e-mail: user-h...@commons.apache.org
[math] DerivativeStructure and erf/erfx
I'd like to use the org.apache.commons.math3.special.Erf with a DerivativeStructure but is seems there is no pre-defined function for this (like exp()...). So the question is, can Erf transformed inside a DerivativeStructure somehow or does the DerivativeStructure need to support that function directly? - To unsubscribe, e-mail: user-unsubscr...@commons.apache.org For additional commands, e-mail: user-h...@commons.apache.org
Re: [math] example for constrain parameters for Least squares
LSB = LeasSquaresBuilder :-) It seems to work with INF in my testings for one exception: If the starting point is in an invalid range, an exception is thrown (unable to perform Q.R decomposition) so if using this technique one must take care that the initial guess is always inside valid bounds! Am 11.08.20 um 16:11 schrieb Gilles Sadowski: Hello. Le mar. 11 août 2020 à 12:08, Christoph Läubrich a écrit : Thanks for your patience, maybe a better/simpler example would be [1], I want to find the best fit using LSB "LSB" ? under the constraint that the height of the curve never is above 0.7 (even though without constrains the optimizer would find a better solution around 8.5). It occurs to me that this would be more properly defined as a least-square problem by assigning an error bar (weight) to each data point. So my first idea way to copy/extend GausianCurveFitter to accept some kind of "maxHeight(...)", adjust the function to return INF for h>maxHeight, I was just unsure that return INF is allowed (per definition) as it is not mentioned anywhere in the userdocs. And if I want to limit others (like max position) it would work the same way. Hopefully "infinity" will not cause an issue; you could try and check. [Then if it does, just use any suitably large value.] In my final problem I need to limit the height, width and position of the bell-curve to fall into certain bounds, but there is no direct relation (e.g. width must be three times the height). Then, width is totally correlated (to height) and there are only 2 parameters to fit (height and mean); hence it's probably better (and more efficient) to use that fact instead of defining the correlation as a constraint i.e. define (untested): ---CUT--- public class MyFunc implements ParametricUnivariateFunction { public double value(double x, double ... param) { final double diff = x - param[1]; // param[1] is the mean. final double norm = param[0]; // param[0] is the height. final double s = 3 * norm; // "constraint". final double i2s2 = 1 / (2 * s * s); return Gaussian.value(diff, norm, i2s2); } // And similarly for the "gradient": // https://gitbox.apache.org/repos/asf?p=commons-math.git;a=blob;f=src/main/java/org/apache/commons/math4/analysis/function/Gaussian.java;h=08dcac0d4d37179bc85a7071c84a6cb289c09f02;hb=HEAD#l143 } ---CUT--- to be passed to "SimpleCurveFitter". Regards, Gilles [1] https://crlbucophysics101.files.wordpress.com/2015/02/gaussian.png?w=538&h=294 Am 11.08.20 um 11:18 schrieb Gilles Sadowski: Hi. 2020-08-11 8:51 UTC+02:00, Christoph Läubrich : Hi Gilles, Just to make clear I don't suspect any error with GausianCurveFitter, I just don't understand how the advice in the user-doc to restrict parameter (for general problems) could be applied to a concrete problem and thus chosen GausianCurvefitter as an example as it uses LeastSquaresBuilder. I also noticed that Gaussian Fitter has a restriction on parameters (norm can't be negative) that is handled in a third way (returning Double.POSITIVE_INFINITY instead of Parameter Validator) not mentioned in the userdoc at all, so I wonder if this is a general purpose solution for restricting parameters (seems the simplest approach). I'd indeed suggest to first try the same trick as in "GaussianCurveFitter" (i.e. return a "high" value for arguments outside a known range). That way, you only have to define a suitable "ParametricUnivariateFunction" and pass it to "SimpleCurveFitter". One case for the "ParameterValidator" is when some of the model parameters might be correlated to others. But using it makes it necessary that you handle yourself all the arguments to be passed to the "LeastSquaresProblem". To take the gausian example for my use case, consider an observed signal similar to [1], given I know (from other source as the plain data) for example that the result must be found in the range of 2...3 and I wanted to restrict valid solutions to this area. The same might apply to the norm: I know it must be between a given range and I want to restrict the optimizer here even though there might be a solution outside of the range that (compared of the R^2) fits better, e.g. a gausian fit well inside the -1..1. I hope it is a little bit clearer. I'm not sure. The picture shows a function that is not a Gaussian. Do you mean that you want to fit only *part* of the data with a function that would not fit well *all* the data? Regards, Gilles [1] https://ascelibrary.org/cms/asset/6ca2b016-1a4f-4eed-80da-71219777cac1/1.jpg Am 11.08.20 um 00:42 schrieb Gilles Sadowski: Hello. Le lun. 10 août 2020 à 17:09, Christoph Läubrich a écrit : The userguide [1] mentions that it is currently not directly possible to contrain paramet
Re: [math] example for constrain parameters for Least squares
Thanks for your patience, maybe a better/simpler example would be [1], I want to find the best fit using LSB under the constraint that the height of the curve never is above 0.7 (even though without constrains the optimizer would find a better solution around 8.5). So my first idea way to copy/extend GausianCurveFitter to accept some kind of "maxHeight(...)", adjust the function to return INF for h>maxHeight, I was just unsure that return INF is allowed (per definition) as it is not mentioned anywhere in the userdocs. And if I want to limit others (like max position) it would work the same way. In my final problem I need to limit the height, width and position of the bell-curve to fall into certain bounds, but there is no direct relation (e.g. width must be three times the height). [1] https://crlbucophysics101.files.wordpress.com/2015/02/gaussian.png?w=538&h=294 Am 11.08.20 um 11:18 schrieb Gilles Sadowski: Hi. 2020-08-11 8:51 UTC+02:00, Christoph Läubrich : Hi Gilles, Just to make clear I don't suspect any error with GausianCurveFitter, I just don't understand how the advice in the user-doc to restrict parameter (for general problems) could be applied to a concrete problem and thus chosen GausianCurvefitter as an example as it uses LeastSquaresBuilder. I also noticed that Gaussian Fitter has a restriction on parameters (norm can't be negative) that is handled in a third way (returning Double.POSITIVE_INFINITY instead of Parameter Validator) not mentioned in the userdoc at all, so I wonder if this is a general purpose solution for restricting parameters (seems the simplest approach). I'd indeed suggest to first try the same trick as in "GaussianCurveFitter" (i.e. return a "high" value for arguments outside a known range). That way, you only have to define a suitable "ParametricUnivariateFunction" and pass it to "SimpleCurveFitter". One case for the "ParameterValidator" is when some of the model parameters might be correlated to others. But using it makes it necessary that you handle yourself all the arguments to be passed to the "LeastSquaresProblem". To take the gausian example for my use case, consider an observed signal similar to [1], given I know (from other source as the plain data) for example that the result must be found in the range of 2...3 and I wanted to restrict valid solutions to this area. The same might apply to the norm: I know it must be between a given range and I want to restrict the optimizer here even though there might be a solution outside of the range that (compared of the R^2) fits better, e.g. a gausian fit well inside the -1..1. I hope it is a little bit clearer. I'm not sure. The picture shows a function that is not a Gaussian. Do you mean that you want to fit only *part* of the data with a function that would not fit well *all* the data? Regards, Gilles [1] https://ascelibrary.org/cms/asset/6ca2b016-1a4f-4eed-80da-71219777cac1/1.jpg Am 11.08.20 um 00:42 schrieb Gilles Sadowski: Hello. Le lun. 10 août 2020 à 17:09, Christoph Läubrich a écrit : The userguide [1] mentions that it is currently not directly possible to contrain parameters directly but suggest one can use the ParameterValidator, is there any example code for both mentioned alternatives? For example GaussianCurveFitter uses LeastSquaresBuilder and I wan't to archive that the mean is within a closed bound e.g from 5 to 6 where my datapoints ranges from 0..90, how would this be archived? Could you set up a unit test as a practical example of what you need to achieve? I'm especially interested because the FUNCTION inside GaussianCurveFitter seems to reject invalid values (e.g. negative valuenorm) by simply return Double.POSITIVE_INFINITY instead of using either approach described in the user-docs. What I don't quite get is why you need to force the mean within a certain range; if the data match a Gaussian with a mean within that range, I would assume that the fitter will find the correct value... Sorry if I missed something. Hopefully the example will clarify. Best, Gilles [1] https://commons.apache.org/proper/commons-math/userguide/leastsquares.html - To unsubscribe, e-mail: user-unsubscr...@commons.apache.org For additional commands, e-mail: user-h...@commons.apache.org - To unsubscribe, e-mail: user-unsubscr...@commons.apache.org For additional commands, e-mail: user-h...@commons.apache.org
Re: [math] example for constrain parameters for Least squares
Hi Gilles, Just to make clear I don't suspect any error with GausianCurveFitter, I just don't understand how the advice in the user-doc to restrict parameter (for general problems) could be applied to a concrete problem and thus chosen GausianCurvefitter as an example as it uses LeastSquaresBuilder. I also noticed that Gaussian Fitter has a restriction on parameters (norm can't be negative) that is handled in a third way (returning Double.POSITIVE_INFINITY instead of Parameter Validator) not mentioned in the userdoc at all, so I wonder if this is a general purpose solution for restricting parameters (seems the simplest approach). To take the gausian example for my use case, consider an observed signal similar to [1], given I know (from other source as the plain data) for example that the result must be found in the range of 2...3 and I wanted to restrict valid solutions to this area. The same might apply to the norm: I know it must be between a given range and I want to restrict the optimizer here even though there might be a solution outside of the range that (compared of the R^2) fits better, e.g. a gausian fit well inside the -1..1. I hope it is a little bit clearer. [1] https://ascelibrary.org/cms/asset/6ca2b016-1a4f-4eed-80da-71219777cac1/1.jpg Am 11.08.20 um 00:42 schrieb Gilles Sadowski: Hello. Le lun. 10 août 2020 à 17:09, Christoph Läubrich a écrit : The userguide [1] mentions that it is currently not directly possible to contrain parameters directly but suggest one can use the ParameterValidator, is there any example code for both mentioned alternatives? For example GaussianCurveFitter uses LeastSquaresBuilder and I wan't to archive that the mean is within a closed bound e.g from 5 to 6 where my datapoints ranges from 0..90, how would this be archived? Could you set up a unit test as a practical example of what you need to achieve? I'm especially interested because the FUNCTION inside GaussianCurveFitter seems to reject invalid values (e.g. negative valuenorm) by simply return Double.POSITIVE_INFINITY instead of using either approach described in the user-docs. What I don't quite get is why you need to force the mean within a certain range; if the data match a Gaussian with a mean within that range, I would assume that the fitter will find the correct value... Sorry if I missed something. Hopefully the example will clarify. Best, Gilles [1] https://commons.apache.org/proper/commons-math/userguide/leastsquares.html - To unsubscribe, e-mail: user-unsubscr...@commons.apache.org For additional commands, e-mail: user-h...@commons.apache.org - To unsubscribe, e-mail: user-unsubscr...@commons.apache.org For additional commands, e-mail: user-h...@commons.apache.org
[math] example for constrain parameters for Least squares
The userguide [1] mentions that it is currently not directly possible to contrain parameters directly but suggest one can use the ParameterValidator, is there any example code for both mentioned alternatives? For example GaussianCurveFitter uses LeastSquaresBuilder and I wan't to archive that the mean is within a closed bound e.g from 5 to 6 where my datapoints ranges from 0..90, how would this be archived? I'm especially interested because the FUNCTION inside GaussianCurveFitter seems to reject invalid values (e.g. negative valuenorm) by simply return Double.POSITIVE_INFINITY instead of using either approach described in the user-docs. [1] https://commons.apache.org/proper/commons-math/userguide/leastsquares.html - To unsubscribe, e-mail: user-unsubscr...@commons.apache.org For additional commands, e-mail: user-h...@commons.apache.org
Re: [net] FTPClient doesn't download full images
Don't know if the code you have showed is your original one but in general its not a good idea to "catch and ignore" exceptions, yo either: - simply buble up to the caller - wrap into other exceptions if required by the API - explicitly handle/document I'm using comons-ftp for large files and small files and it always has worked quite well. Am 04.06.20 um 14:10 schrieb Julia Ruzicka: @Robert Paasche At first I tried to do it with a `FileOutputStream` and `ftpClient.retrieveFile(fileName,fos)` but that's how I noticed the problem. I then switched to using ` InputStream is = ftpClient.retrieveFileStream(filename)` and when that didn't work properly either, I added ` BufferedInputStream bis = new BufferedInputStream(is)`. What type of files are you downloading? What's your `content` and how do you write the data? With a DataOutputStream? - To unsubscribe, e-mail: user-unsubscr...@commons.apache.org For additional commands, e-mail: user-h...@commons.apache.org - To unsubscribe, e-mail: user-unsubscr...@commons.apache.org For additional commands, e-mail: user-h...@commons.apache.org
[math] Question about GaussianCurveFitter.ParameterGuesser
I'm currently looking into some curve-fitting and have examined the basicGuess(WeightedObservedPoint[]) in GaussianCurveFitter.ParameterGuesser, and there is one thing I don't really understand. in Line 301 [1] the "halfY" is computed by n (what is the maximal y value of the observation) + half the sum of n+m (where m is the x value of the maximum) applying this to some of my datapoints that fit relatively good as a Gaussian later, it seems that this condition never works, and the default in the catch clause is used instead. I also wonder why x is taken into account for computing the half-y value.. thanks in advance. [1] https://github.com/apache/commons-math/blob/master/src/main/java/org/apache/commons/math4/fitting/GaussianCurveFitter.java#L301 - To unsubscribe, e-mail: user-unsubscr...@commons.apache.org For additional commands, e-mail: user-h...@commons.apache.org