Sorry to not weigh in on this earlier, but the distributed SVD work we do currently relies on the still-deprecated old COLT EigenvalueDecomposition.java code, which in turn relies on DoubleMatrix1D and DoubleMatrix2D.
Now, it turns out that currently, we only use the special case of an eigenvalue decomposition for small, tri-diagonal matrices, and is not a bottleneck in terms of algorithmic complexity by any means, which means if someone were to write (or port) tri-diagonal eigendecomposition to run on Mahout standard data structures, we could remove the current dependency on DoubleMatrix1D and DoubleMatrix2D. On the other hand, some of the approaches to stochastic decomposition which have been suggested, require eigendecomposition of medium-sized fully general dense symmetric matrices, and COLT's code is the only form of that we have, so if we only pulled out COLT's special-case of tri-diagonal decomposition, we'd suddenly be missing some key implementation. What would be really really nice is if some kind soul out there ported the following classes in their entirety over to use Mahout Matrix/Vector classes: math/src/main/java/org/apache/mahout/math/matrix/linalg/LUDecomposition.java math/src/main/java/org/apache/mahout/math/matrix/linalg/LUDecompositionQuick.java math/src/main/java/org/apache/mahout/math/matrix/linalg/QRDecomposition.java math/src/main/java/org/apache/mahout/math/matrix/linalg/EigenvalueDecomposition.java Then I'd be fine if we nuked the entire remaining COLT codebase, in fact (although I'm not sure if we really are completely up to par with the jet stats stuff, are we?). -jake On Tue, Aug 31, 2010 at 11:53 AM, Ted Dunning <[email protected]> wrote: > > Here is my classification of the remaining deprecated math code: > > Code to nuke now unless needed for compile: > > math/src/main/java/org/apache/mahout/math/GenericPermuting.java > math/src/main/java/org/apache/mahout/math/Partitioning.java > math/src/main/java/org/apache/mahout/math/function/IntIntIntProcedure.java > math/src/main/java/org/apache/mahout/math/jet/math/IntFunctions.java > > math/src/main/java/org/apache/mahout/math/jet/random/sampling/RandomSampler.java > > math/src/main/java/org/apache/mahout/math/jet/random/sampling/RandomSamplingAssistant.java > > math/src/main/java/org/apache/mahout/math/jet/random/sampling/WeightedRandomSampler.java > > math/src/main/java/org/apache/mahout/math/jet/stat/quantile/DoubleQuantileFinder.java > > math/src/main/java/org/apache/mahout/math/jet/stat/quantile/EquiDepthHistogram.java > > math/src/main/java/org/apache/mahout/math/jet/stat/quantile/QuantileFinderFactory.java > math/src/main/java/org/apache/mahout/math/matrix/DoubleFactory1D.java > math/src/main/java/org/apache/mahout/math/matrix/DoubleFactory2D.java > math/src/main/java/org/apache/mahout/math/matrix/DoubleFactory3D.java > math/src/main/java/org/apache/mahout/math/matrix/DoubleMatrix1D.java > > math/src/main/java/org/apache/mahout/math/matrix/DoubleMatrix1DProcedure.java > math/src/main/java/org/apache/mahout/math/matrix/DoubleMatrix2D.java > > math/src/main/java/org/apache/mahout/math/matrix/DoubleMatrix2DProcedure.java > math/src/main/java/org/apache/mahout/math/matrix/DoubleMatrix3D.java > > math/src/main/java/org/apache/mahout/math/matrix/DoubleMatrix3DProcedure.java > > math/src/main/java/org/apache/mahout/math/matrix/doublealgo/DoubleMatrix1DComparator.java > > math/src/main/java/org/apache/mahout/math/matrix/doublealgo/DoubleMatrix2DComparator.java > math/src/main/java/org/apache/mahout/math/matrix/doublealgo/Formatter.java > > math/src/main/java/org/apache/mahout/math/matrix/doublealgo/Partitioning.java > math/src/main/java/org/apache/mahout/math/matrix/doublealgo/Sorting.java > math/src/main/java/org/apache/mahout/math/matrix/doublealgo/Statistic.java > math/src/main/java/org/apache/mahout/math/matrix/doublealgo/Stencil.java > math/src/main/java/org/apache/mahout/math/matrix/doublealgo/Transform.java > > math/src/main/java/org/apache/mahout/math/matrix/impl/AbstractFormatter.java > math/src/main/java/org/apache/mahout/math/matrix/impl/AbstractMatrix.java > math/src/main/java/org/apache/mahout/math/matrix/impl/AbstractMatrix1D.java > math/src/main/java/org/apache/mahout/math/matrix/impl/AbstractMatrix2D.java > math/src/main/java/org/apache/mahout/math/matrix/impl/AbstractMatrix3D.java > > math/src/main/java/org/apache/mahout/math/matrix/impl/DenseDoubleMatrix1D.java > > math/src/main/java/org/apache/mahout/math/matrix/impl/DenseDoubleMatrix2D.java > > math/src/main/java/org/apache/mahout/math/matrix/impl/DenseDoubleMatrix3D.java > math/src/main/java/org/apache/mahout/math/matrix/impl/Former.java > math/src/main/java/org/apache/mahout/math/matrix/impl/FormerFactory.java > math/src/main/java/org/apache/mahout/math/matrix/impl/RCDoubleMatrix2D.java > > math/src/main/java/org/apache/mahout/math/matrix/impl/SparseDoubleMatrix1D.java > > math/src/main/java/org/apache/mahout/math/matrix/impl/SparseDoubleMatrix2D.java > > math/src/main/java/org/apache/mahout/math/matrix/impl/SparseDoubleMatrix3D.java > math/src/main/java/org/apache/mahout/math/matrix/linalg/Algebra.java > > Code to fix soon: > > math/src/main/java/org/apache/mahout/math/jet/random/Beta.java > math/src/main/java/org/apache/mahout/math/jet/random/Binomial.java > math/src/main/java/org/apache/mahout/math/jet/random/ChiSquare.java > math/src/main/java/org/apache/mahout/math/jet/random/Distributions.java > math/src/main/java/org/apache/mahout/math/jet/random/Exponential.java > math/src/main/java/org/apache/mahout/math/jet/random/ExponentialPower.java > math/src/main/java/org/apache/mahout/math/jet/random/HyperGeometric.java > math/src/main/java/org/apache/mahout/math/jet/random/Hyperbolic.java > math/src/main/java/org/apache/mahout/math/jet/random/Logarithmic.java > math/src/main/java/org/apache/mahout/math/jet/random/Normal.java > math/src/main/java/org/apache/mahout/math/jet/random/PoissonSlow.java > math/src/main/java/org/apache/mahout/math/jet/random/StudentT.java > math/src/main/java/org/apache/mahout/math/jet/random/VonMises.java > > math/src/main/java/org/apache/mahout/math/matrix/linalg/LUDecomposition.java > > math/src/main/java/org/apache/mahout/math/matrix/linalg/LUDecompositionQuick.java > > math/src/main/java/org/apache/mahout/math/matrix/linalg/QRDecomposition.java > > Code somebody might want: > > math/src/main/java/org/apache/mahout/math/jet/math/Bessel.java > math/src/main/java/org/apache/mahout/math/jet/random/BreitWigner.java > > math/src/main/java/org/apache/mahout/math/jet/random/BreitWignerMeanSquare.java > math/src/main/java/org/apache/mahout/math/jet/random/Empirical.java > math/src/main/java/org/apache/mahout/math/jet/random/EmpiricalWalker.java > math/src/main/java/org/apache/mahout/math/jet/random/Zeta.java > math/src/main/java/org/apache/mahout/math/jet/stat/Descriptive.java > > math/src/main/java/org/apache/mahout/math/matrix/linalg/EigenvalueDecomposition.java > math/src/main/java/org/apache/mahout/math/matrix/linalg/Property.java > > > On Tue, Aug 31, 2010 at 11:29 AM, Sean Owen <[email protected]> wrote: > > > I don't want to do this now. I don't even want to do it for 0.4. But > > wanted to poll sentiment: what would happen if we removed any > > deprecated, unused code from the math libraries? This is the source > > of, well, a lot of unused code and most style / findbugs / pmd > > warnings now. > > > > The argument against of course is that it could be used at some point. > > I might note that code can always be resurrected from SVN, that it's > > easier to remove code earlier than later, and that un-deprecating this > > code needs a bit of rewrite anyway. > > >
