In case we need to do need multithread all the algos should be reusable in that framework without any code modification. And I have a feeling hadoop will strive to improve multicore processor utilisation.
Robin On Wed, Feb 10, 2010 at 2:13 PM, Jake Mannix <jake.man...@gmail.com> wrote: > On Wed, Feb 10, 2010 at 12:39 AM, Robin Anil <robin.a...@gmail.com> wrote: > > > Smp.java is not used anywhere. > > SmpBlas is used at one place and could be replaced by Sequential version. > > In > > Mahout we dont need to run multithreading anyways. Assuming our > allegiance > > is to Hadoop M/R. and a map job shouldn't be doing further spliting of > work > > > > I wouldn't go so far as to say we don't care about multithreading, when it > can help, > because we're not *all* Hadoop M/R, but in this case I agree. > > > > May I kick them out? > > > > +1 > > -jake > > > > > > Robin > > > > On Wed, Feb 10, 2010 at 1:24 PM, Jake Mannix <jake.man...@gmail.com> > > wrote: > > > > > The lovely named "EDU.oswego.*" stuff from Doug Lea's concurrent lib I > > > had tried really hard to figure out how to pull out when I first > brought > > > colt > > > into the fold, but it turns out that these are parts of concurrent > which > > > didn't make it into java.util.concurrent, and so actually aren't > > available > > > in stock java yet (they're in java 7, not that this is relevant). > > > > > > Of course, we don't use Smp.java/SmbBlas.java anywhere else in our > > > usage of colt (we actually don't hardly use much of colt at all, > really. > > > We use their small-matrix eigen decomposition as part of Lanczos, > > > but that's all I know of). We can probably remove those entirely. > > > > > > EasyMock is nice, I'd like to use it more, fwiw. > > > > > > Don't know about the rest. > > > > > > -jake > > > > > > On Tue, Feb 9, 2010 at 11:38 PM, Robin Anil <robin.a...@gmail.com> > > wrote: > > > > > > > There are some libaries in mahout only in very special place for only > a > > > few > > > > classes. Cant we do without it? all these stats are courtesy of this > > > > wonderful eclipse plugin STAN > > > > > > > > http://stan4j.com/dependencies/dependency-analysis.html > > > > > > > > > > > > Only 3 classes used for the EDU.oswego library. > > > > > > > > > > > > > > ./math/src/main/java/org/apache/mahout/math/matrix/linalg/Smp.java:11:import > > > > EDU.oswego.cs.dl.util.concurrent.FJTask; > > > > > > > > > > > > > > ./math/src/main/java/org/apache/mahout/math/matrix/linalg/Smp.java:12:import > > > > EDU.oswego.cs.dl.util.concurrent.FJTaskRunnerGroup; > > > > > > > > > > > > > > ./math/src/main/java/org/apache/mahout/math/matrix/linalg/SmpBlas.java:11:import > > > > EDU.oswego.cs.dl.util.concurrent.FJTask; > > > > > > > > Google commons MapMaker (com.google.common is used at one place > > > indirectly > > > > by watchmaker.framework.CachingFitnessEvaluator<T>. > > > > > > > > com.thoughtworks.xtream used only in mahout.commong.StringUtils. > > > > > > > > contribs.mx only used by apache commons logging and jets3t > > > > > > > > > > > > net.sf.cglib depends only on xtream and easymock. > > > > > > > > > > > > org.xmlcommons is not used anywhere > > > > > > > > > > > > mahout. nlp, lda and ga use easy mock for test rest dont > > > > > > > > > > > > objenesis also pulled in only by easymock > > > > > > > > > > > > Any thoughts > > > > > > > > Robin > > > > > > > > > >
