Hi Dmitriy:
thanks for your stochastic SVD, I am trying to use it.
the hadoop command line is as follows:
hadoop jar mahout-core-0.6-SNAPSHOT-job.jar
org.apache.mahout.math.hadoop.stochasticsvd.SSVDCli
-Dmapred.job.queue.name=unfunded
-Dmapred.job.map.memory.mb=4096 -Dmapred.job.reduce.memory.mb=4096
-Dmapred.child.java.opts=-Xmx3072M -k 10 -p 490 -r 589769 -s 100 --input
input --output output
is it correct? and the process seems very slow especially the Q-job
(only reading 72 records in 6 minutes).
and how to set the parameter s?
Best
Wei
On Sat, Jun 4, 2011 at 5:58 AM, Dmitriy Lyubimov <[email protected]> wrote:
> For stochastic SVD, i have a tex-ified help document here.
>
> http://weatheringthrutechdays.blogspot.com/2011/03/ssvd-command-line-usage.html
> .
> let me know if it doesn't open/save.
>
> if we had mathjax setup on wiki, i probably could drop a little bit
> more details there.
>
> -d
>
>
> On Thu, Jun 2, 2011 at 2:23 AM, Dan Brickley <[email protected]> wrote:
> > On 2 June 2011 09:25, Sean Owen <[email protected]> wrote:
> >> Apache Mahout has reached version 0.5. All developers are encouraged to
> >> begin using version 0.5, as again much has changed and been fixed since
> >> version 0.4. Many APIs have been changed, added or removed, and will
> >> continue before version 1.0. Highlights of version 0.5 include:
> >>
> >> - Improved Lanczos solver: graceful restarts, better scalability
> >> - LDA improvements: document-topic distribution output, graceful
> restarts
> >> - Stochastic Singular Value Decomposition implementation
> >> - Incremental SVD implementation
> >> - Alternating Least Squares with Weighted Regularization collaborative
> >> filtering implementation, both distributed and non-distributed
> >> - SVDRecommender enhancements
> >> - Initial work at merging clustering and classification infrastructure
> >> - Better control over candidate item selection in item-based
> recommenders
> >> - Significant removal of deprecated or dead code
> >> - Many bug fixes, refactorings and other small improvements
> >>
> >> Changes in 0.5 are detailed in the release notes (
> >>
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12315255&version=12314396
> >> )
> >>
> >> Downloads of all releases are available from Apache Mirrors (
> >> http://www.apache.org/dyn/closer.cgi/mahout/).
> >
> > Congratulations on shipping! Lots of hard work in there...
> >
> > May I ask for more already? :) It seems there are quite a few
> > SVD-related pieces of Mahout now.
> >
> > Just in this list we have mentions of the Lanczos solver; Stochastic
> > and Incremental SVD implementations; and the Taste SVDRecommender.
> >
> > A few words on how they all fit together would go a long way.
> > Apologies if I've missed them. I've tried tried Lanczos via Hadoop,
> > and SVDRecommender on a single machine, and they seem quite separate
> > components of Mahout. But if the family of SVD-related pieces is
> > growing, it would be great to have a summary for the Wiki: do they
> > share APIs, command line tools, ...?
> >
> > https://cwiki.apache.org/MAHOUT/svd-singular-value-decomposition.html
> > or https://cwiki.apache.org/MAHOUT/dimensional-reduction.html would be
> > a good home. I'm happy to wikify if people follow up in this thread
> > with the raw materials.
> >
> > cheers,
> >
> > Dan
> >
>
