Re: Mahout dependency problem with asm-1.3

2012-11-10 Thread Ted Dunning
Why do you have maven.glassfish.org in your repo path? On Fri, Nov 9, 2012 at 7:17 PM, Lance Norskog wrote: > I'm getting this from the current git checkout. There are 301 > (redirections) but there is nothing at the target either. > > Downloading: > https://repository.apache.org/content/reposit

Re: Jobs Hadoop-Mahout: Full Capacity

2012-11-10 Thread Dmitriy Lyubimov
I would imagine optimizing Mahout jobs are not fundamentally different from optiimizing any Hadoop job. Make sure you have optimal amount of task per node configured, as well as optimal amount of memory to prevent GC thrashing. (Iterative Mahout batches tend to create GC churn at somewhat respectab

Re: Jobs Hadoop-Mahout: Full Capacity

2012-11-10 Thread pricila rr
I am running kmeans algorithm. Increasing the number of tasktrackers and datanodes, increase the speed? Thank you 2012/11/10 Dmitriy Lyubimov > I would imagine optimizing Mahout jobs are not fundamentally different from > optiimizing any Hadoop job. Make sure you have optimal amount of task per

Re: Jobs Hadoop-Mahout: Full Capacity

2012-11-10 Thread Sean Owen
Did you set -Dmapred.reduce.tasks ? it defaults to 1. On Sat, Nov 10, 2012 at 7:22 PM, pricila rr wrote: > I am running kmeans algorithm. > Increasing the number of tasktrackers and datanodes, increase the speed? > > Thank you > > 2012/11/10 Dmitriy Lyubimov > > > I would imagine optimizing Ma

Re: Jobs Hadoop-Mahout: Full Capacity

2012-11-10 Thread pricila rr
No, as is the default 2012/11/10 Sean Owen > Did you set -Dmapred.reduce.tasks ? it defaults to 1. > > > On Sat, Nov 10, 2012 at 7:22 PM, pricila rr wrote: > > > I am running kmeans algorithm. > > Increasing the number of tasktrackers and datanodes, increase the speed? > > > > Thank you > > > >

Re: Jobs Hadoop-Mahout: Full Capacity

2012-11-10 Thread Sean Owen
That means you are having Hadoop run at most 1 reducer at a time across the whole cluster. In any Hadoop job this needs to be set to about the number of open reduce slots. On Sat, Nov 10, 2012 at 7:28 PM, pricila rr wrote: > No, as is the default > > 2012/11/10 Sean Owen > > > Did you set -Dma

Re: Submitting mahout jobs to map/reduce cluster with fair scheduling

2012-11-10 Thread Yazan Boshmaf
Thanks, Sean. So I added the line: MAHOUT_OPTS="$MAHOUT_OPTS -Dmapred.fairscheduler.pool=si.highpri_pipelines" to $MAHOUT_HOME/bin/mahout and then issued $MAHOUT_HOME/bin/mahout org.apache.mahout.clustering.syntheticcontrol.kmeans.Job but I still ended up with the same error. Moreover, I am st

Re: Jobs Hadoop-Mahout: Full Capacity

2012-11-10 Thread Ted Dunning
If you want k-means speed see the new k-means code: https://github.com/tdunning/knn Can you describe your data a bit? On Sat, Nov 10, 2012 at 11:22 AM, pricila rr wrote: > I am running kmeans algorithm. > Increasing the number of tasktrackers and datanodes, increase the speed? > > Thank you > >

Re: Mahout dependency problem with asm-1.3

2012-11-10 Thread Lance Norskog
I do not. No idea why it is going there. I removed .m2 and everything downloaded correctly. But now, I have the slf4j bindings problem mentioned in another email. - Original Message - | From: "Ted Dunning" | To: user@mahout.apache.org | Sent: Saturday, November 10, 2012 12:03:15 AM | Su