Hello,
I am using Mahout Spectral clustering example which internally calls a map
reduce job. Right now, it is using */tmp/hadoop-/mapred/..*
directory by default for its operations.
Can someone please let me know how to make mahout to use a different path?
Thanks
Vikas
Can you let us know which code/scripts you're using?
On Tuesday, March 31, 2015, Vikas Kumar wrote:
> Hello,
>
> I am using Mahout Spectral clustering example which internally calls a map
> reduce job. Right now, it is using */tmp/hadoop-/mapred/..*
> directory by default for its operations.
>
>
If u running Spectral KMeans via Command Line, u should be able to set the
parameter -tempDir to point to a different path
On Wed, Apr 1, 2015 at 1:55 AM, Andrew Musselman wrote:
> Can you let us know which code/scripts you're using?
>
> On Tuesday, March 31, 2015, Vikas Kumar wrote:
>
> > Hell
The following line specifically:
SpectralKMeansDriver.run(conf, affinities, output, vectors.size(),
noOfClusters, measure, onvergenceDelta, maxIterations, tmp, false);
where other variables are set accordingly. I can send the whole file if
required.
It shows the following in the log which helped
That was helpful to figure out what was required.
I had to set the right path for variable *tmp* in the function from :
Path tmp = new Path("tmp")
to
Path tmp = new Path("<>");
Silly mistake. Thanks for the clue :)
-Vikas
On Wed, Apr 1, 2015 at 1:34 AM, Suneel Marthi
wrote:
> If u run
Sorry, it didn't solved the problem.
What it changed was the *tmp* directory for the following (taken from the
log attached above):
15/04/01 01:18:13 INFO mapred.MapTask: Processing split:
file:/export/scratch/vikas/<<<
/tmp/calculations/seqfile/part-r-0:0+86000
However, the *tmp* directo
You need to set the temp path in ur Configuration and pass the
Configuration object to the subsequent calls.
IIRC, Spectral KMeans internally calls other MapReduce jobs like
MatrixDiagnolizeJob, VectorMatrixMultiplicationJob, SSVD.
So ensure that you are passing common parameters like tempDir, out
Thanks for the reply.
I think I have found the parameter that was required to set. The
configuration object is required to set the parameter *"mapred.local.dir"*
which is used by
*org.apache.hadoop.filecache.TrackerDistributedCacheManager.*
conf.set("mapred.local.dir", "tmpDirectory");
It is wor
