Seeking Advice on Upgrading a Cluster

2011-04-22 Thread Geoffry Roberts
All, I am a developer, not a super networking guy or hardware guy, and new to Hadoop. I'm working a research project. Funds are limited. I have a compute problem where I need to get the performance up on the processing of large text files and no doubt Hadoop can help if I do things well. I am

Re: Hadoop 0.21.0 and eclipse europa compatibility issue

2011-04-22 Thread SuhasKulkarni
I ran into exact same issue and installed Eclipse 3.3.2. The problem did not go away. -- View this message in context: http://lucene.472066.n3.nabble.com/Hadoop-0-21-0-and-eclipse-europa-compatibility-issue-tp2824306p2848838.html Sent from the Hadoop lucene-users mailing list archive at

how to display a specific text line in dfs

2011-04-22 Thread Shi Yu
I use hadoop dfs -text path | head-n hadoop dfs -text path | tail-n to browse the n-th line from the head or from the tail. But it is slow when the file is large. Is there any command that goes directly to a specific line in dfs? Shi

Re: providing the same input to more than one Map task

2011-04-22 Thread Mehmet Tepedelenlioglu
There is a way: http://hadoop.apache.org/common/docs/r0.18.3/mapred_tutorial.html#DistributedCache Are you working with a sparse matrix, or a full one? On Apr 22, 2011, at 2:33 PM, aanghelescu wrote: Hi all, I am trying to perform matrix-vector multiplication using Hadoop. So I have

How do I create a sequence file on my local harddrive?

2011-04-22 Thread W.P. McNeill
I want to create a sequence file on my local harddrive. I want to write something like this: LocalFileSystem fs = new LocalFileSystem(); Configuration configuration = new Configuration(); SequenceFile.Writer writer = SequenceFile.createWriter(fs, configuration, new

Re: providing the same input to more than one Map task

2011-04-22 Thread Ted Dunning
I would recommend taking this question to the Mahout mailing list. The short answer is that matrix multiplication by a column vector is pretty easy. Each mapper reads the vector in the configure method and then does a dot product for each row of the input matrix. Results are reassembled into a

Re: How do I create a sequence file on my local harddrive?

2011-04-22 Thread Joey Echeverria
Did you try calling fs.setConf(configuration)? On Apr 22, 2011 9:09 PM, W.P. McNeill bill...@gmail.com wrote: