Re: Very strange Java Collection behavior in Hadoop
On Mon, Mar 19, 2012 at 11:05 PM, madhu phatak wrote: > Hi Owen O'Malley, > Thank you for that Instant reply. It's working now. Can you explain me > what you mean by "input to reducer is reused" in little detail? Each time the statement "Text value = values.next();" is executed it always returns the same Text object with the contents of that object changed. When you add the Text to the list, you are adding a pointer to the same Text object. At the end you have 6 copies of the same pointer instead of 6 different Text objects. The reason that I said it is my fault, is because I added the optimization that causes it. If you are interested in Hadoop archeology, it was HADOOP-2399 that made the change. I also did HADOOP-3522 to improve the documentation in the area. -- Owen
Re: Very strange Java Collection behavior in Hadoop
Hi Owen O'Malley, Thank you for that Instant reply. It's working now. Can you explain me what you mean by "input to reducer is reused" in little detail? On Tue, Mar 20, 2012 at 11:28 AM, Owen O'Malley wrote: > On Mon, Mar 19, 2012 at 10:52 PM, madhu phatak > wrote: > > > Hi All, > > I am using Hadoop 0.20.2 . I am observing a Strange behavior of Java > > Collection's . I have following code in reducer > > > That is my fault. *sigh* The input to the reducer is reused. Replace: > > list.add(value); > > with: > > list.add(new Text(value)); > > and the problem will go away. > > -- Owen > -- https://github.com/zinnia-phatak-dev/Nectar
Re: Very strange Java Collection behavior in Hadoop
On Mon, Mar 19, 2012 at 10:52 PM, madhu phatak wrote: > Hi All, > I am using Hadoop 0.20.2 . I am observing a Strange behavior of Java > Collection's . I have following code in reducer That is my fault. *sigh* The input to the reducer is reused. Replace: list.add(value); with: list.add(new Text(value)); and the problem will go away. -- Owen
Very strange Java Collection behavior in Hadoop
Hi All,
I am using Hadoop 0.20.2 . I am observing a Strange behavior of Java
Collection's . I have following code in reducer
public void reduce(Text text, Iterator values,
OutputCollector collector, Reporter reporter)
throws IOException {
// TODO Auto-generated method stub
List list = new ArrayList();
while(values.hasNext())
{
Text value = values.next();
list.add(value);
System.out.println(value.toString()); }
for(Text value : list)
{
System.out.println(value.toString());
}
}
The first sysout prints following
4 5 6
1 2 3
But when I print from the List, it prints following
1 2 3
1 2 3
*
*The all List values are getting replaced by last added value.
I am not able to understand this behavior. Did anyone seen this behavior ?
Regards,
Madhukara Phatak
--
https://github.com/zinnia-phatak-dev/Nectar
Re: Using mapred.reducer (old api) but don't write anything to HDFS using output collector
Hi,
If you do not want to output anything, use NullOutputFormat and simply
do not call output.collect(…) anywhere. The framework does not demand
that you call output.collect(…).
On Tue, Mar 20, 2012 at 3:03 AM, aji1705 wrote:
> I am using the old api and my Reducer class has the following signature (only
> relevant parts shown):
>
>
> import org.apache.hadoop.mapred.MapReduceBase;
> import org.apache.hadoop.mapred.OutputCollector;
> import org.apache.hadoop.mapred.Reducer;
> import org.apache.hadoop.mapred.Reporter;
>
> public class FlagDropComparisionReducer extends MapReduceBase
> implements Reducer {
>
> public void reduce(Text key, Iterator values,
> OutputCollector output, Reporter reporter)
> {
>
> ..
> String result = some code here to compute result using key and value
> Use HTTP Client to POST "result" to server
>
>
> }
> }
>
> I don't want to write the output of reducer to HDFS ie no
> "output.collect(key, somevalue). Is there a way for me to do this using the
> old api? Note: I must have a reducer because I am doing some calculation
> here (not shown).
>
> Suggestions? Thanks.
>
>
>
> --
> View this message in context:
> http://hadoop-common.472056.n3.nabble.com/Using-mapred-reducer-old-api-but-don-t-write-anything-to-HDFS-using-output-collector-tp3840442p3840442.html
> Sent from the Users mailing list archive at Nabble.com.
--
Harsh J
question about processing large zip
Hi, I have a large (~300 gig) zip of images that I need to process. My current workflow is to copy the zip to HDFS, use a custom input format to read the zip entries, do the processing in a map, and then generate a processing report in the reduce. I'm struggling to tune params right now with my cluster to make everything run smoothly, but I'm also worried that I'm missing a better way of processing. Does anybody have suggestions for how to make the processing of a zip more parallel? The only other idea I had was uploading the zip as a sequence file, but that proved incredibly slow (~30 hours on my 3 node cluster to upload). Thanks in advance. -Andrew
Using mapred.reducer (old api) but don't write anything to HDFS using output collector
I am using the old api and my Reducer class has the following signature (only
relevant parts shown):
import org.apache.hadoop.mapred.MapReduceBase;
import org.apache.hadoop.mapred.OutputCollector;
import org.apache.hadoop.mapred.Reducer;
import org.apache.hadoop.mapred.Reporter;
public class FlagDropComparisionReducer extends MapReduceBase
implements Reducer {
public void reduce(Text key, Iterator values,
OutputCollector output, Reporter reporter) {
..
String result = some code here to compute result using key and value
Use HTTP Client to POST "result" to server
}
}
I don't want to write the output of reducer to HDFS ie no
"output.collect(key, somevalue). Is there a way for me to do this using the
old api? Note: I must have a reducer because I am doing some calculation
here (not shown).
Suggestions? Thanks.
--
View this message in context:
http://hadoop-common.472056.n3.nabble.com/Using-mapred-reducer-old-api-but-don-t-write-anything-to-HDFS-using-output-collector-tp3840442p3840442.html
Sent from the Users mailing list archive at Nabble.com.
Re: Problem setting super user
The right property for your version of Hadoop is "dfs.permissions.supergroup". Change the property name, restart NN, and your 'root' user should behave as a superuser afterwards. Or, you can just do "sudo -u hdfs " as root. On Mon, Mar 19, 2012 at 11:02 PM, Olivier Sallou wrote: > Hi, > I have installed Hadoop 1.0 using .deb package. > I tried to configure superuser groups but it somehow fail. I do not know > what's wrong: > > I expect root to be able to run hadoop dfsadmin -report command. > > # id hdfs > uid=201(hdfs) gid=201(hdfs) groupes=123(hadoop),201(hdfs) > # id root > uid=0(root) gid=0(root) groupes=123(hadoop),0(root) > > Both my hdfs user (superuser by default) and my root user are in group > hadoop. > > In hdfs-site.xml I have: > > dfs.permissions.superusergroup > hadoop > The name of the group of super-users. > > > I tried running refreshServiceAcl and > refreshSuperUserGroupsConfiguration as hdfs, but I still have the error: > root# hadoop dfsadmin -report > report: org.apache.hadoop.security.AccessControlException: Access denied > for user root. Superuser privilege is required > > Has anyone an idea? > > Thanks! > > Olivier > > -- > > gpg key id: 4096R/326D8438 (keyring.debian.org) > Key fingerprint = 5FB4 6F83 D3B9 5204 6335 D26D 78DC 68DB 326D 8438 > > -- Harsh J
Re: _temporary doesn't exist
I found that "_temporary" directory is getting created and getting deleted
it self by hadoop. So is there any timeout for "_temporary" folder? I
noticed that if the job finishes within 5 minutes then I am not getting
this problem, but when it takes more than 50 minutes, the
"/output/_temporary" directory is missing.
Just to remind, I am working on 0.17.2 version.
-Vipul
On Fri, Mar 16, 2012 at 11:29 AM, Vipul Bharakhada wrote:
> Hi,
> I am using the old hadoop version 0.17.2 and I am getting the following
> exception when I am trying to run a Job. But it only happens at particular
> time, as cron jobs do run those task at particular intervals but it only
> fails at one particular time in day.
>
> Mar 14 06:49:23 7 08884: java.io.IOException: The directory
> hdfs://<{IPADDRESS}>:<{PORT}>/myserver/matcher/output/_temporary doesnt
> exist
> Mar 14 06:49:23 7 08884: \09at
> org.apache.hadoop.mapred.TaskTracker$TaskInProgress.localizeTask(TaskTracker.java:1439)
> Mar 14 06:49:23 7 08884: \09at
> org.apache.hadoop.mapred.TaskTracker$TaskInProgress.launchTask(TaskTracker.java:1511)
> Mar 14 06:49:23 7 08884: \09at
> org.apache.hadoop.mapred.TaskTracker.launchTaskForJob(TaskTracker.java:723)
> Mar 14 06:49:23 7 08884: \09at
> org.apache.hadoop.mapred.TaskTracker.localizeJob(TaskTracker.java:716)
> Mar 14 06:49:23 7 08884: \09at
> org.apache.hadoop.mapred.TaskTracker.startNewTask(TaskTracker.java:1274)
> Mar 14 06:49:23 7 08884: \09at
> org.apache.hadoop.mapred.TaskTracker.offerService(TaskTracker.java:915)
> Mar 14 06:49:23 7 08884: \09at
> org.apache.hadoop.mapred.TaskTracker.run(TaskTracker.java:1310)
> Mar 14 06:49:23 7 08884: \09at
> org.apache.hadoop.mapred.TaskTracker.main(TaskTracker.java:2251)
>
> What can be a problem? as this folder is created by hadoop internally and
> used internally, clean up is also done internally by hadoop. So why this
> directory is missing at some particular time?
> Any clue?
> -Vipul
>
Re: Problem setting super user
does it work under user hdfs? On Mar 19, 2012 6:32 PM, "Olivier Sallou" wrote: > Hi, > I have installed Hadoop 1.0 using .deb package. > I tried to configure superuser groups but it somehow fail. I do not know > what's wrong: > > I expect root to be able to run hadoop dfsadmin -report command. > > # id hdfs > uid=201(hdfs) gid=201(hdfs) groupes=123(hadoop),201(hdfs) > # id root > uid=0(root) gid=0(root) groupes=123(hadoop),0(root) > > Both my hdfs user (superuser by default) and my root user are in group > hadoop. > > In hdfs-site.xml I have: > >dfs.permissions.superusergroup >hadoop >The name of the group of super-users. > > > I tried running refreshServiceAcl and > refreshSuperUserGroupsConfiguration as hdfs, but I still have the error: > root# hadoop dfsadmin -report > report: org.apache.hadoop.security.AccessControlException: Access denied > for user root. Superuser privilege is required > > Has anyone an idea? > > Thanks! > > Olivier > > -- > > gpg key id: 4096R/326D8438 (keyring.debian.org) > Key fingerprint = 5FB4 6F83 D3B9 5204 6335 D26D 78DC 68DB 326D 8438 > > >
Problem setting super user
Hi, I have installed Hadoop 1.0 using .deb package. I tried to configure superuser groups but it somehow fail. I do not know what's wrong: I expect root to be able to run hadoop dfsadmin -report command. # id hdfs uid=201(hdfs) gid=201(hdfs) groupes=123(hadoop),201(hdfs) # id root uid=0(root) gid=0(root) groupes=123(hadoop),0(root) Both my hdfs user (superuser by default) and my root user are in group hadoop. In hdfs-site.xml I have: dfs.permissions.superusergroup hadoop The name of the group of super-users. I tried running refreshServiceAcl and refreshSuperUserGroupsConfiguration as hdfs, but I still have the error: root# hadoop dfsadmin -report report: org.apache.hadoop.security.AccessControlException: Access denied for user root. Superuser privilege is required Has anyone an idea? Thanks! Olivier -- gpg key id: 4096R/326D8438 (keyring.debian.org) Key fingerprint = 5FB4 6F83 D3B9 5204 6335 D26D 78DC 68DB 326D 8438
Re: need help regarding eclipse plugin for hadoop
Hi, On top of the Hadoop eclipse plugin, you already need to have a Hadoop Virtual Machine, where eclipse can run the code, and a local library of hadoop, which will be used by Eclipse to compile your code. Below are the steps that I used to run a sample wordcount example on the Hadoop VM. 1. Download this virtual appliance: http://shared.zenfs.com/hadoop-vm-appliance-0-20-S.zip Also, download a relevant version of hadoop onto your computer, you will need to provide the path to the hadoop library to Eclipse. 2. Login using: hadoop-user, pwd: hadoop 3. Try this command: hadoop dfs -ls if you get an error, then use this command kinit -k -t /home/hadoop-user/hadoop-user.keytab hadoop-user/localhost@LOCALDOMAIN This will setup your Hadoop VM Next, to install the plugin on eclipse, 1. Download this plugin, and place it in the plugin directory in eclipse http://code.google.com/edu/parallel/tools/hadoopvm/hadoop-eclipse-plugin.jar 2. Use the steps mentioned here to configure the plugin in eclipse: http://code.google.com/edu/parallel/tools/hadoopvm/index.html Host id is the IP address that is printed on VM startup Hadoop directory: /home/hadoop-user/hadoop 3. Create input directory in hadoop DFS, which will be input directory for map reduce, and create a text file which will be the input file. a) vi temp.txt b) mkdir input c) mv temp.txt input/temp.txt d) hadoop dfs copyFromLocal input . e) mkdir output f) hadoop dfs copyFromLoacl output . 4. Then run the code attached on the hadoop server, if it runs successfully, the output will be stored in the output directory. a) vi output/part-r-0 On Mon, Mar 19, 2012 at 12:29 PM, tabraiz anwer wrote: > Hi, > I am newbie to the hadoop, and trying to configure eclipse plugin of > hadoop but its resposnse is very awkward and giving me the error. > Error : Unable to login. when trying to connect to hadoop DFS using > Hadoop plugin for eclipse > > > using hadoop : hadoop-0.20.203.0 > eclipse plugin : hadoop-eclipse-plugin-0.20.203.0.jar > eclipse : eclipse indigo > using Ubuntu 11.04 64bit.
need help regarding eclipse plugin for hadoop
Hi, I am newbie to the hadoop, and trying to configure eclipse plugin of hadoop but its resposnse is very awkward and giving me the error. Error : Unable to login. when trying to connect to hadoop DFS using Hadoop plugin for eclipse using hadoop : hadoop-0.20.203.0 eclipse plugin : hadoop-eclipse-plugin-0.20.203.0.jar eclipse : eclipse indigo using Ubuntu 11.04 64bit.
Re: EOFException
I guess I am trying to see how to debug such problems? I don't see enough info in the logs. On Mon, Mar 19, 2012 at 12:48 AM, madhu phatak wrote: > Hi, > Seems like HDFS is in safemode. > > On Fri, Mar 16, 2012 at 1:37 AM, Mohit Anchlia >wrote: > > > This is actually just hadoop job over HDFS. I am assuming you also know > why > > this is erroring out? > > > > On Thu, Mar 15, 2012 at 1:02 PM, Gopal wrote: > > > > > On 03/15/2012 03:06 PM, Mohit Anchlia wrote: > > > > > >> When I start a job to read data from HDFS I start getting these > errors. > > >> Does anyone know what this means and how to resolve it? > > >> > > >> 2012-03-15 10:41:31,402 [Thread-5] INFO > > org.apache.hadoop.hdfs.**DFSClient > > >> - > > >> Exception in createBlockOutputStream 164.28.62.204:50010java.io.** > > >> EOFException > > >> 2012-03-15 10:41:31,402 [Thread-5] INFO > > org.apache.hadoop.hdfs.**DFSClient > > >> - > > >> Abandoning block blk_-6402969611996946639_11837 > > >> 2012-03-15 10:41:31,403 [Thread-5] INFO > > org.apache.hadoop.hdfs.**DFSClient > > >> - > > >> Excluding datanode 164.28.62.204:50010 > > >> 2012-03-15 10:41:31,406 [Thread-5] INFO > > org.apache.hadoop.hdfs.**DFSClient > > >> - > > >> Exception in createBlockOutputStream 164.28.62.198:50010java.io.** > > >> EOFException > > >> 2012-03-15 10:41:31,406 [Thread-5] INFO > > org.apache.hadoop.hdfs.**DFSClient > > >> - > > >> Abandoning block blk_-5442664108986165368_11838 > > >> 2012-03-15 10:41:31,407 [Thread-5] INFO > > org.apache.hadoop.hdfs.**DFSClient > > >> - > > >> Exception in createBlockOutputStream 164.28.62.197:50010java.io.** > > >> EOFException > > >> 2012-03-15 10:41:31,407 [Thread-5] INFO > > org.apache.hadoop.hdfs.**DFSClient > > >> - > > >> Abandoning block blk_-3373089616877234160_11838 > > >> 2012-03-15 10:41:31,407 [Thread-5] INFO > > org.apache.hadoop.hdfs.**DFSClient > > >> - > > >> Excluding datanode 164.28.62.198:50010 > > >> 2012-03-15 10:41:31,409 [Thread-5] INFO > > org.apache.hadoop.hdfs.**DFSClient > > >> - > > >> Excluding datanode 164.28.62.197:50010 > > >> 2012-03-15 10:41:31,410 [Thread-5] INFO > > org.apache.hadoop.hdfs.**DFSClient > > >> - > > >> Exception in createBlockOutputStream 164.28.62.204:50010java.io.** > > >> EOFException > > >> 2012-03-15 10:41:31,410 [Thread-5] INFO > > org.apache.hadoop.hdfs.**DFSClient > > >> - > > >> Abandoning block blk_4481292025401332278_11838 > > >> 2012-03-15 10:41:31,411 [Thread-5] INFO > > org.apache.hadoop.hdfs.**DFSClient > > >> - > > >> Excluding datanode 164.28.62.204:50010 > > >> 2012-03-15 10:41:31,412 [Thread-5] INFO > > org.apache.hadoop.hdfs.**DFSClient > > >> - > > >> Exception in createBlockOutputStream 164.28.62.200:50010java.io.** > > >> EOFException > > >> 2012-03-15 10:41:31,412 [Thread-5] INFO > > org.apache.hadoop.hdfs.**DFSClient > > >> - > > >> Abandoning block blk_-5326771177080888701_11838 > > >> 2012-03-15 10:41:31,413 [Thread-5] INFO > > org.apache.hadoop.hdfs.**DFSClient > > >> - > > >> Excluding datanode 164.28.62.200:50010 > > >> 2012-03-15 10:41:31,414 [Thread-5] INFO > > org.apache.hadoop.hdfs.**DFSClient > > >> - > > >> Exception in createBlockOutputStream 164.28.62.197:50010java.io.** > > >> EOFException > > >> 2012-03-15 10:41:31,414 [Thread-5] INFO > > org.apache.hadoop.hdfs.**DFSClient > > >> - > > >> Abandoning block blk_-8073750683705518772_11839 > > >> 2012-03-15 10:41:31,415 [Thread-5] INFO > > org.apache.hadoop.hdfs.**DFSClient > > >> - > > >> Excluding datanode 164.28.62.197:50010 > > >> 2012-03-15 10:41:31,416 [Thread-5] INFO > > org.apache.hadoop.hdfs.**DFSClient > > >> - > > >> Exception in createBlockOutputStream 164.28.62.199:50010java.io.** > > >> EOFException > > >> 2012-03-15 10:41:31,416 [Thread-5] INFO > > org.apache.hadoop.hdfs.**DFSClient > > >> - > > >> Exception in createBlockOutputStream 164.28.62.198:50010java.io.** > > >> EOFException > > >> 2012-03-15 10:41:31,416 [Thread-5] INFO > > org.apache.hadoop.hdfs.**DFSClient > > >> - > > >> Abandoning block blk_441003866688859169_11838 > > >> 2012-03-15 10:41:31,416 [Thread-5] INFO > > org.apache.hadoop.hdfs.**DFSClient > > >> - > > >> Abandoning block blk_-466858474055876377_11839 > > >> 2012-03-15 10:41:31,417 [Thread-5] INFO > > org.apache.hadoop.hdfs.**DFSClient > > >> - > > >> Excluding datanode 164.28.62.198:50010 > > >> 2012-03-15 10:41:31,417 [Thread-5] WARN > > org.apache.hadoop.hdfs.**DFSClient > > >> - > > >> > > >> > > > Try shutting down and restarting hbase. > > > > > > > > > -- > https://github.com/zinnia-phatak-dev/Nectar >
Re: EOFException
Hi, Seems like HDFS is in safemode. On Fri, Mar 16, 2012 at 1:37 AM, Mohit Anchlia wrote: > This is actually just hadoop job over HDFS. I am assuming you also know why > this is erroring out? > > On Thu, Mar 15, 2012 at 1:02 PM, Gopal wrote: > > > On 03/15/2012 03:06 PM, Mohit Anchlia wrote: > > > >> When I start a job to read data from HDFS I start getting these errors. > >> Does anyone know what this means and how to resolve it? > >> > >> 2012-03-15 10:41:31,402 [Thread-5] INFO > org.apache.hadoop.hdfs.**DFSClient > >> - > >> Exception in createBlockOutputStream 164.28.62.204:50010java.io.** > >> EOFException > >> 2012-03-15 10:41:31,402 [Thread-5] INFO > org.apache.hadoop.hdfs.**DFSClient > >> - > >> Abandoning block blk_-6402969611996946639_11837 > >> 2012-03-15 10:41:31,403 [Thread-5] INFO > org.apache.hadoop.hdfs.**DFSClient > >> - > >> Excluding datanode 164.28.62.204:50010 > >> 2012-03-15 10:41:31,406 [Thread-5] INFO > org.apache.hadoop.hdfs.**DFSClient > >> - > >> Exception in createBlockOutputStream 164.28.62.198:50010java.io.** > >> EOFException > >> 2012-03-15 10:41:31,406 [Thread-5] INFO > org.apache.hadoop.hdfs.**DFSClient > >> - > >> Abandoning block blk_-5442664108986165368_11838 > >> 2012-03-15 10:41:31,407 [Thread-5] INFO > org.apache.hadoop.hdfs.**DFSClient > >> - > >> Exception in createBlockOutputStream 164.28.62.197:50010java.io.** > >> EOFException > >> 2012-03-15 10:41:31,407 [Thread-5] INFO > org.apache.hadoop.hdfs.**DFSClient > >> - > >> Abandoning block blk_-3373089616877234160_11838 > >> 2012-03-15 10:41:31,407 [Thread-5] INFO > org.apache.hadoop.hdfs.**DFSClient > >> - > >> Excluding datanode 164.28.62.198:50010 > >> 2012-03-15 10:41:31,409 [Thread-5] INFO > org.apache.hadoop.hdfs.**DFSClient > >> - > >> Excluding datanode 164.28.62.197:50010 > >> 2012-03-15 10:41:31,410 [Thread-5] INFO > org.apache.hadoop.hdfs.**DFSClient > >> - > >> Exception in createBlockOutputStream 164.28.62.204:50010java.io.** > >> EOFException > >> 2012-03-15 10:41:31,410 [Thread-5] INFO > org.apache.hadoop.hdfs.**DFSClient > >> - > >> Abandoning block blk_4481292025401332278_11838 > >> 2012-03-15 10:41:31,411 [Thread-5] INFO > org.apache.hadoop.hdfs.**DFSClient > >> - > >> Excluding datanode 164.28.62.204:50010 > >> 2012-03-15 10:41:31,412 [Thread-5] INFO > org.apache.hadoop.hdfs.**DFSClient > >> - > >> Exception in createBlockOutputStream 164.28.62.200:50010java.io.** > >> EOFException > >> 2012-03-15 10:41:31,412 [Thread-5] INFO > org.apache.hadoop.hdfs.**DFSClient > >> - > >> Abandoning block blk_-5326771177080888701_11838 > >> 2012-03-15 10:41:31,413 [Thread-5] INFO > org.apache.hadoop.hdfs.**DFSClient > >> - > >> Excluding datanode 164.28.62.200:50010 > >> 2012-03-15 10:41:31,414 [Thread-5] INFO > org.apache.hadoop.hdfs.**DFSClient > >> - > >> Exception in createBlockOutputStream 164.28.62.197:50010java.io.** > >> EOFException > >> 2012-03-15 10:41:31,414 [Thread-5] INFO > org.apache.hadoop.hdfs.**DFSClient > >> - > >> Abandoning block blk_-8073750683705518772_11839 > >> 2012-03-15 10:41:31,415 [Thread-5] INFO > org.apache.hadoop.hdfs.**DFSClient > >> - > >> Excluding datanode 164.28.62.197:50010 > >> 2012-03-15 10:41:31,416 [Thread-5] INFO > org.apache.hadoop.hdfs.**DFSClient > >> - > >> Exception in createBlockOutputStream 164.28.62.199:50010java.io.** > >> EOFException > >> 2012-03-15 10:41:31,416 [Thread-5] INFO > org.apache.hadoop.hdfs.**DFSClient > >> - > >> Exception in createBlockOutputStream 164.28.62.198:50010java.io.** > >> EOFException > >> 2012-03-15 10:41:31,416 [Thread-5] INFO > org.apache.hadoop.hdfs.**DFSClient > >> - > >> Abandoning block blk_441003866688859169_11838 > >> 2012-03-15 10:41:31,416 [Thread-5] INFO > org.apache.hadoop.hdfs.**DFSClient > >> - > >> Abandoning block blk_-466858474055876377_11839 > >> 2012-03-15 10:41:31,417 [Thread-5] INFO > org.apache.hadoop.hdfs.**DFSClient > >> - > >> Excluding datanode 164.28.62.198:50010 > >> 2012-03-15 10:41:31,417 [Thread-5] WARN > org.apache.hadoop.hdfs.**DFSClient > >> - > >> > >> > > Try shutting down and restarting hbase. > > > -- https://github.com/zinnia-phatak-dev/Nectar
Re: Setting up MapReduce 2 on a test cluster
Try to check if all necessary files exist in /share/hdfs and /share/mapreduce folder.
