Re: Help in running hadoop from eclipse

Thanks to you Aaron, but whats the procedure for external debugging ? what project name should be specified ? and what more information i need to fill in remote debugging options Regards, Ashish. On Thu, Aug 6, 2009 at 11:22 PM, Aaron Kimball wrote: > May I ask why you're trying to run the Name

Re: Help in running hadoop from eclipse

thanx 2009/8/7 Aaron Kimball > May I ask why you're trying to run the NameNode in Eclipse? This is > likely going to cause you lots of classpath headaches. I think your > current problem is that it can't find its config files, so it's not > able to read in the strings for what addresses it shoul

Help with Hbase and Hadoop on S3

I can't seem to get Hbase to run using the hadoop i have connected to my s3 bucket Running Hbase 0.19.2 Hadoop 0.19.2 Hadoop-site.xml < configuration> fs.default.name s3://hbase fs.s3.awsAccessKeyId ID fs.s3.awsSecretAccessKey SECRET and it seems to start up no problem m

Re: Maps running - how to increase?

I don't know that your load-in speed is going to dramatically increase. There's a number of parameters that adjust aspects of MapReduce, but HDFS more or less works out of the box. You should run some monitoring on your nodes (ganglia, nagios) or check out what they're doing with top, iotop and ift

Re: Maps running - how to increase?

Thanks Aaron, I changed the settings in hadoop-site.xml file on all the machines. BTW, some settings are only reflected on the job level when I change the hadoop-default file, not sure why hadoop-site is being ignored (ex: mapred.tasktracker.map.tasks.maximum). The files I am trying load are fair

Re: Help in running hadoop from eclipse

May I ask why you're trying to run the NameNode in Eclipse? This is likely going to cause you lots of classpath headaches. I think your current problem is that it can't find its config files, so it's not able to read in the strings for what addresses it should listen on. If you want to see what's

Re: Maps running - how to increase?

Is that setting in the hadoop-site.xml file on every node? Each tasktracker reads in that file once and sets its max map tasks from that. There's no way to control this setting on a per-job basis or from the client (submitting) system. If you've changed hadoop-site.xml after starting the tasktracke

Re: Help in running hadoop from eclipse

May I ask why you're trying to run the NameNode in Eclipse? This is likely going to cause you lots of classpath headaches. I think your current problem is that it can't find its config files, so it's not able to read in the strings for what addresses it should listen on. If you want to see what's

HADOOP-4539 question

Hi. I checked this ticket and I like what I found. Had question about it, and hoped someone can answer it: If I have a NN, and BN, and the NN fails, how the DFS clients will know how to connect to the new IP? It will be a config level setting? Or it needs to be achieved via external Linux HA s

Re: Accessing HDFS over network

Why can't you use -copyFromLocal to copy stuff directly from the Windows machines to HDFS? If you install hadoop on the windows machine, you should be able to push log files to HDFS (btw the Windows machine doesn't have to be part of the cluster). - Harish On Thu, Aug 6, 2009 at 7:38 PM, thkunkel

Accessing HDFS over network

Hello, I have an HDFS cluster using Ubuntu. I want to load IIS logs into the HDFS and then do some processing with them. The IIS logs come from Windows 2003 servers. Currently in Ubuntu I have mounts set up that point to the IIS logs on the Windows servers. From there I copy the logs from the mo

Re: how to dump data from a mysql cluster to hdfs?

Sounds like you don't have many other options other than finding a machine that can get to the mysql gateway, do some mysql export, copy to a machine which can talk to the Hadoop gateway (if necessary) and then copy it in to HDFS. I'm not sure what else you could do. Cheers Tim On Thu, Aug 6,

Re: how to dump data from a mysql cluster to hdfs?

I guess I havenot expressed clearly. Neither datanodes nor namenodes is allowed to be directly connected. Even though, namenode is often in heavy work, it would be burdened dumping data on them. On Thu, Aug 6, 2009 at 1:52 PM, Yang Zhou wrote: > Write a Java program which will dump data from mys

Help in building and running hadoop from eclipse

Hi All, I am trying to learn building Hadoop as I have to make few changes to FShell code . I tried the link given in the http://www.*wiki*.apache.org/*hadoop*/*Eclipse*Environment but always landed up in errors. Then there was error like .eclipse.template missing

0.19.2 Native Libraries Issue

Hi, When using native compression libraries that come with 0.19.2 there seem to be an issue on certain OS that wasn't there on 0.19.1. Some of our machines are RHEL4 and some RHEL5 and when using compression on RHEL4 the JVMs are having critical errors with the native code, RHEL5 is working fine.