Re: 2011-06-10 13:14:29,767 FATAL org.apache.hadoop.mapred.JobTracker: java.net.BindException: Problem binding to server1/10.0.3.85:8021 : Address already in use

lex Kozlov wrote: > Can you be more explicit for the benefit of others? How can a history > folder result in the "Address already in use" exception? > > On Fri, Jun 10, 2011 at 2:33 PM, Shuja Rehman wrote: > >> Thanks Alex, >> the problem was due to histor

Re: 2011-06-10 13:14:29,767 FATAL org.apache.hadoop.mapred.JobTracker: java.net.BindException: Problem binding to server1/10.0.3.85:8021 : Address already in use

ing server1 returns with? > 2. Try `telnet server1 8022` as well as `telnet 10.0.3.85 8022`: does it > connect? > > There is another unrelated error about /var/log/hadoop-0.20/history: make > sure your have x permissions along the whole path. > > Alex K > > > On Fri, Ju

Re: 2011-06-10 13:14:29,767 FATAL org.apache.hadoop.mapred.JobTracker: java.net.BindException: Problem binding to server1/10.0.3.85:8021 : Address already in use

J wrote: > On face value, it appears to look like (1) A JT instance is already up > and bound to that port (2) Something else is using 8021 and has bound > it to itself. > > On Sat, Jun 11, 2011 at 1:57 AM, Shuja Rehman > wrote: > > Hi All, > > > > i am not su

2011-06-10 13:14:29,767 FATAL org.apache.hadoop.mapred.JobTracker: java.net.BindException: Problem binding to server1/10.0.3.85:8021 : Address already in use

Hi All, i am not sure why i start getting this error. can anybody let me know? 2011-06-10 13:14:29,767 FATAL org.apache.hadoop.mapred.JobTracker: java.net.BindException: Problem binding to server1/10.0.3.85:8021 : Address already in use at org.apache.hadoop.ipc.Server.bind(Server.java:23

Profiling Hadoop Code

Hi All, I was investigating the ways to profile the hadoop code. All I found is to use JobConf.setProfileEnabled(boolean)but i believe this is not available in the new api. so c

iterate through Reducer values more than one time?

Hi I need to iterate the values of reducer more than one time but it seems that it allow only for once. Does anybody know how to achieve it? Thanks -- Regards Shuja-ur-Rehman Baig

Re: java.lang.NullPointerException at org.apache.hadoop.conf.Configuration.getLocalPath(Configuration.java:1089)

1 at 1:50 AM, Shuja Rehman > wrote: > > Hi All, > > > > I am trying to run a map reduce job and it is running perfectly from cmd > > using following command > > > > hadoop jar Processor.jar arg1 arg2 > > > > but when i schedule the same job in

java.lang.NullPointerException at org.apache.hadoop.conf.Configuration.getLocalPath(Configuration.java:1089)

Hi All, I am trying to run a map reduce job and it is running perfectly from cmd using following command hadoop jar Processor.jar arg1 arg2 but when i schedule the same job in oozie, it is giving me following exception. Apr 20, 2011 12:49:34 PM NightProcessor.ProcessorDriver main SEVERE: null

Distributed Cache File Not Found Exception

Hi, I m getting FileNotFound Exception while using distribute cache. here is the details. Configuration config = new Configuration(); config.clear(); config.set("hbase.zookeeper.quorum", HBASE_ZOOKEEPER_QUORUM); config.set("hbase.zookeeper.property.clientPort", HBASE_ZOOKEEPER_PROPERTY_CLIENTPORT)

Distributed Cache Problem

Hi All, Does anybody know about this problem? SEVERE: null java.io.FileNotFoundException: File does not exist: /home/shuja/extract/2e8baca8-67e7-4da6-8253-2ae9e6d3fb8a at org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:519) I am using CDH3Bb3. --

Distributed Cache not working

Hi All, I have implemented the distributed cache according to following article. http://chasebradford.wordpress.com/2011/02/05/distributed-cache-static-objects-and-fast-setup/ but when i run the program over the cluster, i am getting the following exceptions. SEVERE: null java.io.FileNotFoundExc

Job Configuration for MAP only Jobs

Hi I want to configure a map only job where i need to read from hbase table 1 and do some processing in mapper and then save to some other hbase table and i do not need reducer for it. i have configure the job in this way. Job job = new Job(config, "Detector"); job.setJarByClass(Driver.class);

Best way to access same data into mapper/reducer class

Hi All, I have a simple question. I have a arraylist which i am populating through db in main class. now i want to use the same list in my map and reduce class so the question is how to access/send this array list in map and reduce class. one way is to again fetch the data in mapper and reducer cl

Best way to Merge small XML files

Hi Folks, I am having hundreds of small xml files coming each hour. The size varies from 5 Mb to 15 Mb. As Hadoop did not work well with small files so i want to merge these small files. So what is the best option to merge these xml files? -- Regards Shuja-ur-Rehman Baig

Global Sorting and Multiple Reducers ?

Hi All, I have a question about map reduce. Suppose I have set of small files (say 100) usually having size 8-15 MB and need to process in a single job. For each file, there will be 1 map process and hence 100 map process will be initiated for 100 files. Now the question is about number of reducer

Re: Job without Output files

hi all what does speculative execution of tasks (if it is turned on)? means??? and how to turn off it and what is the advantage/disadvantage of it? I am not using Tableoutput format because i need to use put statement millions of times in single job and if i use tableoutput format then the same j

Configure Ganglia with Hadoop

Hi I have cluster of 4 machines and want to configure ganglia for monitoring purpose. I have read the wiki and add the following lines to hadoop-metrics.properties on each machine. dfs.class=org.apache.hadoop.metrics.ganglia.GangliaContext dfs.period=10 dfs.servers=10.10.10.2:8649 mapred.class=or

Re: Job without Output files

Thanks Nullouput format works. On Mon, Nov 8, 2010 at 1:46 PM, Harsh J wrote: > Hi, > > On Mon, Nov 8, 2010 at 1:19 AM, Shuja Rehman > wrote: > > Hi > > > > I have a job where i did not need any reducers. I am using only mappers. > At > > the moment,

Re: Job without Output files

lt and > what's the meaning of this job ? > > > On Mon, Nov 8, 2010 at 3:49 AM, Shuja Rehman > wrote: > > Hi > > > > I have a job where i did not need any reducers. I am using only mappers. > At > > the moment, the output of job is generated in fi

Job without Output files

Hi I have a job where i did not need any reducers. I am using only mappers. At the moment, the output of job is generated in files. But i want to use only java api to do some calculation and i want that there should be no output from the mappers. So is it possible to make a job which did not produ

Re: Single Job to put Data into Hbase+MySQL

hbase in map reduce manually? On Thu, Oct 28, 2010 at 1:47 AM, Jean-Daniel Cryans wrote: > Do both insertions in your reducer by either not using the output > formats at all or use one of them and do the other insert by hand. > > J-D > > On Wed, Oct 27, 2010 at 1:44 PM, Shu

Single Job to put Data into Hbase+MySQL

Hi Folks I am wondering if anyone has the answer of this question. I am processing log files using Map reduce and get data to put some part into mysql and rest of hbase. At the moment, i am running two separate jobs to do this so reading single file for 2 times to dump the data. My questions is th

Custom Input Format in New API (Convert Mahaout XMLInput Format to New API)

Hi I am trying to convert Mahout xmlInputFormat to new API but this is not working. The problem which i think is that in old api we have next method which takes key and value and we can set it in the method public boolean next(LongWritable key, Text value) throws IOException { but in new API,

java.lang.OutOfMemoryError: Java heap space

Hi All I am facing a hard problem. I am running a map reduce job using streaming but it fails and it gives the following error. Caught: java.lang.OutOfMemoryError: Java heap space at Nodemapper5.parseXML(Nodemapper5.groovy:25) java.lang.RuntimeException: PipeMapRed.waitOutputThreads(): su