Re: Can jobs be configured to be sequential

[Paco NATHAN]

Hi Ravion,

The problem you are describing sounds like a workflow where you must
be careful to verify certain conditions before proceeding to a next
step.

We have similar kinds of use cases for Hadoop apps at work, which are
essentially ETL.  I recommend that you look at http://cascading.org as
an abstraction layer for managing these kinds of workflows. We've
found it quite useful.

Best,
Paco


On Fri, Oct 17, 2008 at 8:29 PM, Ravion <[EMAIL PROTECTED]> wrote:
> Dear all,
>
> We have in our Data Warehouse System, about 600  ETL( Extract Transform Load) 
> jobs to create interim data model. SOme jobs are dependent on completion of 
> others.
>
> Assume that I create a group id intdependent jobs. Say a group G1 contains 
> 100 jobs , G2 contains another 200 jobs which are dependent on completion of 
> Group G1 and so on.
>
> Can we leverage on Haddop so that Hadoop executed G1 first, on failure it 
> wont execute G2 otherwise will continue for G2 and so  on.. ?
>
> Or do I need to configure "N" ( where N =  total number of groups) Haddop 
> jobs independently and handle by ourselves?
>
> Please share your thoughts, thanks
>
> Warmest regards,
> Ravion

Can jobs be configured to be sequential

[Ravion]

Dear all,

We have in our Data Warehouse System, about 600  ETL( Extract Transform Load) 
jobs to create interim data model. SOme jobs are dependent on completion of 
others.

Assume that I create a group id intdependent jobs. Say a group G1 contains 100 
jobs , G2 contains another 200 jobs which are dependent on completion of Group 
G1 and so on.

Can we leverage on Haddop so that Hadoop executed G1 first, on failure it wont 
execute G2 otherwise will continue for G2 and so  on.. ?

Or do I need to configure "N" ( where N =  total number of groups) Haddop jobs 
independently and handle by ourselves?

Please share your thoughts, thanks

Warmest regards,
Ravion

Expected disk space use for files in mapred/local?

[Per Jacobsson]

Hi all,
We've been running a pretty big job on 20 extra-large high-CPU EC2 servers
(Hadoop version 0.18, Java 1.6, the standard AMIs), and started getting the
dreaded "Could not find any valid local directory" error during the final
reduce phase.

I've confirmed that some of the boxes are running out of space, but the disk
usage seems to be very uneven across the servers. The datanodes report 50%
of available space used on all servers, which matches what I'm seeing in the
/mnt/hadoop/dfs/data/folder (an even ~200 Gb / server). But the space used
by files in /mnt/hadoop/mapred/local differs a lot from server to server
(going from 70 Gb to 190 Gb).

Is there any way to predict how much space will be used by the temporary
data stored outside of HDFS? We're only running a total of 20 reducers which
I suspect is very low, since there are a few thousand map tasks. Could that
be the cause or is there anything else we're doing that's obviously wrong?

Besides this, we're also getting this error: "java.lang.OutOfMemoryError: GC
overhead limit exceeded"

Thanks for any help,
/ Per

Re: Any one successfully ran the c++ pipes example?

[Zhengguo 'Mike' SUN]

Hi, Owen,

Did you mean that the example with a C++ record reader is not complete? I have 
to run this example with the class file of that WordCountInputFormat.java. 
Also, it seemed that the semantics of the C++ Pipes API is different from Java. 
A InputSplit is a chunk of a file in Java, while in C++, it seemed to be the 
name of a file. I got this conclution from that wordcount example.

Thanks,
Mike



- Original Message 
From: Owen O'Malley <[EMAIL PROTECTED]>
To: core-user@hadoop.apache.org
Sent: Friday, October 17, 2008 2:02:20 PM
Subject: Re: Any one successfully ran the c++ pipes example?


On Oct 16, 2008, at 1:40 PM, Zhengguo 'Mike' SUN wrote:

> I was trying to write an application using the pipes api. But it  
> seemed the serialization part is not working correctly. More  
> specifically, I can't deserialize a string from an StringInStream  
> constructed from context.getInputSplit(). Even with the examples  
> bundled in the distribution archive(wordcount-nopipe.cc), it threw  
> exceptions. If anyone had experience on that, please kindly give  
> some advise.

So you mean the example with a C++ record reader? You have to use the  
InputFormat that generates input splits that consists of strings. Look  
at src/test/org/apache/hadoop/pipes/WordCountInputFormat.java

It would be useful to have a C++ impl of FileInputSplit too...

-- Owen


__
Do You Yahoo!?
Tired of spam?  Yahoo! Mail has the best spam protection around 
http://mail.yahoo.com 

Re: Pushing jar files on slave machines

[Tarandeep Singh]

Thanks Kyle, I tried libjars option but it didn't work. I tried on 0.18
version.
But I guess I had not set the classpath. So I will try again.

Anyways, putting jars in distributed cache solved my problem. But
-libjars options seems lot more useful and easy to use :)

thanks,
Taran

On Fri, Oct 17, 2008 at 10:09 AM, Kyle Lau <[EMAIL PROTECTED]> wrote:

> If I understand your problem correctly, one solution that worked for me
> is to use the -libjars flag when launching your hadoop job:
>
> bin/hadoop jar -libjars  yourMainClass.jar
> ...
>
> I used this solution on my 5-slave cluster.  I needed to have the third
> party jar files to become available to all nodes without me manually
> distributing them from the master node where I launch my job.
>
> Kyle
>
>
>
>
> On Mon, 2008-10-13 at 12:11 -0700, Allen Wittenauer wrote:
> > On 10/13/08 11:06 AM, "Tarandeep Singh" <[EMAIL PROTECTED]> wrote:
> > > I want to push third party jar files that are required to execute my
> job, on
> > > slave machines. What is the best way to do this?
> >
> > Use a DistributedCache as part of your job submission.
> >
>

Re: Any one successfully ran the c++ pipes example?

[Owen O'Malley]

On Oct 16, 2008, at 1:40 PM, Zhengguo 'Mike' SUN wrote:

I was trying to write an application using the pipes api. But it  
seemed the serialization part is not working correctly. More  
specifically, I can't deserialize a string from an StringInStream  
constructed from context.getInputSplit(). Even with the examples  
bundled in the distribution archive(wordcount-nopipe.cc), it threw  
exceptions. If anyone had experience on that, please kindly give  
some advise.


So you mean the example with a C++ record reader? You have to use the  
InputFormat that generates input splits that consists of strings. Look  
at src/test/org/apache/hadoop/pipes/WordCountInputFormat.java


It would be useful to have a C++ impl of FileInputSplit too...

-- Owen

Re: Chukwa Support

[Jerome Boulon]

We're trying to get all the patches available for end of next week.
Regards,
  Jerome.

On 10/17/08 1:24 AM, "Alex Loddengaard" <[EMAIL PROTECTED]> wrote:

> Thanks, Jerome.  Any ETA on these patches and twiki updates?
> 
> I'm mostly interested in using Chukwa for log analysis.  That is, I want to
> get a better idea of how Hadoop works, and to familiarize myself with Hadoop
> logs.
> 
> Alex
> 
> On Thu, Oct 16, 2008 at 3:12 PM, Jerome Boulon <[EMAIL PROTECTED]>wrote:
> 
>> Hi Alex,
>> Chukwa has recently been deployed at Yahoo and now we are in the process of
>> building a new series of patches to update the hadoop repository.
>> Along with those patches, we're going to update the twiki and the
>> deployment
>> procedure.
>> 
>> For the licensing issue (HICC), we're also working on it but it will take
>> more time since we have to change the library that we are using.
>> 
>> Could you give us more information on you're planning to use chukwa?
>> 
>> Regards,
>>   Jerome B.
>> 
>> On 10/16/08 12:16 PM, "Ariel Rabkin" <[EMAIL PROTECTED]> wrote:
>> 
>>> Hey, glad to see that Chukwa is getting some attention and interest.
>>> 
>>> An adaptor is a Java class that implements
>>> org.apache.hadoop.chukwa.datacollection.adaptor.Adaptor.  The Adaptor
>>> javadoc should tell you what the methods need to do.
>>> 
>>> You start an adaptor by sending a command of the form "add [classname]
>>> [parameters] 0" to the Chukwa agent over TCP.  By default, Chukwa
>>> listens on port 9093.
>>> 
>>> I don't believe HICC has been publicly released yet, due to annoying
>>> GPL/Apache license incompatibilities.
>>> 
>>> On Wed, Oct 15, 2008 at 3:27 PM, Alex Loddengaard
>>> <[EMAIL PROTECTED]> wrote:
 I'm trying to play with Chukwa, but I'm struggling to get anything
>> going.
 
 I've been operating off of the wiki entry (<
 http://wiki.apache.org/hadoop/Chukwa_Quick_Start>), making revisions as
>> I go
 along.  It's  unclear to me how to 1) create an adapter and 2) start
>> HICC
 (see the wiki for more information).
 
 I've gone through the wiki and created 'Document TODO:' items for each
>> issue
 that I've run in to.  Could someone familiar with Chukwa either comment
>> on
 this issues on the mailing list or update the wiki?
 
 Chukwa seems like a great tool, but it's unclear exactly how to get it
>> up
 and running.
 
>> 
>> 

Re: Add jar file via -libjars - giving errors

[Kyle Lau]

That's right.  I have success with the 0.18 (from EC2) and 0.18.1 (my
local installation) as well.

Kyle


On Tue, 2008-10-07 at 09:13 +0530, Amareshwari Sriramadasu wrote:
> Hi,
> 
>  From 0.19, the jars added using -libjars are available on the client 
> classpath also, fixed by HADOOP-3570.
> 
> Thanks
> Amareshwari
> 
> Mahadev Konar wrote:
> > HI Tarandeep,
> >  the libjars options does not add the jar on the client side. Their is an
> > open jira for that ( id ont remember which one)...
> >
> > Oyu have to add the jar to the
> >
> > HADOOP_CLASSPATH on the client side so that it gets picked up on the client
> > side as well.
> >
> >
> > mahadev
> >
> >
> > On 10/6/08 2:30 PM, "Tarandeep Singh" <[EMAIL PROTECTED]> wrote:
> >
> >   
> >> Hi,
> >>
> >> I want to add a jar file (that is required by mappers and reducers) to the
> >> classpath. Initially I had copied the jar file to all the slave nodes in 
> >> the
> >> $HADOOP_HOME/lib directory and it was working fine.
> >>
> >> However when I tried the libjars option to add jar files -
> >>
> >> $HADOOP_HOME/bin/hadoop  jar myApp.jar -conf $MY_CONF_FILE -libjars 
> >> jdom.jar
> >>
> >>
> >> I got this error-
> >>
> >> java.lang.NoClassDefFoundError: org/jdom/input/SAXBuilder
> >>
> >> Can someone please tell me what needs to be fixed here ?
> >>
> >> Thanks,
> >> Taran
> >> 
> >
> >   
> 

Re: How to change number of mappers in Hadoop streaming?

[Erik Holstad]

Hi Steve!
I you can pass -jobconf mapred.map.tasks=$MAPPERS  -jobconf
mapred.reduce.tasks=$REDUCERS
to the streaming job to set the number of reducers and mappers.

Regards Erik

On Wed, Oct 15, 2008 at 4:25 PM, Steve Gao <[EMAIL PROTECTED]> wrote:

> Is there a way to change number of mappers in Hadoop streaming command
> line?
> I know I can change hadoop-default.xml:
>
> 
>   mapred.map.tasks
>   10
>   The default number of map tasks per job.  Typically set
>   to a prime several times greater than number of available hosts.
>   Ignored when mapred.job.tracker is "local".
>   
> 
>
> But that's for all jobs. What if I just want each job has different
> NUM_OF_Mappers themselves? Thanks
>
>
>
>
>

Re: Pushing jar files on slave machines

[Kyle Lau]

If I understand your problem correctly, one solution that worked for me
is to use the -libjars flag when launching your hadoop job:

bin/hadoop jar -libjars  yourMainClass.jar
...

I used this solution on my 5-slave cluster.  I needed to have the third
party jar files to become available to all nodes without me manually
distributing them from the master node where I launch my job.

Kyle




On Mon, 2008-10-13 at 12:11 -0700, Allen Wittenauer wrote:
> On 10/13/08 11:06 AM, "Tarandeep Singh" <[EMAIL PROTECTED]> wrote:
> > I want to push third party jar files that are required to execute my job, on
> > slave machines. What is the best way to do this?
> 
> Use a DistributedCache as part of your job submission.
> 

NY Hadoop User Group meetup 11/10

[Alex Dorman]

What: Katta and a Case Study 



When: November 10, 2008 6:30 PM



Location:

 

ContextWeb, 

9th floor  

22 Cortlandt Street

New York, NY 10007

 

Learn more here and RSVP:
http://www.meetup.com/Hadoop-NYC/calendar/8979383/
 


Meetup Description: 

Stefan Groschupf will present two topics:


"Katta - how to distribute Lucene indexes in a grid."
Katta is a young open source project that helps to serve very large
indexes or very heavy loaded indexes.
Stefan will give an overview of the architecture, discuss functionality
and API. Further more Stefan will explain how to use katta as large
distributed data storage with xpath style queries support.


"A Case Study - An experience report and architectural overview of a
production system"
Stefan will share his experience building a production system to process
millions of events with a hadoop cluster, generate trend alerts and
reports with hadoop, pig and katta.

About the speaker:

Stefan is an active member of the Open Source Community working on
distributed file system, map reduce and search engine implementation
projects. Stefan also contributed to "Nutch" a set of things like the
nutch plugin system.
Over the past 10 years Stefan is consulting on Internet and database
projects for BMW, Intel, Siemens and Hoffmann La Roche, Krugle many
more. Currently Stefan is CTO at Sproose (a user powered search engine).

Re: Career Opportunity in Hadoop

[Edward J. Yoon]

http://people.apache.org/~edwardyoon/resume.html

On Fri, Oct 17, 2008 at 5:43 AM, JoshuaRec <[EMAIL PROTECTED]> wrote:
>
> I posted about a position with a client of mine a few days ago, and got some
> great responses, people who I think are qualified for the position.  Of
> course, the process takes a little time.
>
> Is anyone else interested in a career opportunity as a "Hadoop Guru" for a
> firm in New York City?
>
> If you are, please respond to me by sending me a copy of your resume.  Send
> it to [EMAIL PROTECTED]  When I receive your resume, I will call
> or email you further details about the job.
>
> Thanks.
> --
> View this message in context: 
> http://www.nabble.com/Career-Opportunity-in-Hadoop-tp20016797p20016797.html
> Sent from the Hadoop core-user mailing list archive at Nabble.com.
>
>



-- 
Best regards, Edward J. Yoon
[EMAIL PROTECTED]
http://blog.udanax.org

Re: Why can't I use hostname in hadoop-site.xml?

[imcaptor]

If use change the hostname, you must change /etc/hosts and
/etc/sysconfig/network

for example:

-bash-3.00$ more /etc/hosts
# Do not remove the following line, or various programs
# that require network functionality will fail.
127.0.0.1 localhost.localdomain localhost
192.168.102.205 hadoop205.test.com

-bash-3.00$ more /etc/sysconfig/network
NETWORKING=yes
HOSTNAME=hadoop205.test.com


If you want to resolve other hosts, you must add the ip address and
hostname pair, for example:
-bash-3.00$ more /etc/hosts
# Do not remove the following line, or various programs
# that require network functionality will fail.
127.0.0.1 localhost.localdomain localhost
192.168.102.205 hadoop205.test.com
192.168.102.206 slave.test.com

David Wei 写道:
> Dear all,
>
> I had configured all the nodes(master/slaves) with the correct hostname
> and all the slaves can be reached with hostname from master, and vice versa.
>
> But in my hadoop-site.xml file, if I configure master's
> "fs.default.name" and "mapred.job.tracker" with hostname, e.g.
> datacenter5:9000 and datacenter5:9001. All the slaves will not be able
> to connect to master:
>
> /
> 9 2008-10-17 14:58:10,940 INFO org.apache.hadoop.ipc.Client: Retrying
> connect to server: datacenter5/192.168.52.129:9000. Already tried 0 time(s).
> 10 2008-10-17 14:58:11,943 INFO org.apache.hadoop.ipc.Client: Retrying
> connect to server: datacenter5/192.168.52.129:9000. Already tried 1 time(s).
>
> If changed the settings with IP, e.g. 192.168.52.129. All the slaves
> could be mounted, but when you try to run something, you will get
> following exceptions:
> FAILED
> Error initializing attempt_200810170708_0003_m_00_0:
> java.lang.IllegalArgumentException: Wrong FS:
> hdfs://192.168.52.129:9000/tmp/hadoop-root/mapred/system/job_200810170708_0003/job.xml,
> expected: hdfs://datacenter5:9000
>
> Does anybody can help?
>
> Thx!
>
> David
>
>
>   

Why can't I use hostname in hadoop-site.xml?

[David Wei]

Dear all,

I had configured all the nodes(master/slaves) with the correct hostname
and all the slaves can be reached with hostname from master, and vice versa.

But in my hadoop-site.xml file, if I configure master's
"fs.default.name" and "mapred.job.tracker" with hostname, e.g.
datacenter5:9000 and datacenter5:9001. All the slaves will not be able
to connect to master:

/
9 2008-10-17 14:58:10,940 INFO org.apache.hadoop.ipc.Client: Retrying
connect to server: datacenter5/192.168.52.129:9000. Already tried 0 time(s).
10 2008-10-17 14:58:11,943 INFO org.apache.hadoop.ipc.Client: Retrying
connect to server: datacenter5/192.168.52.129:9000. Already tried 1 time(s).

If changed the settings with IP, e.g. 192.168.52.129. All the slaves
could be mounted, but when you try to run something, you will get
following exceptions:
FAILED
Error initializing attempt_200810170708_0003_m_00_0:
java.lang.IllegalArgumentException: Wrong FS:
hdfs://192.168.52.129:9000/tmp/hadoop-root/mapred/system/job_200810170708_0003/job.xml,
expected: hdfs://datacenter5:9000

Does anybody can help?

Thx!

David

Re: Chukwa Support

[Alex Loddengaard]

Thanks, Jerome.  Any ETA on these patches and twiki updates?

I'm mostly interested in using Chukwa for log analysis.  That is, I want to
get a better idea of how Hadoop works, and to familiarize myself with Hadoop
logs.

Alex

On Thu, Oct 16, 2008 at 3:12 PM, Jerome Boulon <[EMAIL PROTECTED]>wrote:

> Hi Alex,
> Chukwa has recently been deployed at Yahoo and now we are in the process of
> building a new series of patches to update the hadoop repository.
> Along with those patches, we're going to update the twiki and the
> deployment
> procedure.
>
> For the licensing issue (HICC), we're also working on it but it will take
> more time since we have to change the library that we are using.
>
> Could you give us more information on you're planning to use chukwa?
>
> Regards,
>   Jerome B.
>
> On 10/16/08 12:16 PM, "Ariel Rabkin" <[EMAIL PROTECTED]> wrote:
>
> > Hey, glad to see that Chukwa is getting some attention and interest.
> >
> > An adaptor is a Java class that implements
> > org.apache.hadoop.chukwa.datacollection.adaptor.Adaptor.  The Adaptor
> > javadoc should tell you what the methods need to do.
> >
> > You start an adaptor by sending a command of the form "add [classname]
> > [parameters] 0" to the Chukwa agent over TCP.  By default, Chukwa
> > listens on port 9093.
> >
> > I don't believe HICC has been publicly released yet, due to annoying
> > GPL/Apache license incompatibilities.
> >
> > On Wed, Oct 15, 2008 at 3:27 PM, Alex Loddengaard
> > <[EMAIL PROTECTED]> wrote:
> >> I'm trying to play with Chukwa, but I'm struggling to get anything
> going.
> >>
> >> I've been operating off of the wiki entry (<
> >> http://wiki.apache.org/hadoop/Chukwa_Quick_Start>), making revisions as
> I go
> >> along.  It's  unclear to me how to 1) create an adapter and 2) start
> HICC
> >> (see the wiki for more information).
> >>
> >> I've gone through the wiki and created 'Document TODO:' items for each
> issue
> >> that I've run in to.  Could someone familiar with Chukwa either comment
> on
> >> this issues on the mailing list or update the wiki?
> >>
> >> Chukwa seems like a great tool, but it's unclear exactly how to get it
> up
> >> and running.
> >>
>
>

Re: bad connection ack with firstbadlink 10.23.....

[chandravadana]

ashishgandhe wrote:
> 
> Hi Chandra,
> 
> Were you able to resolve this error? I'm facing the exact same issue.
> 
> hi..
> yes was able to fix this.. it was a firewall issue... try disabling
> firewall on all nodes in the cluster...
> 
> Thanks,
> S.Chandravadana
> 
> Thanks,
> Ashish
> 
> 
> 
> chandravadana wrote:
>> 
>> 
>> hi..
>> 
>> i'm using a cluster of 3 systems.. 1 being the master and the other 2
>> are slaves.
>> 
>> i don get any errors when i cluster 2 systems. when i add the 3rd one, i
>> get the following error..
>> 
>> master: 10.232.25.197
>> slave1: 10.232.25.96
>> slave2:10.232.25.69
>> 
>> wordcount/hi/ dir is the input dir
>> 
>> when i execute :
>> 
>> # bin/hadoop dfs -copyFromLocal wordcount/hi wordcount/ins
>> 
>> 08/08/25 13:43:30 INFO dfs.DFSClient: Exception in
>> createBlockOutputStream java.io.IOException: Bad connect ack with
>> firstBadLink 10.232.25.69:50010
>> 08/08/25 13:43:30 INFO dfs.DFSClient: Abandoning block
>> blk_-3916191835981679734
>> 08/08/25 13:43:36 INFO dfs.DFSClient: Exception in
>> createBlockOutputStream java.io.IOException: Bad connect ack with
>> firstBadLink 10.232.25.69:50010
>> 08/08/25 13:43:36 INFO dfs.DFSClient: Abandoning block
>> blk_-7058774921272589893
>> 08/08/25 13:43:42 INFO dfs.DFSClient: Exception in
>> createBlockOutputStream java.io.IOException: Bad connect ack with
>> firstBadLink 10.232.25.69:50010
>> 08/08/25 13:43:42 INFO dfs.DFSClient: Abandoning block
>> blk_3767065959322874247
>> 08/08/25 13:43:48 INFO dfs.DFSClient: Exception in
>> createBlockOutputStream java.io.IOException: Bad connect ack with
>> firstBadLink 10.232.25.69:50010
>> 08/08/25 13:43:48 INFO dfs.DFSClient: Abandoning block
>> blk_-8330992315825789947
>> 08/08/25 13:43:54 WARN dfs.DFSClient: DataStreamer Exception:
>> java.io.IOException: Unable to create new block.
>> 08/08/25 13:43:54 WARN dfs.DFSClient: Error Recovery for block
>> blk_-8330992315825789947 bad datanode[1]
>> copyFromLocal: Could not get block locations. Aborting...
>> 
>> when i examine the log file of the slave, i see this
>> 
>> 2008-08-25 13:42:18,140 INFO org.apache.hadoop.dfs.DataNode:
>> STARTUP_MSG:
>> /
>> STARTUP_MSG: Starting DataNode
>> STARTUP_MSG:   host = slave1/10.232.25.96 STARTUP_MSG:   args = []
>> STARTUP_MSG:   version = 0.16.4 STARTUP_MSG:   build =
>> http://svn.apache.org/repos/asf/hadoop/core/branches/branch-0.16 -r
>> 652614; compiled by 'hadoopqa' on Fri May  2 00:18:12 UTC 2008
>> /
>> 2008-08-25 13:42:18,634 INFO org.apache.hadoop.dfs.Storage: Storage
>> directory /etc/hadoop_install/hadoop-0.16.4/datanodedir is not
>> formatted.
>> 2009-08-25 13:42:18,634 INFO org.apache.hadoop.dfs.Storage:
>> Formatting ...
>> 2008-08-25 13:42:18,701 INFO org.apache.hadoop.dfs.DataNode: Registered
>> FSDatasetStatusMBean
>> 2008-08-25 13:42:18,701 INFO org.apache.hadoop.dfs.DataNode: Opened
>> server at 50010
>> 2008-08-25 13:42:18,705 INFO org.apache.hadoop.dfs.DataNode: Balancing
>> bandwith is 1048576 bytes/s
>> 2008-08-25 13:42:18,911 INFO org.mortbay.util.Credential: Checking
>> Resource aliases
>> 2008-08-25 13:42:19,013 INFO org.mortbay.http.HttpServer: Version
>> Jetty/5.1.4 2008-08-25 13:42:19,014 INFO org.mortbay.util.Container:
>> Started HttpContext[/static,/static]
>> 2008-08-25 13:42:19,014 INFO org.mortbay.util.Container: Started
>> HttpContext[/logs,/logs]
>> 2008-08-25 13:42:19,579 INFO org.mortbay.util.Container: Started
>> [EMAIL PROTECTED]
>> 2008-08-25 13:42:19,658 INFO org.mortbay.util.Container: Started
>> WebApplicationContext[/,/]
>> 2008-08-25 13:42:19,661 INFO org.mortbay.http.SocketListener: Started
>> SocketListener on 0.0.0.0:50075
>> 2008-08-25 13:42:19,661 INFO org.mortbay.util.Container: Started
>> [EMAIL PROTECTED]
>> 2008-08-25 13:42:19,706 INFO org.apache.hadoop.dfs.DataNode: New storage
>> id DS-860242092-10.232.25.96-50010-1219651939700 is assigned to data-
>> node 10.232.25.96:50010
>> 2008-08-25 13:42:19,733 INFO org.apache.hadoop.metrics.jvm.JvmMetrics:
>> Initializing JVM Metrics with processName=DataNode, sessionId=null
>> 2008-08-25 13:42:19,755 INFO org.apache.hadoop.dfs.DataNode:
>> 10.232.25.96:50010In DataNode.run, data = FSDataset
>> {dirpath='/etc/hadoop_install/hadoop-0.16.4/datanodedir/current'}
>> 2008-08-25 13:42:19,755 INFO org.apache.hadoop.dfs.DataNode: using
>> BLOCKREPORT_INTERVAL of 3538776msec Initial delay: 6msec
>> 2008-08-25 13:42:19,828 INFO org.apache.hadoop.dfs.DataNode: BlockReport
>> of 0 blocks got processed in 20 msecs
>> 2008-08-25 13:45:43,982 INFO org.apache.hadoop.dfs.DataNode: Receiving
>> block blk_1031802361447574775 src: /10.232.25.197:40282
>> dest: /10.232.25.197:50010
>> 2008-08-25 13:45:44,032 INFO org.apache.hadoop.dfs.DataNode: Datanode 0
>> forwarding connect ack to upstream firstbadlink is
>> 2008-08-25 13:45:44,081 INFO org.apache.hadoop.dfs.DataNode: Received
>> block bl