Re: Doubt from the book "Definitive Guide"

Hi Mohit, What would be the advantage? Reducers in most cases read data from all the mappers. In the case where mappers were to write to HDFS, a reducer would still require to read data from other datanodes across the cluster. Prashant On Apr 4, 2012, at 9:55 PM, Mohit Anchlia wrote: > On Wed,

Re: Doubt from the book "Definitive Guide"

On Wed, Apr 4, 2012 at 8:42 PM, Harsh J wrote: > Hi Mohit, > > On Thu, Apr 5, 2012 at 5:26 AM, Mohit Anchlia > wrote: > > I am going through the chapter "How mapreduce works" and have some > > confusion: > > > > 1) Below description of Mapper says that reducers get the output file > using > > HT

Re: Doubt from the book "Definitive Guide"

Hi Mohit, On Thu, Apr 5, 2012 at 5:26 AM, Mohit Anchlia wrote: > I am going through the chapter "How mapreduce works" and have some > confusion: > > 1) Below description of Mapper says that reducers get the output file using > HTTP call. But the description under "The Reduce Side" doesn't specifi

Re: how to fine tuning my map reduce job that is generating a lot of intermediate key-value pairs (a lot of I/O operations)

serge, i specify 15 instances, but only 14 end up being data/tasks nodes. 1 instance is reserved as the name node (job tracker). On Wed, Apr 4, 2012 at 1:17 PM, Serge Blazhievsky wrote: > How many datanodes do you use fir your job? > > On 4/3/12 8:11 PM, "Jane Wayne" wrote: > >>i don't have the

Re: Doubt from the book "Definitive Guide"

On Wed, Apr 4, 2012 at 5:23 PM, Prashant Kommireddi wrote: > Answers inline. > > On Wed, Apr 4, 2012 at 4:56 PM, Mohit Anchlia >wrote: > > > I am going through the chapter "How mapreduce works" and have some > > confusion: > > > > 1) Below description of Mapper says that reducers get the output f

Re: Doubt from the book "Definitive Guide"

Answers inline. On Wed, Apr 4, 2012 at 4:56 PM, Mohit Anchlia wrote: > I am going through the chapter "How mapreduce works" and have some > confusion: > > 1) Below description of Mapper says that reducers get the output file using > HTTP call. But the description under "The Reduce Side" doesn't s

Doubt from the book "Definitive Guide"

I am going through the chapter "How mapreduce works" and have some confusion: 1) Below description of Mapper says that reducers get the output file using HTTP call. But the description under "The Reduce Side" doesn't specifically say if it's copied using HTTP. So first confusion, Is the output cop

Re: Yahoo Hadoop Tutorial with new APIs?

Nathan but together the steps together on this blog. http://blog.milford.io/2012/01/kicking-the-tires-on-hadoop-0-23-pseudo-distributed-mode/ Which fills out the missing "details" such as yarn.nodemanager.local-dirs the local directories used by the nodemanager in the official

Re: Yahoo Hadoop Tutorial with new APIs?

Ok, Robert, I will be waiting for you then. There are many folks that use this tutorial, so I think this a good effort in favor of the Hadoop community.It would be nice if Yahoo! donate this work, because, I have some ideas behind this, for example: to release a Spanish version of the tutorial.

Re: Yahoo Hadoop Tutorial with new APIs?

Hi, any interest in joining with this effort of mine? http://hadoopilluminated.com/ - I am also doing only for community benefit. I have more chapters that I am putting out. But, I want to keep the fun, informal style. Thanks, Mark On Wed, Apr 4, 2012 at 4:29 PM, Robert Evans wrote: > I am dro

Re: Yahoo Hadoop Tutorial with new APIs?

I am dropping the cross posts and leaving this on common-user with the others BCCed. Marcos, That is a great idea to be able to update the tutorial, especially if the community is interested in helping to do so. We are looking into the best way to do this. The idea right now is to donate thi

Re: opensuse 12.1

Like OpenSUSE is a RPM-based distribution, you can try with the Apache BigTop project [1], and look for the RPM packages and give them a try. You have noticed that the RPM specification between OpenSUSE and Red Hat-based distributions () change a little, but it can be a starting point. See the

Re: how to fine tuning my map reduce job that is generating a lot of intermediate key-value pairs (a lot of I/O operations)

How many datanodes do you use fir your job? On 4/3/12 8:11 PM, "Jane Wayne" wrote: >i don't have the option of setting the map heap size to 2 GB since my >real environment is AWS EMR and the constraints are set. > >http://hadoop.apache.org/common/docs/r0.20.2/mapred_tutorial.html this >link is w

Re: opensuse 12.1

Lots of people seem to start with this. http://www.michael-noll.com/tutorials/running-hadoop-on-ubuntu-linux-multi-node-cluster/  Raj > > From: "Barry, Sean F" >To: "common-user@hadoop.apache.org" >Sent: Wednesday, April 4, 2012 9:12 AM >Subject: FW: opensus

FW: opensuse 12.1

-Original Message- From: Barry, Sean F [mailto:sean.f.ba...@intel.com] Sent: Wednesday, April 04, 2012 9:10 AM To: common-user@hadoop.apache.org Subject: opensuse 12.1 What is the best way to install hadoop on opensuse 12.1 for a small two node cluster. -SB

opensuse 12.1

What is the best way to install hadoop on opensuse 12.1 for a small two node cluster. -SB

Re: Yahoo Hadoop Tutorial with new APIs?

On 04/04/2012 09:15 AM, Jagat Singh wrote: Hello Marcos Yes , Yahoo tutorials are pretty old but still they explain the concepts of Map Reduce , HDFS beautifully. The way in which tutorials have been defined into sub sections , each builing on previous one is awesome. I remember when i star

Re: Yahoo Hadoop Tutorial with new APIs?

Hello Marcos Yes , Yahoo tutorials are pretty old but still they explain the concepts of Map Reduce , HDFS beautifully. The way in which tutorials have been defined into sub sections , each builing on previous one is awesome. I remember when i started i was digged in there for many days. The t

Yahoo Hadoop Tutorial with new APIs?

Regards to all the list. There are many people that use the Hadoop Tutorial released by Yahoo at http://developer.yahoo.com/hadoop/tutorial/ The main issue here is that, this tutorial is written with the old APIs? (Hadoop 0.18 I