Re: hadoop installation in pseudo distributed mode regular user vs dedicated user

2015-08-06 Thread Alexander Striffeler
Hi Arvind I can't fully answer your questions on how to install Hadoop in pseudo-distributed mode, but I can kind of invalidate your cons: By using sudo su user in your shell, you can easily users during a session. Giving the hadoop-user access to your directories should then be an issue

hadoop installation in pseudo distributed mode regular user vs dedicated user

2015-08-05 Thread Arvind Sundararajan
Hi All, I have a laptop running Ubuntu 14.04 LTS and am trying to install hadoop 2.7.1 (current stable version) in pseudo-distributed mode. I have a regular user account on my laptop, but am confused if i should install hadoop using a dedicated hadoop user on my laptop. NOTE: By 'regular user

Re: Connection Refused error on Hadoop2.6 running Ubuntu 15.04 Desktop on Pseudo-distributed mode

2015-04-27 Thread Anand Murali
Susheel: Since I am new to this, what log file should I look for in the log dir and what should I be looking for. Thanks Sent from my iPhone On 27-Apr-2015, at 2:07 pm, Susheel Kumar Gadalay skgada...@gmail.com wrote: jps listing is not showing namenode daemon. Verify why namenode is

Re: Connection Refused error on Hadoop2.6 running Ubuntu 15.04 Desktop on Pseudo-distributed mode

2015-04-27 Thread Anand Murali
Many thanks Wellington , but what should I look for. Regards Anand Sent from my iPhone On 27-Apr-2015, at 2:34 pm, Wellington Chevreuil wellington.chevre...@gmail.com wrote: Hello Anand, Per your original email, this would be:

Re: Connection Refused error on Hadoop2.6 running Ubuntu 15.04 Desktop on Pseudo-distributed mode

2015-04-27 Thread Wellington Chevreuil
There might be some FATAL/ERROR/WARN or Exception messages in this log file that can explain why NN process is dying. Can you paste some of the last lines on the log file? On 27 Apr 2015, at 09:37, Susheel Kumar Gadalay skgada...@gmail.com wrote: jps listing is not showing namenode daemon.

Re: Connection Refused error on Hadoop2.6 running Ubuntu 15.04 Desktop on Pseudo-distributed mode

2015-04-27 Thread Wellington Chevreuil
Hello Anand, This error means NN could not find it's metadata directory. You probably need to run hadoop namenode -format command before trying to start hdfs. … 2015-04-27 15:21:42,696 WARN org.apache.hadoop.hdfs.server.namenode.FSNamesystem: Encountered exception loading fsimage

Re: Connection Refused error on Hadoop2.6 running Ubuntu 15.04 Desktop on Pseudo-distributed mode

2015-04-27 Thread Wellington Chevreuil
Because you are probably not defining dfs.namenode.name.dir, the NN metadata directory is being created at tmp and getting deleted once the process is restarted. On 27 Apr 2015, at 11:50, Anand Murali anand_vi...@yahoo.com wrote: Wellington: I have done it at installation time. I shall

Re: Connection Refused error on Hadoop2.6 running Ubuntu 15.04 Desktop on Pseudo-distributed mode

2015-04-27 Thread Anand Murali
Dear Wellington: Many thanks for your help. Deeply appreciate it. It seems to work. I have tried shutting down and starting up twice and tested hdfs dfs -ls /, and it connects to hdfs. Once again many thanks.  Anand Murali  11/7, 'Anand Vihar', Kandasamy St, MylaporeChennai - 600 004, IndiaPh:

Re: Connection Refused error on Hadoop2.6 running Ubuntu 15.04 Desktop on Pseudo-distributed mode

2015-04-27 Thread Anand Murali
Dear Wellington: You were right. There is a error with respect to temp files. Find attached log file. Appreciate your help. Thanks  Anand Murali  11/7, 'Anand Vihar', Kandasamy St, MylaporeChennai - 600 004, IndiaPh: (044)- 28474593/ 43526162 (voicemail) On Monday, April 27, 2015 2:46

Re: Connection Refused error on Hadoop2.6 running Ubuntu 15.04 Desktop on Pseudo-distributed mode

2015-04-27 Thread Anand Murali
Wellington: I have done it at installation time. I shall try once again. However, request you look at this URL, and maybe let me know your views/suggestions. BTW, if I uninstall and re-install this error goes away for that session. Thanks.  Anand Murali  11/7, 'Anand Vihar', Kandasamy St,

Re: Connection Refused error on Hadoop2.6 running Ubuntu 15.04 Desktop on Pseudo-distributed mode

2015-04-27 Thread Susheel Kumar Gadalay
jps listing is not showing namenode daemon. Verify why namenode is not up from the logs. On 4/27/15, Anand Murali anand_vi...@yahoo.com wrote: Dear All: Please find below. and_vihar@Latitude-E5540:~/hadoop-2.6.0/sbin$ start-dfs.sh Starting namenodes on [localhost] localhost: starting

Connection Refused error on Hadoop2.6 running Ubuntu 15.04 Desktop on Pseudo-distributed mode

2015-04-27 Thread Anand Murali
Dear All: Please find below. and_vihar@Latitude-E5540:~/hadoop-2.6.0/sbin$ start-dfs.sh     Starting namenodes on [localhost] localhost: starting

Re: Connection Refused error on Hadoop2.6 running Ubuntu 15.04 Desktop on Pseudo-distributed mode

2015-04-27 Thread Wellington Chevreuil
Hello Anand, Per your original email, this would be: /home/anand_vihar/hadoop-2.6.0/logs/hadoop-anand_vihar-namenode-Latitude-E5540.out Cheers. On 27 Apr 2015, at 09:41, Anand Murali anand_vi...@yahoo.com wrote: Susheel: Since I am new to this, what log file should I look for in the log

Re: Can Pseudo-Distributed Mode take advantage of multi-core structure?

2015-03-24 Thread Michael Segel
Short answer yes. On Mar 24, 2015, at 11:53 AM, Xuzhan Sun sunxuz...@outlook.com wrote: Hello, I want to do some test on my single node cluster for Speed. I know it is easy to set up the Pseudo-Distributed Mode, and Hadoop will start one Java process for each single map/reduce. My

Can Pseudo-Distributed Mode take advantage of multi-core structure?

2015-03-24 Thread Xuzhan Sun
Hello, I want to do some test on my single node cluster for Speed. I know it is easy to set up the Pseudo-Distributed Mode, and Hadoop will start one Java process for each single map/reduce. My question is: is it parallel enough on multi-core CPU? I mean if I have 4 mappers at the same time

答复: Can Pseudo-Distributed Mode take advantage of multi-core structure?

2015-03-24 Thread Xuzhan Sun
So I can set up the pseudo distributed mode without virtual machine and get the same performance? Am I right? Sent from my Windows Phone 发件人: Michael Segelmailto:msegel_had...@hotmail.com 发送时间: ‎2015/‎3/‎25 1:17 收件人: user@hadoop.apache.orgmailto:user

Re: core-site.xml not found while installing CDH 5 with YARN on a Single Linux Node in Pseudo-distributed mode

2014-09-18 Thread Ravindra
-distributed mode on CentOS 6 64 bit. But whenever i am running any hdfs command i am getting error: core-site.xml not found. I am using a proxy server. Please help how to solve this problem. The error file is attached herewith. -- Thanks and regards Vandana kumari

Re: core-site.xml not found while installing CDH 5 with YARN on a Single Linux Node in Pseudo-distributed mode

2014-09-18 Thread Vandana kumari
then export enviroment variable HADOOP_CONF_DIR set to this path. On Thu, Sep 18, 2014 at 2:46 PM, Vandana kumari kvandana1...@gmail.com wrote: Hello all I am manually Installing CDH 5 with YARN on a Single Linux Node in Pseudo-distributed mode on CentOS 6 64 bit. But whenever i am running

Re: core-site.xml not found while installing CDH 5 with YARN on a Single Linux Node in Pseudo-distributed mode

2014-09-18 Thread Ravindra
check if /etc/hadoop/conf/ exists. If it exists then export enviroment variable HADOOP_CONF_DIR set to this path. On Thu, Sep 18, 2014 at 2:46 PM, Vandana kumari kvandana1...@gmail.com wrote: Hello all I am manually Installing CDH 5 with YARN on a Single Linux Node in Pseudo-distributed

Re: core-site.xml not found while installing CDH 5 with YARN on a Single Linux Node in Pseudo-distributed mode

2014-09-18 Thread Vandana kumari
. If it exists then export enviroment variable HADOOP_CONF_DIR set to this path. On Thu, Sep 18, 2014 at 2:46 PM, Vandana kumari kvandana1...@gmail.com wrote: Hello all I am manually Installing CDH 5 with YARN on a Single Linux Node in Pseudo-distributed mode on CentOS 6 64 bit. But whenever i am

Re: core-site.xml not found while installing CDH 5 with YARN on a Single Linux Node in Pseudo-distributed mode

2014-09-18 Thread Ritesh Kumar Singh
...@gmail.com wrote: Hello all I am manually Installing CDH 5 with YARN on a Single Linux Node in Pseudo-distributed mode on CentOS 6 64 bit. But whenever i am running any hdfs command i am getting error: core-site.xml not found. I am using a proxy server. Please help how to solve this problem

Re: core-site.xml not found while installing CDH 5 with YARN on a Single Linux Node in Pseudo-distributed mode

2014-09-18 Thread Ravindra
kvandana1...@gmail.com wrote: Hello all I am manually Installing CDH 5 with YARN on a Single Linux Node in Pseudo-distributed mode on CentOS 6 64 bit. But whenever i am running any hdfs command i am getting error: core-site.xml not found. I am using a proxy server. Please help how to solve

Re: core-site.xml not found while installing CDH 5 with YARN on a Single Linux Node in Pseudo-distributed mode

2014-09-18 Thread Vandana kumari
Installing CDH 5 with YARN on a Single Linux Node in Pseudo-distributed mode on CentOS 6 64 bit. But whenever i am running any hdfs command i am getting error: core-site.xml not found. I am using a proxy server. Please help how to solve this problem. The error file is attached herewith. -- Thanks

Re: Started learning Hadoop. Which distribution is best for native install in pseudo distributed mode?

2014-08-15 Thread Mohan Radhakrishnan
: Re: Started learning Hadoop. Which distribution is best for native install in pseudo distributed mode? He didn¹t ask for the best and nobody framed up their answer like that. He asked what people were using. Out of the 10 responses only four of them actually answered his question

Re: Started learning Hadoop. Which distribution is best for native install in pseudo distributed mode?

2014-08-14 Thread Chris MacKenzie
\Bob\ Wakefield, MBA adaryl.wakefi...@hotmail.com Reply-To: user@hadoop.apache.org Date: Thursday, 14 August 2014 01:13 To: user@hadoop.apache.org Subject: Re: Started learning Hadoop. Which distribution is best for native install in pseudo distributed mode? He didn¹t ask for the best

Re: Started learning Hadoop. Which distribution is best for native install in pseudo distributed mode?

2014-08-14 Thread Kai Wähner
: Thursday, 14 August 2014 01:13 To: user@hadoop.apache.org Subject: Re: Started learning Hadoop. Which distribution is best for native install in pseudo distributed mode? He didn¹t ask for the best and nobody framed up their answer like that. He asked what people were using. Out of the 10

Re: Pseudo -distributed mode

2014-08-13 Thread Sergey Murylev
same process as hadoop client. 2. pseudo distributed mode http://hadoop.apache.org/docs/r1.2.1/single_node_setup.html- you have simple configuration with namenode, secondarynamenode, datanode, jobtracker and tasktracker daemons. In this case map-reduce jobs would be processed

Re: Started learning Hadoop. Which distribution is best for native install in pseudo distributed mode?

2014-08-13 Thread Andre Kelpe
Why don't you just use the apache tarball? We even have that automated, if vagrant is your thing: https://github.com/Cascading/vagrant-cascading-hadoop-cluster - André On Tue, Aug 12, 2014 at 10:12 PM, mani kandan mankand...@gmail.com wrote: Which distribution are you people using? Cloudera

Re: Started learning Hadoop. Which distribution is best for native install in pseudo distributed mode?

2014-08-13 Thread Sebastiano Di Paola
Hi, I'm a newbie too and I'm not using any particular distribution. Just download the component I need / want to try for my deploiment and use them. It's a slow process but allows me to better understand what I'm doing under the hood. Regards, Seba On Tue, Aug 12, 2014 at 10:12 PM, mani kandan

Re: Started learning Hadoop. Which distribution is best for native install in pseudo distributed mode?

2014-08-13 Thread Kilaru, Sambaiah
for native install in pseudo distributed mode? Hi, I'm a newbie too and I'm not using any particular distribution. Just download the component I need / want to try for my deploiment and use them. It's a slow process but allows me to better understand what I'm doing under the hood. Regards, Seba

Re: Started learning Hadoop. Which distribution is best for native install in pseudo distributed mode?

2014-08-13 Thread Adaryl Bob Wakefield, MBA
distribution is best for native install in pseudo distributed mode? Engough wars on going on which is best. You choose one of it and try to learn and there is nothing that x is better or y is better. It is upto your choice. Thanks, Sam From: Sebastiano Di Paola sebastiano.dipa...@gmail.com Reply

Pseudo -distributed mode

2014-08-12 Thread sindhu hosamane
Can Setting up 2 datanodes on same machine be considered as pseudo-distributed mode hadoop ? Thanks, Sindhu

Re: Pseudo -distributed mode

2014-08-12 Thread Sergey Murylev
Yes :) Pseudo-distributed mode is such configuration when we have some Hadoop environment on single computer. On 12/08/14 18:25, sindhu hosamane wrote: Can Setting up 2 datanodes on same machine be considered as pseudo-distributed mode hadoop ? Thanks, Sindhu signature.asc Description

Re: Pseudo -distributed mode

2014-08-12 Thread sindhu hosamane
I have read By default, Hadoop is configured to run in a non-distributed mode, as a single Java process . But if my hadoop is pseudo distributed mode , why does it still runs as a single Java process and utilizes only 1 cpu core even if there are many more ? On Tue, Aug 12, 2014 at 4:32 PM

Started learning Hadoop. Which distribution is best for native install in pseudo distributed mode?

2014-08-12 Thread mani kandan
Which distribution are you people using? Cloudera vs Hortonworks vs Biginsights?

Re: Started learning Hadoop. Which distribution is best for native install in pseudo distributed mode?

2014-08-12 Thread Kai Voigt
www.linkedin.com/in/bobwakefieldmba Twitter: @BobLovesData From: mani kandan Sent: Tuesday, August 12, 2014 3:12 PM To: user@hadoop.apache.org Subject: Started learning Hadoop. Which distribution is best for native install in pseudo distributed mode? Which distribution are you people using

Re: Started learning Hadoop. Which distribution is best for native install in pseudo distributed mode?

2014-08-12 Thread Adaryl Bob Wakefield, MBA
: Re: Started learning Hadoop. Which distribution is best for native install in pseudo distributed mode? 3. seems a biased and incomplete statement. Cloudera’s distribution CDH is fully open source. The proprietary „stuff you refer to is most likely Cloudera Manager, an additional tool to make

Re: Started learning Hadoop. Which distribution is best for native install in pseudo distributed mode?

2014-08-12 Thread Aaron Eng
distribution is best for native install in pseudo distributed mode? Which distribution are you people using? Cloudera vs Hortonworks vs Biginsights? -- *Kai Voigt* Am Germaniahafen 1 k...@123.org 24143 Kiel +49 160 96683050 Germany @KaiVoigt

Re: Started learning Hadoop. Which distribution is best for native install in pseudo distributed mode?

2014-08-12 Thread Jay Vyas
: Tuesday, August 12, 2014 3:12 PM To: user@hadoop.apache.org Subject: Started learning Hadoop. Which distribution is best for native install in pseudo distributed mode? Which distribution are you people using? Cloudera vs Hortonworks vs Biginsights? Kai VoigtAm

Re: Started learning Hadoop. Which distribution is best for native install in pseudo distributed mode?

2014-08-12 Thread Adaryl Bob Wakefield, MBA
Subject: Re: Started learning Hadoop. Which distribution is best for native install in pseudo distributed mode? On that note, 2 is also misleading/incomplete. You might want to explain which specific features you are referencing so the original poster can figure out if those features are relevant

Re: Install Hadoop on Linux Pseudo Distributed Mode - Root Required?

2013-05-18 Thread Amal G Jose
, but post-port-bind we switch back down to the actual user. On Mon, May 13, 2013 at 8:12 PM, Raj Hadoop hadoop...@yahoo.com wrote: Hi, I am planning to install Hadoop on Linux in a Pseudo Distributed Mode ( One Machine ). Do I require 'root' privileges for install ? Please advise

Re: Install Hadoop on Linux Pseudo Distributed Mode - Root Required?

2013-05-13 Thread Nitin Pawar
You do not root if you want to install everything in your home directory and assuming sun jdk is installed On May 13, 2013 8:13 PM, Raj Hadoop hadoop...@yahoo.com wrote: Hi, I am planning to install Hadoop on Linux in a Pseudo Distributed Mode ( One Machine ). Do I require 'root' privileges

Re: Install Hadoop on Linux Pseudo Distributed Mode - Root Required?

2013-05-13 Thread Mohammad Tariq
On Mon, May 13, 2013 at 8:12 PM, Raj Hadoop hadoop...@yahoo.com wrote: Hi, I am planning to install Hadoop on Linux in a Pseudo Distributed Mode ( One Machine ). Do I require 'root' privileges for install ? Please advise. Thanks, Raj

Re: Install Hadoop on Linux Pseudo Distributed Mode - Root Required?

2013-05-13 Thread Raj Hadoop
, May 13, 2013 10:47 AM Subject: Re: Install Hadoop on Linux Pseudo Distributed Mode - Root Required? Hello Raj,      Install in what sense?Are you planning to use Apache's package?If that is the case you just have to download and unzip it. And you don't need root privilege for that.Or something

Re: Install Hadoop on Linux Pseudo Distributed Mode - Root Required?

2013-05-13 Thread Nitin Pawar
require root privielges? *From:* Mohammad Tariq donta...@gmail.com *To:* user@hadoop.apache.org user@hadoop.apache.org; Raj Hadoop hadoop...@yahoo.com *Sent:* Monday, May 13, 2013 10:47 AM *Subject:* Re: Install Hadoop on Linux Pseudo Distributed Mode - Root Required? Hello Raj

Re: Install Hadoop on Linux Pseudo Distributed Mode - Root Required?

2013-05-13 Thread Raj Hadoop
Subject: Re: Install Hadoop on Linux Pseudo Distributed Mode - Root Required? if you want to install CDH, then you will need root access as it needs to install RPMs  for apache downloads, its not needed  On Mon, May 13, 2013 at 8:25 PM, Raj Hadoop hadoop...@yahoo.com wrote: I am thinking

Re: Install Hadoop on Linux Pseudo Distributed Mode - Root Required?

2013-05-13 Thread shashwat shriparv
If you are installing CDH version of hadoop tell your admi that you need root access as yoiu need to install RPM :) *Thanks Regards* ∞ Shashwat Shriparv

Re: Install Hadoop on Linux Pseudo Distributed Mode - Root Required?

2013-05-13 Thread Harsh J
planning to install Hadoop on Linux in a Pseudo Distributed Mode ( One Machine ). Do I require 'root' privileges for install ? Please advise. Thanks, Raj -- Harsh J

pseudo distributed mode

2013-05-03 Thread mouna laroussi
Hi, I want to configure my Hadoop in tne pseudo distributed mode. when i arrive to the step to format namenode, i foind at the web page 50070 there are no namenode in the cluster. what shouled i do? is there any path to change? Thanks -- LAROUSSI Mouna Élève ingénieur en Génie Logiciel - INSAT

Re: pseudo distributed mode

2013-05-03 Thread Nitin Pawar
once you format the namenode, it will need to started again for the normal purpose usage On Fri, May 3, 2013 at 12:45 PM, mouna laroussi mouna.larou...@gmail.comwrote: Hi, I want to configure my Hadoop in tne pseudo distributed mode. when i arrive to the step to format namenode, i foind

Re: pseudo distributed mode

2013-05-03 Thread Mohammad Tariq
the namenode, it will need to started again for the normal purpose usage On Fri, May 3, 2013 at 12:45 PM, mouna laroussi mouna.larou...@gmail.comwrote: Hi, I want to configure my Hadoop in tne pseudo distributed mode. when i arrive to the step to format namenode, i foind at the web page

Re: pseudo distributed mode

2013-05-03 Thread Roman Shaposhnik
On Fri, May 3, 2013 at 12:15 AM, mouna laroussi mouna.larou...@gmail.com wrote: Hi, I want to configure my Hadoop in tne pseudo distributed mode. when i arrive to the step to format namenode, i foind at the web page 50070 there are no namenode in the cluster. what shouled i do? is there any

Error for Pseudo-distributed Mode

2013-02-12 Thread yeyu1899
Hi all, I installed a redhat_enterprise-linux-x86 in VMware Workstation, and set the virtual machine 1G memory. Then I followed steps guided by Installing CDH4 on a Single Linux Node in Pseudo-distributed Mode —— https://ccp.cloudera.com/display/CDH4DOC/Installing+CDH4+on+a+Single+Linux+Node

RE: Error for Pseudo-distributed Mode

2013-02-12 Thread Vijay Thakorlal
realise that you're running in pseudo-distributed mode): Caused by: com.google.protobuf.ServiceException: java.net.SocketTimeoutException: Call From localhost.localdomain/127.0.0.1 to localhost.localdomain:54113 failed on socket timeout exception: java.net.SocketTimeoutException: 6 millis

Re: setting hadoop for pseudo distributed mode.

2012-12-27 Thread Mohammad Tariq
://mtariq.jux.com/ On Fri, Dec 28, 2012 at 1:33 AM, jamal sasha jamalsha...@gmail.comwrote: Hi, So I am still in process of learning hadoop. I tried to run wordcount.java (by writing my own mapper reducer.. creating jar and then running it in a pseudo distributed mode). At that time I got

Re: Pseudo distributed mode : How to increase no of concurrent map task

2012-09-29 Thread Harsh J
Did you restart your TaskTrackers after increasing the mapred.tasktracker.map.tasks.maximum value in mapred-site.xml? It is a TaskTracker property, not a per-job one. On Sun, Sep 30, 2012 at 12:36 AM, Shing Hing Man mat...@yahoo.com wrote: Hi, I am running Hadoop 1.03 in Pseudo distributed

Re: Pseudo distributed mode : How to increase no of concurrent map task

2012-09-29 Thread Shing Hing Man
my problem ! Thanks! Shing   - Original Message - From: Harsh J ha...@cloudera.com To: user@hadoop.apache.org; Shing Hing Man mat...@yahoo.com Cc: Sent: Saturday, September 29, 2012 8:23 PM Subject: Re: Pseudo distributed mode : How to increase no of concurrent map task Did you

Re: Pseudo distributed mode : How to increase no of concurrent map task

2012-09-29 Thread Jay Vyas
Hmmm... I always make this mistake on my hadoop vm -- trying to set parameters which require xml settings in the conf.setInt(...) API at runtime, which sometimes has no effect. How can we know, (without having to individually troubleshoot a parameter) which parameters CAN versus CANNOT be set

Re: Hadoop in Pseudo-Distributed mode on Mac OS X 10.8

2012-08-31 Thread Stone
running hadoop v1.0.3 in Mac OS X 10.8 with Java_1.6.0_33-b03-424 When running hadoop on pseudo-distributed mode, the map seems to work, but it cannot compute the reduce. 12/08/13 08:58:12 INFO mapred.JobClient: Running job: job_201208130857_0001 12/08/13 08:58:13 INFO mapred.JobClient

Re: Hadoop in Pseudo-Distributed mode on Mac OS X 10.8

2012-08-31 Thread Stone
wrote: Hello, I am running hadoop v1.0.3 in Mac OS X 10.8 with Java_1.6.0_33-b03-424 When running hadoop on pseudo-distributed mode, the map seems to work, but it cannot compute the reduce. 12/08/13 08:58:12 INFO mapred.JobClient: Running job: job_201208130857_0001 12/08/13

Re: Hadoop in Pseudo-Distributed mode on Mac OS X 10.8

2012-08-31 Thread Håvard Wahl Kongsgård
Pseudo-distributed mode is good for developing and testing hadoop code. But instead of experimenting with hadoop on your mac, I would go for hadoop on EC2. With starcluster http://web.mit.edu/star/cluster/ it takes just a single command to start hadoop. You also get a fixed environment. -Håvard

Re: Hadoop in Pseudo-Distributed mode on Mac OS X 10.8

2012-08-30 Thread Stone
v1.0.3 in Mac OS X 10.8 with Java_1.6.0_33-b03-424 When running hadoop on pseudo-distributed mode, the map seems to work, but it cannot compute the reduce. 12/08/13 08:58:12 INFO mapred.JobClient: Running job: job_201208130857_0001 12/08/13 08:58:13 INFO mapred.JobClient: map 0% reduce 0% 12/08

Re: Hadoop in Pseudo-Distributed mode on Mac OS X 10.8

2012-08-30 Thread Harsh J
X 10.8 with Java_1.6.0_33-b03-424 When running hadoop on pseudo-distributed mode, the map seems to work, but it cannot compute the reduce. 12/08/13 08:58:12 INFO mapred.JobClient: Running job: job_201208130857_0001 12/08/13 08:58:13 INFO mapred.JobClient: map 0% reduce 0% 12/08/13 08:58:27

Re: Hadoop in Pseudo-Distributed mode on Mac OS X 10.8

2012-08-30 Thread Hemanth Yamijala
The mapred.local.dir is local directories on the file system of slave nodes. In pseudo distributed mode, this would be your own machine. If you've specified any configuration for it, it should be in your mapred-site.xml. If not, it defaults to value of hadoop.tmp.dir/mapred/local. The default

Re: Hadoop in Pseudo-Distributed mode on Mac OS X 10.8

2012-08-13 Thread Mohammad Tariq
, Subho Banerjee subs.z...@gmail.com wrote: Hello, I am running hadoop v1.0.3 in Mac OS X 10.8 with Java_1.6.0_33-b03-424 When running hadoop on pseudo-distributed mode, the map seems to work, but it cannot compute the reduce. 12/08/13 08:58:12 INFO mapred.JobClient: Running job

Re: Hadoop in Pseudo-Distributed mode on Mac OS X 10.8

2012-08-13 Thread Subho Banerjee
is not accessible. As a result 403 is thrown. Regards, Mohammad Tariq On Mon, Aug 13, 2012 at 9:51 AM, Subho Banerjee subs.z...@gmail.comwrote: Hello, I am running hadoop v1.0.3 in Mac OS X 10.8 with Java_1.6.0_33-b03-424 When running hadoop on pseudo-distributed mode, the map seems to work

Re: Configuring Multiple Data Nodes on Pseudo-distributed mode ?

2011-06-10 Thread Kumar Kandasami
...@gmail.com wrote: Environment: Mac 10.6.x. Hadoop version: hadoop-0.20.2-cdh3u0 Is there any good reference/link that provides configuration of additional data-nodes on a single machine (in pseudo distributed mode). Thanks for the support. Kumar_/|\_ www.saisk.com ku

Re: Configuring Multiple Data Nodes on Pseudo-distributed mode ?

2011-06-10 Thread Harsh J
data-nodes on a single machine (in pseudo distributed mode). Thanks for the support. Kumar    _/|\_ www.saisk.com ku...@saisk.com making a profound difference with knowledge and creativity... -- Harsh J -- Harsh J

Re: Configuring Multiple Data Nodes on Pseudo-distributed mode ?

2011-06-09 Thread Harsh J
: Environment: Mac 10.6.x.  Hadoop version: hadoop-0.20.2-cdh3u0 Is there any good reference/link that provides configuration of additional data-nodes on a single machine (in pseudo distributed mode). Thanks for the support. Kumar    _/|\_ www.saisk.com ku...@saisk.com making a profound

Masters and Slaves Files For Pseudo-Distributed Mode

2011-01-02 Thread Jon Lederman
I am wondering what should the content be for the masters and slaves files when running in pseudo-distributed mode? The only way I could get my DataNode and Secondary Namenode to start was to have both files contain: localhost. Is this correct? -Jon

Re: Masters and Slaves Files For Pseudo-Distributed Mode

2011-01-02 Thread Hari Sreekumar
...@gmail.com wrote: I am wondering what should the content be for the masters and slaves files when running in pseudo-distributed mode? The only way I could get my DataNode and Secondary Namenode to start was to have both files contain: localhost. Is this correct? -Jon

Re: Masters and Slaves Files For Pseudo-Distributed Mode

2011-01-02 Thread Harsh J
On Sun, Jan 2, 2011 at 10:08 PM, Jon Lederman jon2...@gmail.com wrote: I am wondering what should the content be for the masters and slaves files when running in pseudo-distributed mode? The only way I could get my DataNode and Secondary Namenode to start was to have both files contain

Program is running well in pseudo distributed mode on Hadoop-0.18.3, but it is not running in distributed mode on 4 nodes(each running Redhat linux 9)

2010-01-04 Thread Ravi
Hi, I have designed a mapreduce algorithm for all pairs shortest paths problem. As a part of the implementation of this algorithm, I have written the following mapreduce job. It is running well and producing desired output in pseudo distributed mode. I have used a machine with ubuntu 8.04

Re: Program is running well in pseudo distributed mode on Hadoop-0.18.3, but it is not running in distributed mode on 4 nodes(each running Redhat linux 9)

2010-01-04 Thread Steve Loughran
Ravi wrote: Hi, I have designed a mapreduce algorithm for all pairs shortest paths problem. As a part of the implementation of this algorithm, I have written the following mapreduce job. It is running well and producing desired output in pseudo distributed mode. I have used a machine

Re: Program is running well in pseudo distributed mode on Hadoop-0.18.3, but it is not running in distributed mode on 4 nodes(each running Redhat linux 9)

2010-01-04 Thread Ravi
the following mapreduce job. It is running well and producing desired output in pseudo distributed mode. I have used a machine with ubuntu 8.04 and hadoop-0.18.3 to run the job in pseudo distributed mode. When I tried to run the same program on a cluster of 4 machines(each running Redhat linux 9

I am not able to run this program in distributed mode but I am able to run it in pseudo distributed mode

2010-01-02 Thread Ravi
import java.lang.Integer; import java.util.TreeMap; import java.io.IOException; import java.util.Date; import java.util.Iterator; import java.lang.StringBuilder; import java.util.StringTokenizer; import java.util.Random; import java.lang.String; import java.util.TreeSet; import java.util.HashMap;

Re: I am not able to run this program in distributed mode but I am able to run it in pseudo distributed mode

2010-01-02 Thread Ravi
Someone please go through the code and fix the bug. Thanks in advance. On Sat, Jan 2, 2010 at 10:05 PM, Ravi ravindra.babu.rav...@gmail.comwrote: import java.lang.Integer; import java.util.TreeMap; import java.io.IOException; import java.util.Date; import java.util.Iterator; import

Re: I am not able to run this program in distributed mode but I am able to run it in pseudo distributed mode

2010-01-02 Thread Todd Lipcon
http://catb.org/~esr/faqs/smart-questions.html#id383250 Please go through the above explanation of how to ask questions on a mailing list, and repost your question. Thanks in advance. On Sat, Jan 2, 2010 at 8:35 AM, Ravi ravindra.babu.rav...@gmail.com wrote: import java.lang.Integer; import

Can we configure two or more datanode under pseudo-distributed mode?

2009-09-25 Thread Huang Qian
Hi guys, I just want to simulate a cluster with Hadoop on my laptop, so I chose the pseudo-distribute mode. The example is running well, but now I just want to test getting date from different machines. Unfortunately, I have not found anything on that topic yet. Can Hadoop fit my needs under

Re: Can we configure two or more datanode under pseudo-distributed mode?

2009-09-25 Thread Edward Capriolo
Qian, By definition pseudo-distributed means fake distributed. If you want run hadoop on multiple nodes it IS a distributed cluster. Follow the normal cluster setup documentation. http://hadoop.apache.org/common/docs/r0.20.1/cluster_setup.html Thank you, Edward On Fri, Sep 25, 2009 at 2:00 AM,

Re: Can we configure two or more datanode under pseudo-distributed mode?

2009-09-25 Thread Boris Shkolnik
You can run multiple data nodes on the same machine. You should create a separate config directory for each dn. The following stuff needs to be created hdfs-site.xml Pid/log/data/tmp dirs log4.properties Master/slaves And then start these data nodes ( something like this: bin/hdfs --config

Re: Can we configure two or more datanode under pseudo-distributed mode?

2009-09-25 Thread Todd Lipcon
Hi Huang, Boris's answer should work fine. If it would be useful for you to have a single command line tool to start up a pseudo-distributed cluster for testing, please comment on this JIRA: http://issues.apache.org/jira/browse/MAPREDUCE-987 -Todd On Fri, Sep 25, 2009 at 10:19 AM, Boris

Error in starting Pseudo-Distributed mode hadoop-0.19.2

2009-08-14 Thread arvind subramanian
Hi, I am new to Hadoop, and am trying to get Hadoop started in Pseudo-distributed mode on ubuntu jaunty. In the archives I noticed that someone had a similar issue with hadoop-0.20.0, but the logs are different. As in the quickstart guide ( http://hadoop.apache.org/common/docs/current

Problem with starting Hadoop in Pseudo Distributed Mode

2009-08-03 Thread Onur AKTAS
Hi, I'm having troubles with running Hadoop in RHEL 5, I did everything as documented in: http://hadoop.apache.org/common/docs/r0.20.0/quickstart.html And configured: conf/core-site.xml, conf/hdfs-site.xml, conf/mapred-site.xml. Connected to localhost with ssh (did passphrase stuff etc.),

Re: Problem with starting Hadoop in Pseudo Distributed Mode

2009-08-03 Thread Amandeep Khurana
I'm assuming that you have no data in HDFS since it never came up... So, go ahead and clean up the directory where you are storing the datanode's data and the namenode's metadata. After that format the namenode and restart hadoop. 2009/8/3 Onur AKTAS onur.ak...@live.com Hi, I'm having

Re: Problem with starting Hadoop in Pseudo Distributed Mode

2009-08-03 Thread Amandeep Khurana
/hadoop-oracle/dfs/ /tmp/hadoop-oracle/mapred/ If yes, how can I change the directory to anywhere else? I do not want it to be kept in /tmp folder. From: ama...@gmail.com Date: Mon, 3 Aug 2009 17:02:50 -0700 Subject: Re: Problem with starting Hadoop in Pseudo Distributed Mode To: common-user

RE: Problem with starting Hadoop in Pseudo Distributed Mode

2009-08-03 Thread Onur AKTAS
: Re: Problem with starting Hadoop in Pseudo Distributed Mode To: common-user@hadoop.apache.org I'm assuming that you have no data in HDFS since it never came up... So, go ahead and clean up the directory where you are storing the datanode's data and the namenode's metadata. After

Re: Problem with starting Hadoop in Pseudo Distributed Mode

2009-08-03 Thread Amandeep Khurana
: Problem with starting Hadoop in Pseudo Distributed Mode From: ama...@gmail.com To: common-user@hadoop.apache.org Yes, you need to change these directories. The config is put in the hadoop-site.xml. Or in this case, separately in the 3 xmls. See the default xml for syntax and property

Re: Problem with starting Hadoop in Pseudo Distributed Mode

2009-08-03 Thread Amandeep Khurana
temporary directories./description /property Thank you again.. From: ama...@gmail.com Date: Mon, 3 Aug 2009 17:48:24 -0700 Subject: Re: Problem with starting Hadoop in Pseudo Distributed Mode To: common-user@hadoop.apache.org 1. The default xmls are in $HADOOP_HOME/build/classes 2. You

Re: Pseudo-distributed mode problem

2009-06-02 Thread Vasyl Keretsman
30, 2009 at 2:44 AM, Vasyl Keretsman vasi...@gmail.com wrote: Hi all, I am just getting started with hadoop 0.20 and trying to run a job in pseudo-distributed mode. I configured hadoop according to the tutorial, but it seems it does not work as expected. My map/reduce tasks are running

Re: Pseudo-distributed mode problem

2009-06-02 Thread Aaron Kimball
Aaron Kimball aa...@cloudera.com: Can you post the contents of your hadoop-site.xml file here? - Aaron On Sat, May 30, 2009 at 2:44 AM, Vasyl Keretsman vasi...@gmail.com wrote: Hi all, I am just getting started with hadoop 0.20 and trying to run a job in pseudo-distributed mode

Re: Pseudo-distributed mode problem

2009-06-01 Thread Aaron Kimball
Can you post the contents of your hadoop-site.xml file here? - Aaron On Sat, May 30, 2009 at 2:44 AM, Vasyl Keretsman vasi...@gmail.com wrote: Hi all, I am just getting started with hadoop 0.20 and trying to run a job in pseudo-distributed mode. I configured hadoop according

Pseudo-distributed mode problem

2009-05-30 Thread Vasyl Keretsman
Hi all, I am just getting started with hadoop 0.20 and trying to run a job in pseudo-distributed mode. I configured hadoop according to the tutorial, but it seems it does not work as expected. My map/reduce tasks are running sequencially and output result is stored on local filesystem instead

Hadoop warnings in pseudo-distributed mode

2008-07-29 Thread Arv Mistry
Could anyone tell me, is it normal to get warnings could only be replicated to 0 nodes, instead of 1 when running in a psudo-distributed mode i.e. everything on one machine? It seems to be writing to the files that I expect, just I get this warning. If it isn't normal, just some background; -