Re: Cloudera Vs Hortonworks Vs MapR

So here's an example of marketing FUD at work. On Mon, Sep 16, 2013 at 3:10 PM, Xuri Nagarin wrote: > So I will try to answer the OP's question best I can without deviating too > much into opinions and stick to facts. Disclaimer: I am not an employee of > either vendor or any partner of theirs.

Re: mapred.join package not migrated to mapreduce

mapred is not recommended for use I understand mapred is based interface, but mapreduce is based on an abstract class 2013/9/17 Ivan Balashov > Hi, > > Just wondering if there is any particular reason that 'mapred.join' > package never found its way into 'mapreduce'. Being in the old space make

Re: Cloudera Vs Hortonworks Vs MapR

Our evaluation was similar except we did not consider the "management" tools any vendor provided as that's just as much lock in as any proprietary tool. What if I want trade vendors? I have to re-tool to use there mgmt? Nope, wrote our own. Being in a large enterprise, we went with the "perceiv

mapred.join package not migrated to mapreduce

Hi, Just wondering if there is any particular reason that 'mapred.join' package never found its way into 'mapreduce'. Being in the old space makes its use rather inconvenient when most of its former neighbors now happily live in the new package. Is this package recommended for prod at all, or may

Resource limits with Hadoop and JVM

We recently experienced a couple of situations that brought one or more Hadoop nodes down (unresponsive). One was related to a bug in a utility we use (ffmpeg) that was resolved by compiling a new version. The next, today, occurred after attempting to join a new node to the cluster. A basic

Re: Cloudera Vs Hortonworks Vs MapR

So I will try to answer the OP's question best I can without deviating too much into opinions and stick to facts. Disclaimer: I am not an employee of either vendor or any partner of theirs. Context is important: My team's use case was general data exploration of semi-structured log data and we had

Re: Resource limits with Hadoop and JVM

Yes, I mentioned below we're running RHEL. In this case, when I went to add the node, I ran "hadoop mradmin -refreshNodes" (as user hadoop) and the master node went completely nuts - the system load jumped to 60 ("top" was frozen on the console) and required a hard reboot. Whether or not the

Re: SequenceFile output in Wordcount example

Moving general@ to bcc On Mon, Sep 16, 2013 at 1:20 PM, xeon wrote: > Hi, > > - I want that the wordcount example produces a SequenceFile output with > the result. How I do this? > > - I want also to do a cat to the SequenceFile and read the result. A > simple "hdfs dfs -cat sequencefile" is e

dfs.namenode.edits.dir and dfs.namenode.shared.edits.dir

I am running CDH4.2. I've noticed that my NameNodes are logging edits both locally and to the journalnodes. I took a look at the code, and this doesn't seem to be required -- and also, it's the whole point of QJM right? However, due to the following, we are logging both locally and to the quorum

Re: Resource limits with Hadoop and JVM

I assume you are on Linux. Also assuming that your tasks are so resource intensive that they are taking down nodes. You should enable limits per task, see http://hadoop.apache.org/docs/stable/cluster_setup.html#Memory+monitoring What it does is that jobs are now forced to up front provide their

Re: assign tasks to specific nodes

Potentially you would be able to but I guess you will have to update the partitioning code and correspondingly RMContainerAllocator (YARN-map reduce) code. Today we have same priority for all map task < same priority for all reduce task. What you can do is to change the MAP task priorities based on

Re: Unclear Hadoop 2.1X documentation

Moving general@ to bcc and redirecting this to the appropriate list - user@hadoop.apache.org On Mon, Sep 16, 2013 at 2:18 AM, Jagat Singh wrote: > Hello Mahmoud > > You can run on your machine also. > > I learnt everything on my 3gb 2ghz machine and recently got better machine. > > If you follo

RE: Hadoop - Browsefile system error

Hi, I checked as you said i'm able to ping with my data nodes using domain name as well as with ip address. Thanks, Manickam P Date: Mon, 16 Sep 2013 17:16:07 +0530 Subject: Re: Hadoop - Browsefile system error From: jeetuyadav200...@gmail.com To: user@hadoop.apache.org Hi, Because you were are

hadoop join example

Hi all, Can anyone explains how the join example in the hadoop source code examples folder works? What I don't really understand is how it works the "mapred.join.expr" parameter in the jobconf. Thanks in advance! K.A

Re: Hadoop - Browsefile system error

Hi, Because you were are accessing name node through ip address not by domain name. may be you have some ip/domain resolver issue, can you please use ping by domain name/ip/hostname from all the nodes? Regards Jitendra On Mon, Sep 16, 2013 at 5:08 PM, Manickam P wrote: > hi, > > It works. but

RE: Hadoop - Browsefile system error

hi, It works. but why it is not coming for master node? Thanks, Manickam P Date: Mon, 16 Sep 2013 16:53:31 +0530 Subject: Re: Hadoop - Browsefile system error From: jeetuyadav200...@gmail.com To: user@hadoop.apache.org Can you try this from your web browser? http://10.108.19.69:50075/browseDir

Re: Hadoop - Browsefile system error

Can you try this from your web browser? http:// 10.108.19.69 :50075/browseDirectory.jsp?namenodeInfoPort=50070&dir=/&nnaddr=10.108.19.68:9000

RE: Hadoop - Browsefile system error

Hi Jitendra, The below are the host entry in my name node. Here 68 is the name node and other two are data nodes. i'm having the same entry in data nodes host file. 10.108.19.68lab2-hadoop.eng.com lab2-hadoop10.108.19.69 lab2-hadoop2-vm1.eng.com lab2-hadoop2-vm110.108.19.7

Re: Hadoop - Browsefile system error

If you don't mind can you please share your host entries from all the nodes? Also let me know from which host you are accessing the URL. Regards Jitendra On Mon, Sep 16, 2013 at 3:56 PM, Manickam P wrote: > Hi, > > I checked the host entry. It has all the details. I checked my data nodes > als

RE: Hadoop - Browsefile system error

Hi, I checked the host entry. It has all the details. I checked my data nodes also even It has the proper host entry. I don't have any clue here. Thanks, Manickam P Date: Mon, 16 Sep 2013 15:41:03 +0530 Subject: Re: Hadoop - Browsefile system error From: jeetuyadav200...@gmail.com To: user@hadoo

Re: Hadoop - Browsefile system error

Hi, Looks like your are using hadoop in Virtual environment right? Make sure you have lab2-hadoop2-vm1.eng.dnb.com domain and IP entry in your host file from where your are accessing http:// 10.108.19.68

RE: Hadoop - Browsefile system error

Hi, The 10.108.19.68 machine is my name node. when try open it browser i'm getting that error. Thanks, Manickam P Date: Mon, 16 Sep 2013 15:26:38 +0530 Subject: Re: Hadoop - Browsefile system error From: jeetuyadav200...@gmail.com To: user@hadoop.apache.org Hi, >From where you are accessing t

Re: Hadoop - Browsefile system error

Hi, >From where you are accessing this "http://10.108.19.68:50070 " URL? Regards Jitendra On Mon, Sep 16, 2013 at 3:22 PM, Manickam P wrote: > Hi, > > I've installed hadoop-2.

Hadoop - Browsefile system error

Hi, I've installed hadoop-2.1.0-beta version and configured. When i start http://10.108.19.68:50070 i'm getting the page. While clicking on the browse file system i'm getting dns unresolved host name error. I've given below the browser URL. http://lab2-hadoop2-vm1.eng.com:50075/browseDirectory.