Realtime sensor's tcpip data to hadoop

Sensors' may send tcpip data to server. Each sensor may send tcpip data like a stream to the server, the quatity of the sensors and the data rate of the data is high. Firstly, how the data from tcpip can be put into hadoop. It need to do some process and store in hbase. Does it need through

RE: Wordcount file cannot be located

below to your config - for some reason hadoop-common jar is being overwritten - please share your feedback - thanks config.set(fs.hdfs.impl,org.apache.hadoop.hdfs.DistributedFileSystem.class.getName() On Fri, May 2, 2014 at 12:08 AM, Alex Lee eliy...@hotmail.com wrote: I tried to add the code

Which database should be used

There are many database, such as Hbase, hive and mango etc. I need to choose one to save data big volumn stream data from sensors. Will hbase be good, thanks.

Wordcount file cannot be located

I am using eclipse(kepler) to run the wordcount example on hadoop 2.2 with plugin 2.2. I am trying Run as Configuration, and the Arguments is /tmp/in /tmp/out The console always said: 2014-05-01 21:05:46,280 ERROR [main] security.UserGroupInformation (UserGroupInformation.java:doAs(1494)) -

RE: Wordcount file cannot be located

I tried to add the code, but seems still not working. http://postimg.org/image/6c1dat3jx/ 2014-05-02 11:56:06,780 WARN [main] util.NativeCodeLoader (NativeCodeLoader.java:clinit(62)) - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable

hadoop eclipse plugin compile path

Trying to use the below command to generate hadoop eclipse plugin, but seem the directory =/usr/local/hadoop-2.2.0 not correct. I just used the ambari to installed the hadoop. $ant jar -Dversion=2.2.0 -Declipse.home=/usr/local/eclipse -Dhadoop.home=/usr/local/hadoop-2.2.0 error log BUILD

RE: how to install and distribute master services in 4 nodes.

or if was you I will run the datanode / nodemananger on every nodes. Kind regards Olivier On 6 Apr 2014 05:26, Alex Lee eliy...@hotmail.com wrote: Trying to install hadoop by horton, there some many master services, such as HDFS, Yarn+mapreduce, Nagios, Ganglia, Hive, Hbase and Pig as the image below

RE: how to install and distribute master services in 4 nodes.

Apr 2014 05:26, Alex Lee eliy...@hotmail.com wrote: Trying to install hadoop by horton, there some many master services, such as HDFS, Yarn+mapreduce, Nagios, Ganglia, Hive, Hbase and Pig as the image below. http://postimg.org/image/pwbv6go33/ Currently, only have 4 nodes, and as below, 3

how to install and distribute master services in 4 nodes.

Trying to install hadoop by horton, there some many master services, such as HDFS, Yarn+mapreduce, Nagios, Ganglia, Hive, Hbase and Pig as the image below. http://postimg.org/image/pwbv6go33/ Currently, only have 4 nodes, and as below, 3 nodes may ocuppy by master services, and only 1 node

RE: DNS pre request for hadoop install

/network as well so the hostname is persisted after system reboot. HOSTNAME=n2.myhdp.com --- HOSTNAME=n2 On Thu, Apr 3, 2014 at 6:05 PM, Alex Lee eliy...@hotmail.com wrote: Yes, thanks a lot, it is almost done. From: m...@gopivotal.com Date: Thu, 3 Apr 2014 14:07:24 +0800 Subject: Re: DNS pre

RE: DNS pre request for hadoop install

Yes, thanks a lot, it is almost done. From: m...@gopivotal.com Date: Thu, 3 Apr 2014 14:07:24 +0800 Subject: Re: DNS pre request for hadoop install To: user@hadoop.apache.org So you are on machine n2, right? Run command hostname n2 and run hostname -f, see what's the output. -- Cheers -MJ

DNS pre request for hadoop install

I am trying to use ambari (hortonwoks) to install hadoop. One step is to pre-config DNS as manual link below. http://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.0.9.1/bk_using_Ambari_book/content/ambari-chap1-5-5.html Since I am only using internal network, not sure how to config

RE: DNS pre request for hadoop install

, Apr 2, 2014 at 4:43 PM, Alex Lee eliy...@hotmail.com wrote: I am trying to use ambari (hortonwoks) to install hadoop. One step is to pre-config DNS as manual link below. http://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.0.9.1/bk_using_Ambari_book/content/ambari-chap1-5-5.html Since I

RE: DNS pre request for hadoop install

.localdomain4 ::1 localhost.localdomain localhost localhost6 localhost6.localdomain6 192.168.9.168 devmachine.mydomain devmachine where 192.168.9.168 is my machine ip, devmachine.mydomain is my host fqdn. On Wed, Apr 2, 2014 at 5:13 PM, Alex Lee eliy...@hotmail.com wrote

RE: DNS pre request for hadoop install

command hostname localhost c. in your etc/hosts, change it to the following: 127.0.0.1 localhost.localdomain localhost Now, if you run hostname -f, you will get localhost.localdomain. On Wed, Apr 2, 2014 at 5:59 PM, Alex Lee eliy...@hotmail.com wrote: This is the command result, [root

Newbie, any tutorial for install hadoop 2.3 with proper linux version

Hello, I am quite a newbie here. And want to setup hadoop 2.3 on 4 new PCs. Later may add more PCs into it. Is there any tutorial I can learn from, such as the which linux version I should use, how to setup the linux, and how to install the hadoop step by step. I am trying to setup

RE: Newbie, any tutorial for install hadoop 2.3 with proper linux version

This is the link about cluster setup: https://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-common/ClusterSetup.html - Zhijie On Thu, Feb 27, 2014 at 9:41 PM, Alex Lee eliy...@hotmail.com wrote: Hello, I am quite a newbie here. And want to setup hadoop 2.3 on 4 new PCs. Later may add