Re: HADOOP in Production

2012-10-02 Thread Ruslan Al-Fakikh
have understood the Hadoop and Hadoop Ecosystem(Pig as ETL, Hive as DataWare house, Sqoop as importing tool). I worked and learned on single node cluster with demo data. As Hadoop suits best on Unix platform. Please help me to understand the requirement form start to finish to use Hadoop

Re: HADOOP in Production

2012-10-02 Thread Michael Segel
on Unix platform. Please help me to understand the requirement form start to finish to use Hadoop in production. What would be the things to use Hadoop on real time project. like Hadoop automation on Unix, alert of failure process. Please put some light on using Hadoop on real time

RE: HADOOP in Production

2012-10-02 Thread Hank Cohen
There is an important difference between real time and real fast Real time means that system response must meet a fixed schedule. Real fast just means sooner is better. Real time systems always have hard schedules. The schedule could be in microseconds to control a laser for making masks for

Re: HADOOP in Production

2012-10-02 Thread Ted Dunning
On Tue, Oct 2, 2012 at 7:05 PM, Hank Cohen hank.co...@altior.com wrote: There is an important difference between real time and real fast Real time means that system response must meet a fixed schedule. Real fast just means sooner is better. Good thought, but real-time can also include a

RE: HADOOP in Production

2012-10-02 Thread Hank Cohen
Cohen From: Ted Dunning [mailto:tdunn...@maprtech.com] Sent: Tuesday, October 02, 2012 4:13 PM To: user@hadoop.apache.org Subject: Re: HADOOP in Production On Tue, Oct 2, 2012 at 7:05 PM, Hank Cohen hank.co...@altior.commailto:hank.co...@altior.com wrote: There is an important difference between

HADOOP in Production

2012-10-01 Thread yogesh dhari
Hadoop in production. What would be the things to use Hadoop on real time project. like Hadoop automation on Unix, alert of failure process. Please put some light on using Hadoop on real time and what objectives are recommended. Thanks Regards Yogesh Kumar

Re: NEED HELP:: using Hadoop in Production

2012-10-01 Thread Bertrand Dechoux
(Pig as ETL, Hive as DataWare house, Sqoop as importing tool). I worked and learned on single node cluster with demo data. As Hadoop suits best on Unix platform. Please help me to understand the requirement form start to finish to use Hadoop in production. What would be the things to use Hadoop

RES: NEED HELP:: using Hadoop in Production

2012-10-01 Thread Ferreira, Rafael
: NEED HELP:: using Hadoop in Production Prioridade: Alta Hi all, I have understood the Hadoop and Hadoop Ecosystem(Pig as ETL, Hive as DataWare house, Sqoop as importing tool). I worked and learned on single node cluster with demo data. As Hadoop suits best on Unix platform. Please help me

Re: Experience with Hadoop in production

2012-02-25 Thread Jie Li
Hi Pavel, Seems your team spent some time on the performance and tuning issues. Just wonder whether an automatic Hadoop tuning tool like Starfish would be interesting to you. We'd like to exchange the tuning experience with you. Thanks, Jie Starfish Group, Duke

RE: Experience with Hadoop in production

2012-02-24 Thread GOEKE, MATTHEW (AG/1000)
@hadoop.apache.org Subject: Re: Experience with Hadoop in production Just be sure you have that corporate card available 24x7 when you need to call support ;) Sent from my iPhone On Feb 23, 2012, at 10:30, Serge Blazhievsky serge.blazhiyevs...@nice.com wrote: What I have seen companies do often

Experience with Hadoop in production

2012-02-23 Thread Pavel Frolov
Hi, We are going into 24x7 production soon and we are considering whether we need vendor support or not. We use a free vendor distribution of Cluster Provisioning + Hadoop + HBase and looked at their Enterprise version but it is very expensive for the value it provides (additional functionality

Re: Experience with Hadoop in production

2012-02-23 Thread Jamack, Peter
A lot of it depends on your staff and their experiences. Maybe they don't have hadoop, but if they were involved with large databases, data warehouse, etc they can utilize their skills experiences and provide a lot of help. If you have linux admins, system admins, network admins with years of

Re: Experience with Hadoop in production

2012-02-23 Thread Serge Blazhievsky
What I have seen companies do often is that they will use free version of the commercial vendor and only get their support if there are major problems that they cannot solve on their own. That way you will get free distribution and insurance that you have support if something goes wrong. Serge

Re: Experience with Hadoop in production

2012-02-23 Thread Mike Lyon
Just be sure you have that corporate card available 24x7 when you need to call support ;) Sent from my iPhone On Feb 23, 2012, at 10:30, Serge Blazhievsky serge.blazhiyevs...@nice.com wrote: What I have seen companies do often is that they will use free version of the commercial vendor and

Re: Most Common ways to load data into Hadoop in production systems

2010-07-24 Thread Gautam
On Wed, Jul 21, 2010 at 10:25 PM, Edward Capriolo edlinuxg...@gmail.comwrote: On Wed, Jul 21, 2010 at 12:42 PM, Xavier Stevens xstev...@mozilla.com wrote: Hi Urckle, A lot of the more advanced setups just record data directly to HDFS to start with. You have to write some custom code

Most Common ways to load data into Hadoop in production systems

2010-07-21 Thread Urckle
Hi, I have a newbie question. Scenario: Hadoop version: 0.20.2 MR coding will be done in java. Just starting out with my first Hadoop setup. I would like to know are there any best practice ways to load data into the dfs? I have (obviously) manually put data files into hdfs using the shell

Re: Most Common ways to load data into Hadoop in production systems

2010-07-21 Thread Xavier Stevens
Hi Urckle, A lot of the more advanced setups just record data directly to HDFS to start with. You have to write some custom code using the HDFS API but that way you don't need to import large masses of data. People also use distcp to do large scale imports, but if you're hitting something like

Re: Most Common ways to load data into Hadoop in production systems

2010-07-21 Thread Urckle
Hi Xavier, thanks for replying. Your input is very much appreciated! This is exactly what I need. Thanks again, Regards Enthusiastic Hadoop newbie!! :-D On 21/07/2010 17:42, Xavier Stevens wrote: Hi Urckle, A lot of the more advanced setups just record data directly to HDFS to start

Re: Most Common ways to load data into Hadoop in production systems

2010-07-21 Thread Edward Capriolo
On Wed, Jul 21, 2010 at 12:42 PM, Xavier Stevens xstev...@mozilla.com wrote:  Hi Urckle, A lot of the more advanced setups just record data directly to HDFS to start with.  You have to write some custom code using the HDFS API but that way you don't need to import large masses of data.  

Re: Most Common ways to load data into Hadoop in production systems

2010-07-21 Thread Urckle
On 21/07/2010 17:55, Edward Capriolo wrote: On Wed, Jul 21, 2010 at 12:42 PM, Xavier Stevensxstev...@mozilla.com wrote: Hi Urckle, A lot of the more advanced setups just record data directly to HDFS to start with. You have to write some custom code using the HDFS API but that way you

can you refer me to a User with Hadoop in production

2008-07-10 Thread Bill Boas
Please? Bill Boas VP, Business Development System Fabric Works 510-375-8840 [EMAIL PROTECTED] www.systemfabricworks.com