Re: JSR 107 with Hadoop

Please open a jira, we can discuss there. thanks, Arun On Aug 14, 2012, at 2:22 PM, Dhruv wrote: > Have there been any attempts to integrate JSR 107 compliant caches on mappers > and reducers? > > There are some use cases where this will be beneficial, but I couldn't find > any suitable plug

Re: New Production Cluster Criticisms/Advice

Real clusters are a tad harder than the pseudo cluster. You may want to consider EMR where you can choose between Amazon's Hadoop release (Its Apache), MapR M3 or MapR M5 On Aug 14, 2012, at 8:36 PM, Eli Finkelshteyn wrote: > Hey Mohammad, > Thanks for the reply. I've been using Hadoop and Pi

Re: New Production Cluster Criticisms/Advice

My 2 cents on Hadoop version in Production: If you think you will be deploying your stuff in prod in in 1-2 month then you should note that cdh4 uses Hadoop-2.0.0-Alpha and "Alpha" release means Hadoop-2.0.0 is not production ready. \. So you might need to make a call on which cdh version to use(c

Re: New Production Cluster Criticisms/Advice

Hey Mohammad, Thanks for the reply. I've been using Hadoop and Pig for a while, and I've setup a pseudo-cluster before. I've just never setup anything production-scale yet and wanted advice on that. Cheers, On Tue, Aug 14, 2012 at 6:20 PM, Mohammad Tariq wrote: > Hello Eli, > > If this is y

Re: New Production Cluster Criticisms/Advice

Hello Eli, If this is your first time with Hadoop then I would suggest to configure a cluster locally just to get yourself familiar with Hadoop(a pseudo setup would do). For your analytical stuff you can have a look at Pig, another member of the Hadoop ecosystem. It's a dataflow language that

New Production Cluster Criticisms/Advice

Hey Folks, I'm going to be setting up my first new production cluster soon, and was hoping to get some advice and criticism on my current plan of action. Here's my current plan: *Background/Requirements:* I'm setting this up for a start-up that's not gathering very big data yet, but will be in the

Re: Hello! - Hadoop: System Requirements.

You can visit this link, if you need some help on how to configure Hadoop in pseudo mode : http://cloudfront.blogspot.in/2012/07/how-to-configure-hadoop.html Regards, Mohammad Tariq On Wed, Aug 15, 2012 at 4:54 AM, Jeffrey Silverman wrote: > Cristian, > > You have a basic network problem.

Re: Hello! - Hadoop: System Requirements.

Cristian, You have a basic network problem. You have a single name, RHEL, which points to two IP addresses, 10.9.6.160 and 10.9.0.188. That won't work. The /etc/hosts file is searched sequentially so it always finds the first occurrence of RHEL. By default, any process that listens on all int

Re: Hello! - Hadoop: System Requirements.

Thanks Harsh J! But never heard about binding services to localhost... How can I do that? Could you be so kind to point to a website or the like in order to learn “how to”? As you already know... I am trying to become a “newbie”... Thanks! From: Harsh J Sent: Tuesday, August 14, 2012 7:57 PM

Re: Hello! - Hadoop: System Requirements.

I agree with Harsh..You are anyway using Hadoop in pseudo mode..And it is also obvious that your actual cluster is gonna be stationary. So need of manipulating the hosts file there. Regards, Mohammad Tariq On Wed, Aug 15, 2012 at 4:27 AM, Harsh J wrote: > Can you not bind all your service

RE: Hello! - Hadoop: System Requirements.

Can you not bind all your services, including Hadoop, to the localhost interface? That usually works for my pseudo instances. On Aug 15, 2012 4:22 AM, "Cristian Carranza" wrote: > Thanks Mohammad... > > Yep, the problem is when IP gets changed... And this will be the rule > since I travel a lot

RE: Hello! - Hadoop: System Requirements.

Thanks Mohammad... Yep, the problem is when IP gets changed... And this will be the rule since I travel a lot for business reasons (I am a quality consultant, hotels here and there...). Also I am testing/learning now in order to give a recommendation to my BI manager in near future... single no

Re: pipes(pydoop) and hbase classpath

Hi, Per: > org.apache.hadoop.hbase.mapreduce.TableInputFormat not org.apache.hadoop.mapred.InputFormat Pydoop seems to be expecting you to pass it an old API class for InputFormat/etc. but you've passed in the newer class. I am unsure what part of your code exactly may be at fault since I do not

pipes(pydoop) and hbase classpath

Hi, I'am trying to read hbase key-values with pipes(pydoop). As hadoop is unable to find the hbase jar files. I get Exception in thread "main" java.lang.RuntimeException: java.lang.RuntimeException: class org.apache.hadoop.hbase.mapreduce.TableInputFormat not org.apache.hadoop.mapred.InputFormat

RE: Pending reducers

Alright, thanks, we're already busy rolling out the config for the capacity scheduler. Still, interesting behavior. The fifo scheduler looks at the load on the cores? Seems unnecessary, the kernel is quite good at context switching. Evert From: Harsh J [h

Re: Native not compiling in OS X Mountain Lion

Hi I fixed some of the errors.. then before proceeding i though i would confirm this with others. I can give this a try. ./Zahoor http://blog.zahoor.in On 14-Aug-2012, at 2:07 AM, Colin McCabe wrote: > There hasn't been a lot of demand for Hadoop clusters running on MacOS > X, so in general

Re: Hello! - Hadoop: System Requirements.

Hello Cristian, No question is "dull"..I also do the same thing when stuck. Now,the reason behind your problem is that when you switch from wired to wireless, the IP gets changed. Just use "ifconfig" command to get the IP and paste it in your hosts file along with your hostname. BTW, are you

RE: Hello! - Hadoop: System Requirements.

Thanks again Mohammad! Please help me to go further on your advice, since I am not good dealing with hostnames and network conf... I am using a wired connection to the internet right now and can "ping myself" {ping } and IBI/Hadoop are running fine. But if I change to a wireless connecti

Re: Hello! - Hadoop: System Requirements.

If you don't want to use static IP, use the hostname everywhere in your configuration. But you need to modify the /etc/hosts file everytime to reflect the changes. Regards, Mohammad Tariq On Tue, Aug 14, 2012 at 7:43 PM, Cristian Carranza < cristiancarranz...@hotmail.com> wrote: > Julien

Re: GOODBYE from user@hadoop.apache.org

Try to unsubscribe from both user@hadoop.apache.org and common-u...@hadoop.apache.org (by sending one email to each unsubscribe address, user-unsubscribe@... and common-user-unsubscribe@...). M. On 14.08.2012 14:42, sathyavageeswaran wrote: You are very lucky. I also got the automated message

Re: Hello! - Hadoop: System Requirements.

Julien: Thanks for your prompt response. But... Is there a way to use Hadoop without a static IP address with a VM? Thanks! From: Julien Muller Sent: Tuesday, August 14, 2012 11:05 AM To: user@hadoop.apache.org Subject: Re: Hello! - Hadoop: System Requirements. if your purpose is learn / dev

Re: Hello! - Hadoop: System Requirements.

Hello Cristian, I would suggest you to begin with a pseudo-distributed setup. And once you are comfortable with the settings and configuration procedure, extend it to the fully-distributed setup. As far as requirements are concerned, it's always better to have a Linux machine with Oracle Java

Re: Hello! - Hadoop: System Requirements.

if your purpose is learn / dev / demo, it would be a good idea to use a VM. You will not only be able to use a static IP, but also keep copies of working systems, make some tests, learn how to setup a cluster with only 1 physical machine. And much more. Julien 2012/8/14 Cristian Carranza > He

do comparators need to be thread safe?

Do comparators set via JobConf.setOutputKeyComparator or JobConf.setOutputValueGroupingComparator and not through WritableComparator.define need to be thread safe?

Re: submit a job in a remote jobtracker

But this solution implies that a user must access the remote machine before submit the job. This is not what I want. I want to submit a job in my local machine, and it will be forwarded to the remote JobTracker. On 14 August 2012 14:15, Harsh J wrote: > Hi Pedro, > > This has been asked before.

Hello! - Hadoop: System Requirements.

Hello all in this list! Thank you Harsh J for your help! I decided to post it again in order to properly identify myself in the archive list. My name is Cristián and I am trying to learn Hadoop and to use it in near future... I’ve started to learn Hadoop via Infosphere BigInsights (IBI), Ba

Re: unsubscribe

http://grokbase.com/t/hadoop/hdfs-user/1288vhqqbk/howto-unsubscribe-from-a-mailing-list On Tue, Aug 14, 2012 at 11:17 AM, Maheswaran wrote: > unsubscribe > >

Re: submit a job in a remote jobtracker

Hi Pedro, This has been asked before. See http://search-hadoop.com/m/bikPd1LWhhB1 (or search more on that same site) On Tue, Aug 14, 2012 at 6:32 PM, Pedro Sá da Costa wrote: > I want to submit a job in a remove job tracker, how can I do it? > > -- > Best regards, > -- Harsh J

Re: Pending reducers

I guess this is the regular behavior of the default FIFO task scheduler. It takes into account the reducer load and that may be why it refused to schedule the rest up immediately. You may have better luck using either Fair or Capacity schedulers. On Tue, Aug 14, 2012 at 5:56 PM, Evert Lammerts wr

RE: GOODBYE from user@hadoop.apache.org

You are very lucky. I also got the automated message but continue to be flooded with emails. From: Hung-Sheng Tsao Ph.D. [mailto:laot...@gmail.com] Sent: 14 August 2012 18:04 To: sathyavageeswaran Subject: Fwd: GOODBYE from user@hadoop.apache.org I just unsubscribe ---

RE: Pending reducers

> whats the memory/cpu stats on the machines ? are they exhausted No, they're not. The nodes themselves have more than enough memory available, and the load on the cores sits between 0.8 and 0.9. Is current load in terms other than available slots even taken into account in the default schedule

Re: unsubscribe FYI

Is it a co-incidence that Cutting et al live in California? I think not. :-P (And he is old enough to remember 'Hotel California') On Aug 14, 2012, at 7:14 AM, "sathyavageeswaran" wrote: > Sending mail to user-unsubscr...@hadoop.apache.org is of no use. > > “once a hadoopite, always a had

RE: unsubscribe FYI

Sending mail to user-unsubscr...@hadoop.apache.org is of no use. "once a hadoopite, always a hadoopite" From: Hung-Sheng Tsao Ph.D. [mailto:laot...@gmail.com] Sent: 14 August 2012 17:28 To: user@hadoop.apache.org Cc: sathyavageeswaran Subject: Re: unsubscribe FYI hi just FYI please

Re: unsubscribe FYI

hi just FYI please send mail to user-unsubscr...@hadoop.apache.org not to user@hadoop.apache.org *to unsubscribe* regards If you use Hadoop Common, please subscribe to the Hadoop user mailing list . The Hadoop user mailing list is: use

Re: Pending reducers

whats the memory/cpu stats on the machines ? are they exhausted On Tue, Aug 14, 2012 at 5:20 PM, Evert Lammerts wrote: >> reducers of multiple jobs do run con-currently as long as they have the >> resources available. > > Yep, and that's what's not happening in my situation. 528 reduce slots, 400

RE: Pending reducers

> reducers of multiple jobs do run con-currently as long as they have the > resources available. Yep, and that's what's not happening in my situation. 528 reduce slots, 400 taken by one job, 26 of another job remain in pending state. What could explain this behavior? Evert > > If you want to

RE: unsubscribe

Once into hadoop mailing list you have no option but to continue or change your email ID. From: kalakoti shyam [mailto:shyam.kalak...@gmail.com] Sent: 14 August 2012 16:48 To: user@hadoop.apache.org Subject: unsubscribe Thanks Shyam K +91 92434 21209 _ No virus found in this m

Re: DataNode and Tasttracker communication

Hi James, thank you for your reply! i tried to, but i can only see my own processes, since i am no root user. :( I already sent out a request to the cluster admins to sort this out for me. Regards, Björn Am 14.08.2012 08:51, schrieb James Brown: Hi Bjorn, For the two items below, it is poss

Re: unsubscribe

by any chance the unsubscribe mailing list is added under user list ? can't think so many people who can understand hadoop .. can not understand how to unsubscribe :( On Tue, Aug 14, 2012 at 4:44 PM, Michael Segel wrote: > Welcome to the hotel California... > > Oh wait, many of you are way to yo

unsubscribe

Thanks *Shyam K +91 92434 21209*

Re: unsubscribe

Welcome to the hotel California... Oh wait, many of you are way to young to remember that song On Aug 14, 2012, at 12:55 AM, "sathyavageeswaran" wrote: > You can't unsubscribe. Registering at hadoop is one way ticket. > > -Original Message- > From: Maheswaran [mailto:maheswa...@ik

unsubscribe

unsubscribe

Re: Pending reducers

reducers of multiple jobs do run con-currently as long as they have the resources available. If you want to limit someone overtaking the cluster, then you can create different job queues and assign quota to each queue. You also have the flexibility of allocating max quota per user in a queue as we

Pending reducers

Hi list, I have a cluster running Hadoop 0.20.205 with Kerberos enabled, exposing 528 map slots and 528 reduce slots. Currently somebody is running a NORMAL priority job with 7 mappers and 400 reducers. The mappers have finished and the system is processing the reducers. Another user is running