encountering an error in org.apache.hadoop.security.UserGroupInformation.getOSLoginModuleName(UserGroupInformation.java:303)

2014-01-20 Thread Amit Jain
Hi, I am trying to write a map reduce program on a windows machine and I have imported hadoop-core1.2.1.jar in eclipse on windows I am getting the following error Exception in thread "main" java.lang.NoSuchFieldError: IBM_JAVA at org.apache.hadoop.security.UserGroupInformation.getOSLoginModuleNa

Collecting Big Data with AWS S3/CloudFront Logging

2014-01-20 Thread Moty Michaely
Hey there, Just wanted to share my follow up article about data collection: http://goo.gl/yLJ5Na Comments are more than welcome! Enjoy! -- [image: Inline image 1] *Moty Michaely* VP R&D Cell: +972 (52) 631-1019 Email: m...@xplenty.com Web:www.xplenty.com

Re: Admin Scripts

2014-01-20 Thread hadoop hive
Hey Haohui, Do you have something handy , just for an idea thanks On Mon, Jan 20, 2014 at 10:36 AM, Haohui Mai wrote: > Hi, > > pssh should be able do to the job. > > ~Haohui > > > On Mon, Jan 20, 2014 at 10:34 AM, hadoop hive wrote: > >> Hi Folks, >> >> Does anyone have idea regrading, start

hadoop problem tracking

2014-01-20 Thread Rananavare, Sunil
Hi, we are new to hadoop and trying to use open source apache hadoop (2.2.0) with IBM JDK V7.0. We had to compile hadoop components (e.g. pig, hive, etc.) with IBM JDK (since original distribution worked with Oracle JRE but failed on IBM JRE) however; we discovered a few compilation errors. Here

Re: Hadoop 2 Namenode HA not working properly

2014-01-20 Thread Jing Zhao
Hi Bruno, Could you post your configuration? Also, when you killed one of the NN, you mean only killing the NN process or you shutdown the whole machine? Thanks, -Jing On Mon, Jan 20, 2014 at 4:11 AM, Bruno Andrade wrote: > Hey, > > I have configured a Hadoop v2.2.0 cluster with QJM and Zoo

Re: Admin Scripts

2014-01-20 Thread Nitin Pawar
please look at 1) pdsh 2)clustered shells On Tue, Jan 21, 2014 at 12:04 AM, hadoop hive wrote: > Hi Folks, > > Does anyone have idea regrading, start and stop services on all the nodes > in parallel. i am facing an issue i have a big cluster like 1000 nodes and > i want services to start and st

Re: Admin Scripts

2014-01-20 Thread Haohui Mai
Hi, pssh should be able do to the job. ~Haohui On Mon, Jan 20, 2014 at 10:34 AM, hadoop hive wrote: > Hi Folks, > > Does anyone have idea regrading, start and stop services on all the nodes > in parallel. i am facing an issue i have a big cluster like 1000 nodes and > i want services to start

Admin Scripts

2014-01-20 Thread hadoop hive
Hi Folks, Does anyone have idea regrading, start and stop services on all the nodes in parallel. i am facing an issue i have a big cluster like 1000 nodes and i want services to start and stop , if i do it sequentially it will take around 30-50mins , what can i do to make it parallelly, does anyon

Re: Hadoop migration downtime: Hadoop 1.x to 2.x

2014-01-20 Thread Ravi Mutyala
If you have a mirror of your production environment (not in size but in versions of hardware and software), I suggest you run migration steps on this environment to test and measure the down times. Actual down times are dependent on components in use. You could also do some of your backup steps by

Container's completion issue

2014-01-20 Thread REYANE OUKPEDJO
Hi there, I was using hadoop-2.2.0 to run an application that only lunch 3 containers the application master container and 2 more containers that run the job. one of the 2 containers is returning as completed from the Resource Manager's logs as soon as it is launched. The problem is the process

Hadoop 2 Namenode HA not working properly

2014-01-20 Thread Bruno Andrade
Hey, I have configured a Hadoop v2.2.0 cluster with QJM and Zookeeper for HA and automatic failover. But I'm having a problem. If I test the automatic failover, by killing one of the namenodes, nothing happens. But if I kill the zkfc of that namenode, then zookeeper elects the other namenode a

RE: Hadoop migration downtime: Hadoop 1.x to 2.x

2014-01-20 Thread Nirmal Kumar
Hi All, Any thoughts to share? -Nirmal From: Nirmal Kumar Sent: Thursday, January 16, 2014 5:44 PM To: user@hadoop.apache.org Subject: Hadoop migration downtime: Hadoop 1.x to 2.x All, I want to know while upgrading\migrating from Apache Hadoop 1.x to 2.x(MRv2+YARN) in a production cluster of

Fair scheduler not assigning tasks

2014-01-20 Thread Amit Sela
I've been using Fair Scheduler with Hadoop 1.0.4 for f few months now with no issues what so ever. All of a sudden I have a problem where jobs are in status UNASSIGNED. Jobs submitted are pending for map/reduce slots although the cluster resources are free. In some of the pools only map slots are a