Unsubscribe

Using libhdfspp

Hi, I have downloaded libhdfspp from the following link, and compiled it. https://github.com/apache/hadoop I found that some functions like hdfsWrite, hdfsHSync are not defined in this library. Also, when I was trying to replace the old libhdfs.so with this new library I am seeing some

Using compression support available in native libraries

Hi, I would like to use compression support available in native libraries. I have tried to google but couldn't know how can I use the LZ4 compression algorithm for compressing the data I am writing to the HDFS files I am writing from my C++ code? Is it possible to get the data written in

Socket Timeout Exception while multiple concurrent applications are reading HDFS data through WebHDFS interface

Hi, We are seeing this SocketTImeout exception while a number of concurrent applications (probably, 50 of them) are trying to read HDFS data through WebHDFS interface. Are there any parameters we can tune so it doesn't happen? An exception occurred: java.net.SocketTimeoutException: Read timed

Re: ETL/DW to Hadoop migrations

Abhishek, Are you looking for loading your data into Hadoop? if yes, IBM DataStage has a stage called BDFS that loads/writes your data into Hadoop. Thanks, Kishore On Tue, Sep 8, 2015 at 1:29 AM, <23singhabhis...@gmail.com> wrote: > Hi guys, > > I am looking for pointers on migrating

Re: Question about Block size configuration

The default HDFS block size 64 MB means, it is the maximum size of block of data written on HDFS. So, if you write 4 MB files, they will still be occupying only 1 block of 4 MB size, not more than that. If your file is more than 64MB, it gets split into multiple blocks. If you set the HDFS block

Apache Slider stop function not working

Hi, I am not aware of any Slider specific group, so I am posting it here. We are using Apache Slider 0.60 and implemented the management operations start, status, stop, etc. in python script. Everything else is working but the stop function is not getting invoked when the container is

Re: 100% CPU consumption by Resource Manager process

I think this config is problematic, too small heartbeat-interval will cause NM contact RM too often. I would suggest you can set this value larger like 1000. Thanks, Wangda On Wed, Aug 13, 2014 at 4:42 PM, Krishna Kishore Bonagiri write2kish...@gmail.com wrote: Hi Wangda, Thanks

Re: 100% CPU consumption by Resource Manager process

) Have you set yarn.scheduler.capacity.schedule-asynchronously.enable=true? 4) What's the yarn.resourcemanager.nodemanagers.heartbeat-interval-ms in your configuration? Thanks, Wangda Tan On Sun, Aug 10, 2014 at 11:29 PM, Krishna Kishore Bonagiri write2kish...@gmail.com wrote: Hi

Re: Negative value given by getVirtualCores() or getAvailableResources()

can set yarn.scheduler.minimum-allocation-vcores=0 Hope this helps, Wangda Tan On Thu, Aug 7, 2014 at 7:13 PM, Krishna Kishore Bonagiri write2kish...@gmail.com wrote: Hi, I am calling getAvailableResources() on AMRMClientAsync and getting -ve value for the number virtual cores

100% CPU consumption by Resource Manager process

Hi, My YARN resource manager is consuming 100% CPU when I am running an application that is running for about 10 hours, requesting as many as 27000 containers. The CPU consumption was very low at the starting of my application, and it gradually went high to over 100%. Is this a known issue or

Negative value given by getVirtualCores() or getAvailableResources()

Hi, I am calling getAvailableResources() on AMRMClientAsync and getting -ve value for the number virtual cores as below. Is there something wrong? memory:16110, vCores:-2. I have set the vcores in my yarn-site.xml like this, and just ran an application that requires two containers other than

How to check what is the log directory for container logs

Hi, Is there a way to check what is the log directory for container logs in my currently running instance of YARN from the command line, I mean using the yarn command or hadoop command or so? Thanks, Kishore

Re: priority in the container request

Vavilapalli vino...@hortonworks.com wrote: Yes, priorities are assigned to ResourceRequests and you can ask multiple containers at the same priority level. You may not get all the containers together as today's scheduler lacks gang functionality. +Vinod On Jun 9, 2014, at 12:08 AM, Krishna Kishore

priority in the container request

Hi, Can we give the same value for priority when requesting multiple containers from the Application Master? Basically, I need all of those containers at the same time, and I am requesting them at the same time. So, I am thinking if we can do that? Thanks, Kishore

Getting the name of the host on which Application Master is launched

Hi, Is there a way to get the name of the host where the AM is launched? I have seen that there is a method getHost() in the ApplicationReport that we get in YarnClient, but it is giving null. Is there a way to make it work? or is there any other way to get the host name? 2014-05-09 04:36:05

Specifying a node/host name on which the Application Master should run

Hi, Is there a way to specify a host name on which we want to run our application master. Can we do this when it is being launched from the YarnClient? Thanks, Kishore

Re: Cleanup activity on YARN containers

container's list are sent to ApplicationMaster in heartbeat. Here you can do clean up activities for containers. Hope this will help for you. J!! Thanks Regards Rohith Sharma K S *From:* Krishna Kishore Bonagiri [mailto:write2kish...@gmail.com] *Sent:* 07 April 2014 16:41

Reuse of YARN container

Hi, Does this JIRA issue mean that we can't currently reuse a container for running/launching two different processes one after another? https://issues.apache.org/jira/browse/YARN-373 If that is true, are there any plans for making that possible? Thanks, Kishore

Re: Cleanup activity on YARN containers

Regards Rohith Sharma K S *From:* Krishna Kishore Bonagiri [mailto:write2kish...@gmail.com] *Sent:* 08 April 2014 20:01 *To:* user@hadoop.apache.org *Subject:* Re: Cleanup activity on YARN containers Hi Rohith, Thanks for the reply. Mine is a YARN application. I have some

Cleanup activity on YARN containers

Hi, Is there any callback kind of facility, in which I can write some code to be executed on my container at the end of my application or at the end of that particular container execution? I want to do some cleanup activities at the end of my application, and the clean up is not related to

Value for yarn.nodemanager.address in configuration file

Hi, This is regarding a single node cluster setup If I have a value of 0.0.0.0:8050 for yarn.nodemanager.address in the configuration file yarn-site.xml/yarn-default.xml is it mandatory requirement that ssh 0.0.0.0 should work on my machine for being able to start YARN? Or will I be

Re: Node manager or Resource Manager crash

this question before. Check if your OS' OOM killer is killing it. +Vinod On Mar 4, 2014, at 6:53 AM, Krishna Kishore Bonagiri write2kish...@gmail.com wrote: Hi, I am running an application on a 2-node cluster, which tries to acquire all the containers that are available on one of those

Node manager or Resource Manager crash

Hi, I am running an application on a 2-node cluster, which tries to acquire all the containers that are available on one of those nodes and remaining containers from the other node in the cluster. When I run this application continuously in a loop, one of the NM or RM is getting killed at a

Re: Node manager or Resource Manager crash

is killing it. +Vinod On Mar 4, 2014, at 6:53 AM, Krishna Kishore Bonagiri write2kish...@gmail.com wrote: Hi, I am running an application on a 2-node cluster, which tries to acquire all the containers that are available on one of those nodes and remaining containers from the other node

YARN -- Debug messages in logs

Hi, How can I get the debug log messages from RM and other daemons? For example, Currently I could see messages from LOG.info() only, i.e. something like this: LOG.info(event.getContainerId() + Container Transitioned from + oldState + to + getState()); How can I get those from

Re: Yarn - specify hosts in ContainerRequest

Hi Anand, Which version of Hadoop are you using? It works from 2.2.0 Try like this, and it should work. I am using this feature on 2.2.0 String[] hosts = new String[1]; hosts[0] = node_name; ContainerRequest request = new ContainerRequest(capability, hosts, null, p,

Re: Can we avoid restarting of AM when it fails?

YarnConfiguration or yarn-site.xml. On Fri, Feb 7, 2014 at 5:24 PM, Krishna Kishore Bonagiri write2kish...@gmail.com wrote: Hi, I am having some failure test cases where my Application Master is supposed to fail. But when it fails it is again started with appID_02 . Is there a way

Can we avoid restarting of AM when it fails?

Hi, I am having some failure test cases where my Application Master is supposed to fail. But when it fails it is again started with appID_02 . Is there a way for me to avoid the second instance of the Application Master getting started? Is it re-started automatically by the RM after the first

Re: Command line tools for YARN

node. 2) what are the cluster resources total or available at the time of running the command Not quite sure, you can search possible options in the yarn command menu. And you can always see the resources usage via web UI though. On Mon, Dec 23, 2013 at 10:08 PM, Krishna Kishore Bonagiri

Command line tools for YARN

Hi, Are there any command line tools for things like, 1) checking how many nodes are in my cluster 2) what are the cluster resources total or available at the time of running the command 3) what are the names of nodes in my cluster etc.. Thanks, Kishore

Container exit status for released container

Hi, I am seeing the exit status of released container(released through a call to releaseAssignedContainer()) to be -100. Can my code assume that -100 will always be given as exit status for a released container by YARN? Thanks, Kishore

Re: Yarn -- one of the daemons getting killed

On Dec 17, 2013, at 1:26 AM, Krishna Kishore Bonagiri write2kish...@gmail.com wrote: Hi Jeff, I have run the resource manager in the foreground without nohup and here are the messages when it was killed, it says it is Killed but doesn't say why! 13/12/17 03:14:54 INFO

Re: Yarn -- one of the daemons getting killed

at 11:10 PM, Jeff Stuckman stuck...@umd.edu wrote: What if you open the daemons in a screen session rather than running them in the background -- for example, run yarn resourcemanager. Then you can see exactly when they terminate, and hopefully why. *From: *Krishna Kishore Bonagiri *Sent

Re: Yarn -- one of the daemons getting killed

. If you are running linux, linux OOM killer may be shooting things down. When it happens, you will see something like 'killed process in system's syslog. Thanks, +Vinod On Dec 13, 2013, at 4:52 AM, Krishna Kishore Bonagiri write2kish...@gmail.com wrote: Vinod, One more thing I

Re: Yarn -- one of the daemons getting killed

files as well for daemons. You may find something. Cheers, Vinayakumar B *From:* Krishna Kishore Bonagiri [mailto:write2kish...@gmail.com] *Sent:* 16 December 2013 16:50 *To:* user@hadoop.apache.org *Subject:* Re: Yarn -- one of the daemons getting killed Hi Vinod, Yes, I am

Re: Yarn -- one of the daemons getting killed

that, that is causing them die? If so, how can we resolve this kind of issue? Thanks, Kishore On Fri, Dec 13, 2013 at 10:16 AM, Krishna Kishore Bonagiri write2kish...@gmail.com wrote: No, I am running on 2 node cluster. On Fri, Dec 13, 2013 at 1:52 AM, Vinod Kumar Vavilapalli vino...@hortonworks.com

Yarn -- one of the daemons getting killed

Hi, I am running a small application on YARN (2.2.0) in a loop of 500 times, and while doing so one of the daemons, node manager, resource manager, or data node is getting killed (I mean disappearing) at a random point. I see no information in the corresponding log files. How can I know why is

Re: Yarn -- one of the daemons getting killed

No, I am running on 2 node cluster. On Fri, Dec 13, 2013 at 1:52 AM, Vinod Kumar Vavilapalli vino...@hortonworks.com wrote: Is all of this on a single node? Thanks, +Vinod On Dec 12, 2013, at 3:26 AM, Krishna Kishore Bonagiri write2kish...@gmail.com wrote: Hi, I am running a small

How to find which version of Hadoop or YARN

Hi, Is there any command for finding which version of Hadoop or YARN we are running? If so, what is that? Thanks, Kishore

Re: How to find which version of Hadoop or YARN

---Brahma -- *From:* Krishna Kishore Bonagiri [write2kish...@gmail.com] *Sent:* Tuesday, December 10, 2013 1:30 PM *To:* user@hadoop.apache.org *Subject:* How to find which version of Hadoop or YARN Hi, Is there any command for finding which version of Hadoop

Re: YARN: LocalResources and file distribution

Hi Arun, I have copied a shell script to HDFS and trying to execute it on containers. How do I specify my shell script PATH in setCommands() call on ContainerLaunchContext? I am doing it this way String shellScriptPath = hdfs://isredeng:8020/user/kbonagir/KKDummy/list.ksh;

Re: Time taken for starting AMRMClientAsync

, Krishna Kishore Bonagiri write2kish...@gmail.com wrote: Hi Alejandro, I don't know what are managed and unmanaged AMs, can you please explain me what are the difference and how are each of them launched? I tried to google for these terms and came across hadoop-yarn-applications-unmanaged-am

Re: Time taken for starting AMRMClientAsync

, at 7:16 AM, Krishna Kishore Bonagiri wrote: Hi, I am seeing the following call to start() on AMRMClientAsync taking from 0.9 to 1 second. Why does it take that long? Is there a way to reduce it, I mean does it depend on any of the interval parameters or so in configuration files? I have tried

Re: Time taken for starting AMRMClientAsync

masters one after another and still seeing 800 to 900 ms being taken for the start() call on AMRMClientAsync in all of those applications. Please suggest if you think I am missing something else Thanks, Kishore On Tue, Nov 19, 2013 at 6:07 PM, Krishna Kishore Bonagiri write2kish...@gmail.com wrote

Unmanaged AMs

Hi, I have seen in comments for code in UnmanagedAMLauncher.java that AM can be in any language. What does that mean? Can AM be written in C++ language? If so, how would I be able to be connect to RM and how would I be able to request for containers? I mean what is the interface doing these

Running multiple application from the same Application Master

Hi, I was reading on this link http://hortonworks.com/blog/apache-hadoop-yarn-concepts-and-applications/ that we can implement an Application Master to manage multiple applications. The text reads like this: *It’s useful to remember that, in reality, every application has its own instance of

Re: Time taken for starting AMRMClientAsync

...@hortonworks.com wrote: It is just creating a connection to RM and shouldn't take that long. Can you please file a ticket so that we can look at it? JVM class loading overhead is one possibility but 1 sec is a bit too much. Thanks, +Vinod On Oct 21, 2013, at 7:16 AM, Krishna Kishore Bonagiri

Re: Time taken for starting AMRMClientAsync

Hi Alejandro, Can you please see if you can answer my question above? I would like to reduce the time taken by the above calls made by my Application Master, the way you do. Thanks, Kishore On Tue, Oct 22, 2013 at 3:09 PM, Krishna Kishore Bonagiri write2kish...@gmail.com wrote: Hi

Re: Time taken for starting AMRMClientAsync

of 20 ms. The application master submission throughput (discarding the first submission) tops at approximately 100 application masters per second. I believe there is room for improvement there. Cheers On Mon, Oct 21, 2013 at 7:16 AM, Krishna Kishore Bonagiri write2kish...@gmail.com wrote

Time taken for starting AMRMClientAsync

Hi, I am seeing the following call to start() on AMRMClientAsync taking from 0.9 to 1 second. Why does it take that long? Is there a way to reduce it, I mean does it depend on any of the interval parameters or so in configuration files? I have tried reducing the value of the first argument below

Re: Hadoop-2.0.1 log files deletion

Hi Reyane, Did you try yarn.nodemanager.log.retain-seconds? increasing that might help. The default value is 10800 seconds, that means 3 hours. Thanks, Kishore On Thu, Oct 10, 2013 at 8:27 PM, Reyane Oukpedjo oukped...@gmail.comwrote: Hi there, I was running somme mapreduce jobs on

Single Yarn Client -- multiple applications?

Hi, Can we submit multiple applications from the same Client class? It seems to be allowed now, I just tried it with Distributed Shell example... Is it OK to do so? or does it have any wrong implications? Thanks, Kishore

Can container requests be made paralelly from multiple threads

Hi, Can we submit container requests from multiple threads in parallel to the Resource Manager? Thanks, Kishore

Re: Can container requests be made paralelly from multiple threads

On Fri, Sep 27, 2013 at 8:31 AM, Krishna Kishore Bonagiri write2kish...@gmail.com wrote: Hi, Can we submit container requests from multiple threads in parallel to the Resource Manager? Thanks, Kishore CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual

Re: Container allocation fails randomly

then please file a map reduce bug. Thanks, Omkar Joshi *Hortonworks Inc.* http://www.hortonworks.com On Tue, Sep 17, 2013 at 2:47 AM, Krishna Kishore Bonagiri write2kish...@gmail.com wrote: Hi Omkar, Thanks for the quick reply, and sorry for not being able to get the required logs that you

Re: Container allocation fails randomly

try applying patch https://issues.apache.org/jira/browse/YARN-1053; ? and check if you can see any message? Thanks, Omkar Joshi *Hortonworks Inc.* http://www.hortonworks.com On Thu, Sep 12, 2013 at 6:15 AM, Krishna Kishore Bonagiri write2kish...@gmail.com wrote: Hi, I am using 2.1.0

Container allocation fails randomly

Hi, I am using 2.1.0-beta and have seen container allocation failing randomly even when running the same application in a loop. I know that the cluster has enough resources to give, because it gave the resources for the same application all the other times in the loop and ran it successfully.

Requesting set of containers on a single node

Hi, My application has a group of processes that need to communicate with each other either through Shared Memory or TCP/IP depending on where the containers are allocated, on the same machine or on different machines. Obviously I would like them to get them allocated on the same node whenever

Re: whitelist feature of YARN

confusing. -Sandy On Tue, Jul 9, 2013 at 2:54 AM, Krishna Kishore Bonagiri write2kish...@gmail.com wrote: Hi Sandy, Yes, I have been using AMRMClient APIs. I am planning to shift to whatever way is this white list feature is supported with. But am not sure what is meant by submitting

Re: whitelist feature of YARN

: On Tue, Aug 6, 2013 at 11:55 PM, Krishna Kishore Bonagiri write2kish...@gmail.com wrote: Hi Sandy, Thanks for the reply and it is good to know YARN-521 is done! Please answer my following questions 1) when is 2.1.0-beta going to be released? is it soon or do you suggest me take it from

Re: Extra start-up overhead with hadoop-2.1.0-beta

from this is the other load on node manager same? is the load on hdfs same? did you see any network bottleneck? More information will help a lot. Thanks, Omkar Joshi *Hortonworks Inc.* http://www.hortonworks.com On Thu, Aug 1, 2013 at 2:19 AM, Krishna Kishore Bonagiri write2kish

Re: Extra start-up overhead with hadoop-2.1.0-beta

can imagine it causing degradation if the configuration files are super big / some other weird cases. -- *From:* Krishna Kishore Bonagiri write2kish...@gmail.com *To:* user@hadoop.apache.org *Sent:* Wednesday, August 7, 2013 10:03 AM *Subject:* Re: Extra start

Re: setLocalResources() on ContainerLaunchContext

startContainer call or the one running AM container? Thanks, Omkar Joshi *Hortonworks Inc.* http://www.hortonworks.com On Tue, Aug 6, 2013 at 10:43 PM, Krishna Kishore Bonagiri write2kish...@gmail.com wrote: Hi Harsh, Hitesh Omkar, Thanks for the replies. I tried getting the last

Re: setLocalResources() on ContainerLaunchContext

in the URI transformation possibly breaking a HDFS source, but try passing a local file - does that fail too? The Shell example uses a local file. On Tue, Aug 6, 2013 at 10:54 AM, Krishna Kishore Bonagiri write2kish...@gmail.com wrote: Hi Harsh, Please see if this is useful, I got

Re: setLocalResources() on ContainerLaunchContext

:/ scheme On Aug 6, 2013 4:05 PM, Krishna Kishore Bonagiri write2kish...@gmail.com wrote: Hi Harsh, The setResource() call on LocalResource() is expecting an argument of type org.apache.hadoop.yarn.api.records.URL which is converted from a string in the form of URI. This happens

Re: setLocalResources() on ContainerLaunchContext

it. This is explicitly done to ensure that file is not updated after user makes the call to avoid possible errors. Thanks, Omkar Joshi Hortonworks Inc. On Tue, Aug 6, 2013 at 5:25 AM, Krishna Kishore Bonagiri write2kish...@gmail.com wrote: I tried the following and it works

setLocalResources() on ContainerLaunchContext

Hi, Can someone please tell me what is the use of calling setLocalResources() on ContainerLaunchContext? And, also an example of how to use this will help... I couldn't guess what is the String in the map that is passed to setLocalResources() like below: // Set the local resources

Re: setLocalResources() on ContainerLaunchContext

-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/src/main/java/org/apache/hadoop/yarn/applications/distributedshell/ApplicationMaster.java#L791 On Mon, Aug 5, 2013 at 2:44 PM, Krishna Kishore Bonagiri write2kish...@gmail.com wrote: Hi, Can someone

Re: setLocalResources() on ContainerLaunchContext

at 12:05 AM, Harsh J ha...@cloudera.com wrote: The detail is insufficient to answer why. You should also have gotten a trace after it, can you post that? If possible, also the relevant snippets of code. On Mon, Aug 5, 2013 at 6:36 PM, Krishna Kishore Bonagiri write2kish...@gmail.com wrote

Re: Extra start-up overhead with hadoop-2.1.0-beta

? More information will help a lot. Thanks, Omkar Joshi *Hortonworks Inc.* http://www.hortonworks.com On Thu, Aug 1, 2013 at 2:19 AM, Krishna Kishore Bonagiri write2kish...@gmail.com wrote: Hi, Please share with me if you anyone has an answer or clues to my question regarding the start

Re: Extra start-up overhead with hadoop-2.1.0-beta

on the container. Thanks, Kishore On Thu, Jul 25, 2013 at 8:38 PM, Krishna Kishore Bonagiri write2kish...@gmail.com wrote: Hi, I have been using the hadoop-2.0.1-beta release candidate and observed that it is slower in running my simple application that runs on 2 containers. I have tried

Re: Node manager crashing when running an app requiring 100 containers on hadoop-2.1.0-beta RC0

node? On Jul 25, 2013, at 5:21 AM, Krishna Kishore Bonagiri write2kish...@gmail.com wrote: Hi Devaraj, I used to run this application with the same number of containers successfully on previous version, i.e. hadoop-2.0.4-alpha. Is it failing with the new version, because YARN itself is also

Node manager crashing when running an app requiring 100 containers on hadoop-2.1.0-beta RC0

Hi, I am running an application against hadoop-2.1.0-beta RC, and my app requires 117 containers, I have got all the containers allocated, but while starting those containers, at around 99th container the node manager has gone down with the following kind of error in it's log. Also, I could

Re: Node manager crashing when running an app requiring 100 containers on hadoop-2.1.0-beta RC0

in the system by reducing the no of running processes in the system. ** ** Thanks Devaraj k ** ** *From:* Krishna Kishore Bonagiri [mailto:write2kish...@gmail.com] *Sent:* 25 July 2013 16:09 *To:* user@hadoop.apache.org *Subject:* Node manager crashing when running an app

Extra start-up overhead with hadoop-2.1.0-beta

Hi, I have been using the hadoop-2.0.1-beta release candidate and observed that it is slower in running my simple application that runs on 2 containers. I have tried to find out which parts of it is really having this extra overhead(compared to hadoop-2.0.4-alpha), and here is what I found

Re: Namenode automatically going to safemode with 2.1.0-beta

/hadoop-dsadm/dfs/name I'd expect you haven't set it up and /tmp is being used off of the out-of-box defaults. Could you try to set it to a specific directory thats not on /tmp? On Mon, Jul 15, 2013 at 2:43 PM, Krishna Kishore Bonagiri write2kish...@gmail.com wrote: I don't have it in my hdfs

Re: Namenode automatically going to safemode with 2.1.0-beta

set to? From /tmp/hadoop-dsadm/dfs/name I'd expect you haven't set it up and /tmp is being used off of the out-of-box defaults. Could you try to set it to a specific directory thats not on /tmp? On Mon, Jul 15, 2013 at 2:43 PM, Krishna Kishore Bonagiri write2kish...@gmail.com wrote: I don't

Namenode automatically going to safemode with 2.1.0-beta

Hi, I am doing no activity on my single node cluster which is using 2.1.0-beta, and still observed that it has gone to safe mode by itself after a while. I was looking at the name node log and see many of these kinds of entries.. Can anything be interpreted from these? 2013-07-12 09:06:11,256

Re: Namenode automatically going to safemode with 2.1.0-beta

:14 PM, Krishna Kishore Bonagiri write2kish...@gmail.com wrote: Hi, I am doing no activity on my single node cluster which is using 2.1.0-beta, and still observed that it has gone to safe mode by itself after a while. I was looking at the name node log and see many of these kinds of entries

Re: Namenode automatically going to safemode with 2.1.0-beta

wrote: Hi, Pls send Web UI namemnode page and what is ur total hard disk size On Mon, Jul 15, 2013 at 1:54 PM, Krishna Kishore Bonagiri write2kish...@gmail.com wrote: I have had enough space on the disk that is used, like around 30 Gigs Thanks, Kishore On Mon, Jul 15, 2013 at 1:30 PM

Re: Namenode automatically going to safemode with 2.1.0-beta

Krishna, Can you please send screenshots of namenode web UI. Thanks Aditya. On Mon, Jul 15, 2013 at 1:54 PM, Krishna Kishore Bonagiri write2kish...@gmail.com wrote: I have had enough space on the disk that is used, like around 30 Gigs Thanks, Kishore On Mon, Jul 15, 2013 at 1:30

Re: whitelist feature of YARN

rack-level ResourceRequest to false * setting the relaxLocality flag on the corresponding any-level ResourceRequest to false -Sandy On Mon, Jul 8, 2013 at 6:48 AM, Krishna Kishore Bonagiri write2kish...@gmail.com wrote: Hi, Can someone please point to some example code of how to use

Re: Requesting containers on a specific host

://people.apache.org/~acmurthy/hadoop-2.1.0-beta-rc0/ or you could check the same with trunk build. ** ** Thanks Devaraj k ** ** *From:* Krishna Kishore Bonagiri [mailto:write2kish...@gmail.com] *Sent:* 04 July 2013 21:33 *To:* user@hadoop.apache.org *Subject:* Re: Requesting

whitelist feature of YARN

Hi, Can someone please point to some example code of how to use the whitelist feature of YARN, I have recently got RC1 for hadoop-2.1.0-beta and want to use this feature. It would be great if you can point me to some description of what this white listing feature is, I have gone through some

Re: Requesting containers on a specific host

the same with trunk build. ** ** Thanks Devaraj k ** ** *From:* Krishna Kishore Bonagiri [mailto:write2kish...@gmail.com] *Sent:* 04 July 2013 21:33 *To:* user@hadoop.apache.org *Subject:* Re: Requesting containers on a specific host ** ** Thanks Arun, it seems

Re: Requesting containers on a specific host

are setting is the same in the RM logs… On Jun 21, 2013, at 2:15 AM, Krishna Kishore Bonagiri write2kish...@gmail.com wrote: Hi, I am trying to get container on a specific host, using setHostName(0 call on ResourceRequest, but could not get allocated anything forever, which just works fine

Re: Requesting containers on a specific host

feature we added recently: https://issues.apache.org/jira/browse/YARN-398 Arun On Jul 4, 2013, at 3:14 AM, Krishna Kishore Bonagiri write2kish...@gmail.com wrote: I could get containers on specific nodes using addContainerRequest() on AMRMClient. But there are issues with it. I have two nodes

Re: Requesting containers on a specific host

a date command on it. Is this something already known? or have I been doing something wrong? Thanks, Kishore On Fri, Jun 21, 2013 at 7:25 PM, Arun C Murthy a...@hortonworks.com wrote: Check if the hostname you are setting is the same in the RM logs… On Jun 21, 2013, at 2:15 AM, Krishna Kishore

Re: Requesting containers on a specific host

Yes Arun, the one I am giving is same as the one I see in RM's log also. Thanks, Kishore On Fri, Jun 21, 2013 at 7:25 PM, Arun C Murthy a...@hortonworks.com wrote: Check if the hostname you are setting is the same in the RM logs… On Jun 21, 2013, at 2:15 AM, Krishna Kishore Bonagiri

Re: Container allocation on the same node

. On Wed, Jun 12, 2013 at 6:25 PM, Krishna Kishore Bonagiri write2kish...@gmail.com wrote: Hi, I want to get some containers for my application on the same node, is there a way to make such a request. For example, I have an application which needs 10 containers, but have

Application Master getting started very late

Hi, I have been using YARN since quite sometime, and recently moved to release 2.0.4. I recently started running huge number of Application Masters (applications) one after another, and observed that sometimes in that sequence the Application Master takes around 1 minute or little more than

Re: What else can be built on top of YARN.

are building it . I wanted to know why is that ? I am in just trying to find out the need or why we might need to write the application on yarn. Rahul On Wed, May 29, 2013 at 8:23 PM, Krishna Kishore Bonagiri write2kish...@gmail.com wrote: Hi Rahul, I am porting a distributed application

Re: What else can be built on top of YARN.

Hi Rahul, I am porting a distributed application that runs on a fixed set of given resources to YARN, with the aim of being able to run it on a dynamically selected resources whichever are available at the time of running the application. Thanks, Kishore On Wed, May 29, 2013 at 8:04 PM,

Out of memory error by Node Manager, and shut down

Hi, I have got the following error in node manager's log, and it got shut down, after about 1 application were run after it was started. Any clue why does it occur... or is this a bug? 2013-05-22 11:53:34,456 FATAL org.apache.hadoop.yarn.YarnUncaughtExceptionHandler: Thread Thread[process

Namenode going to safe mode on YARN

Hi, I have been running application on my YARN cluster since around 20 days, about 5000 applications a day. I am getting the following error today. Please let me know how can I avoid this, is this happening because of a bug?

Re: Namenode going to safe mode on YARN

, Krishna Kishore Bonagiri write2kish...@gmail.com wrote: Hi, I have been running application on my YARN cluster since around 20 days, about 5000 applications a day. I am getting the following error today. Please let me know how can I avoid this, is this happening because of a bug

Re: Namenode going to safe mode on YARN

, Krishna Kishore Bonagiri write2kish...@gmail.com wrote: Hi, I have been running application on my YARN cluster since around 20 days, about 5000 applications a day. I am getting the following error today. Please let me know how can I avoid this, is this happening because of a bug

Re: POLL: Using YARN or pre-YARN?

I have been using YARN, i.e. hadopp-2.0.0-alpha to hadoop-2.0.4-alpha, I don't know what you meant by pre-YARN. Thanks, Kishore On Wed, Apr 24, 2013 at 10:41 PM, Otis Gospodnetic otis.gospodne...@gmail.com wrote: Hi, Quick poll, would be great to know how many people are using YARN vs.

Re: Differences hadoop-2.0.0-alpha Vs hadoop-2.0.3-alpha

of helper libs to make it easier to write new applications. OTOH, your existing application should continue to work. hth, Arun On Mar 26, 2013, at 3:21 AM, Krishna Kishore Bonagiri wrote: Hi, I have some YARN application written and running properly against hadoop-2.0.0-alpha but when I

  1   2   >