Re: Yarn HDFS and Yarn Exceptions when processing "larger" datasets.

Hi Just a quick short reply (tomorrow is my prototype presentation). @Omkar Joshi - RM port 8030 already running when I start my AM - I'll do the client thread size AM - Only AM communicates with RM - RM/NM no exceptions there (as far as I remember will check later [sorry]) Furthermore in fully

RE: Yarn HDFS and Yarn Exceptions when processing "larger" datasets.

Blah blah, Can you build and run the DistributedShell example? If it does not run correctly this would tend to implicate your configuration. If it run correctly then your code is suspect. John From: blah blah [mailto:tmp5...@gmail.com] Sent: Tuesday, June 25, 2013 6:09 PM To: user@hadoop.apac

Re: Yarn HDFS and Yarn Exceptions when processing "larger" datasets.

Also due you see any exception in RM / NM logs? Thanks, Omkar Joshi *Hortonworks Inc.* On Mon, Jul 1, 2013 at 11:19 AM, Omkar Joshi wrote: > Hi, > > As I don't know your complete AM code and how your containers are > communicating with each other...Certain things w

Re: Yarn HDFS and Yarn Exceptions when processing "larger" datasets.

Hi, As I don't know your complete AM code and how your containers are communicating with each other...Certain things which might help you in debugging where you are starting your RM (is it really running on 8030 are you sure there is no previously started RM still running there?) Also in y

Re: Yarn HDFS and Yarn Exceptions when processing "larger" datasets.

Hi Sorry to reply so late. I don't have the data you requested (sorry I have no time, my deadline is within 3 days). However I have observed that this issue occurs not only for the "larger" datasets (6.8MB), but for all datasets and all jobs in general. However for smaller datasets (1MB) the AM do

RE: Yarn HDFS and Yarn Exceptions when processing "larger" datasets.

Hi, Could you check the network usage in the cluster when this problem occurs? Probably it is causing due to high network usage. Thanks Devaraj k From: blah blah [mailto:tmp5...@gmail.com] Sent: 26 June 2013 05:39 To: user@hadoop.apache.org Subject: Yarn HDFS and Yarn Exceptions when process