Re: Gathering connection information

2014-06-07 Thread Peyman Mohajerian
In my experience you build a node called Edge Node which has all the
libraries and configuration setting in XML to connect to the cluster, it
just doesn't have any of the Hadoop daemons running.


On Wed, Jun 4, 2014 at 2:46 PM, John Lilley john.lil...@redpoint.net
wrote:

  We’ve found that much of the Hadoop samples assume that running is being
 done form a cluster node, and that the connection information can be
 gleaned directly from a configuration object.  However, we always run our
 client from a remote computer, and our users must manually specify the
 NN/RM addresses and ports.  We’ve found this varies maddeningly between
 distros and especially on hosted virtual implementations.  Getting the
 wrong port results in various inscrutable errors with red-herring messages
 about security.  Is there a prescribed way to get the correct connection
 information more easily, like from a web API (where at least we’d only need
 one address and port)?



 john





Re: Gathering connection information

2014-06-07 Thread Rishi Yadav
Typically users ssh edge node which is co-located with the cluster. It also 
minimizes latency between client and cluster.




—
Sent from Mailbox

On Sat, Jun 7, 2014 at 7:12 AM, Peyman Mohajerian mohaj...@gmail.com
wrote:

 In my experience you build a node called Edge Node which has all the
 libraries and configuration setting in XML to connect to the cluster, it
 just doesn't have any of the Hadoop daemons running.
 On Wed, Jun 4, 2014 at 2:46 PM, John Lilley john.lil...@redpoint.net
 wrote:
  We’ve found that much of the Hadoop samples assume that running is being
 done form a cluster node, and that the connection information can be
 gleaned directly from a configuration object.  However, we always run our
 client from a remote computer, and our users must manually specify the
 NN/RM addresses and ports.  We’ve found this varies maddeningly between
 distros and especially on hosted virtual implementations.  Getting the
 wrong port results in various inscrutable errors with red-herring messages
 about security.  Is there a prescribed way to get the correct connection
 information more easily, like from a web API (where at least we’d only need
 one address and port)?



 john