what about DNS?
if you have 2 computers (nn and dn) how nn knows dn ip?
The script puts only this computer ip to /etc/hosts
On Thu, Mar 5, 2015 at 6:39 PM, max scalf <oracle.bl...@gmail.com> wrote:
> Here is a easy way to go about assigning static name to your ec2
> instance. When you get the launch an EC2-instance from aws console when
> you get to the point of selecting VPC, ip address screen there is a screen
> that says "USER DATA"...put the below in with appropriate host name(change
> CHANGE_HOST_NAME_HERE to whatever you want) and that should be able to get
> you static name.
>
> #!/bin/bash
>
> HOSTNAME_TAG=CHANGE_HOST_NAME_HERE
> cat > /etc/sysconfig/network << EOF
> NETWORKING=yes
> NETWORKING_IPV6=no
> HOSTNAME=${HOSTNAME_TAG}
> EOF
>
> IP=$(curl http://169.254.169.254/latest/meta-data/local-ipv4)
> echo "${IP} ${HOSTNAME_TAG}.localhost ${HOSTNAME_TAG}" >> /etc/hosts
>
> echo ${HOSTNAME_TAG} > /proc/sys/kernel/hostname
> service network restart
>
>
> Also note i was able to do this on couple of spot instance for cheap
> price, only thing is once you shut it down or someone outbids you, you
> loose that instance but its easy/cheap to play around with.... and i have
> used couple of m3.medium for my NN/SNN and couple of them for data nodes...
>
> On Thu, Mar 5, 2015 at 7:19 PM, Jonathan Aquilina <jaquil...@eagleeyet.net
> > wrote:
>
>> I dont know how you would do that to be honest. With EMR you have
>> destinctions master core and task nodes. If you need to change
>> configuration you just ssh into the EMR master node.
>>
>>
>>
>> ---
>> Regards,
>> Jonathan Aquilina
>> Founder Eagle Eye T
>>
>> On 2015-03-06 02:11, Alexander Pivovarov wrote:
>>
>> What is the easiest way to assign names to aws ec2 computers?
>> I guess computer need static hostname and dns name before it can be used
>> in hadoop cluster.
>> On Mar 5, 2015 4:36 PM, "Jonathan Aquilina" <jaquil...@eagleeyet.net>
>> wrote:
>>
>>> When I started with EMR it was alot of testing and trial and error.
>>> HUE is already supported as something that can be installed from the AWS
>>> console. What I need to know is if you need this cluster on all the time or
>>> this is goign ot be what amazon call a transient cluster. Meaning you fire
>>> it up run the job and tear it back down.
>>>
>>>
>>>
>>> ---
>>> Regards,
>>> Jonathan Aquilina
>>> Founder Eagle Eye T
>>>
>>> On 2015-03-06 01:10, Krish Donald wrote:
>>>
>>> Thanks Jonathan,
>>>
>>> I will try to explore EMR option also.
>>> Can you please let me know the configuration which you have used it?
>>> Can you please recommend for me also?
>>> I would like to setup Hadoop cluster using cloudera manager and then
>>> would like to do below things:
>>>
>>> setup kerberos
>>> setup federation
>>> setup monitoring
>>> setup hadr
>>> backup and recovery
>>> authorization using sentry
>>> backup and recovery of individual componenets
>>> performamce tuning
>>> upgrade of cdh
>>> upgrade of CM
>>> Hue User Administration
>>> Spark
>>> Solr
>>>
>>>
>>> Thanks
>>> Krish
>>>
>>>
>>> On Thu, Mar 5, 2015 at 3:57 PM, Jonathan Aquilina <
>>> jaquil...@eagleeyet.net> wrote:
>>>
>>>> krish EMR wont cost you much with all the testing and data we ran
>>>> through the test systems as well as the large amont of data when everythign
>>>> was read we paid about 15.00 USD. I honestly do not think that the specs
>>>> there would be enough as java can be pretty ram hungry.
>>>>
>>>>
>>>>
>>>> ---
>>>> Regards,
>>>> Jonathan Aquilina
>>>> Founder Eagle Eye T
>>>>
>>>> On 2015-03-06 00:41, Krish Donald wrote:
>>>>
>>>> Hi,
>>>>
>>>> I am new to AWS and would like to setup Hadoop cluster using cloudera
>>>> manager for 6-7 nodes.
>>>>
>>>> t2.micro on AWS; Is it enough for setting up Hadoop cluster ?
>>>> I would like to use free service as of now.
>>>>
>>>> Please advise.
>>>>
>>>> Thanks
>>>> Krish
>>>>
>>>>
>
