[
https://issues.apache.org/jira/browse/WHIRR-363?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13084597#comment-13084597
]
Adrian Cole commented on WHIRR-363:
-----------------------------------
tested on HBase090ServiceTest using -Dwhirr.test.provider=aws-ec2
-Dwhirr.test.identity=XXXX -Dwhirr.test.credential=XXXX
2011-08-13 12:36:57,374 INFO
[org.apache.whirr.service.hbase.integration.HBaseServiceController] (main)
Starting up cluster...
2011-08-13 12:37:15,616 INFO [org.apache.whirr.service.ComputeCache] (main)
created new ComputeServiceContext [id=aws-ec2,
endpoint=https://ec2.us-east-1.amazonaws.com, apiVersion=2010-06-15,
identity=067PW7Z9P0FNH7JDPE82, iso3166Codes=[US-VA, US-CA, IE, SG, JP-13]]
2011-08-13 12:37:15,638 INFO [org.apache.whirr.actions.BootstrapClusterAction]
(main) Bootstrapping cluster
2011-08-13 12:37:15,642 INFO [org.apache.whirr.actions.BootstrapClusterAction]
(main) Configuring template
2011-08-13 12:38:38,344 INFO [org.apache.whirr.actions.NodeStarter]
(pool-3-thread-2) Starting 1 node(s) with roles [hadoop-datanode,
hadoop-tasktracker, hbase-regionserver]
2011-08-13 12:38:38,354 INFO [org.apache.whirr.actions.BootstrapClusterAction]
(main) Configuring template
2011-08-13 12:40:30,377 INFO [org.apache.whirr.actions.NodeStarter]
(pool-3-thread-4) Starting 1 node(s) with roles [hadoop-namenode,
hadoop-jobtracker, zookeeper, hbase-master, hbase-thriftserver]
2011-08-13 12:43:55,473 INFO [org.apache.whirr.actions.NodeStarter]
(pool-3-thread-4) Nodes started: [[id=us-east-1/i-eb662b8a,
providerId=i-eb662b8a, group=hbaseclustertest, name=null,
location=[id=us-east-1b, scope=ZONE, description=us-east-1b, parent=us-east-1,
iso3166Codes=[US-VA], metadata={}], uri=null, imageId=us-east-1/ami-aef607c7,
os=[name=null, family=ubuntu, version=10.04, arch=paravirtual, is64Bit=true,
description=411009282317/RightImage_Ubuntu_10.04_x64_v5.6.8.1_EBS],
state=RUNNING, loginPort=22, hostname=domU-12-31-39-04-65-A8,
privateAddresses=[10.240.106.86], publicAddresses=[50.19.194.241],
hardware=[id=m1.large, providerId=m1.large, name=null, processors=[[cores=2.0,
speed=2.0]], ram=7680, volumes=[[id=null, type=LOCAL, size=420.0,
device=/dev/sdb, durable=false, isBootDevice=false], [id=null, type=LOCAL,
size=420.0, device=/dev/sdc, durable=false, isBootDevice=false],
[id=vol-2cf23e46, type=SAN, size=null, device=/dev/sda1, durable=true,
isBootDevice=true]],
supportsImage=And(ALWAYS_TRUE,Or(isWindows(),requiresVirtualizationType(paravirtual)),ALWAYS_TRUE,is64Bit()),
tags=[]], loginUser=root, userMetadata={}, tags=[]]]
2011-08-13 12:44:15,955 INFO [org.apache.whirr.actions.NodeStarter]
(pool-3-thread-2) Nodes started: [[id=us-east-1/i-71662b10,
providerId=i-71662b10, group=hbaseclustertest, name=null,
location=[id=us-east-1b, scope=ZONE, description=us-east-1b, parent=us-east-1,
iso3166Codes=[US-VA], metadata={}], uri=null, imageId=us-east-1/ami-aef607c7,
os=[name=null, family=ubuntu, version=10.04, arch=paravirtual, is64Bit=true,
description=411009282317/RightImage_Ubuntu_10.04_x64_v5.6.8.1_EBS],
state=RUNNING, loginPort=22, hostname=ip-10-72-102-239,
privateAddresses=[10.72.102.239], publicAddresses=[50.16.76.89],
hardware=[id=m1.large, providerId=m1.large, name=null, processors=[[cores=2.0,
speed=2.0]], ram=7680, volumes=[[id=null, type=LOCAL, size=420.0,
device=/dev/sdb, durable=false, isBootDevice=false], [id=null, type=LOCAL,
size=420.0, device=/dev/sdc, durable=false, isBootDevice=false],
[id=vol-88f13de2, type=SAN, size=null, device=/dev/sda1, durable=true,
isBootDevice=true]],
supportsImage=And(ALWAYS_TRUE,Or(isWindows(),requiresVirtualizationType(paravirtual)),ALWAYS_TRUE,is64Bit()),
tags=[]], loginUser=root, userMetadata={}, tags=[]]]
2011-08-13 12:44:17,080 INFO [org.apache.whirr.service.FirewallManager] (main)
Authorizing firewall ingress to [Instance{roles=[hadoop-namenode,
hadoop-jobtracker, zookeeper, hbase-master, hbase-thriftserver],
publicIp=50.19.194.241, privateIp=10.240.106.86, id=us-east-1/i-eb662b8a,
nodeMetadata=[id=us-east-1/i-eb662b8a, providerId=i-eb662b8a,
group=hbaseclustertest, name=null, location=[id=us-east-1b, scope=ZONE,
description=us-east-1b, parent=us-east-1, iso3166Codes=[US-VA], metadata={}],
uri=null, imageId=us-east-1/ami-aef607c7, os=[name=null, family=ubuntu,
version=10.04, arch=paravirtual, is64Bit=true,
description=411009282317/RightImage_Ubuntu_10.04_x64_v5.6.8.1_EBS],
state=RUNNING, loginPort=22, hostname=domU-12-31-39-04-65-A8,
privateAddresses=[10.240.106.86], publicAddresses=[50.19.194.241],
hardware=[id=m1.large, providerId=m1.large, name=null, processors=[[cores=2.0,
speed=2.0]], ram=7680, volumes=[[id=null, type=LOCAL, size=420.0,
device=/dev/sdb, durable=false, isBootDevice=false], [id=null, type=LOCAL,
size=420.0, device=/dev/sdc, durable=false, isBootDevice=false],
[id=vol-2cf23e46, type=SAN, size=null, device=/dev/sda1, durable=true,
isBootDevice=true]],
supportsImage=And(ALWAYS_TRUE,Or(isWindows(),requiresVirtualizationType(paravirtual)),ALWAYS_TRUE,is64Bit()),
tags=[]], loginUser=root, userMetadata={}, tags=[]]}] on ports [50070, 50030]
for [93.158.79.70/32]
2011-08-13 12:44:21,505 INFO [org.apache.whirr.service.FirewallManager] (main)
Authorizing firewall ingress to [Instance{roles=[hadoop-namenode,
hadoop-jobtracker, zookeeper, hbase-master, hbase-thriftserver],
publicIp=50.19.194.241, privateIp=10.240.106.86, id=us-east-1/i-eb662b8a,
nodeMetadata=[id=us-east-1/i-eb662b8a, providerId=i-eb662b8a,
group=hbaseclustertest, name=null, location=[id=us-east-1b, scope=ZONE,
description=us-east-1b, parent=us-east-1, iso3166Codes=[US-VA], metadata={}],
uri=null, imageId=us-east-1/ami-aef607c7, os=[name=null, family=ubuntu,
version=10.04, arch=paravirtual, is64Bit=true,
description=411009282317/RightImage_Ubuntu_10.04_x64_v5.6.8.1_EBS],
state=RUNNING, loginPort=22, hostname=domU-12-31-39-04-65-A8,
privateAddresses=[10.240.106.86], publicAddresses=[50.19.194.241],
hardware=[id=m1.large, providerId=m1.large, name=null, processors=[[cores=2.0,
speed=2.0]], ram=7680, volumes=[[id=null, type=LOCAL, size=420.0,
device=/dev/sdb, durable=false, isBootDevice=false], [id=null, type=LOCAL,
size=420.0, device=/dev/sdc, durable=false, isBootDevice=false],
[id=vol-2cf23e46, type=SAN, size=null, device=/dev/sda1, durable=true,
isBootDevice=true]],
supportsImage=And(ALWAYS_TRUE,Or(isWindows(),requiresVirtualizationType(paravirtual)),ALWAYS_TRUE,is64Bit()),
tags=[]], loginUser=root, userMetadata={}, tags=[]]}] on ports [8020, 8021]
for [50.19.194.241/32]
2011-08-13 12:44:23,121 INFO [org.apache.whirr.service.FirewallManager] (main)
Authorizing firewall ingress to [Instance{roles=[hadoop-namenode,
hadoop-jobtracker, zookeeper, hbase-master, hbase-thriftserver],
publicIp=50.19.194.241, privateIp=10.240.106.86, id=us-east-1/i-eb662b8a,
nodeMetadata=[id=us-east-1/i-eb662b8a, providerId=i-eb662b8a,
group=hbaseclustertest, name=null, location=[id=us-east-1b, scope=ZONE,
description=us-east-1b, parent=us-east-1, iso3166Codes=[US-VA], metadata={}],
uri=null, imageId=us-east-1/ami-aef607c7, os=[name=null, family=ubuntu,
version=10.04, arch=paravirtual, is64Bit=true,
description=411009282317/RightImage_Ubuntu_10.04_x64_v5.6.8.1_EBS],
state=RUNNING, loginPort=22, hostname=domU-12-31-39-04-65-A8,
privateAddresses=[10.240.106.86], publicAddresses=[50.19.194.241],
hardware=[id=m1.large, providerId=m1.large, name=null, processors=[[cores=2.0,
speed=2.0]], ram=7680, volumes=[[id=null, type=LOCAL, size=420.0,
device=/dev/sdb, durable=false, isBootDevice=false], [id=null, type=LOCAL,
size=420.0, device=/dev/sdc, durable=false, isBootDevice=false],
[id=vol-2cf23e46, type=SAN, size=null, device=/dev/sda1, durable=true,
isBootDevice=true]],
supportsImage=And(ALWAYS_TRUE,Or(isWindows(),requiresVirtualizationType(paravirtual)),ALWAYS_TRUE,is64Bit()),
tags=[]], loginUser=root, userMetadata={}, tags=[]]}] on ports [2181] for
[93.158.79.70/32]
2011-08-13 12:44:23,650 INFO
[org.apache.whirr.service.hbase.HBaseMasterClusterActionHandler] (main)
Authorizing firewall
2011-08-13 12:44:24,247 INFO [org.apache.whirr.service.FirewallManager] (main)
Authorizing firewall ingress to [Instance{roles=[hadoop-namenode,
hadoop-jobtracker, zookeeper, hbase-master, hbase-thriftserver],
publicIp=50.19.194.241, privateIp=10.240.106.86, id=us-east-1/i-eb662b8a,
nodeMetadata=[id=us-east-1/i-eb662b8a, providerId=i-eb662b8a,
group=hbaseclustertest, name=null, location=[id=us-east-1b, scope=ZONE,
description=us-east-1b, parent=us-east-1, iso3166Codes=[US-VA], metadata={}],
uri=null, imageId=us-east-1/ami-aef607c7, os=[name=null, family=ubuntu,
version=10.04, arch=paravirtual, is64Bit=true,
description=411009282317/RightImage_Ubuntu_10.04_x64_v5.6.8.1_EBS],
state=RUNNING, loginPort=22, hostname=domU-12-31-39-04-65-A8,
privateAddresses=[10.240.106.86], publicAddresses=[50.19.194.241],
hardware=[id=m1.large, providerId=m1.large, name=null, processors=[[cores=2.0,
speed=2.0]], ram=7680, volumes=[[id=null, type=LOCAL, size=420.0,
device=/dev/sdb, durable=false, isBootDevice=false], [id=null, type=LOCAL,
size=420.0, device=/dev/sdc, durable=false, isBootDevice=false],
[id=vol-2cf23e46, type=SAN, size=null, device=/dev/sda1, durable=true,
isBootDevice=true]],
supportsImage=And(ALWAYS_TRUE,Or(isWindows(),requiresVirtualizationType(paravirtual)),ALWAYS_TRUE,is64Bit()),
tags=[]], loginUser=root, userMetadata={}, tags=[]]}] on ports [60010, 60000]
for [93.158.79.70/32]
2011-08-13 12:44:26,066 INFO [org.apache.whirr.service.FirewallManager] (main)
Authorizing firewall ingress to [Instance{roles=[hadoop-namenode,
hadoop-jobtracker, zookeeper, hbase-master, hbase-thriftserver],
publicIp=50.19.194.241, privateIp=10.240.106.86, id=us-east-1/i-eb662b8a,
nodeMetadata=[id=us-east-1/i-eb662b8a, providerId=i-eb662b8a,
group=hbaseclustertest, name=null, location=[id=us-east-1b, scope=ZONE,
description=us-east-1b, parent=us-east-1, iso3166Codes=[US-VA], metadata={}],
uri=null, imageId=us-east-1/ami-aef607c7, os=[name=null, family=ubuntu,
version=10.04, arch=paravirtual, is64Bit=true,
description=411009282317/RightImage_Ubuntu_10.04_x64_v5.6.8.1_EBS],
state=RUNNING, loginPort=22, hostname=domU-12-31-39-04-65-A8,
privateAddresses=[10.240.106.86], publicAddresses=[50.19.194.241],
hardware=[id=m1.large, providerId=m1.large, name=null, processors=[[cores=2.0,
speed=2.0]], ram=7680, volumes=[[id=null, type=LOCAL, size=420.0,
device=/dev/sdb, durable=false, isBootDevice=false], [id=null, type=LOCAL,
size=420.0, device=/dev/sdc, durable=false, isBootDevice=false],
[id=vol-2cf23e46, type=SAN, size=null, device=/dev/sda1, durable=true,
isBootDevice=true]],
supportsImage=And(ALWAYS_TRUE,Or(isWindows(),requiresVirtualizationType(paravirtual)),ALWAYS_TRUE,is64Bit()),
tags=[]], loginUser=root, userMetadata={}, tags=[]]}] on ports [9090] for
[93.158.79.70/32]
2011-08-13 12:44:27,166 INFO [org.apache.whirr.service.FirewallManager] (main)
Authorizing firewall ingress to [Instance{roles=[hadoop-namenode,
hadoop-jobtracker, zookeeper, hbase-master, hbase-thriftserver],
publicIp=50.19.194.241, privateIp=10.240.106.86, id=us-east-1/i-eb662b8a,
nodeMetadata=[id=us-east-1/i-eb662b8a, providerId=i-eb662b8a,
group=hbaseclustertest, name=null, location=[id=us-east-1b, scope=ZONE,
description=us-east-1b, parent=us-east-1, iso3166Codes=[US-VA], metadata={}],
uri=null, imageId=us-east-1/ami-aef607c7, os=[name=null, family=ubuntu,
version=10.04, arch=paravirtual, is64Bit=true,
description=411009282317/RightImage_Ubuntu_10.04_x64_v5.6.8.1_EBS],
state=RUNNING, loginPort=22, hostname=domU-12-31-39-04-65-A8,
privateAddresses=[10.240.106.86], publicAddresses=[50.19.194.241],
hardware=[id=m1.large, providerId=m1.large, name=null, processors=[[cores=2.0,
speed=2.0]], ram=7680, volumes=[[id=null, type=LOCAL, size=420.0,
device=/dev/sdb, durable=false, isBootDevice=false], [id=null, type=LOCAL,
size=420.0, device=/dev/sdc, durable=false, isBootDevice=false],
[id=vol-2cf23e46, type=SAN, size=null, device=/dev/sda1, durable=true,
isBootDevice=true]],
supportsImage=And(ALWAYS_TRUE,Or(isWindows(),requiresVirtualizationType(paravirtual)),ALWAYS_TRUE,is64Bit()),
tags=[]], loginUser=root, userMetadata={}, tags=[]]}] on ports [60030, 60020]
for [93.158.79.70/32]
2011-08-13 12:44:27,899 INFO [org.apache.whirr.actions.ConfigureClusterAction]
(main) Running configuration script
2011-08-13 12:45:01,012 INFO [org.apache.whirr.actions.ConfigureClusterAction]
(main) Configuration script run completed
2011-08-13 12:45:01,017 INFO [org.apache.whirr.actions.ConfigureClusterAction]
(main) Running configuration script
2011-08-13 12:45:40,906 INFO [org.apache.whirr.actions.ConfigureClusterAction]
(main) Configuration script run completed
2011-08-13 12:45:40,906 INFO
[org.apache.whirr.service.hadoop.HadoopNameNodeClusterActionHandler] (main)
Completed configuration of hbaseclustertest
2011-08-13 12:45:40,906 INFO
[org.apache.whirr.service.hadoop.HadoopNameNodeClusterActionHandler] (main)
Namenode web UI available at
http://ec2-50-19-194-241.compute-1.amazonaws.com:50070
2011-08-13 12:45:40,906 INFO
[org.apache.whirr.service.hadoop.HadoopNameNodeClusterActionHandler] (main)
Jobtracker web UI available at
http://ec2-50-19-194-241.compute-1.amazonaws.com:50030
2011-08-13 12:45:40,907 INFO
[org.apache.whirr.service.hadoop.HadoopNameNodeClusterActionHandler] (main)
Wrote Hadoop site file /Users/adriancole/.whirr/hbaseclustertest/hadoop-site.xml
2011-08-13 12:45:40,948 INFO
[org.apache.whirr.service.hadoop.HadoopNameNodeClusterActionHandler] (main)
Wrote Hadoop proxy script
/Users/adriancole/.whirr/hbaseclustertest/hadoop-proxy.sh
2011-08-13 12:45:40,948 INFO
[org.apache.whirr.service.zookeeper.ZooKeeperClusterActionHandler] (main)
Completed configuration of hbaseclustertest
2011-08-13 12:45:40,948 INFO
[org.apache.whirr.service.zookeeper.ZooKeeperClusterActionHandler] (main)
Hosts: ec2-50-19-194-241.compute-1.amazonaws.com:2181
2011-08-13 12:45:40,948 INFO
[org.apache.whirr.service.hbase.HBaseMasterClusterActionHandler] (main)
Completed configuration of hbaseclustertest
2011-08-13 12:45:40,949 INFO
[org.apache.whirr.service.hbase.HBaseMasterClusterActionHandler] (main) Web UI
available at http://ec2-50-19-194-241.compute-1.amazonaws.com
2011-08-13 12:45:40,949 INFO
[org.apache.whirr.service.hbase.HBaseMasterClusterActionHandler] (main) Wrote
HBase site file /Users/adriancole/.whirr/hbaseclustertest/hbase-site.xml
2011-08-13 12:45:40,965 INFO
[org.apache.whirr.service.hbase.HBaseMasterClusterActionHandler] (main) Wrote
HBase proxy script /Users/adriancole/.whirr/hbaseclustertest/hbase-proxy.sh
2011-08-13 12:45:41,465 INFO [org.apache.whirr.service.FileClusterStateStore]
(main) Wrote instances file /Users/adriancole/.whirr/hbaseclustertest/instances
2011-08-13 12:45:41,525 INFO
[org.apache.whirr.service.hbase.integration.HBaseServiceController] (main)
Waiting for master...
2011-08-13 12:45:42,179 INFO
[org.apache.whirr.service.hbase.integration.HBaseServiceController] (main)
Connected to thrift server.
2011-08-13 12:45:42,179 INFO
[org.apache.whirr.service.hbase.integration.HBaseServiceController] (main)
Waiting for .META. table...
Warning: Permanently added
'ec2-50-19-194-241.compute-1.amazonaws.com,50.19.194.241' (RSA) to the list of
known hosts.
2011-08-13 12:45:46,794 INFO
[org.apache.whirr.service.hbase.integration.HBaseServiceController] (main)
Master reported in. Continuing.
2011-08-13 12:45:51,806 INFO
[org.apache.whirr.service.hbase.integration.HBaseServiceController] (main)
Shutting down cluster...
2011-08-13 12:45:51,809 INFO [org.apache.whirr.actions.DestroyClusterAction]
(main) Destroying hbaseclustertest cluster
2011-08-13 12:46:32,775 INFO [org.apache.whirr.actions.DestroyClusterAction]
(main) Cluster hbaseclustertest destroyed
> ComputeCache redundantly creates ComputeServiceContexts
> -------------------------------------------------------
>
> Key: WHIRR-363
> URL: https://issues.apache.org/jira/browse/WHIRR-363
> Project: Whirr
> Issue Type: Bug
> Affects Versions: 0.6.0
> Reporter: Adrian Cole
> Assignee: Adrian Cole
> Fix For: 0.6.0
>
> Attachments: ComputeCache.java
>
> Original Estimate: 1h
> Remaining Estimate: 1h
>
> Previously, ComputeCache used a stringified Key to ensure we don't create
> another ComputeService for the same provider. At some point, Key was
> modified to use both a string and a commons configuration object. Since
> commons configuration doesn't implement hashCode in a predictable way, the
> ComputeCache creates a new ComputeServiceContext each time it is used. Since
> we don't log or test this, it was easy to miss.
> Adding a simple log statement inside ComputeCache shows this:
> 2011-08-13 12:29:22,532 INFO
> [org.apache.whirr.service.hbase.integration.HBaseServiceController] (main)
> Starting up cluster...
> 2011-08-13 12:29:37,760 INFO [org.apache.whirr.service.ComputeCache] (main)
> created new ComputeServiceContext [id=aws-ec2,
> endpoint=https://ec2.us-east-1.amazonaws.com, apiVersion=2010-06-15,
> identity=067PW7Z9P0FNH7JDPE82, iso3166Codes=[US-VA, US-CA, IE, SG, JP-13]]
> 2011-08-13 12:29:46,115 INFO [org.apache.whirr.service.ComputeCache] (main)
> created new ComputeServiceContext [id=aws-ec2,
> endpoint=https://ec2.us-east-1.amazonaws.com, apiVersion=2010-06-15,
> identity=067PW7Z9P0FNH7JDPE82, iso3166Codes=[US-VA, US-CA, IE, SG, JP-13]]
> 2011-08-13 12:29:46,118 INFO
> [org.apache.whirr.actions.BootstrapClusterAction] (main) Bootstrapping cluster
> --snip--
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
