And make sure that the ports are open. Instance activated event will be
sent by the agent only if ports are open.
Thanks.
On Mon, Dec 22, 2014 at 1:15 PM, Udara Liyanage <ud...@wso2.com> wrote:
> Hi Raj,
>
> Logs are missed since instance is killed as an obsolete member. I will
> provide the logs next time.
>
> On Mon, Dec 22, 2014 at 1:13 PM, Rajkumar Rajaratnam <rajkum...@wso2.com>
> wrote:
>>
>> Hi Udara,
>>
>> Can you attach the agent log?
>>
>> Thanks.
>>
>> On Mon, Dec 22, 2014 at 1:04 PM, Udara Liyanage <ud...@wso2.com> wrote:
>>
>>> Hi raj,
>>>
>>> I tested with lates python code base. This time member is in topology
>>> and instance spawned event is also received. But yet instance activated is
>>> not sent.
>>>
>>> *1) Instance started event received*
>>>
>>> 2014-12-22 12:53:49,904:DEBUG:Property added: [key] STRATOS_
>>> MEMBER_PROPERTY_PRIMARY[value] false
>>> 2014-12-22 12:53:49,911:DEBUG:Message received:
>>> 'instance/status/InstanceStartedEvent':
>>> '{\n "clusterId": "mytomcat.tomcat.domain", \n
>>> "clusterInstanceId": "app_cartridge_v1-1", \n "instanceId": "", \n
>>> "memberId": "mytomcat.tomcat.domain5dd544b1-d2d6-4ace-8e31-ecb07e116130",
>>> \n "networkPartitionId": "p1", \n "partitionId": "P1", \n
>>> "serviceName": "tomcat"\n}'
>>>
>>> 2) Member in topology
>>>
>>> *Complete Topology:*
>>>
>>>
>>> '{"topology":{"serviceMap":{"tomcat":{"serviceName":"tomcat","serviceType":"SingleTenant","clusterIdClusterMap":{"mytomcat.tomcat.domain":{"serviceName":"tomcat","clusterId":"mytomcat.tomcat.domain","autoscalePolicyName":"autoscale_policy_1","hostNames":["
>>> mytomcat.isuruh.lk
>>> "],"tenantRange":"*","isLbCluster":false,"isKubernetesCluster":false,"memberMap":{"mytomcat.tomcat.domain5dd544b1-d2d6-4ace-8e31-ecb07e116130":{"serviceName":"tomcat","clusterId":"mytomcat.tomcat.domain","memberId":"
>>> *mytomcat.tomcat.domain5dd544b1-d2d6-4ace-8e31-ecb07e116130*
>>> ","instanceId":"ap-southeast-1/i-0147e4cc","clusterInstanceId":"app_cartridge_v1-1","networkPartitionId":"p1","partitionId":"P1","initTime":1419232682397,"portMap":{"443":{"protocol":"https","value":443,"proxy":443},"80":{"protocol":"http","value":80,"proxy":80}},"memberPublicIPs":["54.254.118.165"],"defaultPublicIP":"54.254.118.165","memberPrivateIPs":["10.144.150.199"],"defaultPrivateIP":"10.144.150.199","properties":{"PRIMARY":"false","MIN_COUNT":"1"},"memberStateManager":{"stateStack":["Created","Starting"],"identifier":"mytomcat.tomcat.domain5dd544b1-d2d6-4ace-8e31-ecb07e116130"}}},"appId":"app_cartridge_v1","properties":{},"instanceIdToInstanceContextMap":{"app_cartridge_v1-1":{"instanceId":"app_cartridge_v1-1","instanceProperties":{},"lifeCycleStateManager":{"stateStack":["Created"],"identifier":"mytomcat.tomcat.domain_app_cartridge_v1-1"},"networkPartitionId":"p1"}}}},"portMap":{"443":{"protocol":"https","value":443,"proxy":443},"80":{"protocol":"http","value":80,"proxy":80}},"properties":{}}},"initialized":false}}'
>>>
>>> *Payload:*
>>> APP_ID=app_cartridge_v1,GROUP_NAME=null,SERVICE_NAME=tomcat,HOST_NAME=
>>> mytomcat.isuruh.lk
>>> ,MULTITENANT=false,TENANT_ID=-1234,TENANT_RANGE=*,CARTRIDGE_ALIAS=mytomcat,CLUSTER_ID=mytomcat.tomcat.domain,CARTRIDGE_KEY=kYcZ4B4DPAeA1RGz,REPO_URL=
>>> https://github.com/udaraliyanage/php,PORTS=80|443,PROVIDER=apache,PUPPET_IP=10.148.159.109,PUPPET_HOSTNAME=ip-10-148-159-109.ap-southeast-1.compute.internal,PUPPET_DNS_AVAILABLE=null,PUPPET_ENV=null,,,DEPLOYMENT=default,TOKEN=eyJhbGciOiJSUzI1NiJ9.eyJleHAiOjE4OTY4OTAwNDgsInN1YiI6ImFkbWluIiwiYXpwIjoiMjU1UUZET1UwbFlONmY5NE44aUVhQm5LdlBrYSIsImFwcElkIjoiYXBwX2NhcnRyaWRnZV92MSIsImF1ZCI6WyIyNTVRRkRPVTBsWU42Zjk0TjhpRWFCbkt2UGthIl0sImlzcyI6Imh0dHBzOlwvXC9sb2NhbGhvc3Q6OTQ0M1wvb2F1dGgyZW5kcG9pbnRzXC90b2tlbiIsImlhdCI6MTg5MzI5MDA0OH0.XB4wg8gQxzyH0jv5EIiWFjOG0VDZuFBswYG0G6zX3sjOEgXauywoga4pHmb9DWkKZU9nD5qRFLRyyiHi2onWr-TGtLNBn4qgRq94VDwjVgx5in4ND3f7q-xNSpLipMr_g_gtkbp9st-gZVeuB8y3u7jnjE8fuDEs63pQ75Drrhs,MEMBER_ID=
>>> *mytomcat.tomcat.domain5dd544b1-d2d6-4ace-8e31-ecb07e116130*
>>> ,INSTANCE_ID=null,CLUSTER_INSTANCE_ID=app_cartridge_v1-1,LB_CLUSTER_ID=null,NETWORK_PARTITION_ID=p1,PARTITION_ID=P1,PRIMARY=false,MIN_COUNT=1
>>>
>>> Carbon logs:
>>>
>>> [2014-12-22 07:18:41,444] INFO
>>> {org.apache.stratos.cloud.controller.messaging.publisher.TopologyEventPublisher}
>>> - Publishing instance spawned event: [service-name] tomcat [cluster-id]
>>> mytomcat.tomcat.domain [cluster-instance-id] app_cartridge_v1-1 [member-id]
>>> mytomcat.tomcat.domain5dd544b1-d2d6-4ace-8e31-ecb07e116130 [instance-id]
>>> ap-southeast-1/i-0147e4cc [network-partition-id] p1 [partition-id] P1
>>> [lb-cluster-id] null
>>> [2014-12-22 07:18:41,464] INFO
>>> {org.apache.stratos.messaging.domain.topology.lifecycle.LifeCycleStateManager}
>>> - Life Cycle State Manager started for Element [
>>> mytomcat.tomcat.domain5dd544b1-d2d6-4ace-8e31-ecb07e116130 ], initial
>>> state: Created
>>> [2014-12-22 07:18:41,469] INFO
>>> {org.apache.stratos.messaging.message.processor.topology.InstanceSpawnedMessageProcessor}
>>> - Member created: [service] tomcat [cluster] mytomcat.tomcat.domain
>>> [member] mytomcat.tomcat.domain5dd544b1-d2d6-4ace-8e31-ecb07e116130
>>> [instance] ap-southeast-1/i-0147e4cc [hierarchy-instance] app_cartridge_v1-1
>>> [2014-12-22 07:18:41,480] INFO
>>> {org.apache.stratos.manager.topology.receiver.StratosManagerTopologyEventReceiver}
>>> - [InstanceSpawnedEventListener] Received: class
>>> org.apache.stratos.messaging.event.topology.InstanceSpawnedEvent
>>> [2014-12-22 07:24:01,218] INFO
>>> {org.apache.stratos.manager.listener.InstanceStatusListener} - Instance
>>> status message received
>>> [2014-12-22 07:24:01,218] INFO
>>> {org.apache.stratos.manager.listener.InstanceStatusListener} - Event class
>>> name:
>>> org.apache.stratos.messaging.event.instance.status.InstanceStartedEvent
>>> [2014-12-22 07:24:01,219] INFO
>>> {org.apache.stratos.manager.listener.InstanceStatusListener} - Cluster id:
>>> mytomcat.tomcat.domain
>>> [2014-12-22 07:24:01,220] INFO
>>> {org.apache.stratos.cloud.controller.messaging.topology.TopologyBuilder} -
>>> member started event adding status started
>>> [2014-12-22 07:24:01,235] INFO
>>> {org.apache.stratos.cloud.controller.messaging.publisher.TopologyEventPublisher}
>>> - Publishing member started event: [service-name] tomcat [cluster-id]
>>> mytomcat.tomcat.domain [cluster-instance-id] app_cartridge_v1-1 [member-id]
>>> mytomcat.tomcat.domain5dd544b1-d2d6-4ace-8e31-ecb07e116130 [instance-id]
>>> [network-partition-id] p1 [partition-id] P1
>>> [2014-12-22 07:24:01,250] INFO
>>> {org.apache.stratos.messaging.message.processor.topology.MemberStartedMessageProcessor}
>>> - Member started: [service] tomcat [cluster] mytomcat.tomcat.domain
>>> [member] mytomcat.tomcat.domain5dd544b1-d2d6-4ace-8e31-ecb07e116130
>>> [2014-12-22 07:24:01,250] INFO
>>> {org.apache.stratos.manager.topology.receiver.StratosManagerTopologyEventReceiver}
>>> - [MemberStartedEventListener] Received: class
>>> org.apache.stratos.messaging.event.topology.MemberStartedEvent
>>>
>>>
>>>
>>> On Mon, Dec 22, 2014 at 12:26 PM, Udara Liyanage <ud...@wso2.com> wrote:
>>>>
>>>> Hi Raj,
>>>>
>>>> Thanks for the info. I will check with updated agent.
>>>>
>>>> On Mon, Dec 22, 2014 at 12:24 PM, Rajkumar Rajaratnam <
>>>> rajkum...@wso2.com> wrote:
>>>>>
>>>>> Hi Udara,
>>>>>
>>>>> Are you using the latest python agent?
>>>>>
>>>>> Chamila changed the agent yesterday. Member object doesn't have an
>>>>> attribute called "memberIp anymore. It has been changed to
>>>>> defaultPrivateIP.
>>>>>
>>>>> Latest python agent has this changes.
>>>>>
>>>>> Thanks.
>>>>>
>>>>> On Mon, Dec 22, 2014 at 12:22 PM, Rajkumar Rajaratnam <
>>>>> rajkum...@wso2.com> wrote:
>>>>>
>>>>>> Just noticed that there is an error in parsing complete topology
>>>>>> event. So initialization didn't happen as Isuru mentioned.
>>>>>>
>>>>>> Thanks.
>>>>>>
>>>>>> On Mon, Dec 22, 2014 at 12:19 PM, Rajkumar Rajaratnam <
>>>>>> rajkum...@wso2.com> wrote:
>>>>>>
>>>>>>> Hi,
>>>>>>>
>>>>>>> AFAIR, agent will wait for two conditions to happen. It will proceed
>>>>>>> if either of two conditions becomes true.
>>>>>>>
>>>>>>> 1. Member should be in the topology
>>>>>>> 2. Agent should get instance spawned event from CC
>>>>>>>
>>>>>>> So either of them didn't happen for you.
>>>>>>>
>>>>>>> Thanks.
>>>>>>>
>>>>>>> On Mon, Dec 22, 2014 at 12:17 PM, Isuru Haththotuwa <
>>>>>>> isu...@apache.org> wrote:
>>>>>>>
>>>>>>>> HI Udara,
>>>>>>>>
>>>>>>>> There is an error log about parsing the complete Topology event.
>>>>>>>> Maybe the event did not get parsed correctly, and the required
>>>>>>>> initialization did not happen.
>>>>>>>>
>>>>>>>> On Mon, Dec 22, 2014 at 12:12 PM, Udara Liyanage <ud...@wso2.com>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> Hi Chamila,
>>>>>>>>>
>>>>>>>>> I get the ":Waiting for Cartridge Agent to be initialized..." logs
>>>>>>>>> continuously even though complete topology and complete tenant event
>>>>>>>>> is
>>>>>>>>> already received.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> 2014-12-22 12:04:47,595:DEBUG:Waiting for Cartridge Agent to be
>>>>>>>>> initialized...
>>>>>>>>> 2014-12-22 12:04:48,596:DEBUG:Waiting for Cartridge Agent to be
>>>>>>>>> initialized...
>>>>>>>>> 2014-12-22 12:04:49,597:DEBUG:Waiting for Cartridge Agent to be
>>>>>>>>> initialized...
>>>>>>>>> 2014-12-22 12:04:49,781:DEBUG:Message received:
>>>>>>>>> 'topology/CompleteTopologyEvent':
>>>>>>>>>
>>>>>>>>> '{"topology":{"serviceMap":{"tomcat":{"serviceName":"tomcat","serviceType":"SingleTenant","clusterIdClusterMap":{"mytomcat.tomcat.domain":{"serviceName":"tomcat","clusterId":"mytomcat.tomcat.domain","autoscalePolicyName":"autoscale_policy_1","hostNames":["
>>>>>>>>> mytomcat.isuruh.lk
>>>>>>>>> "],"tenantRange":"*","isLbCluster":false,"isKubernetesCluster":false,"memberMap":{"mytomcat.tomcat.domainf1ee1c36-db95-452d-a1e7-1daf936e9101":{"serviceName":"tomcat","clusterId":"mytomcat.tomcat.domain","memberId":"mytomcat.tomcat.domainf1ee1c36-db95-452d-a1e7-1daf936e9101","instanceId":"ap-southeast-1/i-df7dde12","clusterInstanceId":"app_cartridge_v1-1","networkPartitionId":"p1","partitionId":"P1","initTime":1419229605140,"portMap":{"443":{"protocol":"https","value":443,"proxy":443},"80":{"protocol":"http","value":80,"proxy":80}},"memberPublicIPs":["54.151.190.32"],"defaultPublicIP":"54.151.190.32","memberPrivateIPs":["10.144.68.9"],"defaultPrivateIP":"10.144.68.9","properties":{"PRIMARY":"false","MIN_COUNT":"1"},"memberStateManager":{"stateStack":["Created"],"identifier":"mytomcat.tomcat.domainf1ee1c36-db95-452d-a1e7-1daf936e9101"}}},"appId":"app_cartridge_v1","properties":{},"instanceIdToInstanceContextMap":{"app_cartridge_v1-1":{"instanceId":"app_cartridge_v1-1","instanceProperties":{},"lifeCycleStateManager":{"stateStack":["Created"],"identifier":"mytomcat.tomcat.domain_app_cartridge_v1-1"},"networkPartitionId":"p1"}}}},"portMap":{"443":{"protocol":"https","value":443,"proxy":443},"80":{"protocol":"http","value":80,"proxy":80}},"properties":{}}},"initialized":false}}'
>>>>>>>>> 2014-12-22 12:04:49,781:DEBUG:Executing handler for event
>>>>>>>>> 'CompleteTopologyEvent'
>>>>>>>>> 2014-12-22 12:04:49,781:DEBUG:Complete topology event received
>>>>>>>>> 2014-12-22 12:04:49,782:ERROR:Error processing
>>>>>>>>> 'CompleteTopologyEvent' event
>>>>>>>>> Traceback (most recent call last):
>>>>>>>>> File
>>>>>>>>> "/mnt/apache-stratos-python-cartridge-agent-4.1.0-SNAPSHOT/modules/subscriber/eventsubscriber.py",
>>>>>>>>> line 81, in on_message
>>>>>>>>> handler(msg)
>>>>>>>>> File "agent.py", line 255, in on_complete_topology
>>>>>>>>> event_obj = CompleteTopologyEvent.create_from_json(msg.payload)
>>>>>>>>> File
>>>>>>>>> "/mnt/apache-stratos-python-cartridge-agent-4.1.0-SNAPSHOT/modules/event/topology/events.py",
>>>>>>>>> line 190, in create_from_json
>>>>>>>>> member_obj.member_ip = member_str["memberIp"]
>>>>>>>>> KeyError: 'memberIp'
>>>>>>>>> 2014-12-22 12:04:49,820:DEBUG:Message received:
>>>>>>>>> 'tenant/CompleteTenantEvent':
>>>>>>>>> '{"tenants":[]}'
>>>>>>>>> 2014-12-22 12:04:49,820:DEBUG:Executing handler for event
>>>>>>>>> 'CompleteTenantEvent'
>>>>>>>>> 2014-12-22 12:04:49,820:INFO:Complete tenant event updating task
>>>>>>>>> disabled
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> *Carbon logs:*
>>>>>>>>>
>>>>>>>>> TID: [0] [STRATOS] [2014-12-22 06:27:21,240] INFO
>>>>>>>>> {org.apache.stratos.cloud.controller.messaging.publisher.TopologyEventPublisher}
>>>>>>>>> - Publishing instance spawned event: [service-name] tomcat
>>>>>>>>> [cluster-id]
>>>>>>>>> mytomcat.tomcat.domain [cluster-instance-id] app_cartridge_v1-1
>>>>>>>>> [member-id]
>>>>>>>>> mytomcat.tomcat.domainf1ee1c36-db95-452d-a1e7-1daf936e9101
>>>>>>>>> [instance-id]
>>>>>>>>> ap-southeast-1/i-df7dde12 [network-partition-id] p1 [partition-id] P1
>>>>>>>>> [lb-cluster-id] null
>>>>>>>>> TID: [0] [STRATOS] [2014-12-22 06:27:21,261] INFO
>>>>>>>>> {org.apache.stratos.messaging.domain.topology.lifecycle.LifeCycleStateManager}
>>>>>>>>> - Life Cycle State Manager started for Element [
>>>>>>>>> mytomcat.tomcat.domainf1ee1c36-db95-452d-a1e7-1daf936e9101 ], initial
>>>>>>>>> state: Created
>>>>>>>>> TID: [0] [STRATOS] [2014-12-22 06:27:21,264] INFO
>>>>>>>>> {org.apache.stratos.messaging.message.processor.topology.InstanceSpawnedMessageProcessor}
>>>>>>>>> - Member created: [service] tomcat [cluster] mytomcat.tomcat.domain
>>>>>>>>> [member] mytomcat.tomcat.domainf1ee1c36-db95-452d-a1e7-1daf936e9101
>>>>>>>>> [instance] ap-southeast-1/i-df7dde12 [hierarchy-instance]
>>>>>>>>> app_cartridge_v1-1
>>>>>>>>> TID: [0] [STRATOS] [2014-12-22 06:27:21,268] INFO
>>>>>>>>> {org.apache.stratos.manager.topology.receiver.StratosManagerTopologyEventReceiver}
>>>>>>>>> - [InstanceSpawnedEventListener] Received: class
>>>>>>>>> org.apache.stratos.messaging.event.topology.InstanceSpawnedEvent
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>>
>>>>>>>>> Udara Liyanage
>>>>>>>>> Software Engineer
>>>>>>>>> WSO2, Inc.: http://wso2.com
>>>>>>>>> lean. enterprise. middleware
>>>>>>>>>
>>>>>>>>> web: http://udaraliyanage.wordpress.com
>>>>>>>>> phone:
>>>>>>>>> +94 71 443 6897
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> Thanks and Regards,
>>>>>>>>>
>>>>>>>>> Isuru H.
>>>>>>>>> +94 716 358 048* <http://wso2.com/>*
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> * <http://wso2.com/>*
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> Rajkumar Rajaratnam
>>>>>>> Committer & PMC Member, Apache Stratos
>>>>>>> Software Engineer, WSO2
>>>>>>>
>>>>>>> Mobile : +94777568639
>>>>>>> Blog : rajkumarr.com
>>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Rajkumar Rajaratnam
>>>>>> Committer & PMC Member, Apache Stratos
>>>>>> Software Engineer, WSO2
>>>>>>
>>>>>> Mobile : +94777568639
>>>>>> Blog : rajkumarr.com
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Rajkumar Rajaratnam
>>>>> Committer & PMC Member, Apache Stratos
>>>>> Software Engineer, WSO2
>>>>>
>>>>> Mobile : +94777568639
>>>>> Blog : rajkumarr.com
>>>>>
>>>>
>>>>
>>>> --
>>>>
>>>> Udara Liyanage
>>>> Software Engineer
>>>> WSO2, Inc.: http://wso2.com
>>>> lean. enterprise. middleware
>>>>
>>>> web: http://udaraliyanage.wordpress.com
>>>> phone: +94 71 443 6897
>>>>
>>>
>>>
>>> --
>>>
>>> Udara Liyanage
>>> Software Engineer
>>> WSO2, Inc.: http://wso2.com
>>> lean. enterprise. middleware
>>>
>>> web: http://udaraliyanage.wordpress.com
>>> phone: +94 71 443 6897
>>>
>>
>>
>>
>> --
>> Rajkumar Rajaratnam
>> Committer & PMC Member, Apache Stratos
>> Software Engineer, WSO2
>>
>> Mobile : +94777568639
>> Blog : rajkumarr.com
>>
>
>
> --
>
> Udara Liyanage
> Software Engineer
> WSO2, Inc.: http://wso2.com
> lean. enterprise. middleware
>
> web: http://udaraliyanage.wordpress.com
> phone: +94 71 443 6897
>
--
Rajkumar Rajaratnam
Committer & PMC Member, Apache Stratos
Software Engineer, WSO2
Mobile : +94777568639
Blog : rajkumarr.com
