Hi Raj,
I guess this error occurred when I tried to refresh the browser before the
startup. If you check the log, in line number 1702 there is a graceful
application undeployment.
Thank you!
On Wed, Nov 18, 2015 at 12:52 PM, Rajkumar Rajaratnam <rajkum...@wso2.com>
wrote:
> Hi Pubudu,
>
> In your case, the monitors are not started/scheduled after server restart
> because of the following error. There seems to be a communication issue
> between AS and CC, may be CC web service was not ready at that time.
>
>
> TID: [0] [STRATOS] [2015-11-18 12:08:41,675] INFO
> {org.apache.stratos.autoscaler.monitor.component.ParentComponentMonitor} -
> Starting dependent monitor: [application]
> c3c955b2-b459-ebd1-ab6b-2ada34d5fd50 [component]
> c3c955b2-b459-ebd1-ab6b-2ada34d5fd50.alias2.wso2brs-210.domain
> TID: [0] [STRATOS] [2015-11-18 12:08:41,675] INFO
> {org.apache.stratos.autoscaler.monitor.component.ParentComponentMonitor} -
> Monitor scheduled: [type] cluster [component]
> c3c955b2-b459-ebd1-ab6b-2ada34d5fd50.alias2.wso2brs-210.domain
> TID: [0] [STRATOS] [2015-11-18 12:08:41,676] INFO
> {org.apache.stratos.autoscaler.monitor.component.ParentComponentMonitor} -
> Starting monitor: [type] cluster [component]
> c3c955b2-b459-ebd1-ab6b-2ada34d5fd50.alias2.wso2brs-210.domain
> TID: [0] [STRATOS] [2015-11-18 12:08:41,680] INFO
> {org.apache.stratos.autoscaler.util.AutoscalerUtil} - Monitor started
> successfully: [application] c3c955b2-b459-ebd1-ab6b-2ada34d5fd50
> [dependents] c3c955b2-b459-ebd1-ab6b-2ada34d5fd50.alias2.wso2brs-210.domain
> [startup-time] 0 seconds
> TID: [0] [STRATOS] [2015-11-18 12:08:41,830] INFO
> {org.apache.axis2.transport.http.HTTPSender} - Unable to sendViaPost to
> url[https://localhost:9443/services/CloudControllerService/]
> org.apache.axis2.AxisFault: Transport error: 404 Error: Not Found
> at
> org.apache.axis2.transport.http.HTTPSender.handleResponse(HTTPSender.java:329)
> at
> org.apache.axis2.transport.http.HTTPSender.sendViaPost(HTTPSender.java:196)
> at org.apache.axis2.transport.http.HTTPSender.send(HTTPSender.java:77)
> at
> org.apache.axis2.transport.http.CommonsHTTPTransportSender.writeMessageWithCommons(CommonsHTTPTransportSender.java:451)
> at
> org.apache.axis2.transport.http.CommonsHTTPTransportSender.invoke(CommonsHTTPTransportSender.java:278)
> at org.apache.axis2.engine.AxisEngine.send(AxisEngine.java:442)
> at
> org.apache.axis2.description.OutInAxisOperationClient.send(OutInAxisOperation.java:430)
> at
> org.apache.axis2.description.OutInAxisOperationClient.executeImpl(OutInAxisOperation.java:225)
> at
> org.apache.axis2.client.OperationClient.execute(OperationClient.java:149)
> at
> org.apache.stratos.cloud.controller.stub.CloudControllerServiceStub.validateDeploymentPolicyNetworkPartition(CloudControllerServiceStub.java:3070)
> at
> org.apache.stratos.common.client.CloudControllerServiceClient.validateNetworkPartitionOfDeploymentPolicy(CloudControllerServiceClient.java:201)
> at
> org.apache.stratos.autoscaler.monitor.MonitorFactory.getClusterMonitor(MonitorFactory.java:270)
> at
> org.apache.stratos.autoscaler.monitor.MonitorFactory.getMonitor(MonitorFactory.java:83)
> at
> org.apache.stratos.autoscaler.monitor.component.ParentComponentMonitor$MonitorAdder.run(ParentComponentMonitor.java:1001)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
> at java.util.concurrent.FutureTask.run(FutureTask.java:166)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
> at java.lang.Thread.run(Thread.java:722)
> TID: [0] [STRATOS] [2015-11-18 12:08:41,831] ERROR
> {org.apache.stratos.autoscaler.monitor.MonitorFactory} - Error while
> validating deployment policy from cloud controller [network-partition-id]
> network-partition-openstack
> org.apache.axis2.AxisFault: Transport error: 404 Error: Not Found
> at
> org.apache.axis2.transport.http.HTTPSender.handleResponse(HTTPSender.java:329)
> at
> org.apache.axis2.transport.http.HTTPSender.sendViaPost(HTTPSender.java:196)
> at org.apache.axis2.transport.http.HTTPSender.send(HTTPSender.java:77)
> at
> org.apache.axis2.transport.http.CommonsHTTPTransportSender.writeMessageWithCommons(CommonsHTTPTransportSender.java:451)
> at
> org.apache.axis2.transport.http.CommonsHTTPTransportSender.invoke(CommonsHTTPTransportSender.java:278)
> at org.apache.axis2.engine.AxisEngine.send(AxisEngine.java:442)
> at
> org.apache.axis2.description.OutInAxisOperationClient.send(OutInAxisOperation.java:430)
> at
> org.apache.axis2.description.OutInAxisOperationClient.executeImpl(OutInAxisOperation.java:225)
> at
> org.apache.axis2.client.OperationClient.execute(OperationClient.java:149)
> at
> org.apache.stratos.cloud.controller.stub.CloudControllerServiceStub.validateDeploymentPolicyNetworkPartition(CloudControllerServiceStub.java:3070)
> at
> org.apache.stratos.common.client.CloudControllerServiceClient.validateNetworkPartitionOfDeploymentPolicy(CloudControllerServiceClient.java:201)
> at
> org.apache.stratos.autoscaler.monitor.MonitorFactory.getClusterMonitor(MonitorFactory.java:270)
> at
> org.apache.stratos.autoscaler.monitor.MonitorFactory.getMonitor(MonitorFactory.java:83)
> at
> org.apache.stratos.autoscaler.monitor.component.ParentComponentMonitor$MonitorAdder.run(ParentComponentMonitor.java:1001)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
> at java.util.concurrent.FutureTask.run(FutureTask.java:166)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
> at java.lang.Thread.run(Thread.java:722)
> TID: [0] [STRATOS] [2015-11-18 12:08:41,831] ERROR
> {org.apache.stratos.autoscaler.monitor.component.ParentComponentMonitor} -
> An error occurred while starting monitor: [type] cluster [component]
> c3c955b2-b459-ebd1-ab6b-2ada34d5fd50.alias2.wso2brs-210.domain
> java.lang.RuntimeException: Error while validating deployment policy from
> cloud controller [network-partition-id] network-partition-openstack
> at
> org.apache.stratos.autoscaler.monitor.MonitorFactory.getClusterMonitor(MonitorFactory.java:278)
> at
> org.apache.stratos.autoscaler.monitor.MonitorFactory.getMonitor(MonitorFactory.java:83)
> at
> org.apache.stratos.autoscaler.monitor.component.ParentComponentMonitor$MonitorAdder.run(ParentComponentMonitor.java:1001)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
> at java.util.concurrent.FutureTask.run(FutureTask.java:166)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
> at java.lang.Thread.run(Thread.java:722)
> Caused by: org.apache.axis2.AxisFault: Transport error: 404 Error: Not
> Found
> at
> org.apache.axis2.transport.http.HTTPSender.handleResponse(HTTPSender.java:329)
> at
> org.apache.axis2.transport.http.HTTPSender.sendViaPost(HTTPSender.java:196)
> at org.apache.axis2.transport.http.HTTPSender.send(HTTPSender.java:77)
> at
> org.apache.axis2.transport.http.CommonsHTTPTransportSender.writeMessageWithCommons(CommonsHTTPTransportSender.java:451)
> at
> org.apache.axis2.transport.http.CommonsHTTPTransportSender.invoke(CommonsHTTPTransportSender.java:278)
> at org.apache.axis2.engine.AxisEngine.send(AxisEngine.java:442)
> at
> org.apache.axis2.description.OutInAxisOperationClient.send(OutInAxisOperation.java:430)
> at
> org.apache.axis2.description.OutInAxisOperationClient.executeImpl(OutInAxisOperation.java:225)
> at
> org.apache.axis2.client.OperationClient.execute(OperationClient.java:149)
> at
> org.apache.stratos.cloud.controller.stub.CloudControllerServiceStub.validateDeploymentPolicyNetworkPartition(CloudControllerServiceStub.java:3070)
> at
> org.apache.stratos.common.client.CloudControllerServiceClient.validateNetworkPartitionOfDeploymentPolicy(CloudControllerServiceClient.java:201)
> at
> org.apache.stratos.autoscaler.monitor.MonitorFactory.getClusterMonitor(MonitorFactory.java:270)
>
> Thanks,
> Raj.
>
> On Wed, Nov 18, 2015 at 12:27 PM, Pubudu Gunatilaka <pubu...@wso2.com>
> wrote:
>
>> Hi,
>>
>> I did not get the mentioned exception and application came to created
>> state when I undeploy the application. But instance did not get terminated
>> and it is running on openstack. I enabled debug logs in cloud controller
>> and the log is attached.
>>
>> Thank you!
>>
>> On Wed, Nov 18, 2015 at 11:09 AM, Rajkumar Rajaratnam <rajkum...@wso2.com
>> > wrote:
>>
>>> Hi Akila,
>>>
>>> I haven't tested it with ec2/openstack/gce, but the issue you got is not
>>> bound to any specific iaases, I guess.
>>>
>>> According to your stack trace, the cartridge "php-gce" is not found in
>>> CloudControllerContext#cartridgeTypeToCartridgeMap when Stratos tried to
>>> terminate the instance. That means, something gone wrong in server restart,
>>> and the map didn't get populated correctly. But, I have restarted Stratos
>>> and verified that this map is updated correctly.
>>>
>>> If you have complete debug logs after server restart, please attach it
>>> here. Otherwise, lets wait for Pubudu's test results.
>>>
>>> Thanks,
>>> Raj.
>>>
>>> On Wed, Nov 18, 2015 at 10:38 AM, Gayan Gunarathne <gay...@wso2.com>
>>> wrote:
>>>
>>>> Hi Akila,
>>>>
>>>> Seems like cartridge type "php-gce" not available in the cloud
>>>> controller context. To simplify this did you check whether cartridge "
>>>> php-gce" is available in that case?
>>>>
>>>> Thanks,
>>>> Gayan
>>>>
>>>> On Wed, Nov 18, 2015 at 12:35 AM, Akila Ravihansa Perera <
>>>> raviha...@wso2.com> wrote:
>>>>
>>>>> Hi,
>>>>>
>>>>> I faced an issue where if I restart the Stratos server after
>>>>> successfully deploying an app then try to undeploy the app, it will throw
>>>>> the following error;
>>>>>
>>>>> [2015-11-18 00:31:55,537] ERROR
>>>>> {org.apache.stratos.cloud.controller.services.impl.InstanceTerminator} -
>>>>> Instance termination failed! MemberContext
>>>>> [applicationId=single-cartridge-app-gce, cartridgeType=php-gce,
>>>>> clusterId=single-cartridge-app-gce.my-php-app-gce.php-gce.domain,
>>>>> memberId=single-cartridge-app-gce.my-php-app-gce.php-gce.domainbead7a03-8589-4c0c-a421-b713491d51ca,
>>>>> instanceId=null, partition=Partition [id=partition-1, description=null,
>>>>> isPublic=false, provider=gce, properties=Properties [properties=[Property
>>>>> [name=region, value=asia-east1], Property [name=zone,
>>>>> value=asia-east1-a]]]], defaultPrivateIP=null, defaultPublicIP=null,
>>>>> allocatedIPs=null, publicIPs=null, privateIPs=null,
>>>>> initTime=1447786797325,
>>>>> lbClusterId=null, networkPartitionId=network-partition-gce,
>>>>> kubernetesPodId=null, kubernetesPodName=null, loadBalancingIPType=Private,
>>>>> instanceMetadata=null, properties=Properties [properties=[Property
>>>>> [name=MIN_COUNT, value=1], Property [name=SCALING_DECISION_ID,
>>>>> value=single-cartridge-app-gce.my-php-app-gce.php-gce.domain-52271c86-c13e-4db1-8499-8c28c149b7b2],
>>>>> Property [name=EC2_REGION, value=asia-east1], Property
>>>>> [name=EC2_AVAILABILITY_ZONE, value=asia-east1-a]]]]
>>>>> org.apache.stratos.cloud.controller.exception.InvalidCartridgeTypeException:
>>>>> Member termination failed, could not find cartridge in cloud controller
>>>>> context: [cartridge-type] php-gce [member-id]
>>>>> single-cartridge-app-gce.my-php-app-gce.php-gce.domainbead7a03-8589-4c0c-a421-b713491d51ca
>>>>> at
>>>>> org.apache.stratos.cloud.controller.iaases.JcloudsIaas.terminateInstance(JcloudsIaas.java:307)
>>>>> at
>>>>> org.apache.stratos.cloud.controller.services.impl.InstanceTerminator.run(InstanceTerminator.java:56)
>>>>> at
>>>>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>>>>> at
>>>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>>>>> at java.lang.Thread.run(Thread.java:745)
>>>>> Exception in thread "pool-32-thread-3"
>>>>> org.apache.stratos.cloud.controller.exception.CloudControllerException:
>>>>> Instance termination failed! MemberContext
>>>>> [applicationId=single-cartridge-app-gce, cartridgeType=php-gce,
>>>>> clusterId=single-cartridge-app-gce.my-php-app-gce.php-gce.domain,
>>>>> memberId=single-cartridge-app-gce.my-php-app-gce.php-gce.domainbead7a03-8589-4c0c-a421-b713491d51ca,
>>>>> instanceId=null, partition=Partition [id=partition-1, description=null,
>>>>> isPublic=false, provider=gce, properties=Properties [properties=[Property
>>>>> [name=region, value=asia-east1], Property [name=zone,
>>>>> value=asia-east1-a]]]], defaultPrivateIP=null, defaultPublicIP=null,
>>>>> allocatedIPs=null, publicIPs=null, privateIPs=null,
>>>>> initTime=1447786797325,
>>>>> lbClusterId=null, networkPartitionId=network-partition-gce,
>>>>> kubernetesPodId=null, kubernetesPodName=null, loadBalancingIPType=Private,
>>>>> instanceMetadata=null, properties=Properties [properties=[Property
>>>>> [name=MIN_COUNT, value=1], Property [name=SCALING_DECISION_ID,
>>>>> value=single-cartridge-app-gce.my-php-app-gce.php-gce.domain-52271c86-c13e-4db1-8499-8c28c149b7b2],
>>>>> Property [name=EC2_REGION, value=asia-east1], Property
>>>>> [name=EC2_AVAILABILITY_ZONE, value=asia-east1-a]]]]
>>>>> at
>>>>> org.apache.stratos.cloud.controller.services.impl.InstanceTerminator.run(InstanceTerminator.java:63)
>>>>> at
>>>>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>>>>> at
>>>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>>>>> at java.lang.Thread.run(Thread.java:745)
>>>>> Caused by:
>>>>> org.apache.stratos.cloud.controller.exception.InvalidCartridgeTypeException:
>>>>> Member termination failed, could not find cartridge in cloud controller
>>>>> context: [cartridge-type] php-gce [member-id]
>>>>> single-cartridge-app-gce.my-php-app-gce.php-gce.domainbead7a03-8589-4c0c-a421-b713491d51ca
>>>>> at
>>>>> org.apache.stratos.cloud.controller.iaases.JcloudsIaas.terminateInstance(JcloudsIaas.java:307)
>>>>> at
>>>>> org.apache.stratos.cloud.controller.services.impl.InstanceTerminator.run(InstanceTerminator.java:56)
>>>>> ... 3 more
>>>>>
>>>>>
>>>>> Has anyone faced this before?
>>>>>
>>>>> --
>>>>> Akila Ravihansa Perera
>>>>> WSO2 Inc.; http://wso2.com/
>>>>>
>>>>> Blog: http://ravihansa3000.blogspot.com
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>>
>>>> Gayan Gunarathne
>>>> Technical Lead, WSO2 Inc. (http://wso2.com)
>>>> Committer & PMC Member, Apache Stratos
>>>> email : gay...@wso2.com | mobile : +94 775030545 <%2B94%20766819985>
>>>>
>>>>
>>>>
>>>
>>>
>>>
>>> --
>>> Rajkumar Rajaratnam
>>> Committer & PMC Member, Apache Stratos
>>> Software Engineer, WSO2
>>>
>>> Mobile : +94777568639
>>> Blog : rajkumarr.com
>>>
>>
>>
>>
>> --
>> *Pubudu Gunatilaka*
>> Committer and PMC Member - Apache Stratos
>> Software Engineer
>> WSO2, Inc.: http://wso2.com
>> mobile : +94774079049 <%2B94772207163>
>>
>>
>
>
> --
> Rajkumar Rajaratnam
> Committer & PMC Member, Apache Stratos
> Software Engineer, WSO2
>
> Mobile : +94777568639
> Blog : rajkumarr.com
>
--
*Pubudu Gunatilaka*
Committer and PMC Member - Apache Stratos
Software Engineer
WSO2, Inc.: http://wso2.com
mobile : +94774079049 <%2B94772207163>
