Hi Isuru, When I undeploy application for docker scenario, this is what I am getting in the logs. Any idea why I am getting cluster terminating and terminated events?
TID: [0] [STRATOS] [2014-11-10 14:25:37,319] INFO {org.apache.stratos.autoscaler.applications.topic.ApplicationBuilder} - Handling application terminating event: [application-id] test_app_2 TID: [0] [STRATOS] [2014-11-10 14:25:37,319] DEBUG {org.apache.stratos.autoscaler.applications.ApplicationHolder} - Write lock acquired TID: [0] [STRATOS] [2014-11-10 14:25:37,320] INFO {org.apache.stratos.autoscaler.applications.topic.ApplicationBuilder} - Updating application status: [application-id] test_app_2 [status] Terminating TID: [0] [STRATOS] [2014-11-10 14:25:37,320] INFO {org.apache.stratos.autoscaler.monitor.application.ApplicationMonitor} - [ApplicationMonitor] test_app_2 state changes from Terminating to Terminating TID: [0] [STRATOS] [2014-11-10 14:25:37,329] DEBUG {org.apache.stratos.autoscaler.registry.RegistryManager} - Application [ test_app_2 ] persisted successfully in the Autoscaler Registry TID: [0] [STRATOS] [2014-11-10 14:25:37,332] DEBUG {org.apache.stratos.autoscaler.applications.ApplicationHolder} - Topology updated: {"applicationMap":{"test_app_2":{"id":"test_app_2","key":"pmZafljPD7OJqm4j","tenantId":-1234,"tenantDomain":"carbon.super","tenantAdminUserName":"admin","applicationStateManager":{"stateStack":["Created","Terminating"],"identifier":"test_app_2"},"aliasToGroupMap":{},"aliasToClusterDataMap":{"myphp":{"serviceType":"php","clusterId":"myphp.php.domain"}}}},"initialized":false} TID: [0] [STRATOS] [2014-11-10 14:25:37,332] INFO {org.apache.stratos.autoscaler.applications.topic.ApplicationsEventPublisher} - Publishing Application activated event for [application]: test_app_2 TID: [0] [STRATOS] [2014-11-10 14:25:37,338] ERROR {org.apache.stratos.messaging.message.processor.applications.ApplicationActivatedMessageProcessor} - Invalid State transfer from [ Terminating ] to [ Active ] TID: [0] [STRATOS] [2014-11-10 14:25:37,339] INFO {org.apache.stratos.messaging.domain.topology.lifecycle.LifeCycleStateManager} - Topology element [ test_app_2 ]'s life Cycle State changed from [ Terminating ] to [ Active ] TID: [0] [STRATOS] [2014-11-10 14:25:37,339] DEBUG {org.apache.stratos.autoscaler.applications.ApplicationHolder} - Write lock released TID: [0] [STRATOS] [2014-11-10 14:25:37,340] INFO {org.apache.stratos.autoscaler.applications.topic.ApplicationsEventPublisher} - Publishing Application terminated event for [application]: test_app_2 TID: [0] [STRATOS] [2014-11-10 14:25:37,346] INFO {org.apache.stratos.messaging.domain.topology.lifecycle.LifeCycleStateManager} - Topology element [ test_app_2 ]'s life Cycle State changed from [ Active ] to [ Terminating ] TID: [0] [STRATOS] [2014-11-10 14:26:01,001] DEBUG {org.apache.stratos.autoscaler.applications.ApplicationSynchronizeTask} - Applications Complete Event publisher task has been started... TID: [0] [STRATOS] [2014-11-10 14:26:01,002] DEBUG {org.apache.stratos.autoscaler.applications.ApplicationSynchronizeTask} - Executing topology synchronization task TID: [0] [STRATOS] [2014-11-10 14:26:01,002] DEBUG {org.apache.stratos.autoscaler.applications.topic.ApplicationBuilder} - Handling complete application event TID: [0] [STRATOS] [2014-11-10 14:26:01,003] DEBUG {org.apache.stratos.autoscaler.applications.ApplicationHolder} - Read lock acquired TID: [0] [STRATOS] [2014-11-10 14:26:01,003] DEBUG {org.apache.stratos.manager.publisher.TenantSynzhronizerTask} - Publishing complete tenant event TID: [0] [STRATOS] [2014-11-10 14:26:01,004] DEBUG {org.apache.stratos.cloud.controller.topology.TopologySynchronizerTask} - Executing topology synchronization task TID: [0] [STRATOS] [2014-11-10 14:26:01,004] DEBUG {org.apache.stratos.cloud.controller.topology.TopologyEventPublisher} - Publishing complete topology event TID: [0] [STRATOS] [2014-11-10 14:26:01,018] DEBUG {org.apache.stratos.autoscaler.applications.ApplicationHolder} - Read lock released TID: [0] [STRATOS] [2014-11-10 14:26:12,316] DEBUG {org.apache.stratos.autoscaler.monitor.cluster.KubernetesServiceClusterMonitor} - KubernetesServiceClusterMonitor is running...KubernetesServiceClusterMonitor [ kubernetesHostClusterId=KubGrp1, clusterId=myphp.php.domain] As you can see, cluster monitor is running even after application undeployment. Thanks. On Mon, Nov 10, 2014 at 9:23 AM, Rajkumar Rajaratnam <rajkum...@wso2.com> wrote: > Hi all, > > Please note that Nirmal has updated kubernetes binaries to latest version > in master branch. > > Please use vagrant scripts at [1] if you are going to test kubernetes > scenario on master branch. Earlier one might not work. > > 1. > https://github.com/nirmal070125/vagrant-kubernetes-setup/archive/v2.0.zip > <https://www.google.com/url?q=https%3A%2F%2Fgithub.com%2Fnirmal070125%2Fvagrant-kubernetes-setup%2Farchive%2Fv2.0.zip&sa=D&sntz=1&usg=AFQjCNFibNQQI3b0jsVybDXGXYHZQ_u7Cg> > > Thanks. > > On Mon, Nov 10, 2014 at 12:40 AM, Isuru Haththotuwa <isu...@apache.org> > wrote: > >> Tested undeployment and worked for simple and complex apps. There is an >> issue in re-deploying the same app, where it did not get re-activated. Will >> look in to this. >> >> On Sun, Nov 9, 2014 at 4:24 PM, Chamila De Alwis <chami...@wso2.com> >> wrote: >> >>> Hi, >>> >>> Lasindu and I are still trying to get the Python CA working. The puppet >>> module is operational. But at the moment the CA is not getting the events >>> published to the MB. >>> >>> >>> Regards, >>> Chamila de Alwis >>> Software Engineer | WSO2 | +94772207163 >>> Blog: code.chamiladealwis.com >>> >>> >>> >>> On Sun, Nov 9, 2014 at 3:30 PM, Lahiru Sandaruwan <lahi...@wso2.com> >>> wrote: >>> >>>> Shall we merge to the master now? >>>> >>>> On Sun, Nov 9, 2014 at 3:04 PM, Lahiru Sandaruwan <lahi...@wso2.com> >>>> wrote: >>>> >>>>> >>>>> >>>>> On Sun, Nov 9, 2014 at 3:02 PM, Reka Thirunavukkarasu <r...@wso2.com> >>>>> wrote: >>>>> >>>>>> Fixed the serviceId getting NULL by setting it to the ClusterContext >>>>>> and committed it.. >>>>>> >>>>> >>>>> Fine then :) >>>>> >>>>>> >>>>>> On Sun, Nov 9, 2014 at 2:51 PM, Reka Thirunavukkarasu <r...@wso2.com> >>>>>> wrote: >>>>>> >>>>>>> Hi Lahiru, >>>>>>> >>>>>>> As you have introduced ClusteContext in ClusterMonitor, the >>>>>>> serviceId got to removed. Is this done purposely? If so, can we update >>>>>>> the >>>>>>> getServiceId() (which referred by several other places) in monitors to >>>>>>> return the correct serviceId by reading ClusterContext? >>>>>>> >>>>>>> Due to this, ClusterMonitor returns serviceId as null and it causes >>>>>>> issue in the cluster monitor creation and in the event publishing.. >>>>>>> >>>>>>> Thanks, >>>>>>> Reka >>>>>>> >>>>>>> On Sun, Nov 9, 2014 at 2:39 PM, Reka Thirunavukkarasu <r...@wso2.com >>>>>>> > wrote: >>>>>>> >>>>>>>> Hi >>>>>>>> >>>>>>>> On Sun, Nov 9, 2014 at 2:19 PM, Isuru Haththotuwa < >>>>>>>> isu...@apache.org> wrote: >>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> On Sun, Nov 9, 2014 at 1:13 PM, Isuru Haththotuwa < >>>>>>>>> isu...@apache.org> wrote: >>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On Sun, Nov 9, 2014 at 1:09 PM, Reka Thirunavukkarasu < >>>>>>>>>> r...@wso2.com> wrote: >>>>>>>>>> >>>>>>>>>>> Hi all, >>>>>>>>>>> >>>>>>>>>>> After fixing few issues, i could get the application >>>>>>>>>>> undeployment working with one cluster now. So the basic >>>>>>>>>>> undeployment is >>>>>>>>>>> working fine...Will continue testing with a complex application... >>>>>>>>>>> >>>>>>>>>> Great! I will test the same with a complex application. >>>>>>>>>> >>>>>>>>> Getting the following exception with a complex app (when sending >>>>>>>>> cluster activated event), will check and fix asap. >>>>>>>>> Exception in thread "Thread-61" >>>>>>>>> java.lang.IllegalMonitorStateException >>>>>>>>> at >>>>>>>>> java.util.concurrent.locks.ReentrantReadWriteLock$Sync.tryReleaseShared(ReentrantReadWriteLock.java:363) >>>>>>>>> at >>>>>>>>> java.util.concurrent.locks.AbstractQueuedSynchronizer.releaseShared(AbstractQueuedSynchronizer.java:1317) >>>>>>>>> at >>>>>>>>> java.util.concurrent.locks.ReentrantReadWriteLock$ReadLock.unlock(ReentrantReadWriteLock.java:745) >>>>>>>>> at >>>>>>>>> org.apache.stratos.messaging.domain.topology.locking.TopologyLock.releaseReadLock(TopologyLock.java:61) >>>>>>>>> at >>>>>>>>> org.apache.stratos.messaging.message.receiver.topology.TopologyManager.releaseReadLockForCluster(TopologyManager.java:196) >>>>>>>>> at >>>>>>>>> org.apache.stratos.autoscaler.event.publisher.ClusterStatusEventPublisher.sendClusterActivatedEvent(ClusterStatusEventPublisher.java:109) >>>>>>>>> at >>>>>>>>> org.apache.stratos.autoscaler.status.checker.StatusChecker$1.run(StatusChecker.java:77) >>>>>>>>> at java.lang.Thread.run(Thread.java:662) >>>>>>>>> >>>>>>>> >>>>>>>> I too got this issue as well as a NPE while getting locks..Later >>>>>>>> found that serviceId was null when getting the locks..I have fixed >>>>>>>> ServiceId issue..Can you try now and see whether you are still getting >>>>>>>> this >>>>>>>> issue? >>>>>>>> >>>>>>>> Thanks, >>>>>>>> Reka >>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>>> >>>>>>>>>>> Thanks, >>>>>>>>>>> Reka >>>>>>>>>>> >>>>>>>>>>> On Sun, Nov 9, 2014 at 1:05 PM, Isuru Haththotuwa < >>>>>>>>>>> isu...@apache.org> wrote: >>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> On Sun, Nov 9, 2014 at 8:38 AM, Imesh Gunaratne < >>>>>>>>>>>> im...@apache.org> wrote: >>>>>>>>>>>> >>>>>>>>>>>>> No, there seems to be a problem with application un-deploy >>>>>>>>>>>>> function. For me the un-deploy call hangs in the autoscaler. This >>>>>>>>>>>>> is >>>>>>>>>>>>> currerntly under investigation. >>>>>>>>>>>>> >>>>>>>>>>>> Got the same issue, will check in this. >>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> Since the basic workflow of service grouping and docker are >>>>>>>>>>>>> working now shall we merge docker-grouping-merge branch to master >>>>>>>>>>>>> branch >>>>>>>>>>>>> now? WDYT? >>>>>>>>>>>>> >>>>>>>>>>>>> On Sun, Nov 9, 2014 at 4:13 AM, Rajaratnam Rajkumar < >>>>>>>>>>>>> rraju1...@gmail.com> wrote: >>>>>>>>>>>>> >>>>>>>>>>>>>> Hi all, >>>>>>>>>>>>>> Did anyone undeploy the application successfully? Were all >>>>>>>>>>>>>> clusters wiped out? >>>>>>>>>>>>>> >>>>>>>>>>>>>> Thanks >>>>>>>>>>>>>> >>>>>>>>>>>>>> On Saturday, November 8, 2014, Imesh Gunaratne < >>>>>>>>>>>>>> im...@apache.org> wrote: >>>>>>>>>>>>>> >>>>>>>>>>>>>>> On Sat, Nov 8, 2014 at 11:13 PM, Chamila De Alwis < >>>>>>>>>>>>>>> chami...@wso2.com> wrote: >>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Don't we also need the event processing chain since it >>>>>>>>>>>>>>>> constructs the event objects from the Jsons? Additionally this >>>>>>>>>>>>>>>> is done >>>>>>>>>>>>>>>> through the Gson library, so we would have some difficulty in >>>>>>>>>>>>>>>> getting that >>>>>>>>>>>>>>>> functionality in to python as well. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> A good point. May be we need to think little bit here, shall >>>>>>>>>>>>>>> we first try to convert the required classes and see how good >>>>>>>>>>>>>>> they are? >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Thanks >>>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> -- >>>>>>>>>>>>> Imesh Gunaratne >>>>>>>>>>>>> >>>>>>>>>>>>> Technical Lead, WSO2 >>>>>>>>>>>>> Committer & PMC Member, Apache Stratos >>>>>>>>>>>>> >>>>>>>>>>>>> -- >>>>>>>>>>>>> Thanks and Regards, >>>>>>>>>>>>> >>>>>>>>>>>>> Isuru H. >>>>>>>>>>>>> +94 716 358 048* <http://wso2.com/>* >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> * <http://wso2.com/>* >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> -- >>>>>>>>>>> Reka Thirunavukkarasu >>>>>>>>>>> Senior Software Engineer, >>>>>>>>>>> WSO2, Inc.:http://wso2.com, >>>>>>>>>>> Mobile: +94776442007 >>>>>>>>>>> >>>>>>>>>>> -- >>>>>>>>>>> <%2B94776442007> >>>>>>>>>>> <%2B94776442007> >>>>>>>>>>> Thanks and Regards, >>>>>>>>>>> >>>>>>>>>>> Isuru H. >>>>>>>>>>> <%2B94776442007> >>>>>>>>>>> +94 716 358 048 >>>>>>>>>>> >>>>>>>>>>> -- >>>>>>>>>>> <%2B94776442007> >>>>>>>>>>> <%2B94776442007> >>>>>>>>>>> Thanks and Regards, >>>>>>>>>>> >>>>>>>>>>> Isuru H. >>>>>>>>>>> <%2B94776442007> >>>>>>>>>>> +94 716 358 048 <%2B94776442007>* <http://wso2.com/>* >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> * <http://wso2.com/>* >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> -- >>>>>>>> Reka Thirunavukkarasu >>>>>>>> Senior Software Engineer, >>>>>>>> WSO2, Inc.:http://wso2.com, >>>>>>>> Mobile: +94776442007 >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> Reka Thirunavukkarasu >>>>>>> Senior Software Engineer, >>>>>>> WSO2, Inc.:http://wso2.com, >>>>>>> Mobile: +94776442007 >>>>>>> >>>>>>> >>>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> Reka Thirunavukkarasu >>>>>> Senior Software Engineer, >>>>>> WSO2, Inc.:http://wso2.com, >>>>>> Mobile: +94776442007 >>>>>> >>>>>> >>>>>> >>>>> >>>>> >>>>> -- >>>>> -- >>>>> Lahiru Sandaruwan >>>>> Committer and PMC member, Apache Stratos, >>>>> Senior Software Engineer, >>>>> WSO2 Inc., http://wso2.com >>>>> lean.enterprise.middleware >>>>> >>>>> email: lahi...@wso2.com blog: http://lahiruwrites.blogspot.com/ >>>>> linked-in: http://lk.linkedin.com/pub/lahiru-sandaruwan/16/153/146 >>>>> >>>>> >>>> >>>> >>>> -- >>>> -- >>>> Lahiru Sandaruwan >>>> Committer and PMC member, Apache Stratos, >>>> Senior Software Engineer, >>>> WSO2 Inc., http://wso2.com >>>> lean.enterprise.middleware >>>> >>>> email: lahi...@wso2.com blog: http://lahiruwrites.blogspot.com/ >>>> linked-in: http://lk.linkedin.com/pub/lahiru-sandaruwan/16/153/146 >>>> >>>> -- >>>> <http://lk.linkedin.com/pub/lahiru-sandaruwan/16/153/146> >>>> <http://lk.linkedin.com/pub/lahiru-sandaruwan/16/153/146> >>>> Thanks and Regards, >>>> >>>> Isuru H. >>>> <http://lk.linkedin.com/pub/lahiru-sandaruwan/16/153/146> >>>> +94 716 358 048 >>>> <http://lk.linkedin.com/pub/lahiru-sandaruwan/16/153/146>* >>>> <http://wso2.com/>* >>>> >>>> >>>> * <http://wso2.com/>* >>>> >>>> >>>> > > > -- > Raj > -- Raj