Hi, There is an issue with undeployment where the clusters do not get properly removed from Topology. I'm working on a fix now.
On Mon, Nov 10, 2014 at 2:31 PM, Rajkumar Rajaratnam <rajkum...@wso2.com> wrote: > Hi Isuru, > > When I undeploy application for docker scenario, this is what I am getting > in the logs. Any idea why I am getting cluster terminating and terminated > events? > > TID: [0] [STRATOS] [2014-11-10 14:25:37,319] INFO > {org.apache.stratos.autoscaler.applications.topic.ApplicationBuilder} - > Handling application terminating event: [application-id] test_app_2 > TID: [0] [STRATOS] [2014-11-10 14:25:37,319] DEBUG > {org.apache.stratos.autoscaler.applications.ApplicationHolder} - Write > lock acquired > TID: [0] [STRATOS] [2014-11-10 14:25:37,320] INFO > {org.apache.stratos.autoscaler.applications.topic.ApplicationBuilder} - > Updating application status: [application-id] test_app_2 [status] > Terminating > TID: [0] [STRATOS] [2014-11-10 14:25:37,320] INFO > {org.apache.stratos.autoscaler.monitor.application.ApplicationMonitor} - > [ApplicationMonitor] test_app_2 state changes from Terminating to > Terminating > TID: [0] [STRATOS] [2014-11-10 14:25:37,329] DEBUG > {org.apache.stratos.autoscaler.registry.RegistryManager} - Application [ > test_app_2 ] persisted successfully in the Autoscaler Registry > TID: [0] [STRATOS] [2014-11-10 14:25:37,332] DEBUG > {org.apache.stratos.autoscaler.applications.ApplicationHolder} - Topology > updated: > {"applicationMap":{"test_app_2":{"id":"test_app_2","key":"pmZafljPD7OJqm4j","tenantId":-1234,"tenantDomain":"carbon.super","tenantAdminUserName":"admin","applicationStateManager":{"stateStack":["Created","Terminating"],"identifier":"test_app_2"},"aliasToGroupMap":{},"aliasToClusterDataMap":{"myphp":{"serviceType":"php","clusterId":"myphp.php.domain"}}}},"initialized":false} > TID: [0] [STRATOS] [2014-11-10 14:25:37,332] INFO > {org.apache.stratos.autoscaler.applications.topic.ApplicationsEventPublisher} > - Publishing Application activated event for [application]: test_app_2 > TID: [0] [STRATOS] [2014-11-10 14:25:37,338] ERROR > {org.apache.stratos.messaging.message.processor.applications.ApplicationActivatedMessageProcessor} > - Invalid State transfer from [ Terminating ] to [ Active ] > TID: [0] [STRATOS] [2014-11-10 14:25:37,339] INFO > {org.apache.stratos.messaging.domain.topology.lifecycle.LifeCycleStateManager} > - Topology element [ test_app_2 ]'s life Cycle State changed from [ > Terminating ] to [ Active ] > TID: [0] [STRATOS] [2014-11-10 14:25:37,339] DEBUG > {org.apache.stratos.autoscaler.applications.ApplicationHolder} - Write > lock released > TID: [0] [STRATOS] [2014-11-10 14:25:37,340] INFO > {org.apache.stratos.autoscaler.applications.topic.ApplicationsEventPublisher} > - Publishing Application terminated event for [application]: test_app_2 > TID: [0] [STRATOS] [2014-11-10 14:25:37,346] INFO > {org.apache.stratos.messaging.domain.topology.lifecycle.LifeCycleStateManager} > - Topology element [ test_app_2 ]'s life Cycle State changed from [ Active > ] to [ Terminating ] > TID: [0] [STRATOS] [2014-11-10 14:26:01,001] DEBUG > {org.apache.stratos.autoscaler.applications.ApplicationSynchronizeTask} - > Applications Complete Event publisher task has been started... > TID: [0] [STRATOS] [2014-11-10 14:26:01,002] DEBUG > {org.apache.stratos.autoscaler.applications.ApplicationSynchronizeTask} - > Executing topology synchronization task > TID: [0] [STRATOS] [2014-11-10 14:26:01,002] DEBUG > {org.apache.stratos.autoscaler.applications.topic.ApplicationBuilder} - > Handling complete application event > TID: [0] [STRATOS] [2014-11-10 14:26:01,003] DEBUG > {org.apache.stratos.autoscaler.applications.ApplicationHolder} - Read lock > acquired > TID: [0] [STRATOS] [2014-11-10 14:26:01,003] DEBUG > {org.apache.stratos.manager.publisher.TenantSynzhronizerTask} - Publishing > complete tenant event > TID: [0] [STRATOS] [2014-11-10 14:26:01,004] DEBUG > {org.apache.stratos.cloud.controller.topology.TopologySynchronizerTask} - > Executing topology synchronization task > TID: [0] [STRATOS] [2014-11-10 14:26:01,004] DEBUG > {org.apache.stratos.cloud.controller.topology.TopologyEventPublisher} - > Publishing complete topology event > TID: [0] [STRATOS] [2014-11-10 14:26:01,018] DEBUG > {org.apache.stratos.autoscaler.applications.ApplicationHolder} - Read lock > released > TID: [0] [STRATOS] [2014-11-10 14:26:12,316] DEBUG > {org.apache.stratos.autoscaler.monitor.cluster.KubernetesServiceClusterMonitor} > - KubernetesServiceClusterMonitor is > running...KubernetesServiceClusterMonitor [ > kubernetesHostClusterId=KubGrp1, clusterId=myphp.php.domain] > > As you can see, cluster monitor is running even after application > undeployment. > > Thanks. > > On Mon, Nov 10, 2014 at 9:23 AM, Rajkumar Rajaratnam <rajkum...@wso2.com> > wrote: > >> Hi all, >> >> Please note that Nirmal has updated kubernetes binaries to latest version >> in master branch. >> >> Please use vagrant scripts at [1] if you are going to test kubernetes >> scenario on master branch. Earlier one might not work. >> >> 1. >> https://github.com/nirmal070125/vagrant-kubernetes-setup/archive/v2.0.zip >> <https://www.google.com/url?q=https%3A%2F%2Fgithub.com%2Fnirmal070125%2Fvagrant-kubernetes-setup%2Farchive%2Fv2.0.zip&sa=D&sntz=1&usg=AFQjCNFibNQQI3b0jsVybDXGXYHZQ_u7Cg> >> >> Thanks. >> >> On Mon, Nov 10, 2014 at 12:40 AM, Isuru Haththotuwa <isu...@apache.org> >> wrote: >> >>> Tested undeployment and worked for simple and complex apps. There is an >>> issue in re-deploying the same app, where it did not get re-activated. Will >>> look in to this. >>> >>> On Sun, Nov 9, 2014 at 4:24 PM, Chamila De Alwis <chami...@wso2.com> >>> wrote: >>> >>>> Hi, >>>> >>>> Lasindu and I are still trying to get the Python CA working. The puppet >>>> module is operational. But at the moment the CA is not getting the events >>>> published to the MB. >>>> >>>> >>>> Regards, >>>> Chamila de Alwis >>>> Software Engineer | WSO2 | +94772207163 >>>> Blog: code.chamiladealwis.com >>>> >>>> >>>> >>>> On Sun, Nov 9, 2014 at 3:30 PM, Lahiru Sandaruwan <lahi...@wso2.com> >>>> wrote: >>>> >>>>> Shall we merge to the master now? >>>>> >>>>> On Sun, Nov 9, 2014 at 3:04 PM, Lahiru Sandaruwan <lahi...@wso2.com> >>>>> wrote: >>>>> >>>>>> >>>>>> >>>>>> On Sun, Nov 9, 2014 at 3:02 PM, Reka Thirunavukkarasu <r...@wso2.com> >>>>>> wrote: >>>>>> >>>>>>> Fixed the serviceId getting NULL by setting it to the ClusterContext >>>>>>> and committed it.. >>>>>>> >>>>>> >>>>>> Fine then :) >>>>>> >>>>>>> >>>>>>> On Sun, Nov 9, 2014 at 2:51 PM, Reka Thirunavukkarasu <r...@wso2.com >>>>>>> > wrote: >>>>>>> >>>>>>>> Hi Lahiru, >>>>>>>> >>>>>>>> As you have introduced ClusteContext in ClusterMonitor, the >>>>>>>> serviceId got to removed. Is this done purposely? If so, can we update >>>>>>>> the >>>>>>>> getServiceId() (which referred by several other places) in monitors to >>>>>>>> return the correct serviceId by reading ClusterContext? >>>>>>>> >>>>>>>> Due to this, ClusterMonitor returns serviceId as null and it causes >>>>>>>> issue in the cluster monitor creation and in the event publishing.. >>>>>>>> >>>>>>>> Thanks, >>>>>>>> Reka >>>>>>>> >>>>>>>> On Sun, Nov 9, 2014 at 2:39 PM, Reka Thirunavukkarasu < >>>>>>>> r...@wso2.com> wrote: >>>>>>>> >>>>>>>>> Hi >>>>>>>>> >>>>>>>>> On Sun, Nov 9, 2014 at 2:19 PM, Isuru Haththotuwa < >>>>>>>>> isu...@apache.org> wrote: >>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On Sun, Nov 9, 2014 at 1:13 PM, Isuru Haththotuwa < >>>>>>>>>> isu...@apache.org> wrote: >>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> On Sun, Nov 9, 2014 at 1:09 PM, Reka Thirunavukkarasu < >>>>>>>>>>> r...@wso2.com> wrote: >>>>>>>>>>> >>>>>>>>>>>> Hi all, >>>>>>>>>>>> >>>>>>>>>>>> After fixing few issues, i could get the application >>>>>>>>>>>> undeployment working with one cluster now. So the basic >>>>>>>>>>>> undeployment is >>>>>>>>>>>> working fine...Will continue testing with a complex application... >>>>>>>>>>>> >>>>>>>>>>> Great! I will test the same with a complex application. >>>>>>>>>>> >>>>>>>>>> Getting the following exception with a complex app (when sending >>>>>>>>>> cluster activated event), will check and fix asap. >>>>>>>>>> Exception in thread "Thread-61" >>>>>>>>>> java.lang.IllegalMonitorStateException >>>>>>>>>> at >>>>>>>>>> java.util.concurrent.locks.ReentrantReadWriteLock$Sync.tryReleaseShared(ReentrantReadWriteLock.java:363) >>>>>>>>>> at >>>>>>>>>> java.util.concurrent.locks.AbstractQueuedSynchronizer.releaseShared(AbstractQueuedSynchronizer.java:1317) >>>>>>>>>> at >>>>>>>>>> java.util.concurrent.locks.ReentrantReadWriteLock$ReadLock.unlock(ReentrantReadWriteLock.java:745) >>>>>>>>>> at >>>>>>>>>> org.apache.stratos.messaging.domain.topology.locking.TopologyLock.releaseReadLock(TopologyLock.java:61) >>>>>>>>>> at >>>>>>>>>> org.apache.stratos.messaging.message.receiver.topology.TopologyManager.releaseReadLockForCluster(TopologyManager.java:196) >>>>>>>>>> at >>>>>>>>>> org.apache.stratos.autoscaler.event.publisher.ClusterStatusEventPublisher.sendClusterActivatedEvent(ClusterStatusEventPublisher.java:109) >>>>>>>>>> at >>>>>>>>>> org.apache.stratos.autoscaler.status.checker.StatusChecker$1.run(StatusChecker.java:77) >>>>>>>>>> at java.lang.Thread.run(Thread.java:662) >>>>>>>>>> >>>>>>>>> >>>>>>>>> I too got this issue as well as a NPE while getting locks..Later >>>>>>>>> found that serviceId was null when getting the locks..I have fixed >>>>>>>>> ServiceId issue..Can you try now and see whether you are still >>>>>>>>> getting this >>>>>>>>> issue? >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> Reka >>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>> Thanks, >>>>>>>>>>>> Reka >>>>>>>>>>>> >>>>>>>>>>>> On Sun, Nov 9, 2014 at 1:05 PM, Isuru Haththotuwa < >>>>>>>>>>>> isu...@apache.org> wrote: >>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> On Sun, Nov 9, 2014 at 8:38 AM, Imesh Gunaratne < >>>>>>>>>>>>> im...@apache.org> wrote: >>>>>>>>>>>>> >>>>>>>>>>>>>> No, there seems to be a problem with application un-deploy >>>>>>>>>>>>>> function. For me the un-deploy call hangs in the autoscaler. >>>>>>>>>>>>>> This is >>>>>>>>>>>>>> currerntly under investigation. >>>>>>>>>>>>>> >>>>>>>>>>>>> Got the same issue, will check in this. >>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> Since the basic workflow of service grouping and docker are >>>>>>>>>>>>>> working now shall we merge docker-grouping-merge branch to >>>>>>>>>>>>>> master branch >>>>>>>>>>>>>> now? WDYT? >>>>>>>>>>>>>> >>>>>>>>>>>>>> On Sun, Nov 9, 2014 at 4:13 AM, Rajaratnam Rajkumar < >>>>>>>>>>>>>> rraju1...@gmail.com> wrote: >>>>>>>>>>>>>> >>>>>>>>>>>>>>> Hi all, >>>>>>>>>>>>>>> Did anyone undeploy the application successfully? Were all >>>>>>>>>>>>>>> clusters wiped out? >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Thanks >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> On Saturday, November 8, 2014, Imesh Gunaratne < >>>>>>>>>>>>>>> im...@apache.org> wrote: >>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> On Sat, Nov 8, 2014 at 11:13 PM, Chamila De Alwis < >>>>>>>>>>>>>>>> chami...@wso2.com> wrote: >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Don't we also need the event processing chain since it >>>>>>>>>>>>>>>>> constructs the event objects from the Jsons? Additionally >>>>>>>>>>>>>>>>> this is done >>>>>>>>>>>>>>>>> through the Gson library, so we would have some difficulty in >>>>>>>>>>>>>>>>> getting that >>>>>>>>>>>>>>>>> functionality in to python as well. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> A good point. May be we need to think little bit here, >>>>>>>>>>>>>>>> shall we first try to convert the required classes and see how >>>>>>>>>>>>>>>> good they >>>>>>>>>>>>>>>> are? >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Thanks >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> -- >>>>>>>>>>>>>> Imesh Gunaratne >>>>>>>>>>>>>> >>>>>>>>>>>>>> Technical Lead, WSO2 >>>>>>>>>>>>>> Committer & PMC Member, Apache Stratos >>>>>>>>>>>>>> >>>>>>>>>>>>>> -- >>>>>>>>>>>>>> Thanks and Regards, >>>>>>>>>>>>>> >>>>>>>>>>>>>> Isuru H. >>>>>>>>>>>>>> +94 716 358 048* <http://wso2.com/>* >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> * <http://wso2.com/>* >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> -- >>>>>>>>>>>> Reka Thirunavukkarasu >>>>>>>>>>>> Senior Software Engineer, >>>>>>>>>>>> WSO2, Inc.:http://wso2.com, >>>>>>>>>>>> Mobile: +94776442007 >>>>>>>>>>>> >>>>>>>>>>>> -- >>>>>>>>>>>> <%2B94776442007> >>>>>>>>>>>> <%2B94776442007> >>>>>>>>>>>> Thanks and Regards, >>>>>>>>>>>> >>>>>>>>>>>> Isuru H. >>>>>>>>>>>> <%2B94776442007> >>>>>>>>>>>> +94 716 358 048 >>>>>>>>>>>> >>>>>>>>>>>> -- >>>>>>>>>>>> <%2B94776442007> >>>>>>>>>>>> <%2B94776442007> >>>>>>>>>>>> Thanks and Regards, >>>>>>>>>>>> >>>>>>>>>>>> Isuru H. >>>>>>>>>>>> <%2B94776442007> >>>>>>>>>>>> +94 716 358 048 <%2B94776442007>* <http://wso2.com/>* >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> * <http://wso2.com/>* >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> -- >>>>>>>>> Reka Thirunavukkarasu >>>>>>>>> Senior Software Engineer, >>>>>>>>> WSO2, Inc.:http://wso2.com, >>>>>>>>> Mobile: +94776442007 >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> -- >>>>>>>> Reka Thirunavukkarasu >>>>>>>> Senior Software Engineer, >>>>>>>> WSO2, Inc.:http://wso2.com, >>>>>>>> Mobile: +94776442007 >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> Reka Thirunavukkarasu >>>>>>> Senior Software Engineer, >>>>>>> WSO2, Inc.:http://wso2.com, >>>>>>> Mobile: +94776442007 >>>>>>> >>>>>>> >>>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> -- >>>>>> Lahiru Sandaruwan >>>>>> Committer and PMC member, Apache Stratos, >>>>>> Senior Software Engineer, >>>>>> WSO2 Inc., http://wso2.com >>>>>> lean.enterprise.middleware >>>>>> >>>>>> email: lahi...@wso2.com blog: http://lahiruwrites.blogspot.com/ >>>>>> linked-in: http://lk.linkedin.com/pub/lahiru-sandaruwan/16/153/146 >>>>>> >>>>>> >>>>> >>>>> >>>>> -- >>>>> -- >>>>> Lahiru Sandaruwan >>>>> Committer and PMC member, Apache Stratos, >>>>> Senior Software Engineer, >>>>> WSO2 Inc., http://wso2.com >>>>> lean.enterprise.middleware >>>>> >>>>> email: lahi...@wso2.com blog: http://lahiruwrites.blogspot.com/ >>>>> linked-in: http://lk.linkedin.com/pub/lahiru-sandaruwan/16/153/146 >>>>> >>>>> -- >>>>> <http://lk.linkedin.com/pub/lahiru-sandaruwan/16/153/146> >>>>> <http://lk.linkedin.com/pub/lahiru-sandaruwan/16/153/146> >>>>> Thanks and Regards, >>>>> >>>>> Isuru H. >>>>> <http://lk.linkedin.com/pub/lahiru-sandaruwan/16/153/146> >>>>> +94 716 358 048 >>>>> <http://lk.linkedin.com/pub/lahiru-sandaruwan/16/153/146>* >>>>> <http://wso2.com/>* >>>>> >>>>> >>>>> * <http://wso2.com/>* >>>>> >>>>> >>>>> >> >> >> -- >> Raj >> > > > > -- > Raj > > -- > Thanks and Regards, > > Isuru H. > +94 716 358 048* <http://wso2.com/>* > > > * <http://wso2.com/>* > > >