Hi Nirmal, This turned out to be a non-issue for me w.r.t. launching cartridge. When I subscribed to the cartridge in CLI, I had specified a deployment policy that no longer exists. After fixing that, I was able to launch the cartridge successfully.
I think for this case, Stratos CLI should give an error instead of showing "You have successfully subscribed…" message and not able to launch the cartridge later on. There was no error anywhere (CLI console and WSO2 logs) for us to tell why the instance was not spawned. Essentially we need to add some sort of validation in place. Thanks, -Jeffrey From: jeffrngu <[email protected]<mailto:[email protected]>> Reply-To: "[email protected]<mailto:[email protected]>" <[email protected]<mailto:[email protected]>> Date: Thursday, April 3, 2014 9:42 AM To: "[email protected]<mailto:[email protected]>" <[email protected]<mailto:[email protected]>> Subject: Re: Does this look like config issue? Hi Nirmal, We're not using Stratos setup script. We're deploying the Stratos binaries manually. But I'll try out the single multi-profile setup when I have some cycles :-) I looked at the logs for MB, CC and SM. There's not much except the "No routes for message" warnings from MB. Please see log snippets below. I'm suspecting this has something to do with the MB's ActiveMQ setup. Below are all copies of MB client lib jars on my test bed: root@octl-01:~# find /opt/wso2/ -name 'andes-client*.jar'|xargs ls -l -rw-r--r-- 1 root root 2140848 Feb 19 00:30 /opt/wso2/apache-stratos-autoscaler-4.0.0-SNAPSHOT/repository/components/dropins/andes-client-0.13.wso2v8.jar -rw-r--r-- 1 root root 2140848 Feb 19 00:30 /opt/wso2/apache-stratos-cartridge-agent-4.0.0-SNAPSHOT/lib/andes-client-0.13.wso2v8.jar -rw-r--r-- 1 root root 2140848 Feb 19 00:30 /opt/wso2/apache-stratos-cc-4.0.0-SNAPSHOT/client-lib/andes-client-0.13.wso2v8.jar -rw-r--r-- 1 root root 2140848 Feb 19 00:30 /opt/wso2/apache-stratos-cc-4.0.0-SNAPSHOT/repository/components/dropins/andes-client-0.13.wso2v8.jar -rw-r--r-- 1 root root 2140848 Feb 19 00:30 /opt/wso2/apache-stratos-load-balancer-4.0.0-SNAPSHOT/repository/components/dropins/andes-client-0.13.wso2v8.jar -rw-r--r-- 1 root root 2140848 Feb 19 00:30 /opt/wso2/apache-stratos-manager-4.0.0-SNAPSHOT/repository/components/dropins/andes-client-0.13.wso2v8.jar -rw-r--r-- 1 root root 2140848 Mar 26 15:41 /opt/wso2/apache-stratos-manager-4.0.0-SNAPSHOT/repository/components/patches/patch0000/andes-client_0.13.0.wso2v8.jar -rw-r--r-- 1 root root 2140848 Mar 26 15:41 /opt/wso2/apache-stratos-manager-4.0.0-SNAPSHOT/repository/components/plugins/andes-client_0.13.0.wso2v8.jar -rw-r--r-- 1 root root 2140848 Feb 19 00:30 /opt/wso2/wso2cep-3.0.0/repository/components/dropins/andes-client-0.13.wso2v8.jar -rw-r--r-- 1 root root 2139249 Mar 20 09:28 /opt/wso2/wso2cep-3.0.0/repository/components/lib/andes-client-0.13.wso2v8.jar -rw-r--r-- 1 root root 2139249 Jul 2 2013 /opt/wso2/wso2mb-2.1.0/client-lib/andes-client-0.13.wso2v8.jar root@octl-01:~# root@octl-01:~# grep ERROR /opt/wso2/wso2mb-2.1.0/repository/logs/wso2carbon.log root@octl-01:~# grep WARN /opt/wso2/wso2mb-2.1.0/repository/logs/wso2carbon.log TID: [0] [MB] [2014-04-03 03:06:53,120] WARN {org.wso2.carbon.core.bootup.validator.SystemValidator} - Could not validate the system for configuration parameter : CPU {org.wso2.carbon.core.bootup.validator.SystemValidator} TID: [0] [MB] [2014-04-03 03:06:53,121] WARN {org.wso2.carbon.core.bootup.validator.util.ValidationResultPrinter} - Swap Memory size (MB): 1021 of the system is below the recommended minimum size :2048 {org.wso2.carbon.core.bootup.validator.util.ValidationResultPrinter} TID: [0] [MB] [2014-04-03 03:06:53,791] WARN {net.sf.ehcache.config.ConfigurationFactory} - No configuration found. Configuring ehcache from ehcache-failsafe.xml found in the classpath: bundleresource://38.fwk456338722/ehcache-failsafe.xml {net.sf.ehcache.config.ConfigurationFactory} TID: [0] [MB] [2014-04-03 03:07:02,556] WARN {org.apache.cassandra.service.StorageService} - Generated random token 75959671262994002948453465246431048258. Random tokens will result in an unbalanced ring; see http://wiki.apache.org/cassandra/Operations {org.apache.cassandra.service.StorageService} TID: [0] [MB] [2014-04-03 03:07:02,556] WARN {org.apache.cassandra.service.StorageService} - Generated random token 75959671262994002948453465246431048258. Random tokens will result in an unbalanced ring; see http://wiki.apache.org/cassandra/Operations {org.apache.cassandra.service.StorageService} TID: [0] [MB] [2014-04-03 03:08:06,707] WARN {org.wso2.andes.server.AMQChannel} - MESSAGE DISCARDED: No routes for message - Message[(HC:205894544 ID:365588970905011200 Ref:0)]: 365588970905011200; ref count: 0 {org.wso2.andes.server.AMQChannel} TID: [0] [MB] [2014-04-03 03:09:06,723] WARN {org.wso2.andes.server.AMQChannel} - MESSAGE DISCARDED: No routes for message - Message[(HC:733045095 ID:365588986639680512 Ref:0)]: 365588986639680512; ref count: 0 {org.wso2.andes.server.AMQChannel} TID: [0] [MB] [2014-04-03 03:09:57,394] WARN {org.wso2.andes.server.AMQChannel} - MESSAGE DISCARDED: No routes for message - Message[(HC:1914176150 ID:365588999923303424 Ref:0)]: 365588999923303424; ref count: 0 {org.wso2.andes.server.AMQChannel} TID: [0] [MB] [2014-04-03 03:10:01,057] WARN {org.wso2.andes.server.AMQChannel} - MESSAGE DISCARDED: No routes for message - Message[(HC:545495255 ID:365589000881177600 Ref:0)]: 365589000881177600; ref count: 0 {org.wso2.andes.server.AMQChannel} TID: [0] [MB] [2014-04-03 03:11:01,023] WARN {org.wso2.andes.server.AMQChannel} - MESSAGE DISCARDED: No routes for message - Message[(HC:2070517861 ID:365589016603526144 Ref:0)]: 365589016603526144; ref count: 0 {org.wso2.andes.server.AMQChannel} TID: [0] [MB] [2014-04-03 03:12:01,042] WARN {org.wso2.andes.server.AMQChannel} - MESSAGE DISCARDED: No routes for message - Message[(HC:2135480317 ID:365589032337146880 Ref:0)]: 365589032337146880; ref count: 0 {org.wso2.andes.server.AMQChannel} TID: [0] [MB] [2014-04-03 03:13:01,034] WARN {org.wso2.andes.server.AMQChannel} - MESSAGE DISCARDED: No routes for message - Message[(HC:699522331 ID:365589048063689728 Ref:0)]: 365589048063689728; ref count: 0 {org.wso2.andes.server.AMQChannel} TID: [0] [MB] [2014-04-03 03:14:01,046] WARN {org.wso2.andes.server.AMQChannel} - MESSAGE DISCARDED: No routes for message - Message[(HC:1395979658 ID:365589063795475456 Ref:0)]: 365589063795475456; ref count: 0 {org.wso2.andes.server.AMQChannel} TID: [0] [MB] [2014-04-03 03:15:01,033] WARN {org.wso2.andes.server.AMQChannel} - MESSAGE DISCARDED: No routes for message - Message[(HC:1677663149 ID:365589079520707584 Ref:0)]: 365589079520707584; ref count: 0 {org.wso2.andes.server.AMQChannel} TID: [0] [MB] [2014-04-03 03:16:01,040] WARN {org.wso2.andes.server.AMQChannel} - MESSAGE DISCARDED: No routes for message - Message[(HC:1432199149 ID:365589095251182592 Ref:0)]: 365589095251182592; ref count: 0 {org.wso2.andes.server.AMQChannel} TID: [0] [MB] [2014-04-03 03:17:01,035] WARN {org.wso2.andes.server.AMQChannel} - MESSAGE DISCARDED: No routes for message - Message[(HC:1619866108 ID:365589110978511872 Ref:0)]: 365589110978511872; ref count: 0 {org.wso2.andes.server.AMQChannel} TID: [0] [MB] [2014-04-03 03:18:01,040] WARN {org.wso2.andes.server.AMQChannel} - MESSAGE DISCARDED: No routes for message - Message[(HC:2111042371 ID:365589126708462592 Ref:0)]: 365589126708462592; ref count: 0 {org.wso2.andes.server.AMQChannel} TID: [0] [MB] [2014-04-03 03:19:01,031] WARN {org.wso2.andes.server.AMQChannel} - MESSAGE DISCARDED: No routes for message - Message[(HC:545178066 ID:365589142435005440 Ref:0)]: 365589142435005440; ref count: 0 {org.wso2.andes.server.AMQChannel} TID: [0] [MB] [2014-04-03 03:20:01,039] WARN {org.wso2.andes.server.AMQChannel} - MESSAGE DISCARDED: No routes for message - Message[(HC:702544124 ID:365589158165742592 Ref:0)]: 365589158165742592; ref count: 0 {org.wso2.andes.server.AMQChannel} TID: [0] [MB] [2014-04-03 03:21:01,033] WARN {org.wso2.andes.server.AMQChannel} - MESSAGE DISCARDED: No routes for message - Message[(HC:396379545 ID:365589173892547584 Ref:0)]: 365589173892547584; ref count: 0 {org.wso2.andes.server.AMQChannel} root@octl-01:~# root@octl-01:~# root@octl-01:~# root@octl-01:~# root@octl-01:~# grep ERROR /opt/wso2/apache-stratos-cc-4.0.0-SNAPSHOT/repository/logs/wso2carbon.log root@octl-01:~# grep WARN /opt/wso2/apache-stratos-cc-4.0.0-SNAPSHOT/repository/logs/wso2carbon.log TID: [0] [CC] [2014-04-03 03:07:55,315] WARN {org.wso2.carbon.core.bootup.validator.SystemValidator} - Could not validate the system for configuration parameter : CPU {org.wso2.carbon.core.bootup.validator.SystemValidator} TID: [0] [CC] [2014-04-03 03:07:55,316] WARN {org.wso2.carbon.core.bootup.validator.util.ValidationResultPrinter} - The default keystore (wso2carbon.jks) is currently being used. To maximize security when deploying to a production environment, configure a new keystore with a unique password in the production server profile. {org.wso2.carbon.core.bootup.validator.util.ValidationResultPrinter} TID: [0] [CC] [2014-04-03 03:07:55,316] WARN {org.wso2.carbon.core.bootup.validator.util.ValidationResultPrinter} - Swap Memory size (MB): 1021 of the system is below the recommended minimum size :2048 {org.wso2.carbon.core.bootup.validator.util.ValidationResultPrinter} TID: [0] [CC] [2014-04-03 03:08:03,803] WARN {org.apache.stratos.cloud.controller.registry.RegistryManager} - Registry Service is null. Hence unable to fetch data from registry. {org.apache.stratos.cloud.controller.registry.RegistryManager} TID: [0] [CC] [2014-04-03 03:08:03,807] WARN {org.apache.stratos.cloud.controller.axiom.parser.IaasProviderConfigParser} - Unable to find a value for identity element from Secure Vault.Hence we will try to assign the plain text value (if specified). {org.apache.stratos.cloud.controller.axiom.parser.IaasProviderConfigParser} TID: [0] [CC] [2014-04-03 03:08:03,807] WARN {org.apache.stratos.cloud.controller.axiom.parser.IaasProviderConfigParser} - Unable to find a value for credential element from Secure Vault.Hence we will try to assign the plain text value (if specified). {org.apache.stratos.cloud.controller.axiom.parser.IaasProviderConfigParser} TID: [0] [CC] [2014-04-03 03:08:05,274] WARN {org.apache.synapse.transport.passthru.config.PassThroughConfiguration} - Error loading properties from a file at from the System defined location: passthru-http.properties {org.apache.synapse.transport.passthru.config.PassThroughConfiguration} TID: [0] [CC] [2014-04-03 03:08:06,544] WARN {org.wso2.andes.common.QpidProperties} - Unable to find resource qpidversion.properties from classloader {org.wso2.andes.common.QpidProperties} root@octl-01:~# root@octl-01:~# root@octl-01:~# grep ERROR /opt/wso2/apache-stratos-manager-4.0.0-SNAPSHOT/repository/logs/wso2carbon.log TID: [0] [SCC] [2014-04-03 03:09:42,873] ERROR {org.wso2.stratos.identity.saml2.sso.mgt.ui.Util} - The configuration element 'TenantRegistrationPage' not found in '/opt/wso2/apache-stratos-manager-4.0.0-SNAPSHOT/repository/conf/sso-idp-config.xml' {org.wso2.stratos.identity.saml2.sso.mgt.ui.Util} root@octl-01:~# root@octl-01:~# grep WARN /opt/wso2/apache-stratos-manager-4.0.0-SNAPSHOT/repository/logs/wso2carbon.log TID: [2014-04-03 03:09:05,297] WARN {org.wso2.carbon.server.extensions.DropinsBundleDeployer} - Ignoring the deployment of bundle: ../dropins/andes-client-0.13.wso2v8.jar, because it is already available in the system: ../plugins/andes-client_0.13.0.wso2v8.jar. Bundle-SymbolicName and Bundle-Version headers are identical {org.wso2.carbon.server.extensions.DropinsBundleDeployer} TID: [0] [SCC] [2014-04-03 03:09:13,355] WARN {org.wso2.carbon.core.bootup.validator.SystemValidator} - Could not validate the system for configuration parameter : CPU {org.wso2.carbon.core.bootup.validator.SystemValidator} TID: [0] [SCC] [2014-04-03 03:09:13,356] WARN {org.wso2.carbon.core.bootup.validator.util.ValidationResultPrinter} - The default keystore (wso2carbon.jks) is currently being used. To maximize security when deploying to a production environment, configure a new keystore with a unique password in the production server profile. {org.wso2.carbon.core.bootup.validator.util.ValidationResultPrinter} TID: [0] [SCC] [2014-04-03 03:09:13,357] WARN {org.wso2.carbon.core.bootup.validator.util.ValidationResultPrinter} - Swap Memory size (MB): 1021 of the system is below the recommended minimum size :2048 {org.wso2.carbon.core.bootup.validator.util.ValidationResultPrinter} TID: [0] [SCC] [2014-04-03 03:09:41,527] WARN {org.apache.axis2.description.java2wsdl.DefaultSchemaGenerator} - We don't support method overloading. Ignoring [validateAudienceRestriction] {org.apache.axis2.description.java2wsdl.DefaultSchemaGenerator} TID: [0] [SCC] [2014-04-03 03:09:45,707] WARN {org.wso2.andes.common.QpidProperties} - Unable to find resource qpidversion.properties from classloader {org.wso2.andes.common.QpidProperties} root@octl-01:~# From: Nirmal Fernando <[email protected]<mailto:[email protected]>> Reply-To: "[email protected]<mailto:[email protected]>" <[email protected]<mailto:[email protected]>> Date: Wednesday, April 2, 2014 8:15 PM To: "[email protected]<mailto:[email protected]>" <[email protected]<mailto:[email protected]>> Subject: Re: Does this look like config issue? Hi Jeffrey, Please send the log files of all servers. And I believe it's time for you to try out the single multi-profile product we have built :-) On Thu, Apr 3, 2014 at 9:23 AM, Jeffrey Nguyen (jeffrngu) <[email protected]<mailto:[email protected]>> wrote: Thanks Nirmal. I restarted MB and also restarted CC after that. However, the problem persisted. I'm using wso2mb-2.1.0.zip (same version since m7) -Jeffrey From: Nirmal Fernando <[email protected]<mailto:[email protected]>> Reply-To: "[email protected]<mailto:[email protected]>" <[email protected]<mailto:[email protected]>> Date: Wednesday, April 2, 2014 7:23 PM To: "[email protected]<mailto:[email protected]>" <[email protected]<mailto:[email protected]>> Subject: Re: Does this look like config issue? Seems like your MB hasn't received any subscriptions. Can you please restart MB and see? On Thu, Apr 3, 2014 at 6:30 AM, Jeffrey Nguyen (jeffrngu) <[email protected]<mailto:[email protected]>> wrote: I saw this email thread (attached). It seems like the same issue. Was there any resolution on this? Thanks, -Jeffrey From: jeffrngu <[email protected]<mailto:[email protected]>> Reply-To: "[email protected]<mailto:[email protected]>" <[email protected]<mailto:[email protected]>> Date: Wednesday, April 2, 2014 4:35 PM To: "[email protected]<mailto:[email protected]>" <[email protected]<mailto:[email protected]>> Subject: Does this look like config issue? Hi, I'm debugging an issue where the instance is not launched when I subscribe to a cartridge. In Stratos CLI, when I subscribed, I can see the cartridge listed as subscribed. From CC log, the event that triggers instance launch never came. I checked MB log and saw several entries like one below. This is the same Stratos code where cartridge subscription worked fine before. The only thing that's changed is the testbed. Has anyone seen this type of errors before? TID: [0] [MB] [2014-04-02 08:53:08,330] INFO {org.wso2.andes.server.exchange.TopicExchange} - Message routing key: topology No routes. {org.wso2.andes.server.exchange.TopicExchange} TID: [0] [MB] [2014-04-02 08:53:08,331] WARN {org.wso2.andes.server.AMQChannel} - MESSAGE DISCARDED: No routes for message - Message[(HC:1120244111 ID:365571748472554496 Ref:0)]: 365571748472554496; ref count: 0 {org.wso2.andes.server.AMQChannel} Thanks, -Jeffrey -- Best Regards, Nirmal Nirmal Fernando. PPMC Member & Committer of Apache Stratos, Senior Software Engineer, WSO2 Inc. Blog: http://nirmalfdo.blogspot.com/ -- Best Regards, Nirmal Nirmal Fernando. PPMC Member & Committer of Apache Stratos, Senior Software Engineer, WSO2 Inc. Blog: http://nirmalfdo.blogspot.com/
