Hi David,
It is difficult to find the exact problem with the above logs. Will you be
able to start the load balancer by enabling DEBUG logs as follows:
[load-balancer-home]/repository/conf/log4j.properties
log4j.logger.org.apache.stratos.load.balancer=DEBUG
log4j.logger.org.apache.stratos.load.balancer.common.statistics=DEBUG
Thanks
On Wed, Mar 11, 2015 at 11:01 AM, david hbase <davidhb...@gmail.com> wrote:
> Hi, Imesh
>
> After activemq-client-5.10.0.jar uploaded and JAVA_HOME error fixed, no
> more error in log, but The LB did not send any status to Stratos CEP. The
> Stratos debug log and LB logs are attached as following, Do you think if
> any issue with LB CPU or ValidationResultPrinter memory? If yes, how do we
> configure them?
>
> Please help.
> Thanks
> -D
>
> 1. Stratos Debug Log:
> [2015-03-11 05:09:15,558] DEBUG
> {org.apache.stratos.autoscaler.monitor.ClusterMonitor} - Running minimum
> check for partition zone-1
> [2015-03-11 05:09:15,558] DEBUG
> {org.apache.stratos.autoscaler.rule.RuleLog} - Running minimum rule:
> [network-partition] openstack-region1 [partition] zone-1
> [2015-03-11 05:09:15,558] DEBUG
> {org.apache.stratos.autoscaler.rule.RuleLog} - [min-check]
> [network-partition] openstack-region1 [partition] zone-1 Non terminated
> member count: 1
> [2015-03-11 05:09:15,558] DEBUG
> {org.apache.stratos.autoscaler.rule.RuleLog} - [min-check]
> [network-partition] openstack-region1 [partition] zone-1 Minimum member
> count: 1
> [2015-03-11 05:09:15,558] DEBUG
> {org.apache.stratos.autoscaler.rule.RuleLog} - Running obsolete instances
> rule: [partition] zone-1 [network-partition] openstack-region1
> [2015-03-11 05:09:15,559] DEBUG
> {org.apache.stratos.autoscaler.rule.RuleLog} - [obsolete-check]
> [network-partition] openstack-region1 [partition] zone-1 Obsolete member
> count: 0
> [2015-03-11 05:09:15,559] DEBUG
> {org.apache.stratos.autoscaler.rule.AutoscalerRuleEvaluator} - Minimum
> check executed for : org.apache.stratos.autoscaler.PartitionContext@513b1870
> [2015-03-11 05:09:15,559] DEBUG
> {org.apache.stratos.autoscaler.monitor.ClusterMonitor} - Scale rule will
> not run since the LB statistics have not received before this cycle for
> network partition openstack-region1
> [2015-03-11 05:09:32,811] DEBUG
> {org.apache.stratos.autoscaler.message.receiver.health.AutoscalerHealthStatEventReceiver}
> - Average Rif event: [cluster] php.php301.domain [network-partition]
> zone-1 [value] 0.0
> [2015-03-11 05:09:32,812] DEBUG
> {org.apache.stratos.autoscaler.message.receiver.health.AutoscalerHealthStatEventReceiver}
> - Network partition context is not available for : [network partition]
> zone-1
> [2015-03-11 05:09:32,812] DEBUG
> {org.apache.stratos.autoscaler.message.receiver.health.AutoscalerHealthStatEventReceiver}
> - Average Rif event: [cluster] lb.lb.domain [network-partition] zone-1
> [value] 0.0
> [2015-03-11 05:09:32,812] DEBUG
> {org.apache.stratos.autoscaler.message.receiver.health.AutoscalerHealthStatEventReceiver}
> - Network partition context is not available for : [network partition]
> zone-1
> [2015-03-11 05:09:32,813] DEBUG
> {org.apache.stratos.autoscaler.message.receiver.health.AutoscalerHealthStatEventReceiver}
> - Gradient of Rif event: [cluster] php.php301.domain [network-partition]
> zone-1 [value] 0.0
> [2015-03-11 05:09:32,813] DEBUG
> {org.apache.stratos.autoscaler.message.receiver.health.AutoscalerHealthStatEventReceiver}
> - Network partition context is not available for : [network partition]
> zone-1
> [2015-03-11 05:09:32,985] DEBUG
> {org.apache.stratos.autoscaler.message.receiver.health.AutoscalerHealthStatEventReceiver}
> - Second derivative of Rif event: [cluster] php.php301.domain
> [network-partition] zone-1 [value] 0.0
> [2015-03-11 05:09:32,986] DEBUG
> {org.apache.stratos.autoscaler.message.receiver.health.AutoscalerHealthStatEventReceiver}
> - Network partition context is not available for : [network partition]
> zone-1
> [2015-03-11 05:09:33,208] DEBUG
> {org.apache.stratos.autoscaler.monitor.ClusterMonitor} - Cluster monitor
> is running.. ClusterMonitor [clusterId=lb.lb.domain, serviceId=lb,
> deploymentPolicy=Deployment Policy [id]economyDeployment [partitions]
> [org.apache.stratos.cloud.controller.stub.deployment.partition.Partition@4800d30d],
> autoscalePolicy=ASPolicy [id=economyPolicy, displayName=null,
> description=null], lbReferenceType=null]
> [2015-03-11 05:09:33,208] DEBUG
> {org.apache.stratos.autoscaler.monitor.ClusterMonitor} - Running minimum
> check for partition zone-1
> [2015-03-11 05:09:33,208] DEBUG
> {org.apache.stratos.autoscaler.rule.RuleLog} - Running minimum rule:
> [network-partition] openstack-region1 [partition] zone-1
> [2015-03-11 05:09:33,209] DEBUG
> {org.apache.stratos.autoscaler.rule.RuleLog} - [min-check]
> [network-partition] openstack-region1 [partition] zone-1 Non terminated
> member count: 1
> [2015-03-11 05:09:33,209] DEBUG
> {org.apache.stratos.autoscaler.rule.RuleLog} - [min-check]
> [network-partition] openstack-region1 [partition] zone-1 Minimum member
> count: 1
> [2015-03-11 05:09:33,209] DEBUG
> {org.apache.stratos.autoscaler.rule.RuleLog} - Running obsolete instances
> rule: [partition] zone-1 [network-partition] openstack-region1
> [2015-03-11 05:09:33,209] DEBUG
> {org.apache.stratos.autoscaler.rule.RuleLog} - [obsolete-check]
> [network-partition] openstack-region1 [partition] zone-1 Obsolete member
> count: 0
> [2015-03-11 05:09:33,209] DEBUG
> {org.apache.stratos.autoscaler.rule.AutoscalerRuleEvaluator} - Minimum
> check executed for : org.apache.stratos.autoscaler.PartitionContext@4becc9ef
> [2015-03-11 05:09:33,209] DEBUG
> {org.apache.stratos.autoscaler.monitor.ClusterMonitor} - Scale rule will
> not run since the LB statistics have not received before this cycle for
> network partition openstack-region1
> [2015-03-11 05:09:45,559] DEBUG
> {org.apache.stratos.autoscaler.monitor.ClusterMonitor} - Cluster monitor
> is running.. ClusterMonitor [clusterId=php.php301.domain, serviceId=php301,
> deploymentPolicy=Deployment Policy [id]economyDeployment [partitions]
> [org.apache.stratos.cloud.controller.stub.deployment.partition.Partition@4800d30d],
> autoscalePolicy=ASPolicy [id=economyPolicy, displayName=null,
> description=null], lbReferenceType=null]
>
>
> 2. LB logs
> root@lblbdoma-508:~#
> /mnt/apache-stratos-load-balancer-4.0.0/bin/stratos.sh &
> [1] 1521
> root@lblbdoma-508:~# JAVA_HOME environment variable is set to
> /usr/lib/jvm/java-7-openjdk-amd64
> CARBON_HOME environment variable is set to
> /mnt/apache-stratos-load-balancer-4.0.0
> [2015-03-11 05:25:38,642] INFO - CarbonCoreActivator Starting WSO2
> Carbon...
> [2015-03-11 05:25:38,646] INFO - CarbonCoreActivator Operating System :
> Linux 3.2.0-74-virtual, amd64
> [2015-03-11 05:25:38,646] INFO - CarbonCoreActivator Java Home :
> /usr/lib/jvm/java-7-openjdk-amd64/jre
> [2015-03-11 05:25:38,646] INFO - CarbonCoreActivator Java Version :
> 1.7.0_75
> [2015-03-11 05:25:38,646] INFO - CarbonCoreActivator Java VM :
> OpenJDK 64-Bit Server VM 24.75-b04,Oracle Corporation
> [2015-03-11 05:25:38,646] INFO - CarbonCoreActivator Carbon Home :
> /mnt/apache-stratos-load-balancer-4.0.0
> [2015-03-11 05:25:38,646] INFO - CarbonCoreActivator Java Temp Dir :
> /mnt/apache-stratos-load-balancer-4.0.0/tmp
> [2015-03-11 05:25:38,646] INFO - CarbonCoreActivator User :
> root, en-US, Etc/UTC
> [2015-03-11 05:25:38,948] WARN - SystemValidator Could not validate the
> system for configuration parameter : CPU
> [2015-03-11 05:25:38,955] WARN - ValidationResultPrinter RAM size (MB):
> 2003 of the system is below the recommended minimum size :2048
> [2015-03-11 05:25:38,957] WARN - ValidationResultPrinter The default
> keystore (wso2carbon.jks) is currently being used. To maximize security
> when deploying to a production environment, configure a new keystore with a
> unique password in the production server profile.
> [2015-03-11 05:25:38,957] WARN - ValidationResultPrinter Swap Memory size
> (MB): 0 of the system is below the recommended minimum size :2048
> [2015-03-11 05:25:38,973] INFO - AgentHolder Agent created !
> [2015-03-11 05:25:39,019] INFO - AgentDS Successfully deployed Agent
> Client
> [2015-03-11 05:25:42,353] INFO - EmbeddedRegistryService Configured
> Registry in 107ms
> [2015-03-11 05:25:42,707] INFO - RegistryCoreServiceComponent Registry
> Mode : READ-WRITE
> [2015-03-11 05:25:43,372] INFO - UserStoreMgtDSComponent Carbon
> UserStoreMgtDSComponent activated successfully.
> [2015-03-11 05:25:51,202] INFO - ClusterBuilder Clustering has been
> disabled
> [2015-03-11 05:25:51,794] INFO - CarbonAxisConfigurator No modules
> directory was found under
> /mnt/apache-stratos-load-balancer-4.0.0/repository/deployment/server.
> [2015-03-11 05:25:51,951] INFO - PassThroughHttpSSLSender Initializing
> Pass-through HTTP/S Sender...
> [2015-03-11 05:25:52,012] INFO - ClientConnFactoryBuilder HTTPS Loading
> Identity Keystore from : repository/resources/security/wso2carbon.jks
> [2015-03-11 05:25:52,041] INFO - ClientConnFactoryBuilder HTTPS Loading
> Trust Keystore from : repository/resources/security/client-truststore.jks
> [2015-03-11 05:25:52,144] INFO - PassThroughHttpSSLSender Pass-through
> HTTPS Sender started...
> [2015-03-11 05:25:52,144] INFO - PassThroughHttpSender Initializing
> Pass-through HTTP/S Sender...
> [2015-03-11 05:25:52,148] INFO - PassThroughHttpSender Pass-through HTTP
> Sender started...
> [2015-03-11 05:25:52,177] INFO - PassThroughHttpSSLListener Initializing
> Pass-through HTTP/S Listener...
> [2015-03-11 05:25:52,234] INFO - PassThroughHttpListener Initializing
> Pass-through HTTP/S Listener...
> [2015-03-11 05:25:52,476] INFO - ModuleDeployer Deploying module:
> addressing-1.6.1-wso2v10 -
> file:/mnt/apache-stratos-load-balancer-4.0.0/repository/deployment/client/modules/addressing-1.6.1-wso2v10.mar
> [2015-03-11 05:25:52,508] INFO - ModuleDeployer Deploying module:
> rampart-1.6.1-wso2v1 -
> file:/mnt/apache-stratos-load-balancer-4.0.0/repository/deployment/client/modules/rampart-1.6.1-wso2v1.mar
> [2015-03-11 05:25:52,519] INFO - TCPTransportSender TCP Sender started
> [2015-03-11 05:25:54,032] INFO - CarbonServerManager Repository :
> /mnt/apache-stratos-load-balancer-4.0.0/repository/deployment/server/
> [2015-03-11 05:25:54,105] INFO - PermissionUpdater Permission cache
> updated for tenant -1234
> [2015-03-11 05:25:54,160] INFO - ServiceBusInitializer Starting ESB...
> [2015-03-11 05:25:54,194] INFO - ServiceBusInitializer Initializing
> Apache Synapse...
> [2015-03-11 05:25:54,209] INFO - SynapseControllerFactory Using Synapse
> home : /mnt/apache-stratos-load-balancer-4.0.0/.
> [2015-03-11 05:25:54,209] INFO - SynapseControllerFactory Using
> synapse.xml location :
> /mnt/apache-stratos-load-balancer-4.0.0/././repository/deployment/server/synapse-configs/default
> [2015-03-11 05:25:54,209] INFO - SynapseControllerFactory Using server
> name : localhost
> [2015-03-11 05:25:54,220] INFO - SynapseControllerFactory The timeout
> handler will run every : 15s
> [2015-03-11 05:25:54,231] INFO - Axis2SynapseController Initializing
> Synapse at : Wed Mar 11 05:25:54 UTC 2015
> [2015-03-11 05:25:54,238] INFO - CarbonSynapseController Loading the
> mediation configuration from the file system
> [2015-03-11 05:25:54,240] INFO - MultiXMLConfigurationBuilder Building
> synapse configuration from the synapse artifact repository at :
> ././repository/deployment/server/synapse-configs/default
> [2015-03-11 05:25:54,248] INFO - XMLConfigurationBuilder Generating the
> Synapse configuration model by parsing the XML configuration
> [2015-03-11 05:25:54,544] INFO - SynapseConfigurationBuilder Loaded
> Synapse configuration from the artifact repository at :
> ././repository/deployment/server/synapse-configs/default
> [2015-03-11 05:25:54,555] INFO - Axis2SynapseController Loading mediator
> extensions...
> [2015-03-11 05:25:54,561] INFO - Axis2SynapseController Deploying the
> Synapse service...
> [2015-03-11 05:25:54,565] INFO - Axis2SynapseController Deploying Proxy
> services...
> [2015-03-11 05:25:54,568] INFO - Axis2SynapseController Deploying
> EventSources...
> [2015-03-11 05:25:54,608] INFO - ServerManager Server ready for
> processing...
> [2015-03-11 05:25:54,780] WARN - LoadBalancerConfiguration Endpoint
> timeout not found, using default: 15000
> [2015-03-11 05:25:54,790] INFO - SynapseConfigurator Configuring synapse
> main sequence...
> [2015-03-11 05:25:54,924] INFO - SynapseConfigurator Synapse main
> sequence configured successfully
> [2015-03-11 05:25:54,946] INFO - LoadBalancerServiceComponent Tenant
> receiver thread started
> [2015-03-11 05:25:54,964] INFO - TenantEventMessageDelegator Tenant event
> message delegator started
> [2015-03-11 05:25:54,972] INFO - LoadBalancerServiceComponent Topology
> receiver thread started
> [2015-03-11 05:25:54,980] INFO - LoadBalancerTopologyEventReceiver Load
> balancer topology receiver thread started
> [2015-03-11 05:25:54,996] INFO - TopologyEventMessageDelegator Topology
> event message delegator started
> [2015-03-11 05:25:55,127] INFO - LoadBalancerServiceComponent Load
> balancer statistics notifier thread started
> [2015-03-11 05:25:55,128] INFO - LoadBalancerServiceComponent Load
> balancer service component is activated
> [2015-03-11 05:25:55,554] INFO - PassThroughHttpSSLListener Starting
> Pass-through HTTPS Listener...
> [2015-03-11 05:25:55,596] INFO - PassThroughHttpSSLListener Pass-through
> HTTPS Listener started on 0:0:0:0:0:0:0:0:443
> [2015-03-11 05:25:55,596] INFO - PassThroughHttpListener Starting
> Pass-through HTTP Listener...
> [2015-03-11 05:25:55,604] INFO - PassThroughHttpListener Pass-through
> HTTP Listener started on 0:0:0:0:0:0:0:0:80
> [2015-03-11 05:25:55,762] INFO - TaskServiceImpl Task service starting in
> STANDALONE mode...
> [2015-03-11 05:25:55,869] INFO - JMXServerManager JMX Service URL :
> service:jmx:rmi://localhost:11111/jndi/rmi://localhost:9999/jmxrmi
> [2015-03-11 05:25:55,869] INFO - StartupFinalizerServiceComponent
> Server : Apache Stratos LB-4.0.0
> [2015-03-11 05:25:55,869] INFO - StartupFinalizerServiceComponent WSO2
> Carbon started in 24 sec
> [2015-03-11 05:25:56,286] INFO - CarbonUIServiceComponent Mgt Console
> URL : https://10.203.28.219:9443/carbon/
> [2015-03-11 05:26:02,051] INFO - CompleteTenantMessageProcessor Tenant
> initialized
> [2015-03-11 05:26:02,060] INFO - CompleteTopologyMessageProcessor
> Topology initialized
>
>
>
>
> On Tue, Mar 10, 2015 at 11:32 AM, Imesh Gunaratne <im...@apache.org>
> wrote:
>
>> Hi David,
>>
>> On Tue, Mar 10, 2015 at 10:32 AM, david hbase <davidhb...@gmail.com>
>> wrote:
>>
>>> Hi, Team:
>>>
>>> I found the ERROR with Stratos 4.0 loadbalancer as following, and this
>>> error blocked the LB statistics to CEP,
>>>
>>> [2015-03-10 04:59:46,057] ERROR - TopicSubscriber Error while
>>> subscribing to the topic: topology
>>> javax.naming.NoInitialContextException: Failed to create InitialContext
>>> using factory specified in hash table. [Root exception is
>>> java.lang.ClassNotFoundException: class
>>> org.apache.activemq.jndi.ActiveMQInitialContextFactory not found]
>>> at
>>> org.wso2.carbon.context.internal.CarbonContextDataHolder$CarbonInitialJNDIContextFactoryBuilder.createInitialContextFactory(CarbonContextDataHolder.java:737)
>>> at
>>> javax.naming.spi.NamingManager.getInitialContext(NamingManager.java:681)
>>> at
>>> javax.naming.InitialContext.getDefaultInitCtx(InitialContext.java:307)
>>> at javax.naming.InitialContext.init(InitialContext.java:242)
>>> at javax.naming.InitialContext.<init>(InitialContext.java:216)
>>> at
>>> org.apache.stratos.messaging.broker.connect.TopicConnector.init(TopicConnector.java:59)
>>> at
>>> org.apache.stratos.messaging.broker.subscribe.TopicSubscriber.doSubscribe(TopicSubscriber.java:63)
>>> at
>>> org.apache.stratos.messaging.broker.subscribe.TopicSubscriber.run(TopicSubscriber.java:116)
>>> at java.lang.Thread.run(Thread.java:745)
>>>
>>> As I see the ActiveMQ client JAR file (activemq-client-5.10.0.jar) is
>> not there in the load balancer repository/components/lib folder. If you
>> copy this the above issue may get solved.
>>
>> Regarding making Autoscaling factors configurable, I'm +1 for the fix,
>> just need to check who would like to contribute for this.
>>
>> Thanks
>>
>
>
--
Imesh Gunaratne
Technical Lead, WSO2
Committer & PMC Member, Apache Stratos
