Hi All,
I also observed diego reported issue [1] at ELB startup time and before
issue occurring followed following steps
Steps:
1. Started IS 500 fronted by ELB 2.1.0 server
2. Configure IS to try out scenarios [2]
3. Then loaded the scenario nearly with 300 TPS
After some time observed WARN - PassThroughHttpSSLListener System may be
unstable: HTTPS ListeningIOReactor encountered a checked exception : Too
many open files
java.io.IOException: Too many open files on ELB side. and
ERROR - Axis2Sender Unexpected error during sending message out
java.lang.IllegalStateException: I/O reactor has been shut down
4. Then shutdown the ELB server due to above errors and increase file
limits on /etc/security/limits.conf file
5. When restarting downed ELB server observed issue [1]
6. After clearing and replacing content on /repository/deployment/server
folder from a fresh pack content , we were able to resolve the issue[1]
It seams content on /repository/deployment/server folder have corrupted due
to the load or too many open files issue.
Can anyone have an idea on root cause of this issue and provide a proper
solution than replacing the content.?
[1]
[2014-09-02 09:14:46,993] FATAL - LoadBalanceEndpointServiceComponent
Failed to register Tenant Aware Load Balance Endpoint in Send Mediator.
[2014-09-02 09:14:46,994] ERROR - LoadBalanceEndpointServiceComponent
Failed to activate Endpoint Admin bundle
org.apache.synapse.SynapseException: Failed to register Tenant Aware Load
Balance Endpoint in Send Mediator.
at
org.wso2.carbon.lb.endpoint.internal.LoadBalanceEndpointServiceComponent.activate(LoadBalanceEndpointServiceComponent.java:191)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at
org.eclipse.equinox.internal.ds.model.ServiceComponent.activate(ServiceComponent.java:260)
at
org.eclipse.equinox.internal.ds.model.ServiceComponentProp.activate(ServiceComponentProp.java:146)
at
org.eclipse.equinox.internal.ds.model.ServiceComponentProp.build(ServiceComponentProp.java:347)
at
org.eclipse.equinox.internal.ds.InstanceProcess.buildComponent(InstanceProcess.java:620)
at
org.eclipse.equinox.internal.ds.InstanceProcess.buildComponents(InstanceProcess.java:197)
at
org.eclipse.equinox.internal.ds.Resolver.getEligible(Resolver.java:343)
at
org.eclipse.equinox.internal.ds.SCRManager.serviceChanged(SCRManager.java:222)
at
org.eclipse.osgi.internal.serviceregistry.FilteredServiceListener.serviceChanged(FilteredServiceListener.java:107)
at
org.eclipse.osgi.framework.internal.core.BundleContextImpl.dispatchEvent(BundleContextImpl.java:861)
at
org.eclipse.osgi.framework.eventmgr.EventManager.dispatchEvent(EventManager.java:230)
at
org.eclipse.osgi.framework.eventmgr.ListenerQueue.dispatchEventSynchronous(ListenerQueue.java:148)
at
org.eclipse.osgi.internal.serviceregistry.ServiceRegistry.publishServiceEventPrivileged(ServiceRegistry.java:819)
at
org.eclipse.osgi.internal.serviceregistry.ServiceRegistry.publishServiceEvent(ServiceRegistry.java:771)
at
org.eclipse.osgi.internal.serviceregistry.ServiceRegistrationImpl.register(ServiceRegistrationImpl.java:130)
at
org.eclipse.osgi.internal.serviceregistry.ServiceRegistry.registerService(ServiceRegistry.java:214)
at
org.eclipse.osgi.framework.internal.core.BundleContextImpl.registerService(BundleContextImpl.java:433)
at
org.eclipse.osgi.framework.internal.core.BundleContextImpl.registerService(BundleContextImpl.java:451)
at
org.wso2.carbon.mediation.initializer.ServiceBusInitializer.activate(ServiceBusInitializer.java:245)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at
org.eclipse.equinox.internal.ds.model.ServiceComponent.activate(ServiceComponent.java:260)
at
org.eclipse.equinox.internal.ds.model.ServiceComponentProp.activate(ServiceComponentProp.java:146)
at
org.eclipse.equinox.internal.ds.model.ServiceComponentProp.build(ServiceComponentProp.java:347)
at
org.eclipse.equinox.internal.ds.InstanceProcess.buildComponent(InstanceProcess.java:620)
at
org.eclipse.equinox.internal.ds.InstanceProcess.buildComponents(InstanceProcess.java:197)
at
org.eclipse.equinox.internal.ds.Resolver.getEligible(Resolver.java:343)
at
org.eclipse.equinox.internal.ds.SCRManager.serviceChanged(SCRManager.java:222)
at
org.eclipse.osgi.internal.serviceregistry.FilteredServiceListener.serviceChanged(FilteredServiceListener.java:107)
at
org.eclipse.osgi.framework.internal.core.BundleContextImpl.dispatchEvent(BundleContextImpl.java:861)
at
org.eclipse.osgi.framework.eventmgr.EventManager.dispatchEvent(EventManager.java:230)
at
org.eclipse.osgi.framework.eventmgr.ListenerQueue.dispatchEventSynchronous(ListenerQueue.java:148)
at
org.eclipse.osgi.internal.serviceregistry.ServiceRegistry.publishServiceEventPrivileged(ServiceRegistry.java:819)
at
org.eclipse.osgi.internal.serviceregistry.ServiceRegistry.publishServiceEvent(ServiceRegistry.java:771)
at
org.eclipse.osgi.internal.serviceregistry.ServiceRegistrationImpl.register(ServiceRegistrationImpl.java:130)
at
org.eclipse.osgi.internal.serviceregistry.ServiceRegistry.registerService(ServiceRegistry.java:214)
at
org.eclipse.osgi.framework.internal.core.BundleContextImpl.registerService(BundleContextImpl.java:433)
at
org.eclipse.osgi.framework.internal.core.BundleContextImpl.registerService(BundleContextImpl.java:451)
at
org.wso2.carbon.event.core.internal.builder.EventBrokerHandler.startEventBroker(EventBrokerHandler.java:58)
at
org.wso2.carbon.event.core.internal.builder.EventBrokerBuilderDS.activate(EventBrokerBuilderDS.java:72)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at
org.eclipse.equinox.internal.ds.model.ServiceComponent.activate(ServiceComponent.java:260)
at
org.eclipse.equinox.internal.ds.model.ServiceComponentProp.activate(ServiceComponentProp.java:146)
at
org.eclipse.equinox.internal.ds.model.ServiceComponentProp.build(ServiceComponentProp.java:347)
at
org.eclipse.equinox.internal.ds.InstanceProcess.buildComponent(InstanceProcess.java:620)
at
org.eclipse.equinox.internal.ds.InstanceProcess.buildComponents(InstanceProcess.java:197)
at
org.eclipse.equinox.internal.ds.Resolver.getEligible(Resolver.java:343)
at
org.eclipse.equinox.internal.ds.SCRManager.serviceChanged(SCRManager.java:222)
at
org.eclipse.osgi.internal.serviceregistry.FilteredServiceListener.serviceChanged(FilteredServiceListener.java:107)
at
org.eclipse.osgi.framework.internal.core.BundleContextImpl.dispatchEvent(BundleContextImpl.java:861)
at
org.eclipse.osgi.framework.eventmgr.EventManager.dispatchEvent(EventManager.java:230)
at
org.eclipse.osgi.framework.eventmgr.ListenerQueue.dispatchEventSynchronous(ListenerQueue.java:148)
at
org.eclipse.osgi.internal.serviceregistry.ServiceRegistry.publishServiceEventPrivileged(ServiceRegistry.java:819)
at
org.eclipse.osgi.internal.serviceregistry.ServiceRegistry.publishServiceEvent(ServiceRegistry.java:771)
at
org.eclipse.osgi.internal.serviceregistry.ServiceRegistrationImpl.register(ServiceRegistrationImpl.java:130)
at
org.eclipse.osgi.internal.serviceregistry.ServiceRegistry.registerService(ServiceRegistry.java:214)
at
org.eclipse.osgi.framework.internal.core.BundleContextImpl.registerService(BundleContextImpl.java:433)
at
org.eclipse.osgi.framework.internal.core.BundleContextImpl.registerService(BundleContextImpl.java:451)
at
org.wso2.carbon.core.init.CarbonServerManager.initializeCarbon(CarbonServerManager.java:517)
at
org.wso2.carbon.core.init.CarbonServerManager.start(CarbonServerManager.java:219)
at
org.wso2.carbon.core.internal.CarbonCoreServiceComponent.activate(CarbonCoreServiceComponent.java:77)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at
org.eclipse.equinox.internal.ds.model.ServiceComponent.activate(ServiceComponent.java:260)
at
org.eclipse.equinox.internal.ds.model.ServiceComponentProp.activate(ServiceComponentProp.java:146)
at
org.eclipse.equinox.internal.ds.model.ServiceComponentProp.build(ServiceComponentProp.java:347)
at
org.eclipse.equinox.internal.ds.InstanceProcess.buildComponent(InstanceProcess.java:620)
at
org.eclipse.equinox.internal.ds.InstanceProcess.buildComponents(InstanceProcess.java:197)
at
org.eclipse.equinox.internal.ds.Resolver.getEligible(Resolver.java:343)
at
org.eclipse.equinox.internal.ds.SCRManager.serviceChanged(SCRManager.java:222)
at
org.eclipse.osgi.internal.serviceregistry.FilteredServiceListener.serviceChanged(FilteredServiceListener.java:107)
at
org.eclipse.osgi.framework.internal.core.BundleContextImpl.dispatchEvent(BundleContextImpl.java:861)
at
org.eclipse.osgi.framework.eventmgr.EventManager.dispatchEvent(EventManager.java:230)
at
org.eclipse.osgi.framework.eventmgr.ListenerQueue.dispatchEventSynchronous(ListenerQueue.java:148)
at
org.eclipse.osgi.internal.serviceregistry.ServiceRegistry.publishServiceEventPrivileged(ServiceRegistry.java:819)
at
org.eclipse.osgi.internal.serviceregistry.ServiceRegistry.publishServiceEvent(ServiceRegistry.java:771)
at
org.eclipse.osgi.internal.serviceregistry.ServiceRegistrationImpl.register(ServiceRegistrationImpl.java:130)
at
org.eclipse.osgi.internal.serviceregistry.ServiceRegistry.registerService(ServiceRegistry.java:214)
at
org.eclipse.osgi.framework.internal.core.BundleContextImpl.registerService(BundleContextImpl.java:433)
at
org.eclipse.equinox.http.servlet.internal.Activator.registerHttpService(Activator.java:81)
at
org.eclipse.equinox.http.servlet.internal.Activator.addProxyServlet(Activator.java:60)
at
org.eclipse.equinox.http.servlet.internal.ProxyServlet.init(ProxyServlet.java:40)
at
org.wso2.carbon.tomcat.ext.servlet.DelegationServlet.init(DelegationServlet.java:38)
at
org.apache.catalina.core.StandardWrapper.initServlet(StandardWrapper.java:1267)
at
org.apache.catalina.core.StandardWrapper.loadServlet(StandardWrapper.java:1186)
at
org.apache.catalina.core.StandardWrapper.load(StandardWrapper.java:1081)
at
org.apache.catalina.core.StandardContext.loadOnStartup(StandardContext.java:5027)
at
org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5314)
at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:150)
at
org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1559)
at
org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1549)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:722)
[2]
https://docs.wso2.com/display/IS500/OpenID+Connect+with+the+WSO2+Identity+Server+and+WSO2+OAuth2+Playground
On Thu, Feb 6, 2014 at 3:26 PM, Manoj Kumara <ma...@wso2.com> wrote:
> Hi Diego,
>
> ELB 2.1.0 was released under Carbon 4.2.0 latest version and during Carbon
> 4.2.0 release the clustering implementation has changed from Axis2
> clustering to Hazelcast.
>
> I don't recommend you to test your AS 5.1.0 to be used with ELB 2.1.0. You
> can either use ESB 2.0.3 with AS 5.1.0 or you can use the latest AS 5.2.1
> version on your environment.
>
> I think that may be one issue on your environment. You can find our
> release-matrix on [1],
>
> [1] http://wso2.com/products/carbon/release-matrix/
>
> Thanks,
> Manoj
>
>
> *Manoj Kumara*
> Software Engineer
> WSO2 Inc. http://wso2.com/
> *lean.enterprise.middleware*
> Mobile: +94713448188
>
>
> On Thu, Feb 6, 2014 at 3:14 PM, Diego <dseoan...@gmail.com> wrote:
>
>> Hi all!
>> We have a development environment with a running cluster of ESB and AS
>> using both ELB 2.0.3 (for ES 4.7.0) and ELB 2.1.0 (for AS 5.1.0).
>>
>> Now we need to deploy a testing environment with the same configuration,
>> from a fresh installation, and when we try to start ELB 2.0.3 are getting
>> this FATAL error:
>>
>> [2014-02-05 15:10:07,031] FATAL - LoadBalanceEndpointServiceComponent
>> Failed to register Tenant Aware Load Balance Endpoint in Send Mediator.
>>
>> [2014-02-05 15:10:07,038] ERROR - LoadBalanceEndpointServiceComponent
>> Failed to activate Endpoint Admin bundle
>>
>> org.wso2.carbon.lb.endpoint.TenantAwareLoadBalanceEndpointException:
>> Failed to register Tenant Aware Load Balance Endpoint in Send Mediator.
>>
>> The only config file we changed was the "loadbalancer.conf", and setted
>> it exactly the same as the file on the development environment (actually is
>> a copy from the Development Environmet loadbalancer.conf file with the
>> domains and hostnames changed)
>>
>>
>> Any clues on what's going on?
>>
>>
>> Thank you very much!
>>
>> _______________________________________________
>> Dev mailing list
>> Dev@wso2.org
>> http://wso2.org/cgi-bin/mailman/listinfo/dev
>>
>>
>
> _______________________________________________
> Dev mailing list
> Dev@wso2.org
> http://wso2.org/cgi-bin/mailman/listinfo/dev
>
>
--
Asanka Vithanage
Senior Software Engineer -QA
Mobile: +94 0716286708
Email: asan...@wso2.com
WSO2 Inc. www.wso2.com
_______________________________________________
Dev mailing list
Dev@wso2.org
http://wso2.org/cgi-bin/mailman/listinfo/dev
