Rayees Namathponnan created CLOUDSTACK-5336:
-----------------------------------------------
Summary: [Automation] During regression automation management
server hang with "out of memory error"
Key: CLOUDSTACK-5336
URL: https://issues.apache.org/jira/browse/CLOUDSTACK-5336
Project: CloudStack
Issue Type: Bug
Security Level: Public (Anyone can view this level - this is the default.)
Components: Management Server
Affects Versions: 4.3.0
Environment: Vmware
Branch : 4.3
Reporter: Rayees Namathponnan
Priority: Blocker
Fix For: 4.3.0
Steps to reproduce
Steps to reproduce
Create build from 4.3 branch and install from RPM
Run BVT and Regression automation
Result
BVT complete without any issues; after start regression MS not responding,
observed below "out of memory error" in MS log
2013-12-01 20:06:27,300 ERROR [c.c.h.v.r.VmwareResource]
(DirectAgent-398:ctx-55572862 10.223.250.131) Unable to execute NetworkUsage
command on DomR (10.223.250.17
4), domR may not be ready yet. failure due to Exception:
java.lang.OutOfMemoryError
Message: Java heap space
java.lang.OutOfMemoryError: Java heap space
at java.util.Arrays.copyOf(Arrays.java:2219)
at java.util.ArrayList.grow(ArrayList.java:242)
at java.util.ArrayList.ensureExplicitCapacity(ArrayList.java:216)
at java.util.ArrayList.ensureCapacityInternal(ArrayList.java:208)
at java.util.ArrayList.add(ArrayList.java:440)
at
com.sun.xml.internal.ws.model.wsdl.WSDLOperationImpl.addFault(WSDLOperationImpl.java:126)
at
com.sun.xml.internal.ws.wsdl.parser.RuntimeWSDLParser.parsePortTypeOperationFault(RuntimeWSDLParser.java:740)
at
com.sun.xml.internal.ws.wsdl.parser.RuntimeWSDLParser.parsePortTypeOperation(RuntimeWSDLParser.java:727)
at
com.sun.xml.internal.ws.wsdl.parser.RuntimeWSDLParser.parsePortType(RuntimeWSDLParser.java:697)
at
com.sun.xml.internal.ws.wsdl.parser.RuntimeWSDLParser.parseWSDL(RuntimeWSDLParser.java:340)
at
com.sun.xml.internal.ws.wsdl.parser.RuntimeWSDLParser.parseImport(RuntimeWSDLParser.java:301)
at
com.sun.xml.internal.ws.wsdl.parser.RuntimeWSDLParser.parseImport(RuntimeWSDLParser.java:677)
at
com.sun.xml.internal.ws.wsdl.parser.RuntimeWSDLParser.parseWSDL(RuntimeWSDLParser.java:336)
at
com.sun.xml.internal.ws.wsdl.parser.RuntimeWSDLParser.parse(RuntimeWSDLParser.java:157)
at
com.sun.xml.internal.ws.wsdl.parser.RuntimeWSDLParser.parse(RuntimeWSDLParser.java:120)
at
com.sun.xml.internal.ws.client.WSServiceDelegate.parseWSDL(WSServiceDelegate.java:257)
at
com.sun.xml.internal.ws.client.WSServiceDelegate.<init>(WSServiceDelegate.java:220)
at
com.sun.xml.internal.ws.client.WSServiceDelegate.<init>(WSServiceDelegate.java:168)
at
com.sun.xml.internal.ws.spi.ProviderImpl.createServiceDelegate(ProviderImpl.java:96)
at javax.xml.ws.Service.<init>(Service.java:77)
at com.vmware.vim25.VimService.<init>(VimService.java:46)
at
com.cloud.hypervisor.vmware.util.VmwareClient.connect(VmwareClient.java:129)
at
com.cloud.hypervisor.vmware.resource.VmwareContextFactory.create(VmwareContextFactory.java:67)
at
com.cloud.hypervisor.vmware.resource.VmwareContextFactory.getContext(VmwareContextFactory.java:85)
at
com.cloud.hypervisor.vmware.resource.VmwareResource.getServiceContext(VmwareResource.java:6879)
at
com.cloud.hypervisor.vmware.resource.VmwareResource.getServiceContext(VmwareResource.java:6861)
at
com.cloud.hypervisor.vmware.resource.VmwareResource.networkUsage(VmwareResource.java:6578)
at
com.cloud.hypervisor.vmware.resource.VmwareResource.getNetworkStats(VmwareResource.java:6596)
at
com.cloud.hypervisor.vmware.resource.VmwareResource.execute(VmwareResource.java:672)
at
com.cloud.hypervisor.vmware.resource.VmwareResource.executeRequest(VmwareResource.java:505)
at
com.cloud.agent.manager.DirectAgentAttache$Task.runInContext(DirectAgentAttache.java:216)
at
org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
2013-12-01 20:06:27,409 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl]
(RouterStatusMonitor-1:ctx-32e68e7c) Found 1 routers to update status.
2013-12-01 20:06:27,467 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl]
(RouterStatusMonitor-1:ctx-32e68e7c) Found 0 networks to update RvR status.
2013-12-01 20:06:27,391 DEBUG [c.c.s.StorageManagerImpl]
(StorageManager-Scavenger-4:ctx-e1826ff2) Storage pool garbage collector found
0 templates to clean up in storage pool: PS0
Environment information
[root@auto_ms2 management]# cat /etc/redhat-release
CentOS release 6.2 (Final)
[root@auto_ms2 management]# java -version
java version "1.7.0_45"
OpenJDK Runtime Environment (rhel-2.4.3.2.el6_4-x86_64 u45-b15)
OpenJDK 64-Bit Server VM (build 24.45-b08, mixed mode)
[root@auto_ms2 management]#
Attaching MS Log with this defect; i will create memory dump and attach with
this defect
--
This message was sent by Atlassian JIRA
(v6.1#6144)
