[
https://issues.apache.org/jira/browse/CLOUDSTACK-3441?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13742030#comment-13742030
]
Sowmya Krishnan commented on CLOUDSTACK-3441:
---------------------------------------------
I still see this issue with basic zone, 20K Hosts at < 1K VMs.
2013-08-16 00:15:17,184 DEBUG [cloud.capacity.CapacityManagerImpl]
(Job-Executor-586:job-1792 = [ e83085d7-3ef2-44bf-bd53-0df49664f598 ]) RAM
STATS after allocation: for host: 246, old used: 0, old reserved: 0, total:
8589934592; new used: 6815744000, reserved: 0; requested mem:
6815744000,alloc_from_last:false
2013-08-16 00:15:17,195 DEBUG [cloud.vm.VirtualMachineManagerImpl]
(Job-Executor-586:job-1792 = [ e83085d7-3ef2-44bf-bd53-0df49664f598 ]) VM is
being created in podId: 25
2013-08-16 00:19:37,516 DEBUG [cloud.network.NetworkManagerImpl]
(Job-Executor-586:job-1792 = [ e83085d7-3ef2-44bf-bd53-0df49664f598 ]) Lock is
acquired for network id 204 as a part of network implement
2013-08-16 00:19:37,585 DEBUG [cloud.network.NetworkManagerImpl]
(Job-Executor-586:job-1792 = [ e83085d7-3ef2-44bf-bd53-0df49664f598 ]) Network
id=204 is already implemented
2013-08-16 00:19:37,587 DEBUG [cloud.network.NetworkManagerImpl]
(Job-Executor-586:job-1792 = [ e83085d7-3ef2-44bf-bd53-0df49664f598 ]) Lock is
released for network id 204 as a part of network implement
...
...
2013-08-16 00:25:48,958 DEBUG [cloud.capacity.CapacityManagerImpl]
(Job-Executor-586:job-1792 = [ e83085d7-3ef2-44bf-bd53-0df49664f598 ]) CPU
STATS after allocation: for host: 244, old used
: 0, old reserved: 0, actual total: 32000, total with overprovisioning: 32000;
new used:500, reserved:0; requested cpu:500,alloc_from_last:false
2013-08-16 00:25:48,958 DEBUG [cloud.capacity.CapacityManagerImpl]
(Job-Executor-586:job-1792 = [ e83085d7-3ef2-44bf-bd53-0df49664f598 ]) RAM
STATS after allocation: for host: 244, old used
: 0, old reserved: 0, total: 8589934592; new used: 134217728, reserved: 0;
requested mem: 134217728,alloc_from_last:false
2013-08-16 00:25:49,011 DEBUG [cloud.vm.VirtualMachineManagerImpl]
(Job-Executor-586:job-1792 = [ e83085d7-3ef2-44bf-bd53-0df49664f598 ]) VM is
being created in podId: 25
2013-08-16 00:32:59,693 DEBUG [cloud.network.NetworkManagerImpl]
(Job-Executor-586:job-1792 = [ e83085d7-3ef2-44bf-bd53-0df49664f598 ]) Lock is
acquired for network id 204 as a part of netw
ork implement
2013-08-16 00:32:59,710 DEBUG [cloud.network.NetworkManagerImpl]
(Job-Executor-586:job-1792 = [ e83085d7-3ef2-44bf-bd53-0df49664f598 ]) Network
id=204 is already implemented
2013-08-16 00:32:59,711 DEBUG [cloud.network.NetworkManagerImpl]
(Job-Executor-586:job-1792 = [ e83085d7-3ef2-44bf-bd53-0df49664f598 ]) Lock is
released for network id 204 as a part of netw
ork implement
mysql> select id, job_cmd, job_status, created, last_updated from async_job
where id = 1792\G
*************************** 1. row ***************************
id: 1792
job_cmd: org.apache.cloudstack.api.command.user.vm.DeployVMCmd
job_status: 1
created: 2013-08-16 07:14:59
last_updated: 2013-08-16 07:33:25
1 row in set (0.00 sec)
Attached the job log for the deploy VM which took 18 mins to
complete:job-1792.log
> [Load Test] High delays between VM being allocated to Pod and network
> implementation causing delays in VM deployment
> --------------------------------------------------------------------------------------------------------------------
>
> Key: CLOUDSTACK-3441
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-3441
> Project: CloudStack
> Issue Type: Bug
> Security Level: Public(Anyone can view this level - this is the
> default.)
> Components: Management Server, Network Controller
> Affects Versions: 4.2.0
> Environment: Load test, with simulator setup
> Reporter: Sowmya Krishnan
> Assignee: Koushik Das
> Fix For: 4.2.0
>
> Attachments: job-3014_5mins
>
>
> Scale Test set up with 20K simulated hosts, 20K VMs, basic zone, 3
> Management servers
> Also using security groups, ~5K VMs are deployed in every Security group.
> While deployment of simulator VMs, following is observed after around 4K VMs:
> There's delay between VM being allocated to Pod and network implementation
> causing delays in VM deployment. Following are the logs:
> 2013-06-10 15:51:01,215 DEBUG [cloud.vm.VirtualMachineManagerImpl]
> (Job-Executor-1004:job-3014) VM is being created in podId: 1390
> 2013-06-10 15:55:31,494 DEBUG [cloud.network.NetworkManagerImpl]
> (Job-Executor-1004:job-3014) Lock is acquired for network id 204 as a part of
> network implement
> This causes delay in VM deployment and the async job to complete. The delays
> get higher and higher as more and more VMs are deployed causing an
> unacceptable delay for huge deployments, viz, > 15-20K VMs
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
