[ https://issues.apache.org/jira/browse/CLOUDSTACK-7703?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Anshul Gangwar resolved CLOUDSTACK-7703. ---------------------------------------- Resolution: Fixed > Cloudstack server endless loop when trying to create a volume while storage > pool is full > ---------------------------------------------------------------------------------------- > > Key: CLOUDSTACK-7703 > URL: https://issues.apache.org/jira/browse/CLOUDSTACK-7703 > Project: CloudStack > Issue Type: Bug > Security Level: Public(Anyone can view this level - this is the > default.) > Components: Management Server > Affects Versions: 4.3.0 > Environment: Centos 6.5 > Reporter: JF Vincent > Assignee: Anshul Gangwar > Priority: Critical > Fix For: 4.5.0 > > > When trying to create a VM, and thus a volume for it and the primary storage > is full (over 90%), the managament server enter in and endless loop (extract > below) and we have to restart it to exit this loop. > 2014-10-14 11:39:20,701 DEBUG [cloud.deploy.DeploymentPlanningManagerImpl] > (Job-Executor-10:ctx-02d42f8f ctx-e581af2c) No suitable pools found for > volume: Vol[5436|vm=5855|DATADISK] under cluster: 2 > 2014-10-14 11:39:20,702 DEBUG [cloud.deploy.DeploymentPlanningManagerImpl] > (Job-Executor-10:ctx-02d42f8f ctx-e581af2c) No suitable pools found > 2014-10-14 11:39:20,702 DEBUG [cloud.deploy.DeploymentPlanningManagerImpl] > (Job-Executor-10:ctx-02d42f8f ctx-e581af2c) No suitable storagePools found > under this Cluster: 2 > 2014-10-14 11:39:20,705 DEBUG [cloud.deploy.DeploymentPlanningManagerImpl] > (Job-Executor-10:ctx-02d42f8f ctx-e581af2c) Could not find suitable > Deployment Destination for this VM under any clusters, returning. > 2014-10-14 11:39:20,705 DEBUG [cloud.deploy.FirstFitPlanner] > (Job-Executor-10:ctx-02d42f8f ctx-e581af2c) Searching all possible resources > under this Zone: 2 > 2014-10-14 11:39:20,705 DEBUG [cloud.deploy.FirstFitPlanner] > (Job-Executor-10:ctx-02d42f8f ctx-e581af2c) Listing clusters in order of > aggregate capacity, that have (atleast one host with) enough CPU and RAM > capacity under this Zone: 2 > 2014-10-14 11:39:20,707 DEBUG [cloud.deploy.FirstFitPlanner] > (Job-Executor-10:ctx-02d42f8f ctx-e581af2c) Removing from the clusterId list > these clusters from avoid set: [] > 2014-10-14 11:39:20,714 DEBUG [cloud.deploy.DeploymentPlanningManagerImpl] > (Job-Executor-10:ctx-02d42f8f ctx-e581af2c) Checking resources in Cluster: 2 > under Pod: 2 > 2014-10-14 11:39:20,714 DEBUG [allocator.impl.FirstFitAllocator] > (Job-Executor-10:ctx-02d42f8f ctx-e581af2c FirstFitRoutingAllocator) Looking > for hosts in dc: 2 pod:2 cluster:2 > 2014-10-14 11:39:20,716 DEBUG [allocator.impl.FirstFitAllocator] > (Job-Executor-10:ctx-02d42f8f ctx-e581af2c FirstFitRoutingAllocator) > FirstFitAllocator has 3 hosts to check for allocation: [Host[-79-Routing], > Host[-89-Routing], Host[-77-Routing]] > 2014-10-14 11:39:20,717 DEBUG [allocator.impl.FirstFitAllocator] > (Job-Executor-10:ctx-02d42f8f ctx-e581af2c FirstFitRoutingAllocator) Found 3 > hosts for allocation after prioritization: [Host[-79-Routing], > Host[-89-Routing], Host[-77-Routing]] > 2014-10-14 11:39:20,717 DEBUG [allocator.impl.FirstFitAllocator] > (Job-Executor-10:ctx-02d42f8f ctx-e581af2c FirstFitRoutingAllocator) Looking > for speed=500Mhz, Ram=500 > 2014-10-14 11:39:20,720 DEBUG [cloud.capacity.CapacityManagerImpl] > (Job-Executor-10:ctx-02d42f8f ctx-e581af2c FirstFitRoutingAllocator) Host: 79 > has cpu capability (cpu:8, speed:2399) to support requested CPU: 1 and > requested speed: 500 > 2014-10-14 11:39:20,720 DEBUG [cloud.capacity.CapacityManagerImpl] > (Job-Executor-10:ctx-02d42f8f ctx-e581af2c FirstFitRoutingAllocator) Checking > if host: 79 has enough capacity for requested CPU: 500 and requested RAM: > 524288000 , cpuOverprovisioningFactor: 4.0 > 2014-10-14 11:39:20,721 DEBUG [cloud.capacity.CapacityManagerImpl] > (Job-Executor-10:ctx-02d42f8f ctx-e581af2c FirstFitRoutingAllocator) Hosts's > actual total CPU: 19192 and CPU after applying overprovisioning: 76768 > 2014-10-14 11:39:20,721 DEBUG [cloud.capacity.CapacityManagerImpl] > (Job-Executor-10:ctx-02d42f8f ctx-e581af2c FirstFitRoutingAllocator) Free > CPU: 57268 , Requested CPU: 500 > 2014-10-14 11:39:20,721 DEBUG [cloud.capacity.CapacityManagerImpl] > (Job-Executor-10:ctx-02d42f8f ctx-e581af2c FirstFitRoutingAllocator) Free > RAM: 93916725248 , Requested RAM: 524288000 > 2014-10-14 11:39:20,721 DEBUG [cloud.capacity.CapacityManagerImpl] > (Job-Executor-10:ctx-02d42f8f ctx-e581af2c FirstFitRoutingAllocator) Host has > enough CPU and RAM available > 2014-10-14 11:39:20,721 DEBUG [cloud.capacity.CapacityManagerImpl] > (Job-Executor-10:ctx-02d42f8f ctx-e581af2c FirstFitRoutingAllocator) STATS: > Can alloc CPU from host: 79, used: 19000, reserved: 500, actual total: 19192, > total with overprovisioning: 76768; requested > cpu:500,alloc_from_last_host?:false ,considerReservedCapacity?: true > 2014-10-14 11:39:20,721 DEBUG [cloud.capacity.CapacityManagerImpl] > (Job-Executor-10:ctx-02d42f8f ctx-e581af2c FirstFitRoutingAllocator) STATS: > Can alloc MEM from host: 79, used: 17192452096, reserved: 524288000, total: > 111633465344; requested mem: 524288000,alloc_from_last_host?:false > ,considerReservedCapacity?: true > 2014-10-14 11:39:20,721 DEBUG [allocator.impl.FirstFitAllocator] > (Job-Executor-10:ctx-02d42f8f ctx-e581af2c FirstFitRoutingAllocator) Found a > suitable host, adding to list: 79 > 2014-10-14 11:39:20,724 DEBUG [cloud.capacity.CapacityManagerImpl] > (Job-Executor-10:ctx-02d42f8f ctx-e581af2c FirstFitRoutingAllocator) Host: 89 > has cpu capability (cpu:8, speed:2399) to support requested CPU: 1 and > requested speed: 500 > 2014-10-14 11:39:20,724 DEBUG [cloud.capacity.CapacityManagerImpl] > (Job-Executor-10:ctx-02d42f8f ctx-e581af2c FirstFitRoutingAllocator) Checking > if host: 89 has enough capacity for requested CPU: 500 and requested RAM: > 524288000 , cpuOverprovisioningFactor: 4.0 > 2014-10-14 11:39:20,725 DEBUG [cloud.capacity.CapacityManagerImpl] > (Job-Executor-10:ctx-02d42f8f ctx-e581af2c FirstFitRoutingAllocator) Hosts's > actual total CPU: 19192 and CPU after applying overprovisioning: 76768 > 2014-10-14 11:39:20,725 DEBUG [cloud.capacity.CapacityManagerImpl] > (Job-Executor-10:ctx-02d42f8f ctx-e581af2c FirstFitRoutingAllocator) Free > CPU: 66268 , Requested CPU: 500 > 2014-10-14 11:39:20,725 DEBUG [cloud.capacity.CapacityManagerImpl] > (Job-Executor-10:ctx-02d42f8f ctx-e581af2c FirstFitRoutingAllocator) Free > RAM: 101013487616 , Requested RAM: 524288000 > 2014-10-14 11:39:20,725 DEBUG [cloud.capacity.CapacityManagerImpl] > (Job-Executor-10:ctx-02d42f8f ctx-e581af2c FirstFitRoutingAllocator) Host has > enough CPU and RAM available > 2014-10-14 11:39:20,725 DEBUG [cloud.capacity.CapacityManagerImpl] > (Job-Executor-10:ctx-02d42f8f ctx-e581af2c FirstFitRoutingAllocator) STATS: > Can alloc CPU from host: 89, used: 10500, reserved: 0, actual total: 19192, > total with overprovisioning: 76768; requested > cpu:500,alloc_from_last_host?:false ,considerReservedCapacity?: true > 2014-10-14 11:39:20,725 DEBUG [cloud.capacity.CapacityManagerImpl] > (Job-Executor-10:ctx-02d42f8f ctx-e581af2c FirstFitRoutingAllocator) STATS: > Can alloc MEM from host: 89, used: 10619977728, reserved: 0, total: > 111633465344; requested mem: 524288000,alloc_from_last_host?:false > ,considerReservedCapacity?: true > 2014-10-14 11:39:20,725 DEBUG [allocator.impl.FirstFitAllocator] > (Job-Executor-10:ctx-02d42f8f ctx-e581af2c FirstFitRoutingAllocator) Found a > suitable host, adding to list: 89 > 2014-10-14 11:39:20,727 DEBUG [cloud.capacity.CapacityManagerImpl] > (Job-Executor-10:ctx-02d42f8f ctx-e581af2c FirstFitRoutingAllocator) Host: 77 > has cpu capability (cpu:8, speed:2399) to support requested CPU: 1 and > requested speed: 500 > 2014-10-14 11:39:20,728 DEBUG [cloud.capacity.CapacityManagerImpl] > (Job-Executor-10:ctx-02d42f8f ctx-e581af2c FirstFitRoutingAllocator) Checking > if host: 77 has enough capacity for requested CPU: 500 and requested RAM: > 524288000 , cpuOverprovisioningFactor: 4.0 > 2014-10-14 11:39:20,728 DEBUG [cloud.capacity.CapacityManagerImpl] > (Job-Executor-10:ctx-02d42f8f ctx-e581af2c FirstFitRoutingAllocator) Hosts's > actual total CPU: 19192 and CPU after applying overprovisioning: 76768 > 2014-10-14 11:39:20,728 DEBUG [cloud.capacity.CapacityManagerImpl] > (Job-Executor-10:ctx-02d42f8f ctx-e581af2c FirstFitRoutingAllocator) Free > CPU: 59768 , Requested CPU: 500 > 2014-10-14 11:39:20,728 DEBUG [cloud.capacity.CapacityManagerImpl] > (Job-Executor-10:ctx-02d42f8f ctx-e581af2c FirstFitRoutingAllocator) Free > RAM: 95586058240 , Requested RAM: 524288000 > 2014-10-14 11:39:20,728 DEBUG [cloud.capacity.CapacityManagerImpl] > (Job-Executor-10:ctx-02d42f8f ctx-e581af2c FirstFitRoutingAllocator) Host has > enough CPU and RAM available > 2014-10-14 11:39:20,729 DEBUG [cloud.capacity.CapacityManagerImpl] > (Job-Executor-10:ctx-02d42f8f ctx-e581af2c FirstFitRoutingAllocator) STATS: > Can alloc CPU from host: 77, used: 11500, reserved: 5500, actual total: > 19192, total with overprovisioning: 76768; requested > cpu:500,alloc_from_last_host?:false ,considerReservedCapacity?: true > 2014-10-14 11:39:20,729 DEBUG [cloud.capacity.CapacityManagerImpl] > (Job-Executor-10:ctx-02d42f8f ctx-e581af2c FirstFitRoutingAllocator) STATS: > Can alloc MEM from host: 77, used: 10280239104, reserved: 5767168000, total: > 111633465344; requested mem: 524288000,alloc_from_last_host?:false > ,considerReservedCapacity?: true > 2014-10-14 11:39:20,729 DEBUG [allocator.impl.FirstFitAllocator] > (Job-Executor-10:ctx-02d42f8f ctx-e581af2c FirstFitRoutingAllocator) Found a > suitable host, adding to list: 77 -- This message was sent by Atlassian JIRA (v6.3.4#6332)