就怕的是这个,错误的顺序删除了主机,之后的问题就不好解决了。 集群的主机都删除了,但存储还在?想去存储里删除卷的时候,系统也在不停尝试expung,其实还是需要主机的命令,以及连接。 你试试加个主机吧这个集群恢复看看。 实在不行再改数据库。删除vm,数据卷的操作不是问题,但删除存储节点可能会有问题。
刘宇超 Richard Liu 发件人: 王磊 发送时间: 2013-11-20 11:41:56 收件人: users-cn 抄送: 主题: 回复: Re: 回复: Re: 回复: Re: 实例一直是Expunging状态,无法删除 下面为日志,麻烦了 2013-11-20 11:03:38,338 INFO [cloud.vm.UserVmManagerImpl] (UserVm-Scavenger-1:null) Found 6 vms to expunge. 2013-11-20 11:03:38,338 DEBUG [cloud.vm.VirtualMachineManagerImpl] (UserVm-Scavenger-1:null) Stopped called on VM[User|windows-2008-R2-X64] but the state is Expunging 2013-11-20 11:03:38,345 DEBUG [cloud.capacity.CapacityManagerImpl] (UserVm-Scavenger-1:null) VM state transitted from :Expunging to Expunging with event: ExpungeOperationvm's original host id: 5 new host id: null host id before state transition: null 2013-11-20 11:03:38,345 DEBUG [cloud.vm.VirtualMachineManagerImpl] (UserVm-Scavenger-1:null) Destroying vm VM[User|windows-2008-R2-X64] 2013-11-20 11:03:38,345 DEBUG [cloud.vm.VirtualMachineManagerImpl] (UserVm-Scavenger-1:null) Cleaning up NICS 2013-11-20 11:03:38,345 DEBUG [cloud.network.NetworkManagerImpl] (UserVm-Scavenger-1:null) Cleaning network for vm: 50 2013-11-20 11:03:38,348 DEBUG [cloud.storage.VolumeManagerImpl] (UserVm-Scavenger-1:null) Cleaning storage for vm: 50 2013-11-20 11:03:38,352 DEBUG [db.Transaction.Transaction] (UserVm-Scavenger-1:null) Rolling back the transaction: Time = 2 Name = cleanupVolumes; called by -Transaction.rollback:898-Transaction.removeUpTo:841-Transaction.close:665-TransactionContextBuilder.interceptException:63-ComponentInstantiationPostProcessor$InterceptorDispatcher.intercept:133-VirtualMachineManagerImpl.advanceExpunge:479-UserVmManagerImpl.expunge:1600-UserVmManagerImpl$ExpungeTask.run:1769-Executors$RunnableAdapter.call:471-FutureTask$Sync.innerRunAndReset:351-FutureTask.runAndReset:178-ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201:165 2013-11-20 11:03:38,352 WARN [cloud.vm.UserVmManagerImpl] (UserVm-Scavenger-1:null) Unable to expunge VM[User|windows-2008-R2-X64] com.cloud.utils.exception.CloudRuntimeException: Unable to locate datastore with id 3 at org.apache.cloudstack.storage.datastore.manager.PrimaryDataStoreProviderManagerImpl.getPrimaryDataStore(PrimaryDataStoreProviderManagerImpl.java:60) at org.apache.cloudstack.storage.datastore.DataStoreManagerImpl.getDataStore(DataStoreManagerImpl.java:45) at org.apache.cloudstack.storage.volume.VolumeDataFactoryImpl.getVolume(VolumeDataFactoryImpl.java:89) at org.apache.cloudstack.storage.volume.VolumeServiceImpl.destroyVolume(VolumeServiceImpl.java:581) at com.cloud.storage.VolumeManagerImpl.cleanupVolumes(VolumeManagerImpl.java:2145) at com.cloud.utils.component.ComponentInstantiationPostProcessor$InterceptorDispatcher.intercept(ComponentInstantiationPostProcessor.java:125) at com.cloud.vm.VirtualMachineManagerImpl.advanceExpunge(VirtualMachineManagerImpl.java:479) at com.cloud.vm.UserVmManagerImpl.expunge(UserVmManagerImpl.java:1600) at com.cloud.vm.UserVmManagerImpl$ExpungeTask.run(UserVmManagerImpl.java:1769) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) at java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:351) at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:178) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:165) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:267) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1146) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:701) 2013-11-20 11:03:38,353 DEBUG [cloud.vm.VirtualMachineManagerImpl] (UserVm-Scavenger-1:null) Stopped called on VM[User|win2008-1] but the state is Expunging 2013-11-20 11:03:38,365 DEBUG [cloud.capacity.CapacityManagerImpl] (UserVm-Scavenger-1:null) VM state transitted from :Expunging to Expunging with event: ExpungeOperationvm's original host id: 7 new host id: null host id before state transition: null 2013-11-20 11:03:38,365 DEBUG [cloud.vm.VirtualMachineManagerImpl] (UserVm-Scavenger-1:null) Destroying vm VM[User|win2008-1] 2013-11-20 11:03:38,365 DEBUG [cloud.vm.VirtualMachineManagerImpl] (UserVm-Scavenger-1:null) Cleaning up NICS 2013-11-20 11:03:38,365 DEBUG [cloud.network.NetworkManagerImpl] (UserVm-Scavenger-1:null) Cleaning network for vm: 52 2013-11-20 11:03:38,368 DEBUG [cloud.storage.VolumeManagerImpl] (UserVm-Scavenger-1:null) Cleaning storage for vm: 52 2013-11-20 11:03:38,371 DEBUG [db.Transaction.Transaction] (UserVm-Scavenger-1:null) Rolling back the transaction: Time = 2 Name = cleanupVolumes; called by -Transaction.rollback:898-Transaction.removeUpTo:841-Transaction.close:665-TransactionContextBuilder.interceptException:63-ComponentInstantiationPostProcessor$InterceptorDispatcher.intercept:133-VirtualMachineManagerImpl.advanceExpunge:479-UserVmManagerImpl.expunge:1600-UserVmManagerImpl$ExpungeTask.run:1769-Executors$RunnableAdapter.call:471-FutureTask$Sync.innerRunAndReset:351-FutureTask.runAndReset:178-ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201:165 2013-11-20 11:03:38,372 WARN [cloud.vm.UserVmManagerImpl] (UserVm-Scavenger-1:null) Unable to expunge VM[User|win2008-1] com.cloud.utils.exception.CloudRuntimeException: Unable to locate datastore with id 5 at org.apache.cloudstack.storage.datastore.manager.PrimaryDataStoreProviderManagerImpl.getPrimaryDataStore(PrimaryDataStoreProviderManagerImpl.java:60) at org.apache.cloudstack.storage.datastore.DataStoreManagerImpl.getDataStore(DataStoreManagerImpl.java:45) at org.apache.cloudstack.storage.volume.VolumeDataFactoryImpl.getVolume(VolumeDataFactoryImpl.java:89) at org.apache.cloudstack.storage.volume.VolumeServiceImpl.destroyVolume(VolumeServiceImpl.java:581) at com.cloud.storage.VolumeManagerImpl.cleanupVolumes(VolumeManagerImpl.java:2145) at com.cloud.utils.component.ComponentInstantiationPostProcessor$InterceptorDispatcher.intercept(ComponentInstantiationPostProcessor.java:125) at com.cloud.vm.VirtualMachineManagerImpl.advanceExpunge(VirtualMachineManagerImpl.java:479) at com.cloud.vm.UserVmManagerImpl.expunge(UserVmManagerImpl.java:1600) at com.cloud.vm.UserVmManagerImpl$ExpungeTask.run(UserVmManagerImpl.java:1769) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) at java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:351) at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:178) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:165) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:267) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1146) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:701) 发件人:tanthalas <tanthalas...@hotmail.com> 发送时间:2013-11-20 10:29 主题:Re: 回复: Re: 回复: Re: 实例一直是Expunging状态,无法删除 收件人:"users-cn"<users-cn@cloudstack.apache.org> 抄送: 因为你先违规操作,先删除的主机,但你给的信息还不够,不够判断。 这个集群所有主机都删除了吗?还只是随意删除一台? 但是数据卷不能删除,我感觉问题比较大,log里肯定有对应信息的。 刘宇超 Richard Liu 发件人: 王磊 发送时间: 2013-11-20 10:06:05 收件人: users-cn 抄送: 主题: 回复: Re: 回复: Re: 实例一直是Expunging状态,无法删除 expunge.delay和expunge.interval我已经全都改成600了 发件人:Du Jun <dj199...@gmail.com> 发送时间:2013-11-20 09:19 主题:Re: 回复: Re: 实例一直是Expunging状态,无法删除 收件人:"users-cn"<users-cn@cloudstack.apache.org> 抄送: 实际上你已经删除实例了,只是CS会将删除的实例保存一段时间。你可以在global settings里面搜索expuge关键字,设置expuge的时间,比如120。那么,过2分钟就会在UI上面移除虚拟机。 在 2013年11月19日下午4:23,toudsf <18600601...@163.com>写道: > 到数据库中,有一个表instance吧好像。把状态变为Stopped然后从画面进行删除。 > > 2013-11-19 > > > > toudsf > > > > 发件人:"王磊" <wang...@smartdot.com> > 发送时间:2013-11-19 16:18 > 主题:回复: Re: 实例一直是Expunging状态,无法删除 > 收件人:"users-cn"<users-cn@cloudstack.apache.org> > 抄送: > > 没有错误提示,页面显示删除成功,但就是无法彻底删除,一直在Expunging,从实例点进去也一直在灰色页面转圈圈没有任何显示 > > 实例对应的ip已经释放 > > 实例对应的存储也无法删除 > > > > > > > 发件人:Wei ZHOU <ustcweiz...@gmail.com> > 发送时间:2013-11-19 16:04 > 主题:Re: 实例一直是Expunging状态,无法删除 > 收件人:"users-cn"<users-cn@cloudstack.apache.org> > 抄送: > > 有什么错误提示吗? > 直接改也问题不大 > > > 在 2013年11月19日上午9:02,王磊 <wang...@smartdot.com>写道: > > > 没有正常删除主机(直接把主机格式化重做系统,后在管理端删除的主机和实例),导致实例一直是Expunging状态,无法删除 > > > > expunge.delay和expunge.interval都已经改成了600,其他实例都可以正常删除。 > > > > 应该怎么做,直接到数据库里操作吗? >