rbd磁盘卷做快照？
hi， 我使用cloudstack4.2，ceph做主存储， 现在对ceph上面的卷做快照（卷定义20G，实际空间1.3G），发现在二级存储上的快照文件比20G还大，是raw格式的文件。 但我直接手动使用qemu-img 将rbd文件导出来为raw格式的文件也就1.3G实际空间。 现在我想看看cloudstack源代码里面他是怎么做的这个快照， 找了半天，没找到这段代码在哪儿？ 谁能指点下 谢谢！！
can't create instance when use rbd
HI.i encounter some problem when use rbd. i have successful add rbd to cloudstack primary storage,but it can't create instance with rbd,this is the error infomation: management log: INFO [cloud.ha.HighAvailabilityManagerImpl] (HA-1:) checking health of usage server WARN [apache.cloudstack.alerts] (HA-1:) alertType:: 13 // dataCenterId:: 0 // podId:: 0 // clusterId:: null // message:: No usage server process running INFO [storage.volume.VolumeServiceImpl] (Job-Executor-1:job-36 = [ 0ec0796a-350a-49d4-9b53-11ad41ef7a6e ]) lock is acquired for VMTemplateStoragePool 6 INFO [storage.volume.VolumeServiceImpl] (Job-Executor-1:job-36 = [ 0ec0796a-350a-49d4-9b53-11ad41ef7a6e ]) releasing lock for VMTemplateStoragePool 6 WARN [utils.db.Merovingian2] (Job-Executor-1:job-36 = [ 0ec0796a-350a-49d4-9b53-11ad41ef7a6e ]) Was unable to find lock for the key template_spool_ref6 and thread id 2118921053 INFO [cloud.vm.VirtualMachineManagerImpl] (Job-Executor-1:job-36 = [ 0ec0796a-350a-49d4-9b53-11ad41ef7a6e ]) Unable to contact resource. com.cloud.exception.StorageUnavailableException: Resource [StoragePool:12] is unreachable: Unable to create Vol[19|vm=16|ROOT]:com.cloud.utils.exception.CloudRuntimeException: Failed to copy /mnt/0aedf226-58b4-3ec0-9c29-9f27e3748937/0fe1ab18-71ae-30e3-9738-2d82c5f525e0.qcow2 to 53b0066a-f7b9-4fcc-87e0-2d4dbd1b0db1 at com.cloud.storage.VolumeManagerImpl.recreateVolume(VolumeManagerImpl.java:2544) at com.cloud.storage.VolumeManagerImpl.prepare(VolumeManagerImpl.java:2592) at com.cloud.vm.VirtualMachineManagerImpl.advanceStart(VirtualMachineManagerImpl.java:889) at com.cloud.vm.VirtualMachineManagerImpl.start(VirtualMachineManagerImpl.java:578) at org.apache.cloudstack.engine.cloud.entity.api.VMEntityManagerImpl.deployVirtualMachine(VMEntityManagerImpl.java:227) at org.apache.cloudstack.engine.cloud.entity.api.VirtualMachineEntityImpl.deploy(VirtualMachineEntityImpl.java:209) at com.cloud.vm.UserVmManagerImpl.startVirtualMachine(UserVmManagerImpl.java:3406) at com.cloud.vm.UserVmManagerImpl.startVirtualMachine(UserVmManagerImpl.java:2966) at com.cloud.vm.UserVmManagerImpl.startVirtualMachine(UserVmManagerImpl.java:2952) at com.cloud.utils.component.ComponentInstantiationPostProcessor$InterceptorDispatcher.intercept(ComponentInstantiationPostProcessor.java:125) at org.apache.cloudstack.api.command.user.vm.DeployVMCmd.execute(DeployVMCmd.java:420) at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:158) at com.cloud.async.AsyncJobManagerImpl$1.run(AsyncJobManagerImpl.java:531) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) at java.util.concurrent.FutureTask.run(FutureTask.java:166) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1146) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:679) WARN [apache.cloudstack.alerts] (Job-Executor-1:job-36 = [ 0ec0796a-350a-49d4-9b53-11ad41ef7a6e ]) alertType:: 8 // dataCenterId:: 1 // podId:: 1 // clusterId:: null // message:: Failed to deploy Vm with Id: 16, on Host with Id: null INFO [user.vm.DeployVMCmd] (Job-Executor-1:job-36 = [ 0ec0796a-350a-49d4-9b53-11ad41ef7a6e ]) com.cloud.exception.InsufficientServerCapacityException: Unable to create a deployment for VM[User|2119a010-ea6e-473a-8183-d76cc00d4eb8]Scope=interface com.cloud.dc.DataCenter; id=1 INFO [user.vm.DeployVMCmd] (Job-Executor-1:job-36 = [ 0ec0796a-350a-49d4-9b53-11ad41ef7a6e ]) Unable to create a deployment for VM[User|2119a010-ea6e-473a-8183-d76cc00d4eb8] com.cloud.exception.InsufficientServerCapacityException: Unable to create a deployment for VM[User|2119a010-ea6e-473a-8183-d76cc00d4eb8]Scope=interface com.cloud.dc.DataCenter; id=1 at com.cloud.vm.VirtualMachineManagerImpl.advanceStart(VirtualMachineManagerImpl.java:842) at com.cloud.vm.VirtualMachineManagerImpl.start(VirtualMachineManagerImpl.java:578) at org.apache.cloudstack.engine.cloud.entity.api.VMEntityManagerImpl.deployVirtualMachine(VMEntityManagerImpl.java:237) at org.apache.cloudstack.engine.cloud.entity.api.VirtualMachineEntityImpl.deploy(VirtualMachineEntityImpl.java:209) at com.cloud.vm.UserVmManagerImpl.startVirtualMachine(UserVmManagerImpl.java:3406) at com.cloud.vm.UserVmManagerImpl.startVirtualMachine(UserVmManagerImpl.java:2966) at com.cloud.vm.UserVmManagerImpl.startVirtualMachine(UserVmManagerImpl.java:2952) at com.cloud.utils.component.ComponentInstantiationPostProcessor$InterceptorDispatcher.intercept(ComponentInstantiationPostProcessor.java:125) at org.apache.cloudstack.api.command.user.vm.DeployVMCmd.execute(DeployVMCmd.java:420) at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:158) at com.cloud.async.AsyncJobManagerImpl$1.run(AsyncJobManagerImpl.java:531) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask
Re: 回复：RE: 配置RBD作为主存储时，迁移出现 若干问题
4.2上支持的比较好，可以看下这部分相关的Release Notes： Snaphotting, Backups, Cloning and System VMs for RBD Primary Storage These new RBD features require at least librbd 0.61.7 (Cuttlefish) and libvirt 0.9.14 on the KVM hypervisors. This release of CloudStack will leverage the features of RBD format 2. This allows snapshotting and backing up those snapshots. Backups of snapshots to Secondary Storage are full copies of the RBD snapshot, they are not RBD diffs. This because when restoring a backup of a snapshot it is not mandatory that this backup is deployed on RBD again, it could also be a NFS Primary Storage. Another key feature of RBD format 2 is cloning. With this release templates will be copied to Primary Storage once and by using the cloning mechanism new disks will be cloned from this parent template. This saves space and decreases deployment time for instances dramatically. Before this release, a NFS Primary Storage was still required for running the System VMs from. The reason was a so called 'patch disk' that was generated by the hypervisor which contained metadata for the System VM. The scripts generating this disk didn't support RBD and thus System VMs had to be deployed from NFS. With 4.2 instead of the patch disk a VirtIO serial console is used to pass meta information to System VMs. This enabled the deployment of System VMs on RBD Primary Storage. 2013/10/23 WXR <474745...@qq.com> > 支持的吧，在4.1和4.2上都配置能运行虚拟机的。 > 原来我在4.1上测试的有问题，现在改用4.2，ceph也比原来版本新一点，现在这个问题好像已经不出现了。 > > > > > -- 原始邮件 -- > 发件人: "王耀锋";; > 发送时间: 2013年10月22日(星期二) 下午4:15 > 收件人: "users-cn@cloudstack.apache.org"; > > 主题: RE: 配置RBD作为主存储时，迁移出现 若干问题 > > > > cloudstack4.1 主存储还像不支持ceph？ > > > From: 474745...@qq.com > > To: users-cn@cloudstack.apache.org > > Subject: 配置RBD作为主存储时，迁移出现若干问题 > > Date: Wed, 16 Oct 2013 10:19:45 +0800 > > > > 环境如下： > > cloudstack4.1.0 > > hypervisor是kvm,qemu-kvm版本1,2.0(手动编译)，libvirt版本1.1.2(手动编译) > > ceph版本0.67.3（yum安装） > > > > 遇到的问题如下： > > 1.运行在rbd主存储上的windows 2003虚拟机关闭后,迁移到另外一个rbd主存储，有时候能成功，有时候虚拟机引导不了； > > 2.windows 2003虚拟机关闭后，从nfs主存储迁移到rbd主存储或者从rbd 主存储迁移到nfs主存储，都出现找不到引导设备的情况； > > 3.linux虚拟机在nfs和rbd之间迁移或者两个rbd存储之间迁移都会出现找不到引导设备的情况； > > 4.linux虚拟机在主机之间迁移后，内核会崩溃，重启虚拟机后正常； > > 5.windows虚拟机在主机之间迁移后会出现蓝屏现象，代码是0x077，关闭此虚拟机后重新打开正常 > -- Gavin
??????RE: ????RBD?????????????????????? ????????
4.1??4.2 4.14.2??ceph?? -- -- ??: "??";; : 2013??10??22??(??) 4:15 ??: "users-cn@cloudstack.apache.org"; : RE: RBD?? cloudstack4.1 ceph?? > From: 474745...@qq.com > To: users-cn@cloudstack.apache.org > Subject: RBD?? > Date: Wed, 16 Oct 2013 10:19:45 +0800 > > ?? > cloudstack4.1.0 > hypervisor??kvm,qemu-kvm1,2.0()??libvirt1.1.2() > ceph0.67.3??yum?? > > > 1.??rbd??windows > 2003,??rbd???? > 2.windows 2003nfsrbdrbd > ????nfs???? > 3.linuxnfs??rbdrbd?? > 4.linux?? > 5.windows0x077
RE: 配置RBD作为主存储时，迁移出现若干问题
cloudstack4.1 主存储还像不支持ceph？ > From: 474745...@qq.com > To: users-cn@cloudstack.apache.org > Subject: 配置RBD作为主存储时，迁移出现若干问题 > Date: Wed, 16 Oct 2013 10:19:45 +0800 > > 环境如下： > cloudstack4.1.0 > hypervisor是kvm,qemu-kvm版本1,2.0(手动编译)，libvirt版本1.1.2(手动编译) > ceph版本0.67.3（yum安装） > > 遇到的问题如下： > 1.运行在rbd主存储上的windows 2003虚拟机关闭后,迁移到另外一个rbd主存储，有时候能成功，有时候虚拟机引导不了； > 2.windows 2003虚拟机关闭后，从nfs主存储迁移到rbd主存储或者从rbd 主存储迁移到nfs主存储，都出现找不到引导设备的情况； > 3.linux虚拟机在nfs和rbd之间迁移或者两个rbd存储之间迁移都会出现找不到引导设备的情况； > 4.linux虚拟机在主机之间迁移后，内核会崩溃，重启虚拟机后正常； > 5.windows虚拟机在主机之间迁移后会出现蓝屏现象，代码是0x077，关闭此虚拟机后重新打开正常
????RBD??????????????????????????????
?? cloudstack4.1.0 hypervisor??kvm,qemu-kvm1,2.0()??libvirt1.1.2() ceph0.67.3??yum?? 1.??rbd??windows 2003,??rbd 2.windows 2003nfsrbdrbd nfs 3.linuxnfs??rbdrbd?? 4.linux?? 5.windows0x077
????RBD????????????????
cloudstack??4.1.0,NFS??RBD RBD??mon??mds??4??osd??osdcloudstack-agent?? NFS http://snag.gy/92N3F.jpg ?? http://snag.gy/kyCq1.jpg INFO [cloud.servlet.ConsoleProxyServlet] (catalina-exec-18:) Parse host info returned from executing GetVNCPortCommand. host info: 172.16.1.72 ERROR [cloud.api.ApiServer] (catalina-exec-6:) unhandled exception executing api command: createStoragePool com.cloud.utils.exception.CloudRuntimeException: DB Exception on: com.mysql.jdbc.JDBC4PreparedStatement@6651ccf2: INSERT INTO storage_pool (storage_pool.id, storage_pool.name, storage_pool.uuid, storage_pool.pool_type, storage_pool.created, storage_pool.update_time, storage_pool.data_center_id, storage_pool.pod_id, storage_pool.available_bytes, storage_pool.capacity_bytes, storage_pool.status, storage_pool.scope, storage_pool.storage_provider_id, storage_pool.host_address, storage_pool.path, storage_pool.port, storage_pool.user_info, storage_pool.cluster_id) VALUES (205, _binary'osd1', _binary'c4a563f9-80b5-3b0f-9975-964197bba067', 'RBD', '2013-10-08 00:45:06', null, 1, 1, 0, 0, 'Up', null, null, null, _binary'LSr-A==@mon/osd1', 6789, null, 1) Caused by: com.mysql.jdbc.exceptions.jdbc4.MySQLIntegrityConstraintViolationException: Column 'host_address' cannot be null host?? ??
Cloudstack 4.1+Centos6.4 KVM host 添加RBD ceph主存储失败
环境情况：
Cloudstack 4.1.1, Ceph 0.6.1.7, Qemu 0.12.1.2-2.355 with RBD enable
I have added ceph RBD as primary successfully with ubuntu 12.04 kvm.
but fail on centos 6.4 kvm host in cloudstack.
以下是在centos kvm host上的部分测试，可以通过rbd访问ceph
[root@centos-kvm01 ~]# qemu-img -v | grep rbd
Supported formats: raw cow qcow vdi vmdk cloop dmg bochs vpc vvfat
qcow2 qed parallels nbd blkdebug host_cdrom host_floppy host_device
file rbd
[root@centos-kvm01 ~]# rbd -m 192.168.250.15 -p volumes ls
08b376f0-5af4-4b46-b41e-d83dc0f93a70
1e9f8d76-94d9-4d56-a8a6-fdff32ba9d1b
b0f7fae3-716b-4210-a89d-af63c6e30859
[root@centos-kvm01 ~]# rados lspools
data
metadata
rbd
volumes
cloudstack
[root@centos-kvm01 ~]# rbd create test --size 4096
[root@centos-kvm01 ~]# rbd ls
test
以下是cloudstack的日志
2013-08-19 17:28:25,683 DEBUG [agent.transport.Request]
(AgentManager-Handler-9:null) Seq 26-368640010: Processing: { Ans: ,
MgmtId: 345050143793, via: 26, Ver: v1, Flags: 10,
[{"Answer":{"result":false,"details":"java.lang.NullPointerException\n\tat
com.cloud.hypervisor.kvm.storage.LibvirtStorageAdaptor.createStoragePool(LibvirtStorageAdaptor.java:540)\n\tat
com.cloud.hypervisor.kvm.storage.KVMStoragePoolManager.createStoragePool(KVMStoragePoolManager.java:111)\n\tat
com.cloud.hypervisor.kvm.storage.KVMStoragePoolManager.createStoragePool(KVMStoragePoolManager.java:104)\n\tat
com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.execute(LibvirtComputingResource.java:2304)\n\tat
com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.executeRequest(LibvirtComputingResource.java:1094)\n\tat
com.cloud.agent.Agent.processRequest(Agent.java:525)\n\tat
com.cloud.agent.Agent$AgentRequestHandler.doTask(Agent.java:852)\n\tat
com.cloud.utils.nio.Task.run(Task.java:83)\n\tat
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1146)\n\tat
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)\n\tat
java.lang.Thread.run(Thread.java:679)\n","wait":0}}] }
2013-08-19 17:28:25,683 DEBUG [agent.transport.Request]
(catalina-exec-1:null) Seq 26-368640010: Received: { Ans: , MgmtId:
345050143793, via: 26, Ver: v1, Flags: 10, { Answer } }
2013-08-19 17:28:25,683 DEBUG [agent.manager.AgentManagerImpl]
(catalina-exec-1:null) Details from executing class
com.cloud.agent.api.ModifyStoragePoolCommand:
java.lang.NullPointerException
at
com.cloud.hypervisor.kvm.storage.LibvirtStorageAdaptor.createStoragePool(LibvirtStorageAdaptor.java:540)
at
com.cloud.hypervisor.kvm.storage.KVMStoragePoolManager.createStoragePool(KVMStoragePoolManager.java:111)
at
com.cloud.hypervisor.kvm.storage.KVMStoragePoolManager.createStoragePool(KVMStoragePoolManager.java:104)
at
com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.execute(LibvirtComputingResource.java:2304)
at
com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.executeRequest(LibvirtComputingResource.java:1094)
at com.cloud.agent.Agent.processRequest(Agent.java:525)
at com.cloud.agent.Agent$AgentRequestHandler.doTask(Agent.java:852)
at com.cloud.utils.nio.Task.run(Task.java:83)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1146)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:679)
2013-08-19 17:28:25,689 WARN [cloud.storage.StorageManagerImpl]
(catalina-exec-1:null) Unable to establish a connection between
Host[-26-Routing] and Pool[207|RBD]
com.cloud.exception.StorageUnavailableException: Resource
[StoragePool:207] is unreachable: Unable establish connection from
storage head to storage pool 207 due to java.lang.NullPointerException
at
com.cloud.hypervisor.kvm.storage.LibvirtStorageAdaptor.createStoragePool(LibvirtStorageAdaptor.java:540)
at
com.cloud.hypervisor.kvm.storage.KVMStoragePoolManager.createStoragePool(KVMStoragePoolManager.java:111)
at
com.cloud.hypervisor.kvm.storage.KVMStoragePoolManager.createStoragePool(KVMStoragePoolManager.java:104)
at
com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.execute(LibvirtComputingResource.java:2304)
at
com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.executeRequest(LibvirtComputingResource.java:1094)
at com.cloud.agent.Agent.processRequest(Agent.java:525)
at com.cloud.agent.Agent$AgentRequestHandler.doTask(Agent.java:852)
at com.cloud.utils.nio.Task.run(Task.java:83)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1146)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:679)
at
com.cloud.storage.StorageManagerImpl.connectHostToSharedPool(StorageManagerImpl.java:1637)
at
com.cloud.storage.StorageManagerImpl.c
