[openstack-dev] [QA][Tempest] Asking for reviews from Tempest cores.
Hi Tempest folks, Thank you for your reviews on this patch: https://review.openstack.org/#/c/195443/. However, the current devstack configuration in gate jobs only supports one cinder back-end, but this test is supposed to test the volume retype and migration between two different cinder back-ends. That is the reason why this test is not covered in the gate or the experimental so far. I will take a look at how to turn on the multiple cinder back-ends support in the devstack for the gate, while at the same time, if you find the code in this patch is OK, could you just approve this patch? This test is very valuable for cinder admins to check if the volume retype and migration work correctly. Thanks. Best wishes, Vincent Hou (侯胜博) Staff Software Engineer, Open Standards and Open Source Team, Emerging Technology Institute, IBM China Software Development Lab Tel: 86-10-82450778 Fax: 86-10-82453660 Notes ID: Sheng Bo Hou/China/IBM@IBMCNE-mail: sb...@cn.ibm.com Address:3F Ring, Building 28 Zhongguancun Software Park, 8 Dongbeiwang West Road, Haidian District, Beijing, P.R.C.100193 地址:北京市海淀区东北旺西路8号中关村软件园28号楼环宇大厦3层 邮编：100193 - Forwarded by Sheng Bo Hou/China/IBM on 12/21/2015 10:44 AM - From: Sheng Bo Hou/China/IBM To: <openstack-dev@lists.openstack.org> Date: 12/15/2015 02:48 PM Subject:[QA][Tempest] Asking for reviews from Tempest cores. Hi Tempest folks, https://review.openstack.org/#/c/195443/ I am asking you to take a review on this patch, which is the integration test for volume retype with migration in cinder. It has been taken quite a while and quite some cycles to get mature. It is a very important test for volume migration feature in Cinder. Thank you for your attention. Best wishes, Vincent Hou (侯胜博) Staff Software Engineer, Open Standards and Open Source Team, Emerging Technology Institute, IBM China Software Development Lab Tel: 86-10-82450778 Fax: 86-10-82453660 Notes ID: Sheng Bo Hou/China/IBM@IBMCNE-mail: sb...@cn.ibm.com Address:3F Ring, Building 28 Zhongguancun Software Park, 8 Dongbeiwang West Road, Haidian District, Beijing, P.R.C.100193 地址:北京市海淀区东北旺西路8号中关村软件园28号楼环宇大厦3层 邮编：100193 __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [QA][Tempest] Asking for reviews from Tempest cores.
Hi Tempest folks, https://review.openstack.org/#/c/195443/ I am asking you to take a review on this patch, which is the integration test for volume retype with migration in cinder. It has been taken quite a while and quite some cycles to get mature. It is a very important test for volume migration feature in Cinder. Thank you for your attention. Best wishes, Vincent Hou (侯胜博) Staff Software Engineer, Open Standards and Open Source Team, Emerging Technology Institute, IBM China Software Development Lab Tel: 86-10-82450778 Fax: 86-10-82453660 Notes ID: Sheng Bo Hou/China/IBM@IBMCNE-mail: sb...@cn.ibm.com Address:3F Ring, Building 28 Zhongguancun Software Park, 8 Dongbeiwang West Road, Haidian District, Beijing, P.R.C.100193 地址:北京市海淀区东北旺西路8号中关村软件园28号楼环宇大厦3层 邮编：100193 __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [Cinder] Is there anyone truly working on this issue https://bugs.launchpad.net/cinder/+bug/1520102?
Hi Mitsuhiro, Thang The patch https://review.openstack.org/#/c/228916 is merged, but sadly it does not cover the issue https://bugs.launchpad.net/cinder/+bug/1520102. This bug is still valid. As far as you know, is there someone working on this issue? If not, I am gonna fix it. Best wishes, Vincent Hou (侯胜博) Staff Software Engineer, Open Standards and Open Source Team, Emerging Technology Institute, IBM China Software Development Lab Tel: 86-10-82450778 Fax: 86-10-82453660 Notes ID: Sheng Bo Hou/China/IBM@IBMCNE-mail: sb...@cn.ibm.com Address:3F Ring, Building 28 Zhongguancun Software Park, 8 Dongbeiwang West Road, Haidian District, Beijing, P.R.C.100193 地址:北京市海淀区东北旺西路8号中关村软件园28号楼环宇大厦3层 邮编：100193 __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [Cinder] [Manila] Will NFS stay with Cinder as a reference implementation?
Hi folks, I have a question about the file services in OpenStack. As you know there is a generic NFS driver in Cinder and other file system drivers inherit it, while the project Manila is determined to provide the file system service. Will NFS stay with Cinder as the reference implementation for the coming release or releases? Are all the file system drivers going to move to Manila? What is relation between Manila as FSaaS and NFS in Cinder? Any ideas? Thank you. Best wishes, Vincent Hou (侯胜博) Staff Software Engineer, Open Standards and Open Source Team, Emerging Technology Institute, IBM China Software Development Lab Tel: 86-10-82450778 Fax: 86-10-82453660 Notes ID: Sheng Bo Hou/China/IBM@IBMCNE-mail: sb...@cn.ibm.com Address:3F Ring, Building 28 Zhongguancun Software Park, 8 Dongbeiwang West Road, Haidian District, Beijing, P.R.C.100193 地址:北京市海淀区东北旺西路8号中关村软件园28号楼环宇大厦3层 邮编：100193 __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [Cinder] The devref for volume migration in Cinder
Hi everyone interested in volume migration: I have just drafted the devref for volume migration in Cinder. It consists of everything I can think of for the volume migration. LINK: https://review.openstack.org/#/c/214941 For folks who have much experience in migration: Help me check it to see if it is comprehensive and precise. If there is anything missing or confusing, please comment on it. For folks who are not familiar with experience in migration: Read the devref from the beginning to the end to see if you have got what volume migration is about, how to use, how to configure, etc. Check if it is clear and easy to understand. Thank you very much. Best wishes, Vincent Hou (侯胜博) Staff Software Engineer, Open Standards and Open Source Team, Emerging Technology Institute, IBM China Software Development Lab Tel: 86-10-82450778 Fax: 86-10-82453660 Notes ID: Sheng Bo Hou/China/IBM@IBMCNE-mail: sb...@cn.ibm.com Address:3F Ring, Building 28 Zhongguancun Software Park, 8 Dongbeiwang West Road, Haidian District, Beijing, P.R.C.100193 地址:北京市海淀区东北旺西路8号中关村软件园28号楼环宇大厦3层 邮编：100193 __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [cinder] Need help from folks working on Dell, Storpool and Infortrend drivers
Hi everyone, For folks who are working on Dell, Storpool and Infortrend drivers: I have got a new patch for https://review.openstack.org/#/c/180873. There is a change about how to implement the method update_migrated_volume for each driver. The code needs to return the final values for the key _name_id and provider_location. I have changed accordingly in your driver, but I am no 100% sure of the precision, so I need your reviews on the changes about your driver. If there is any comment, tell me how to change it. You can take the implementation for Storwize as a reference. First, check if the rename works the same for both the 'available' or 'in-use' volumes. It is possible to handle differently. Then check if the rename is successful, it may return different _name_id and provider_location values. There is an explanation about the update_migrated_volume in https://review.openstack.org/#/c/180873/67/cinder/volume/driver.py Thank you. Best wishes, Vincent Hou (侯胜博) Staff Software Engineer, Open Standards and Open Source Team, Emerging Technology Institute, IBM China Software Development Lab Tel: 86-10-82450778 Fax: 86-10-82453660 Notes ID: Sheng Bo Hou/China/IBM@IBMCNE-mail: sb...@cn.ibm.com Address:3F Ring, Building 28 Zhongguancun Software Park, 8 Dongbeiwang West Road, Haidian District, Beijing, P.R.C.100193 地址:北京市海淀区东北旺西路8号中关村软件园28号楼环宇大厦3层 邮编：100193 __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [openstack-infra] [tempest] [qa] How to make it possible to support multiple vendor storage drivers
Hi everyone, I have submitted a spec in cinder and will implement the integration test cases for volume migration for cinder in tempest. The spec link is: https://review.openstack.org/#/c/186327 Volume migration integration tests need a devstack environment configured with multiple back-ends and multiple vendor drivers. Currently I am write the volume migration cases for multiple back-ends and vendor drivers, for example, one back-end for LVM and the other for Storwize. However, we need to MANUALLY configure the devstack environment to both LVM and Storwize and then run the tests. I was wondering if our gate test can AUTOMATICALLY connect to different cinder back-end storage and setup the environment. Is it something that we can make it happen in OpenStack? For example, if I set CINDER_ENABLED_BACKENDS=storwize:storwize-1,lvm:lvmdriver-1 in localrc, the devstack will configure one back-end of LVM and the other back-end of storwize for me. I need help from folks in [openstack-infra] [tempest] [qa] teams for this potential work item. Thank you very much. Best wishes, Vincent Hou (侯胜博) Staff Software Engineer, Open Standards and Open Source Team, Emerging Technology Institute, IBM China Software Development Lab Tel: 86-10-82450778 Fax: 86-10-82453660 Notes ID: Sheng Bo Hou/China/IBM@IBMCNE-mail: sb...@cn.ibm.com Address:3F Ring, Building 28 Zhongguancun Software Park, 8 Dongbeiwang West Road, Haidian District, Beijing, P.R.C.100193 地址:北京市海淀区东北旺西路8号中关村软件园28号楼环宇大厦3层 邮编：100193 __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [tempest] How to contribute volume migration tests into tempest
Hi everyone working for tempest, I am planning to adding the tempest tests for volume migration in Cinder. The spec link is https://review.openstack.org/#/c/186327/ The blueprint for cinder is https://blueprints.launchpad.net/cinder/+spec/migration-improvement I mentioned the tempest work in this BP/SPEC already. I have already started the patch: https://review.openstack.org/#/c/195443/ I would like to contribute the volume migration tests into tempest. What is the process for that? Do I need to submit a new BP or spec in tempest? Thank you. Best wishes, Vincent Hou (侯胜博) Staff Software Engineer, Open Standards and Open Source Team, Emerging Technology Institute, IBM China Software Development Lab Tel: 86-10-82450778 Fax: 86-10-82453660 Notes ID: Sheng Bo Hou/China/IBM@IBMCNE-mail: sb...@cn.ibm.com Address:3F Ring, Building 28 Zhongguancun Software Park, 8 Dongbeiwang West Road, Haidian District, Beijing, P.R.C.100193 地址:北京市海淀区东北旺西路8号中关村软件园28号楼环宇大厦3层 邮编：100193 __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [Cinder] Need Cinder cores to approve the spec for volume migration improvement
Hi Cinder folks, Can any cores in cinder take a review of this patch https://review.openstack.org/#/c/186327/? I am have already received +1's from Jay, Patrick, Mitsuhiro and Avishay. If you have comments, please feedback to me. If not, can any of you approved this spec? Thank you so much. Best wishes, Vincent Hou (侯胜博) Staff Software Engineer, Open Standards and Open Source Team, Emerging Technology Institute, IBM China Software Development Lab Tel: 86-10-82450778 Fax: 86-10-82453660 Notes ID: Sheng Bo Hou/China/IBM@IBMCNE-mail: sb...@cn.ibm.com Address:3F Ring, Building 28 Zhongguancun Software Park, 8 Dongbeiwang West Road, Haidian District, Beijing, P.R.C.100193 地址:北京市海淀区东北旺西路8号中关村软件园28号楼环宇大厦3层 邮编：100193 __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] About the volume status exposure during migration
Hi Avishay, I really appreciate your comments on the spec I submitted for volume migration improvement.(https://review.openstack.org/#/c/186327/) I truly understand your concerns about exposing the migrating status to the end user(not admin). However, there is something I get confused: when we would like to migrate a volume from LVM to Storwize back-end, we need to use the command retype. During this retype, the volume status is already set to retyping, which is also exposed to the end user(not admin). Do you see any issues about that Is this something we need to change as well? Best wishes, Vincent Hou (侯胜博) Staff Software Engineer, Open Standards and Open Source Team, Emerging Technology Institute, IBM China Software Development Lab Tel: 86-10-82450778 Fax: 86-10-82453660 Notes ID: Sheng Bo Hou/China/IBM@IBMCNE-mail: sb...@cn.ibm.com Address:3F Ring, Building 28 Zhongguancun Software Park, 8 Dongbeiwang West Road, Haidian District, Beijing, P.R.C.100193 地址:北京市海淀区东北旺西路8号中关村软件园28号楼环宇大厦3层 邮编：100193 __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [tempest][qa][cinder] How to configure tempest with to test volume migration?
Yes, I put them in the cinder-spec. Best wishes, Vincent Hou (侯胜博) Staff Software Engineer, Open Standards and Open Source Team, Emerging Technology Institute, IBM China Software Development Lab Tel: 86-10-82450778 Fax: 86-10-82453660 Notes ID: Sheng Bo Hou/China/IBM@IBMCNE-mail: sb...@cn.ibm.com Address:3F Ring, Building 28 Zhongguancun Software Park, 8 Dongbeiwang West Road, Haidian District, Beijing, P.R.C.100193 地址:北京市海淀区东北旺西路8号中关村软件园28号楼环宇大厦3层 邮编：100193 Avishay Traeger avis...@stratoscale.com 05/26/2015 08:52 PM Please respond to OpenStack Development Mailing List \(not for usage questions\) openstack-dev@lists.openstack.org To OpenStack Development Mailing List (not for usage questions) openstack-dev@lists.openstack.org cc Subject Re: [openstack-dev] [tempest][qa][cinder] How to configure tempest with to test volume migration? Good to see that you will work on this! In addition to the cases that Duncan mentioned, there is also the case of moving a volume between two pools within the same backend. This probably complicates the setup even further. On Mon, May 25, 2015 at 12:31 PM, Duncan Thomas duncan.tho...@gmail.com wrote: There doesn't seem to be an existing tempest test for this. I suggest writing the test so that it looks if there are two volume types defined, and if so uses them. Ideally you should test migration to and from lvm. There is offline migration (volume is available) and online migration (volume is attached), which are two completely separate code paths. Great to hear somebody I'd writing tests for this! On 25 May 2015 10:24, Sheng Bo Hou sb...@cn.ibm.com wrote: Hi everyone, I am planning to add test cases for volume migration for cinder into tempest. I am wondering how to enable multiple back-ends for cinder in tempest, and connect to different back-ends. For example, I configure one back-end for LVM and the other is for IBM Storwize V7000 driver. Then run the test named like test_volume_migration_LVM_Storwize to test if the migration really works fine. About the configuration, is this something tempest can do so far? Or is this something new we need to add? Thank you very much. Best wishes, Vincent Hou (侯胜博) Staff Software Engineer, Open Standards and Open Source Team, Emerging Technology Institute, IBM China Software Development Lab Tel: 86-10-82450778 Fax: 86-10-82453660 Notes ID: Sheng Bo Hou/China/IBM@IBMCNE-mail: sb...@cn.ibm.com Address:3F Ring, Building 28 Zhongguancun Software Park, 8 Dongbeiwang West Road, Haidian District, Beijing, P.R.C.100193 地址:北京市海淀区东北旺西路8号中关村软件园28号楼环宇大厦3层 邮编：100193 __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- Avishay Traeger Storage RD Mobile: +972 54 447 1475 E-mail: avis...@stratoscale.com Web | Blog | Twitter | Google+ | Linkedin __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [cinder] Spec for volume migration improvement
Hi all folks from Cinder, According to our agreement in the Vancouver summit, I submitted a cinder-spec to do the volume migration improvement. The spec is here for review: https://review.openstack.org/#/c/186327/. Please feel free to give your comments. @Mitsuhiro Tanino, I believe you can submit another spec for the efficient migration as well. Thanks, folks. Best wishes, Vincent Hou (侯胜博) Staff Software Engineer, Open Standards and Open Source Team, Emerging Technology Institute, IBM China Software Development Lab Tel: 86-10-82450778 Fax: 86-10-82453660 Notes ID: Sheng Bo Hou/China/IBM@IBMCNE-mail: sb...@cn.ibm.com Address:3F Ring, Building 28 Zhongguancun Software Park, 8 Dongbeiwang West Road, Haidian District, Beijing, P.R.C.100193 地址:北京市海淀区东北旺西路8号中关村软件园28号楼环宇大厦3层 邮编：100193 __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [cinder] Migrating in-use volumes with swap_volume in Nova
Hi Vivek, Per our discussion about the migration of an in-use volume, I have done the following tests: I took KVM as the hypervisor and LVM as the backend for cinder. Configure two c-vol nodes. Create the image in glance. Create a volume from this image. Boot a VM from this volume, so the volume becomes in-use and the VM is ready. Migrate this in-use volume. The result turns out fine finally and the volume is successfully migrated via the swap_volume method in Nova. I just send you this mail as a confirmation that swap_volume works for your use case. Best wishes, Vincent Hou (侯胜博) Staff Software Engineer, Open Standards and Open Source Team, Emerging Technology Institute, IBM China Software Development Lab Tel: 86-10-82450778 Fax: 86-10-82453660 Notes ID: Sheng Bo Hou/China/IBM@IBMCNE-mail: sb...@cn.ibm.com Address:3F Ring, Building 28 Zhongguancun Software Park, 8 Dongbeiwang West Road, Haidian District, Beijing, P.R.C.100193 地址:北京市海淀区东北旺西路8号中关村软件园28号楼环宇大厦3层 邮编：100193 __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Manila] [Cinder] About how to hide the dummy destination record during migration
Hi Valeriy, Thank you for telling me about the private driver storage feature from Manila. I have reviewed the patch and it can definitely resolve the dummy destination volume issue I have during migration in Cinder. I do not deny that it is a good approach. However, I need to put Patrick in the CC list to make him aware of this. During the previous Cinder IRC, we made an agreement that Cinder will go along this approach to consider all the common issues by introducing a cinder-internal tenant. Please check the cinder-spec: https://review.openstack.org/#/c/186232/. I guess cinder will go along this approach. @Patrick, I am not sure what our implementation is gonna be. Is it possible that similar data model can be used for cinder as it is in Manila? Please check https://review.openstack.org/#/c/176877/ Best wishes, Vincent Hou (侯胜博) Staff Software Engineer, Open Standards and Open Source Team, Emerging Technology Institute, IBM China Software Development Lab Tel: 86-10-82450778 Fax: 86-10-82453660 Notes ID: Sheng Bo Hou/China/IBM@IBMCNE-mail: sb...@cn.ibm.com Address:3F Ring, Building 28 Zhongguancun Software Park, 8 Dongbeiwang West Road, Haidian District, Beijing, P.R.C.100193 地址:北京市海淀区东北旺西路8号中关村软件园28号楼环宇大厦3层 邮编：100193 Valeriy Ponomaryov vponomar...@mirantis.com 05/27/2015 02:12 PM Please respond to OpenStack Development Mailing List \(not for usage questions\) openstack-dev@lists.openstack.org To OpenStack Development Mailing List (not for usage questions) openstack-dev@lists.openstack.org cc rodrigo.barbieri2...@gamil.com Subject Re: [openstack-dev] [Manila] About how to hide the dummy destination record during migration Hello Vincent Hou, We, Manila folks, are about to merge one of new features - private driver storage [1]. That is going to serve for not-user facing data storage related to any resource that can be reached by both - API and share driver. And in case of share migration, it will be possible to avoid creation of temporary share DB record and use this data storage for storing all required data per each share. Please, look at it, and provide feedback, whether such approach can be used in your case or not and why. [1] - https://review.openstack.org/#/c/176877/ Kind regards, Valeriy Ponomaryov On Wed, May 27, 2015 at 7:28 AM, Sheng Bo Hou sb...@cn.ibm.com wrote: Hi everyone working for Manila, This is Vincent Hou from IBM. I am working on all the migration issues in Cinder. I had one session for the Cinder migration issue in Vancouver and some of you folks attended it. The etherpad link is https://etherpad.openstack.org/p/volume-migration-improvement Per the issue that we had better not let the user see the target volume during migration when issuing command cinder list, we can add an additional flag into the volume table, for example, hidden, into it. The default value is 0, meaning that it will display for cinder list. For the target volume during migration. We can set it to 1, so the user will not be able to see it with the command cinder list. I think it is a straightforward approach we can go with. I just sync up with you folks, so that we can have a consistent way to resolve this issue in both Cinder and Manila. I just need to make sure we are on the same page. Is this solution OK with you folks? Especially @Rodrigo Barbieri and @Erlon Cruz, etc. Looking forward to hearing from you. Thanks. Best wishes, Vincent Hou (侯胜博) Staff Software Engineer, Open Standards and Open Source Team, Emerging Technology Institute, IBM China Software Development Lab Tel: 86-10-82450778 Fax: 86-10-82453660 Notes ID: Sheng Bo Hou/China/IBM@IBMCNE-mail: sb...@cn.ibm.com Address:3F Ring, Building 28 Zhongguancun Software Park, 8 Dongbeiwang West Road, Haidian District, Beijing, P.R.C.100193 地址:北京市海淀区东北旺西路8号中关村软件园28号楼环宇大厦3层 邮编：100193 __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [Manila] About how to hide the dummy destination record during migration
Hi everyone working for Manila, This is Vincent Hou from IBM. I am working on all the migration issues in Cinder. I had one session for the Cinder migration issue in Vancouver and some of you folks attended it. The etherpad link is https://etherpad.openstack.org/p/volume-migration-improvement Per the issue that we had better not let the user see the target volume during migration when issuing command cinder list, we can add an additional flag into the volume table, for example, hidden, into it. The default value is 0, meaning that it will display for cinder list. For the target volume during migration. We can set it to 1, so the user will not be able to see it with the command cinder list. I think it is a straightforward approach we can go with. I just sync up with you folks, so that we can have a consistent way to resolve this issue in both Cinder and Manila. I just need to make sure we are on the same page. Is this solution OK with you folks? Especially @Rodrigo Barbieri and @Erlon Cruz, etc. Looking forward to hearing from you. Thanks. Best wishes, Vincent Hou (侯胜博) Staff Software Engineer, Open Standards and Open Source Team, Emerging Technology Institute, IBM China Software Development Lab Tel: 86-10-82450778 Fax: 86-10-82453660 Notes ID: Sheng Bo Hou/China/IBM@IBMCNE-mail: sb...@cn.ibm.com Address:3F Ring, Building 28 Zhongguancun Software Park, 8 Dongbeiwang West Road, Haidian District, Beijing, P.R.C.100193 地址:北京市海淀区东北旺西路8号中关村软件园28号楼环宇大厦3层 邮编：100193 __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [tempest][qa][cinder] How to configure tempest with to test volume migration?
Hi everyone, I am planning to add test cases for volume migration for cinder into tempest. I am wondering how to enable multiple back-ends for cinder in tempest, and connect to different back-ends. For example, I configure one back-end for LVM and the other is for IBM Storwize V7000 driver. Then run the test named like test_volume_migration_LVM_Storwize to test if the migration really works fine. About the configuration, is this something tempest can do so far? Or is this something new we need to add? Thank you very much. Best wishes, Vincent Hou (侯胜博) Staff Software Engineer, Open Standards and Open Source Team, Emerging Technology Institute, IBM China Software Development Lab Tel: 86-10-82450778 Fax: 86-10-82453660 Notes ID: Sheng Bo Hou/China/IBM@IBMCNE-mail: sb...@cn.ibm.com Address:3F Ring, Building 28 Zhongguancun Software Park, 8 Dongbeiwang West Road, Haidian District, Beijing, P.R.C.100193 地址:北京市海淀区东北旺西路8号中关村软件园28号楼环宇大厦3层 邮编：100193 __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [cinder] Some Changes to Cinder Core
I am not a core, but I support with +1 from my heart. Best wishes, Vincent Hou (侯胜博) Staff Software Engineer, Open Standards and Open Source Team, Emerging Technology Institute, IBM China Software Development Lab Tel: 86-10-82450778 Fax: 86-10-82453660 Notes ID: Sheng Bo Hou/China/IBM@IBMCNE-mail: sb...@cn.ibm.com Address:3F Ring, Building 28 Zhongguancun Software Park, 8 Dongbeiwang West Road, Haidian District, Beijing, P.R.C.100193 地址:北京市海淀区东北旺西路8号中关村软件园28号楼环宇大厦3层 邮编：100193 Jay Bryant jsbry...@electronicjungle.net 05/23/2015 11:22 AM Please respond to jsbry...@electronicjungle.net; Please respond to OpenStack Development Mailing List \(not for usage questions\) openstack-dev@lists.openstack.org To OpenStack Development Mailing List (not for usage questions) openstack-dev@lists.openstack.org cc Subject Re: [openstack-dev] [cinder] Some Changes to Cinder Core +1 Well deserved. Welcome to Core Sean! It is a pleasure to work with you! Thanks to Avishay for all his contributions! Sorry to see you go. Jay On May 22, 2015 4:36 PM, Mike Perez thin...@gmail.com wrote: This is long overdue, but it gives me great pleasure to nominate Sean McGinnis for Cinder core. Reviews: https://review.openstack.org/#/q/reviewer:+%22Sean+McGinnis%22,n,z Contributions: https://review.openstack.org/#/q/owner:+%22Sean+McGinnis%22,n,z 30/90 day review stats: http://stackalytics.com/report/contribution/cinder-group/30 http://stackalytics.com/report/contribution/cinder-group/90 As new contributors step up to help in the project, some move onto other things. I would like to recognize Avishay Traeger for his contributions, and now unfortunately departure from the Cinder core team. Cinder core, please reply with a +1 for approval. This will be left open until May 29th. Assuming there are no objections, this will go forward after voting is closed. -- Mike Perez __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [cinder] Thanks to all the folks on the plane tour to glaciers
Hi all the folks on the plane tour to the glaciers, This is Vincent Hou. I am sorry about getting a bit plane-sick during our trip. It was a sort of harsh for me, but the wonderful thing is that I really enjoyed the snow, the mountain, the lake, the glaciers, etc. I hope the uncomfortable me did not bring too much trouble to you folks. It felt like that the blood in my body had stopped flowing. I got no strength to see, no strength to hear, no strength to sit, even no strength to think... Sorry about that I did not even say goodbye and wish you folks a nice trip back to your place. Thank Kurt. This trip is perhaps the only once in my life with the best view I ever see. Thank Sean. You words helped me a lot. I might not be able to make the trip in the last section without them. Thank Jay, Walter... Thank everybody. One more request in the end. Can you guys share some photos you have taken in our tour? I have missed some view. Thank you so much. I am feeling much better now in the hotel. Take care, folks! Safe trip back home! See you. Best wishes, Vincent Hou (侯胜博) Staff Software Engineer, Open Standards and Open Source Team, Emerging Technology Institute, IBM China Software Development Lab Tel: 86-10-82450778 Fax: 86-10-82453660 Notes ID: Sheng Bo Hou/China/IBM@IBMCNE-mail: sb...@cn.ibm.com Address:3F Ring, Building 28 Zhongguancun Software Park, 8 Dongbeiwang West Road, Haidian District, Beijing, P.R.C.100193 地址:北京市海淀区东北旺西路8号中关村软件园28号楼环宇大厦3层 邮编：100193 __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [cinder] Volume Replication and Migration bug triage ...
Hi all, https://bugs.launchpad.net/cinder/+bug/1255622 For this bug, I have added some comments to explain why the orphaned volume that ends up in the end of the migration. @John Griffith, I hope this can resolve your confusion a bit. Best wishes, Vincent Hou (侯胜博) Staff Software Engineer, Open Standards and Open Source Team, Emerging Technology Institute, IBM China Software Development Lab Tel: 86-10-82450778 Fax: 86-10-82453660 Notes ID: Sheng Bo Hou/China/IBM@IBMCNE-mail: sb...@cn.ibm.com Address:3F Ring, Building 28 Zhongguancun Software Park, 8 Dongbeiwang West Road, Haidian District, Beijing, P.R.C.100193 地址:北京市海淀区东北旺西路8号中关村软件园28号楼环宇大厦3层 邮编：100193 Jay S Bryant/Rochester/IBM@IBMUS 02/07/2015 07:50 AM To Weekly Cinder Meeting cc Theresa Backlund/Rochester/IBM@IBMUS, Jacob Morlock/Rochester/IBM@IBMUS, Peter Wassel/Endicott/IBM@IBMUS Subject Fw: [cinder] Volume Replication and Migration bug triage ... All the world's a stage and most of us are desperately unrehearsed. -- Sean O'Casey - Forwarded by Jay S Bryant/Rochester/IBM on 02/06/2015 05:47 PM - From: Jay S. Bryant jsbry...@electronicjungle.net To: openStack Development Mailing List (not for usage questions) openstack-dev@lists.openstack.org, Jay S Bryant/Rochester/IBM@IBMUS Date: 02/06/2015 05:46 PM Subject:[cinder] Volume Replication and Migration bug triage ... Sent by:Jay Bryant jungleb...@gmail.com All, In discussion with Mike Perez earlier this week the following bugs were highlighted in Volume Migration and Volume Replication. IBM is focusing on investigating and resolving these bugs. I will be putting out updates as we progress towards resolution of these issues. Replication: https://bugs.launchpad.net/cinder/+bug/1390001 -- Investigated by Tao and found to be Invalid https://bugs.launchpad.net/cinder/+bug/1384040 -- assigned to Tao https://bugs.launchpad.net/cinder/+bug/1372292 -- assigned to Tao https://bugs.launchpad.net/cinder/+bug/1370311 -- Requires multi-pool scheduler awareness and replica promote supporting multiple pools. BP opened: https://blueprints.launchpad.net/cinder/+spec/storwize-support-muli-pool-within-one-backend-relative-features https://bugs.launchpad.net/cinder/+bug/1383524 -- Currently assigned to Ronen with updates from Avishay. Have a question in to Avishay to see if he can keep investigating. Migration: https://bugs.launchpad.net/cinder/+bug/1404013 -- Fix released for this. https://bugs.launchpad.net/cinder/+bug/1403916 -- Question out to the reporter to see if this is still an issue. (LVM) https://bugs.launchpad.net/cinder/+bug/1403912 -- Question out to the reporter to see if this is still an issue. (LVM) https://bugs.launchpad.net/cinder/+bug/1403904 -- Marked Invalid https://bugs.launchpad.net/cinder/+bug/1391179 -- Assigned to Alon Marx as this is an issue that was seen on XIV. https://bugs.launchpad.net/cinder/+bug/1283313 -- Avishay was looking into this. Asked if he still is doing so. https://bugs.launchpad.net/cinder/+bug/1255957 -- Currently marked incomplete. May warrant futher investigation. https://bugs.launchpad.net/cinder/+bug/1391172 -- Fix released https://bugs.launchpad.net/cinder/+bug/1403902 -- A number of patches have been proposed around this one. Will follow up to understand if it is still a problem. https://bugs.launchpad.net/cinder/+bug/1255622 -- John was the last one looking at this. Appears to work in some situations. https://bugs.launchpad.net/cinder/+bug/1398177 -- Assigned to Vincent Hou https://bugs.launchpad.net/cinder/+bug/1308822 -- Assigned to Tao or Li Min. https://bugs.launchpad.net/cinder/+bug/1278289 -- Assigned to Jay. Will investigate https://bugs.launchpad.net/cinder/+bug/1308315.-- Tao is investigating but hasn't been able to recreate. While triaging all the issues I updated the ones for migration with a 'migration' tag and updated the replication ones with a 'replication' tag. If you have any questions/concerns about this, please let me know. Otherwise we will work towards cleaning these up. Thanks! Jay __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [cinder] Not seeking another term as PTL
Hi John, Thank you very much for leading the Cinder project. It is you who brings me into OpenStack, into Cinder, and helps me to start my contribution to OpenStack. I will remember the joke we share in Portland. (You know what they call a Quarter Pounder in France?) OK, I guess we will go to Paris this time.:-) I hope we continue to see each other in the community. Wish you all the best. Best wishes, Vincent Hou (侯胜博) Staff Software Engineer, Open Standards and Open Source Team, Emerging Technology Institute, IBM China Software Development Lab Tel: 86-10-82450778 Fax: 86-10-82453660 Notes ID: Sheng Bo Hou/China/IBM@IBMCNE-mail: sb...@cn.ibm.com Address:3F Ring, Building 28 Zhongguancun Software Park, 8 Dongbeiwang West Road, Haidian District, Beijing, P.R.C.100193 地址:北京市海淀区东北旺西路8号中关村软件园28号楼环宇大厦3层 邮编：100193 John Griffith john.griffi...@gmail.com 09/23/2014 10:57 PM Please respond to OpenStack Development Mailing List \(not for usage questions\) openstack-dev@lists.openstack.org To OpenStack Development Mailing List (openstack-dev@lists.openstack.org) openstack-dev@lists.openstack.org cc Subject [openstack-dev] [cinder] Not seeking another term as PTL Hey Everyone, I've been kinda mixed on this one, but I think it's a good time for me to not run for Cinder PTL. I've been filling the role since we started the idea back at the Folsom Summit, and it's been an absolute pleasure and honor for me. I don't plan on going anywhere and will still be involved as I am today, but hopefully I'll also now have a good opportunity to contribute elsewhere in OpenStack. We have a couple of good candidates running for Cinder PTL as well as a strong team backing the project so I think it's a good time to let somebody else take the official PTL role for a bit. Thanks, John___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [Nova] [VMware]How can I put my proposals for VMware drivers in the VMware driver session?
Hi VMware folks, How can I put my proposals for VMware drivers in the VMware driver session? Is there someone from VMware responsible for receiving proposals for this session? I submitted two topics: Resource Pool Support for VMware VCenter: http://summit.openstack.org/cfp/details/349, https://blueprints.launchpad.net/nova/+spec/driver-specific-snapshot. and Reinforcement with Native Server Snapshot: http://summit.openstack.org/cfp/details/351, https://blueprints.launchpad.net/nova/+spec/vmware-resource-pool-enablement . The community decided to put the first into the VMware driver session, but the second has got refused. However, I hope both of them can be put in the VMware driver session. Can someone from VMware help me to put them into the VMware driver session for the discussion in the design summit? Thank you very much. Best wishes, Vincent Hou (侯胜博) Staff Software Engineer, Open Standards and Open Source Team, Emerging Technology Institute, IBM China Software Development Lab Tel: 86-10-82450778 Fax: 86-10-82453660 Notes ID: Sheng Bo Hou/China/IBM@IBMCNE-mail: sb...@cn.ibm.com Address:3F Ring, Building 28 Zhongguancun Software Park, 8 Dongbeiwang West Road, Haidian District, Beijing, P.R.C.100193 地址:北京市海淀区东北旺西路8号中关村软件园28号楼环宇大厦3层 邮编：100193 ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova][blueprint] Accelerate the booting process of a number of vms via VMThunder
I have done a little test for the image download and upload. I created an API for the image access, containing copyFrom and sendTo. I moved the image download and upload code from XenApi into the implementation for Http with some modifications, and the code worked for libvirt as well. copyFrom means to download the image and return the image data, and different hypervisors can choose to save it in a file or import it to the datastore; sendTo is used to upload the image and the image data is passed in as a parameter. I also did an investigation about how each hypervisor is doing the image upload and download. For the download: libvirt, hyper-v and baremetal use the code image_service.download to download the image and save it into a file. vmwareapi uses the code image_service.download to download the image and import it into the datastore. XenAPi uses image_service.download to download the image for VHD image. For the upload: They use image_service.upload to upload the image. I think we can conclude that it is possible to have a common image transfer library with different implementations for different protocols. This is a small demo code for the library: https://review.openstack.org/#/c/90601/(Jay, is it close to the library as you mentioned?). I just replaced the upload and download part with the http implementation for the imageapi and it worked fine. Best wishes, Vincent Hou (侯胜博) Staff Software Engineer, Open Standards and Open Source Team, Emerging Technology Institute, IBM China Software Development Lab Tel: 86-10-82450778 Fax: 86-10-82453660 Notes ID: Sheng Bo Hou/China/IBM@IBMCNE-mail: sb...@cn.ibm.com Address:3F Ring, Building 28 Zhongguancun Software Park, 8 Dongbeiwang West Road, Haidian District, Beijing, P.R.C.100193 地址:北京市海淀区东北旺西路8号中关村软件园28号楼环宇大厦3层 邮编：100193 Solly Ross sr...@redhat.com 2014/04/25 01:46 Please respond to OpenStack Development Mailing List \(not for usage questions\) openstack-dev@lists.openstack.org To OpenStack Development Mailing List (not for usage questions) openstack-dev@lists.openstack.org, cc Subject Re: [openstack-dev] [Nova][blueprint] Accelerate the booting process of a number of vms via VMThunder Something to be aware of when planing an image transfer library is that individual drivers might have optimized support for image transfer in certain cases (especially when dealing with transferring between different formats, like raw to qcow2, etc). This builds on what Christopher was saying -- there's actually a reason why we have code for each driver. While having a common image copying library would be nice, I think a better way to do it would be to have some sort of library composed of building blocks, such that each driver could make use of common functionality while still tailoring the operation to the quirks of the particular drivers. Best Regards, Solly Ross - Original Message - From: Christopher Lefelhocz christopher.lefel...@rackspace.com To: OpenStack Development Mailing List (not for usage questions) openstack-dev@lists.openstack.org Sent: Thursday, April 24, 2014 11:17:41 AM Subject: Re: [openstack-dev] [Nova][blueprint] Accelerate the booting process of a number of vms via VMThunder Apologies for coming to this discussion late... On 4/22/14 6:21 PM, Jay Pipes jaypi...@gmail.com wrote: Right, a good solution would allow for some flexibility via multiple transfer drivers. +1. In particular I don't think this discussion should degenerate into zero-copy vs. pre caching. I see both as possible solutions depending on deployer/environment needs. Jay Pipes has suggested we figure out a blueprint for a separate library dedicated to the data(byte) transfer, which may be put in oslo and used by any projects in need (Hoping Jay can come in:-)). Huiba, Zhiyan, everyone else, do you think we come up with a blueprint about the data transfer in oslo can work? Yes, so I believe the most appropriate solution is to create a library -- in oslo or a standalone library like taskflow -- that would offer a simple byte streaming library that could be used by nova.image to expose a neat and clean task-based API. Right now, there is a bunch of random image transfer code spread throughout nova.image and in each of the virt drivers there seems to be different re-implementations of similar functionality. I propose we clean all that up and have nova.image expose an API so that a virt driver could do something like this: from nova.image import api as image_api ... task = image_api.copy(from_path_or_uri, to_path_or_uri) # do some other work copy_task_result = task.wait() Within nova.image.api.copy(), we would use the aforementioned transfer library to move the image bits from the source to the destination using the most appropriate method. If I understand correctly, we'll create some common library around this. It would be good to understand the details a bit better. I've thought a bit about this issue
Re: [openstack-dev] [Nova][blueprint] Accelerate the booting process of a number of vms via VMThunder
Jay, Huiba, Chris, Solly, Zhiyan, and everybody else, I am so excited that two of the proposals: Image Upload Plugin( http://summit.openstack.org/cfp/details/353) and Data transfer service Plugin(http://summit.openstack.org/cfp/details/352) have been merged together and scheduled in the coming design summit. If you show up in Atlanta, please come this session( http://junodesignsummit.sched.org/event/c00119362c07e4cb203d1c4053add187) and start our discussion, on Wednesday, May 14 • 11:50am - 12:30pm. I will propose a common image transfer library for all the OpenStack projects to to upload and download the images. If it is approved, with this library, Huiba, you can feel free to implement the transfer protocols you like. Best wishes, Vincent Hou (侯胜博) Staff Software Engineer, Open Standards and Open Source Team, Emerging Technology Institute, IBM China Software Development Lab Tel: 86-10-82450778 Fax: 86-10-82453660 Notes ID: Sheng Bo Hou/China/IBM@IBMCNE-mail: sb...@cn.ibm.com Address:3F Ring, Building 28 Zhongguancun Software Park, 8 Dongbeiwang West Road, Haidian District, Beijing, P.R.C.100193 地址:北京市海淀区东北旺西路8号中关村软件园28号楼环宇大厦3层 邮编：100193 Sheng Bo Hou/China/IBM@IBMCN 2014/04/27 22:33 Please respond to OpenStack Development Mailing List \(not for usage questions\) openstack-dev@lists.openstack.org To OpenStack Development Mailing List \(not for usage questions\) openstack-dev@lists.openstack.org, cc Subject Re: [openstack-dev] [Nova][blueprint] Accelerate the booting process of a number of vms via VMThunder I have done a little test for the image download and upload. I created an API for the image access, containing copyFrom and sendTo. I moved the image download and upload code from XenApi into the implementation for Http with some modifications, and the code worked for libvirt as well. copyFrom means to download the image and return the image data, and different hypervisors can choose to save it in a file or import it to the datastore; sendTo is used to upload the image and the image data is passed in as a parameter. I also did an investigation about how each hypervisor is doing the image upload and download. For the download: libvirt, hyper-v and baremetal use the code image_service.download to download the image and save it into a file. vmwareapi uses the code image_service.download to download the image and import it into the datastore. XenAPi uses image_service.download to download the image for VHD image. For the upload: They use image_service.upload to upload the image. I think we can conclude that it is possible to have a common image transfer library with different implementations for different protocols. This is a small demo code for the library: https://review.openstack.org/#/c/90601/(Jay, is it close to the library as you mentioned?). I just replaced the upload and download part with the http implementation for the imageapi and it worked fine. Best wishes, Vincent Hou (侯胜博) Staff Software Engineer, Open Standards and Open Source Team, Emerging Technology Institute, IBM China Software Development Lab Tel: 86-10-82450778 Fax: 86-10-82453660 Notes ID: Sheng Bo Hou/China/IBM@IBMCNE-mail: sb...@cn.ibm.com Address:3F Ring, Building 28 Zhongguancun Software Park, 8 Dongbeiwang West Road, Haidian District, Beijing, P.R.C.100193 地址:北京市海淀区东北旺西路8号中关村软件园28号楼环宇大厦3层 邮编：100193 Solly Ross sr...@redhat.com 2014/04/25 01:46 Please respond to OpenStack Development Mailing List \(not for usage questions\) openstack-dev@lists.openstack.org To OpenStack Development Mailing List (not for usage questions) openstack-dev@lists.openstack.org, cc Subject Re: [openstack-dev] [Nova][blueprint] Accelerate the booting process of a number of vms via VMThunder Something to be aware of when planing an image transfer library is that individual drivers might have optimized support for image transfer in certain cases (especially when dealing with transferring between different formats, like raw to qcow2, etc). This builds on what Christopher was saying -- there's actually a reason why we have code for each driver. While having a common image copying library would be nice, I think a better way to do it would be to have some sort of library composed of building blocks, such that each driver could make use of common functionality while still tailoring the operation to the quirks of the particular drivers. Best Regards, Solly Ross - Original Message - From: Christopher Lefelhocz christopher.lefel...@rackspace.com To: OpenStack Development Mailing List (not for usage questions) openstack-dev@lists.openstack.org Sent: Thursday, April 24, 2014 11:17:41 AM Subject: Re: [openstack-dev] [Nova][blueprint] Accelerate the booting process of a number of vms via VMThunder Apologies for coming to this discussion late... On 4/22/14 6:21 PM, Jay Pipes jaypi...@gmail.com wrote: Right, a good solution would allow for some flexibility
Re: [openstack-dev] [Nova][blueprint] Accelerate the booting process of a number of vms via VMThunder
Right, our discussion ends up with two directions to access the image: full-copying(upload and download the whole image) and zero-copying(remotely access the image and copy on demand). For image transfer via full-copying, nova.image module works, because we only care about how the data(bytes/image) is going to send. It does not matter what the hypervisor is and what the storage it is. I think we can take the library approach with task = image_api.copy(from_path_or_uri, to_path_or_uri) as Jay proposed. For image transfer via zero-copying as Zhi Yan mentioned, depends on the hypervisor and the backend storage. nova.virt is able to access hypervisor and storage context, but not nova.image. The image does not have to be in the same location, where to launch the VM. We care about how the data(image) is accessed(how the VM is launched.) Each of them needs one module to put, and it is hard to merge them. Best wishes, Vincent Hou (侯胜博) Staff Software Engineer, Open Standards and Open Source Team, Emerging Technology Institute, IBM China Software Development Lab Tel: 86-10-82450778 Fax: 86-10-82453660 Notes ID: Sheng Bo Hou/China/IBM@IBMCNE-mail: sb...@cn.ibm.com Address:3F Ring, Building 28 Zhongguancun Software Park, 8 Dongbeiwang West Road, Haidian District, Beijing, P.R.C.100193 地址:北京市海淀区东北旺西路8号中关村软件园28号楼环宇大厦3层 邮编：100193 Zhi Yan Liu lzy@gmail.com 2014/04/24 14:30 Please respond to OpenStack Development Mailing List \(not for usage questions\) openstack-dev@lists.openstack.org To OpenStack Development Mailing List (not for usage questions) openstack-dev@lists.openstack.org, cc Subject Re: [openstack-dev] [Nova][blueprint] Accelerate the booting process of a number of vms via VMThunder First, after the discussing between I and Vincent, I'm sure what we talked for zero-copying in above mail threads is different with full-copying/transferring. The transferring/full-copying means image bits duplication, it focuses on using some method to accelerate image bits replication and transferring between Glance/backend-storage and Nova compute host, like P2P, FTP, HTTP, etc., but zero-copying means there is NO any bits duplication and transferring happened between those two sides during the image preparing for VM provisioning, so the latter one focuses on a) how to attaching remote image volume/disk within Glance managed backend-storage from Nova compute host directly, and b) making the VM's root disk from remote template image based on such hypervisor and particular storage technology, btw c) preventing image bits uploading for VM snapshot/capture case. They are totally different to me. (refer: review comments in https://review.openstack.org/#/c/84671/ ) Second, on the implementation level, I consider to put all image handling related code into nova.image namespace sounds neat but I think it can not work (the leak as last mail side here). IMO, the transferring/full-copying logic is more applicable for nova.image namespace, such transferring approach can be implemented based on existing download module plugins structure, e.g. P2P, FTP, but for the zero-copying, regarding to my above points of view, I consider to implement it in nova.virt + nova.virt.hypervisor is make more sense, since it's more related with particular hypervisor and/or storage technology. (refer: inline comments in https://review.openstack.org/#/c/86583/6/specs/juno/image-multiple-location.rst ) zhiyan On Wed, Apr 23, 2014 at 11:02 PM, Jay Pipes jaypi...@gmail.com wrote: On Wed, 2014-04-23 at 13:56 +0800, lihuiba wrote: For live migration, we use shared storage so I don't think it's quite the same as getting/putting image bits from/to arbitrary locations. With a good zero-copy transfer lib, live migration support can be extended to non-shared storage, or cross-datacenter. It's a kind of value. Hmm, I totally see the value of doing this. Not sure that there could be the same kinds of liveness guarantees with non-shared-storage, but I am certainly happy to see a proof of concept in this area! :) task = image_api.copy(from_path_or_uri, to_path_or_uri) # do some other work copy_task_result = task.wait() +1 looks cool! how about zero-copying? It would be an implementation detail within nova.image.api.copy() function (and the aforementioned image bits mover library) :) The key here is to leak as little implementation detail out of the nova.image.api module Best, -jay At 2014-04-23 07:21:27,Jay Pipes jaypi...@gmail.com wrote: Hi Vincent, Zhi, Huiba, sorry for delayed response. See comments inline. On Tue, 2014-04-22 at 10:59 +0800, Sheng Bo Hou wrote: I actually support the idea Huiba has proposed, and I am thinking of how to optimize the large data transfer(for example, 100G in a short time) as well. I registered two blueprints in nova-specs, one is for an image upload plug-in to upload the image to glance(https://review.openstack.org/#/c/84671/), the other is a data
Re: [openstack-dev] [nova] a question about instance snapshot
Hi Jay and Zhao Qin, Thank you for your reply. I have recap my recent ideas about the blueprints and put them in the link: https://etherpad.openstack.org/p/live-snapshot. Waiting for your comments. Thank you folks again. Best wishes, Vincent Hou (侯胜博) Staff Software Engineer, Open Standards and Open Source Team, Emerging Technology Institute, IBM China Software Development Lab Tel: 86-10-82450778 Fax: 86-10-82453660 Notes ID: Sheng Bo Hou/China/IBM@IBMCNE-mail: sb...@cn.ibm.com Address:3F Ring, Building 28 Zhongguancun Software Park, 8 Dongbeiwang West Road, Haidian District, Beijing, P.R.C.100193 地址:北京市海淀区东北旺西路8号中关村软件园28号楼环宇大厦3层 邮编：100193 Jay Pipes jaypi...@gmail.com 2014/03/14 11:40 To Sheng Bo Hou/China/IBM@IBMCN, cc chaoc...@gmail.com Subject Re: 转发: Re: [openstack-dev] [nova] a question about instance snapshot Hi again, Vincent! I'm including Qin Zhao (cc'd) in our conversation, since we were chatting about this on IRC :) Qin helpfully created an Etherpad where we are beginning to discuss this blueprint (and the related half-completed one). https://etherpad.openstack.org/p/live-snapshot See you on the etherpad! :) Best, -jay On Fri, 2014-03-14 at 09:47 +0800, Sheng Bo Hou wrote: Hi Jay, I found you are in the discussion about live snapshot. I came up with a relatively generic solution for Nova in the following mail. Hope you can take a look review and give me your feedbacks. Thank you so much. Best wishes, Vincent Hou (侯胜博) Hi everyone, I got excited to hear that this live snapshot has been taken into discussion in our community. Recently my clients in China came up with this live snapshot requirement as well, because they have already had their legacy environment and expect the original functions work fine when they transfer to use OpenStack. In my opinion, we need to think a little bit about these clients' needs, because it is also a potential market for OpenStack. I registered a new blueprint for Nova https://blueprints.launchpad.net/nova/+spec/driver-specific-snapshot. It is named driver-specific before, but can be changed later. The Nova API could be implemented via the extension, the following API may be added: • CreateSnapshot: create a snapshot from the VM. The snapshot can be live snapshot or other hypervisor native way to create a snapshot. • RestoreFromSnapshot: restore/revert the VM from a snapshot. • DeleteSnapshot: delete a snapshot. • ListSnapshot: list all the snapshots or list all the snapshots if a VM id is given. • SpawnFromSnapshot: spawn a new VM from an existing snapshot, which is the live snapshot or the snapshot of other snapshot created in a hypervisor native way. The features in this blueprint can be optional for any drivers. If a driver does not have a native way to do live snapshot or other kind of snapshots, it is fine to leave the API not implemented; if a driver can provide the native feature to do snapshot, it is an opportunity to reinforce Nova with this snapshot support. I sincerely need your comments and hope we can figure it out in a most favorable way. Thank you so much. Best wishes, Vincent Hou (侯胜博) Staff Software Engineer, Open Standards and Open Source Team, Emerging Technology Institute, IBM China Software Development Lab Tel: 86-10-82450778 Fax: 86-10-82453660 Notes ID: Sheng Bo Hou/China/IBM@IBMCNE-mail: sb...@cn.ibm.com Address:3F Ring, Building 28 Zhongguancun Software Park, 8 Dongbeiwang West Road, Haidian District, Beijing, P.R.C.100193 地址:北京市海淀区东北旺西路8号中关村软件园28号楼环宇大厦3层 邮编： 100193 Jay Pipes jaypi...@gmail.com 2014/03/12 03:15 Please respond to OpenStack Development Mailing List \(not for usage questions\) openstack-dev@lists.openstack.org To openstack-dev@lists.openstack.org, cc Subject Re: [openstack-dev] [nova] a question about instance snapshot On Tue, 2014-03-11 at 06:35 +, Bohai (ricky) wrote: -Original Message- From: Jay Pipes [mailto:jaypi...@gmail.com] Sent: Tuesday, March 11, 2014 3:20 AM To: openstack-dev@lists.openstack.org Subject: Re: [openstack-dev] [nova] a question about instance snapshot On Mon, 2014-03-10 at 12:13 -0400, Shawn Hartsock wrote: We have very strong interest in pursing this feature in the VMware driver as well. I would like to see the revert instance feature implemented at least. When I used to work in multi-discipline roles involving operations it would be common for us to snapshot a vm, run through an upgrade process, then revert if something did not upgrade smoothly. This ability alone can be exceedingly valuable in long-lived virtual machines. I also have some comments from parties interested in refactoring how the VMware drivers handle snapshots but I'm not certain how much that plays into this live snapshot discussion. I think the reason that there isn't much interest
Re: [openstack-dev] [nova] a question about instance snapshot
Hi everyone, I got excited to hear that this live snapshot has been taken into discussion in our community. Recently my clients in China came up with this live snapshot requirement as well, because they have already had their legacy environment and expect the original functions work fine when they transfer to use OpenStack. In my opinion, we need to think a little bit about these clients' needs, because it is also a potential market for OpenStack. I registered a new blueprint for Nova https://blueprints.launchpad.net/nova/+spec/driver-specific-snapshot. It is named driver-specific before, but can be changed later. The Nova API could be implemented via the extension, the following API may be added: • CreateSnapshot: create a snapshot from the VM. The snapshot can be live snapshot or other hypervisor native way to create a snapshot. • RestoreFromSnapshot: restore/revert the VM from a snapshot. • DeleteSnapshot: delete a snapshot. • ListSnapshot: list all the snapshots or list all the snapshots if a VM id is given. • SpawnFromSnapshot: spawn a new VM from an existing snapshot, which is the live snapshot or the snapshot of other snapshot created in a hypervisor native way. The features in this blueprint can be optional for any drivers. If a driver does not have a native way to do live snapshot or other kind of snapshots, it is fine to leave the API not implemented; if a driver can provide the native feature to do snapshot, it is an opportunity to reinforce Nova with this snapshot support. I sincerely need your comments and hope we can figure it out in a most favorable way. Thank you so much. Best wishes, Vincent Hou (侯胜博) Staff Software Engineer, Open Standards and Open Source Team, Emerging Technology Institute, IBM China Software Development Lab Tel: 86-10-82450778 Fax: 86-10-82453660 Notes ID: Sheng Bo Hou/China/IBM@IBMCNE-mail: sb...@cn.ibm.com Address:3F Ring, Building 28 Zhongguancun Software Park, 8 Dongbeiwang West Road, Haidian District, Beijing, P.R.C.100193 地址:北京市海淀区东北旺西路8号中关村软件园28号楼环宇大厦3层 邮编：100193 Jay Pipes jaypi...@gmail.com 2014/03/12 03:15 Please respond to OpenStack Development Mailing List \(not for usage questions\) openstack-dev@lists.openstack.org To openstack-dev@lists.openstack.org, cc Subject Re: [openstack-dev] [nova] a question about instance snapshot On Tue, 2014-03-11 at 06:35 +, Bohai (ricky) wrote: -Original Message- From: Jay Pipes [mailto:jaypi...@gmail.com] Sent: Tuesday, March 11, 2014 3:20 AM To: openstack-dev@lists.openstack.org Subject: Re: [openstack-dev] [nova] a question about instance snapshot On Mon, 2014-03-10 at 12:13 -0400, Shawn Hartsock wrote: We have very strong interest in pursing this feature in the VMware driver as well. I would like to see the revert instance feature implemented at least. When I used to work in multi-discipline roles involving operations it would be common for us to snapshot a vm, run through an upgrade process, then revert if something did not upgrade smoothly. This ability alone can be exceedingly valuable in long-lived virtual machines. I also have some comments from parties interested in refactoring how the VMware drivers handle snapshots but I'm not certain how much that plays into this live snapshot discussion. I think the reason that there isn't much interest in doing this kind of thing is because the worldview that VMs are pets is antithetical to the worldview that VMs are cattle, and Nova tends to favor the latter (where DRS/DPM on vSphere tends to favor the former). There's nothing about your scenario above of being able to revert an instance to a particular state that isn't possible with today's Nova. Snapshotting an instance, doing an upgrade of software on the instance, and then restoring from the snapshot if something went wrong (reverting) is already fully possible to do with the regular Nova snapshot and restore operations. The only difference is that the live-snapshot stuff would include saving the memory view of a VM in addition to its disk state. And that, at least in my opinion, is only needed when you are treating VMs like pets and not cattle. Hi Jay, I read every words in your reply and respect what you said. But i can't agree with you that memory snapshot is a feature for pat not for cattle. I think it's a feature whatever what do you look the instance as. The world doesn't care about what we look the instance as, in fact, currently almost all the mainstream hypervisors have supported the memory snapshot. If it's just a dispensable feature and no users need it, I can't understand why the hypervisors provide it without exception. In the document OPENSTACK OPERATIONS GUIDE section Live snapshots has the below words: To ensure that important services have written their contents to disk (such as, databases), we recommend you read
Re: [openstack-dev] [cinder] Propose to add copying the reference images when creating a volume
Hi Vish, I would like you to review this patch https://review.openstack.org/#/c/35460/. I think this approach takes the least effort to fix this issue. When we boot an instance from a volume, nova can get the volume information by _get_volume. kerel_id and ramdisk_id are already in the volume information. We just need to make nova retrieve them. In the code of creating instance, kernel_id and ramdisk_id are accessed by checking properties(many parts), but the volume information saved them in volume_image_metadata. I just convert the data structure a bit and save two of this params in properties, and it can work. Well, if you do not see it favorable, we can work it out in another way. Thank you. Best wishes, Vincent Hou (侯胜博) Staff Software Engineer, Open Standards and Open Source Team, Emerging Technology Institute, IBM China Software Development Lab Tel: 86-10-82450778 Fax: 86-10-82453660 Notes ID: Sheng Bo Hou/China/IBM@IBMCNE-mail: sb...@cn.ibm.com Address:3F Ring, Building 28 Zhongguancun Software Park, 8 Dongbeiwang West Road, Haidian District, Beijing, P.R.C.100193 地址:北京市海淀区东北旺西路8号中关村软件园28号楼环宇大厦3层 邮编：100193 Vishvananda Ishaya vishvana...@gmail.com 2013/07/02 01:14 Please respond to OpenStack Development Mailing List openstack-dev@lists.openstack.org To OpenStack Development Mailing List openstack-dev@lists.openstack.org, cc jsbry...@us.ibm.com, Duncan Thomas duncan.tho...@gmail.com John Griffith duncan.thomas@gmail.comduncan.thomas Subject Re: [openstack-dev] [cinder] Propose to add copying the reference images when creating a volume On Jul 1, 2013, at 3:35 AM, Sheng Bo Hou sb...@cn.ibm.com wrote: Hi Mate, First, thanks for answering. I was trying to find the way to prepare the bootable volume. Take the default image downloaded by devstack, there are three images: cirros-0.3.0-x86_64-uec, cirros-0.3.0-x86_64-uec-kernel and cirros-0.3.0-x86_64-uec-ramdisk. cirros-0.3.0-x86_64-uec-kernel is referred as the kernel image and cirros-0.3.0-x86_64-uec-ramdisk is referred as the ramdisk image. Issue: If only the image(cirros-0.3.0-x86_64-uec) is copied to the volume when creating a volume) from an image, this volume is unable to boot an instance without the references to the kernel and the ramdisk images. The current cinder only copies the image cirros-0.3.0-x86_64-uec to one targeted volume(Vol-1), which is marked as bootable but unable to do a successful boot with the current nova code, even if image-id is removed in the parameter. Possible solutions: There are two ways in my mind to resolve it. One is we just need the code change in Nova to let it find the reference images for the bootable volume(Vol-1) and there is no need to change anything in cinder, since the kernel and ramdisk id are saved in the volume_glance_metadata, where the references point to the images(kernel and ramdisk) for the volume(Vol-1). You should be able to create an image in glance that references the volume in block device mapping but also has a kernel_id and ramdisk_id parameter so it can boot properly. I know this is kind of an odd way to do things, but this seems like an edge case and I think it is a valid workaround. Vish The other is that if we need multiple images to boot an instance, we need a new way to create the bootable volume. For example, we can create three separate volumes for three of the images and set the new references in volume_glance_metadata with the kernel_volume_id and ramdisk_volume_id. The benefit of this approach is that the volume can live independent of the existence of the original images. Even the images get lost accidentally, the volumes are still sufficient to boot an instance, because all the information have been copied to Cinder part. I am trying to looking for the another way to prepare your bootable volume as you mentioned and asking for the suggestions. And I think the second approach could be one way. Do you think it is a good approach? Best wishes, Vincent Hou (侯胜博) Staff Software Engineer, Open Standards and Open Source Team, Emerging Technology Institute, IBM China Software Development Lab Tel: 86-10-82450778 Fax: 86-10-82453660 Notes ID: Sheng Bo Hou/China/IBM@IBMCNE-mail: sb...@cn.ibm.com Address:3F Ring, Building 28 Zhongguancun Software Park, 8 Dongbeiwang West Road, Haidian District, Beijing, P.R.C.100193 地址:北京市海淀区东北旺西路8号中关村软件园28号楼环宇大厦3层 邮编：100193 Mate Lakat mate.la...@citrix.com 2013/07/01 04:18 Please respond to OpenStack Development Mailing List openstack-dev@lists.openstack.org To OpenStack Development Mailing List openstack-dev@lists.openstack.org, cc jsbry...@us.ibm.com, Duncan Thomas duncan.tho...@gmail.com John Griffith duncan.thomas@gmail.comduncan.thomas Subject Re: [openstack-dev] [cinder] Propose to add copying the reference images when creating a volume Hi, I just proposed a patch for the boot_from_volume_exercise.sh to get rid of --image. To be honest, I did
[openstack-dev] [cinder] Propose to add copying the reference images when creating a volume
Hi Cinder folks, I am currently fixing the bugs related to booting the instance from the volume. I found there are bugs both in Nova and Cinder. Cinder: https://bugs.launchpad.net/cinder/+bug/1159824 Nova: https://bugs.launchpad.net/nova/+bug/1191069 For the volumes created from the image, I propose to copy the reference image during the creation of the main image. For example, an image may refer to a kernel image and a ramdisk image. When we create a volume from this image, we only copied this one to the volume. The kernel and ramdisk images are still in glance, and the volume still refers to the kernel and ramdisk images. I think if an image has other reference images, the reference images also need to be copied to the volumes(kernel volume and ramdisk volume), and then set the volume referring to the kernel volume and the ramdisk volume. This feature will make booting from a volume completely independent of the existence of the glance image. Do you think we can firstly add this feature to cinder? Do folks have any comments on it? Thanks. Best wishes, Vincent Hou (侯胜博) Staff Software Engineer, Open Standards and Open Source Team, Emerging Technology Institute, IBM China Software Development Lab Tel: 86-10-82450778 Fax: 86-10-82453660 Notes ID: Sheng Bo Hou/China/IBM@IBMCNE-mail: sb...@cn.ibm.com Address:3F Ring, Building 28 Zhongguancun Software Park, 8 Dongbeiwang West Road, Haidian District, Beijing, P.R.C.100193 地址:北京市海淀区东北旺西路8号中关村软件园28号楼环宇大厦3层 邮编：100193 ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
