Re: [DISCUSS] extending the libvirt/KVM plugin to also support libvirt/Xen
On 06/08/2014 11:14 PM, Dave Scott wrote: Hi Wido, Thanks for your mail! On 8 Jun 2014, at 19:02, Wido den Hollander w...@widodh.nl wrote: On 06/08/2014 06:23 PM, Dave Scott wrote: Hi, Following on from the earlier [PROPOSAL] Support pure Xen as a hypervisor”, I’ve added a design doc to the wiki: https://cwiki.apache.org/confluence/display/CLOUDSTACK/Allow+hosts+running+the+Xen+hypervisor+to+be+managed+via+libvirt This design would allow people who want to manage their hypervisors purely through the libvirt tools to choose the Xen hypervisor. From the code point of view, I want to maximise sharing between the KVM and Xen code paths, partly to make QA easier and partly to maximise the chance that adding a feature for “Xen” causes it to work for “KVM” and vice-versa. In particular this means that, if a genuinely-useful capability is currently missing from the libvirt libxl driver, I want to implement it rather than work around it. Seems like a great route to me! You also want to support Xen+Qemu with this way? Yes, it should be possible to run fully virtualised VMs with Xen + Qemu. I think we’ll be able to choose whether to run VMs as PV or HVM. Ok, but those will be different code paths at some level. We have to be aware that there might be some storage differences between KVM and Xen like Ceph which is not fully supported yet by Xen. Ceph is an interesting one. Xen itself doesn’t know anything about storage— instead the dom0 takes care of it either via a kernel driver (blkback) or userspace program (qemu or tapdisk). When I tried to make Ceph work about a year ago[1] I hit a bug in libxl (the Xen control library). The good news is the fix made it into Xen 4.4, so with luck we can get it to work. When Xen runs with Qemu as full HVM it's Qemu which takes care of the Ceph storage, so in that case it's fixed. I haven't got a lot of experience with PV Xen. I heard stories of Ceph being integrated in blktap(2), but never tested it. If anything is missing in libvirt or the Java bindings we have to fix that indeed instead of hacking around it. Great :) Cheers, Dave [1] http://xenserver.org/discuss-virtualization/virtualization-blog/entry/tech-preview-of-xenserver-libvirt-ceph.html Wido Comments appreciated! Cheers, Dave [1] http://mail-archives.apache.org/mod_mbox/cloudstack-users/201403.mbox/%3ccajgxtbnbmqtq81ralgh2kma7v5wjyzkr3xnyasmkc_br+uk...@mail.gmail.com%3e
Review Request 22364: Fixed 26 issues reported by coverity
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/22364/ --- Review request for cloudstack and Kelven Yang. Repository: cloudstack-git Description --- NPEs, unused code or dead code, unwritten field access and self assignment Diffs - engine/orchestration/src/com/cloud/vm/VirtualMachineManagerImpl.java 8ca7d1e Diff: https://reviews.apache.org/r/22364/diff/ Testing --- Thanks, Rajani Karuturi
Re: ssh keypair value overridden by template meta data
Hi Hiroki,
We should not override the ssh key pair provided in the deployVM API with the
ssh key in template metadata.
I just able to reproduce this. Please create a ticket for this issue.
Thanks
Harikrishna
On 06-Jun-2014, at 5:18 pm, Hiroki Ohashi hiroki.s...@gmail.com wrote:
Dear guys
I encountered a problem that a ssh public key of ssh_keypairs for a
newly created instance is overridden by another ssh key in template
meta data. I think this leads to security vulnerability because a
template owner can login to other user's instance created from the
template. So, could you fix this issue?
This behavior is caused by meta data import at commitUserVm method. A
ssh key value specified by an instance owner is set to a UserVmVO
object at line 2986-2988 of
server/src/com/cloud/vm/UserVmManagerImpl.java in 4.3 branch [1], but
this value is overridden at line 3035-3038 by template meta data.
Please note a database contains meta data entries related to a
template which you created from an instance in cloud.template_view
like [2]. 2nd row has detail_name and detail_value about ssh key and
CloudStack override the user specified ssh key value by the owner's
detail value in this situation. It results in delivery of the template
owner's ssh key for the instance created from the template to virtual
router inspite of specification of instance owner's ssh key.
You can reproduce this phenomenon like below.
1. Deploy an instance with a ssh key A by specifying 'keypair'
value.
2. Create a template from this instance.
3. Deploy an instance with another ssh key B by specifying
'keypair' value.
[1] server/src/com/cloud/vm/UserVmManagerImpl.java
2971 private UserVmVO commitUserVm(final DataCenter zone, final
VirtualMachineTemplate template, final String hostName, final String
displayName, final Account owner,
2972 final Long diskOfferingId, final Long diskSize, final
String userData, final HypervisorType hypervisor, final Account
caller, final Boolean isDisplayVmEnabled,
2973 final String keyboard, final long accountId, final
ServiceOfferingVO offering, final boolean isIso, final String
sshPublicKey,
2974 final LinkedHashMapString, NicProfile networkNicMap,
final long id, final String instanceName, final String uuidName, final
HypervisorType hypervisorType,
2975 final MapString, String customParameters) throws
InsufficientCapacityException {
2976 return Transaction.execute(new
TransactionCallbackWithExceptionUserVmVO,
InsufficientCapacityException() {
2977 @Override
2978 public UserVmVO doInTransaction(TransactionStatus
status) throws InsufficientCapacityException {
2979 UserVmVO vm = new UserVmVO(id, instanceName,
displayName,
2980 template.getId(), hypervisorType,
template.getGuestOSId(),
2981 offering.getOfferHA(),
offering.getLimitCpuUse(),
2982 owner.getDomainId(), owner.getId(),
offering.getId(), userData,
2983 hostName, diskOfferingId);
2984 vm.setUuid(uuidName);
2985
vm.setDynamicallyScalable(template.isDynamicallyScalable());
2986 if (sshPublicKey != null) {
2987 vm.setDetail(SSH.PublicKey, sshPublicKey);
2988 }
2989
2990 if (keyboard != null !keyboard.isEmpty())
2991 vm.setDetail(VmDetailConstants.KEYBOARD, keyboard);
2992
2993 if (isIso) {
2994 vm.setIsoId(template.getId());
2995 }
2996
2997 if(isDisplayVmEnabled != null){
2998 if(!_accountMgr.isRootAdmin(caller.getType())){
2999 throw new PermissionDeniedException(
Cannot update parameter displayvm, only admin permitted );
3000 }
3001 vm.setDisplayVm(isDisplayVmEnabled);
3002 }else {
3003 vm.setDisplayVm(true);
3004 }
3005
3006 // If hypervisor is vSphere, check for clone
type setting.
3007 if (hypervisorType.equals(HypervisorType.VMware)) {
3008 // retrieve clone flag.
3009 UserVmCloneType cloneType = UserVmCloneType.linked;
3010 String value =
_configDao.getValue(Config.VmwareCreateFullClone.key());
3011 if (value != null) {
3012 if (Boolean.parseBoolean(value) == true)
3013 cloneType = UserVmCloneType.full;
3014 }
3015 UserVmCloneSettingVO vmCloneSettingVO =
new UserVmCloneSettingVO(id, cloneType.toString());
3016
RE: KVM + LXC on the same host
-Original Message-
From: ilya musayev [mailto:ilya.mailing.li...@gmail.com]
Sent: Saturday, 7 June 2014 1:50 AM
To: dev@cloudstack.apache.org
Subject: Re: KVM + LXC on the same host
Tuna
Thanks for the feedback, conceptually, i was not trying to address the issue
of
sysvms not running on LXC.
Here is the use case as i see it.
Assume you roll out a farm of 10 KVM servers and your density with KVM is
200 virtual machines. Most of these VMs dont require independent kernel
and additional layers of virtualization abstraction. As the result, you can
place
400 LXC machines and 20 fully virtualized Linux or even Windows Servers. If
you do chargeback, you can offer the LXC machines for much lower price point
since we can place more LXC containers.
Your density becomes much greater with LXC and yet you still cover the
corner case when end-user needs KVM instance.
Pierre-Luc
If I will get to try this scenario - i may have to alter the KVM/LXC agent
somewhat to make it work with CloudStack, i will let you know.
Ilya,
Some minor changes to KVM/LXC agent will be required to make this work. Below
piece of code in createVMFromSpec (LibvirtComputingResource), deploys systems
Vms in KVM and user Vms in LXC.
If we include some flag in VirtualMachineTO, it should be possible to specify
which userVm has to be deployed (KVM/LXC).
if (HypervisorType.LXC == _hypervisorType VirtualMachine.Type.User
== vmTO.getType()) {
// LXC domain is only valid for user VMs. Use KVM for system VMs.
guest.setGuestType(GuestDef.guestType.LXC);
vm.setHvsType(HypervisorType.LXC.toString().toLowerCase());
} else {
guest.setGuestType(GuestDef.guestType.KVM);
vm.setHvsType(HypervisorType.KVM.toString().toLowerCase());
vm.setLibvirtVersion(_hypervisorLibvirtVersion);
vm.setQemuVersion(_hypervisorQemuVersion);
}
Alternative solution would be to run separate LXC and KVM clusters, which is
also a possibility - but usage and distribution will be uneven.
Regards
ilya
The concept of combining both technologies under one hypervisor On 6/6/14,
8:42 AM, Pierre-Luc Dion wrote:
ilya,
Let us know how it goes your hybrid of LXC+KVM. I'm interested to know
how it's going, I might try that too on the side.
Pierre-Luc Dion
Architecte de Solution Cloud | Cloud Solutions Architect 855-OK-CLOUD
(855-652-5683) x1101
- - -
*CloudOps*420 rue Guy
Montréal QC H3J 1S6
www.cloudops.com
@CloudOps_
On Fri, Jun 6, 2014 at 10:53 AM, Nguyen Anh Tu t...@apache.org wrote:
That should be a good idea, Ilya. At that moment, i'm working on
Docker support. When it's done, we can run only Docker hosts, no need
to use KVM for hosting system vms.
Cheers,
--Tuna
Sent from my GT-N7000
On Jun 5, 2014 7:58 AM, ilya musayev ilya.mailing.li...@gmail.com
wrote:
We are considering running KVM and LXC on the same host and
hopefully control both through cloudstack.
I know there are agents involved for each component, i dont know if
we
can
have a hybrid of LXC+KVM.
The use case is simple, we would like the end user to pick
LXC/Docker for performance, or KVM instance if he really needed all
bells and whistles
of
dedicated kernel in fully virtualized environment.
Is anyone aware why we should not mix 2 workloads on the same host?
Is it possible at this point in time to mix LXC, KVM and CloudStack,
i assume
the
answer is no, but perhaps there is a hack i can try.
Thanks
ilya
Re: Review Request 22364: Fixed 26 issues reported by coverity
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/22364/ --- (Updated June 9, 2014, 7:09 a.m.) Review request for cloudstack and Kelven Yang. Changes --- Fixed checkstyle issue Repository: cloudstack-git Description --- NPEs, unused code or dead code, unwritten field access and self assignment Diffs (updated) - engine/orchestration/src/com/cloud/vm/VirtualMachineManagerImpl.java 25c67db Diff: https://reviews.apache.org/r/22364/diff/ Testing --- Thanks, Rajani Karuturi
Re: Review Request 22354: CLOUDSTACK-6850: Return cpu cores, cpu speed and memory in listUsageRecords
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/22354/#review45061 --- Hi, is it a bug or a new feature ? I see that it alters the usage db schema, will this affects upgrades ? The jira entry says that it applies to 4.3 as well, does you patch applies to 4.3 ? - Sebastien Goasguen On June 8, 2014, 4:40 p.m., Olivier Lemasle wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/22354/ --- (Updated June 8, 2014, 4:40 p.m.) Review request for cloudstack and Kishan Kavala. Bugs: CLOUDSTACK-6850 https://issues.apache.org/jira/browse/CLOUDSTACK-6850 Repository: cloudstack-git Description --- This patch updates VMInstanceUsageParser to process usage details (cpu cores, cpu speed and memory) for dynamic compute offering usages, in order to save them in table cloud_usage.cloud_usage and return them with the API (listUsageRecords). See CLOUDSTACK-6850. A change in database model is required for this patch (three new columns in cloud_usage.cloud_usage). The patch should apply on master and 4.4. Diffs - api/src/org/apache/cloudstack/api/response/UsageRecordResponse.java 5e2e85d api/src/org/apache/cloudstack/usage/Usage.java 20dc189 engine/schema/src/com/cloud/usage/UsageVMInstanceVO.java 06d4876 engine/schema/src/com/cloud/usage/UsageVO.java 67014ef engine/schema/src/com/cloud/usage/dao/UsageVMInstanceDaoImpl.java b94e12f server/src/com/cloud/api/ApiResponseHelper.java 119f56d setup/db/db/schema-430to440.sql d149a65 usage/src/com/cloud/usage/parser/VMInstanceUsageParser.java ba0d4bf Diff: https://reviews.apache.org/r/22354/diff/ Testing --- Packaged in RPMs, installed and manually tested. 1. Create a VM with compute offering Small instance. 2. Stop the VM. 3. Change service offering to a custom offering, with cpu_number=1, cpu_speed=500 MHz, memory=512 MB 4. Start the VM, wait... Stop the VM. 6. Change service offering to an other custom offering, with cpu_number=2, cpu_speed=800 MHz, memory=1024 MB 7. Start the VM, wait... Stop the VM. 8. Change service offering to the first custom offering, with same details (cpu_number=2, cpu_speed=800 MHz, memory=1024 MB) 9. Start the VM, wait... Stop the VM. 10. Change service offering to Medium instance. Usages and details are reflected in database and in API response. There is no regression found for other usage types. Details are returned (and saved in database) only for RUNNING_VM and ALLOCATED_VM, only if the service offering is a custom one. Example of response: usagerecord accountadmin/account accountid184a6564-edab-11e3-b629-0050569ed71c/accountid domainidfbc7578a-edaa-11e3-b629-0050569ed71c/domainid zoneide5047089-c911-40ea-8251-4a76810ac1e0/zoneid descriptioncalifornia allocated (ServiceOffering: 12) (Template: 5)/description usage0.098078 Hrs/usage usagetype2/usagetype rawusage0.098078/rawusage virtualmachineide89c22af-65cf-42a2-8905-6f4bbd7d59a1/virtualmachineid namecalifornia/name offeringidb05b64e9-8e0f-4335-b0e0-acc5bad81938/offeringid templateidfbca2564-edaa-11e3-b629-0050569ed71c/templateid usageide89c22af-65cf-42a2-8905-6f4bbd7d59a1/usageid typeXenServer/type cpunumber1/cpunumber cpuspeed500/cpuspeed memory512/memory startdate2014-06-08'T'17:05:52+02:00/startdate enddate2014-06-08'T'17:15:52+02:00/enddate /usagerecord usagerecord accountadmin/account accountid184a6564-edab-11e3-b629-0050569ed71c/accountid domainidfbc7578a-edaa-11e3-b629-0050569ed71c/domainid zoneide5047089-c911-40ea-8251-4a76810ac1e0/zoneid descriptioncalifornia allocated (ServiceOffering: 1) (Template: 5)/description usage0.068594 Hrs/usage usagetype2/usagetype rawusage0.068594/rawusage virtualmachineide89c22af-65cf-42a2-8905-6f4bbd7d59a1/virtualmachineid namecalifornia/name offeringid84bbf38f-b582-440b-8a8d-20e63e0dbed7/offeringid templateidfbca2564-edaa-11e3-b629-0050569ed71c/templateid usageide89c22af-65cf-42a2-8905-6f4bbd7d59a1/usageid typeXenServer/type startdate2014-06-08'T'17:05:52+02:00/startdate enddate2014-06-08'T'17:15:52+02:00/enddate /usagerecord Thanks, Olivier Lemasle
Re: ssh keypair value overridden by template meta data
Hi Harikrishna
Thank you very much for your reproduction work.
I created a ticket for this issue.
https://issues.apache.org/jira/browse/CLOUDSTACK-6869
Please confirm it.
Best Regards
2014-06-09 15:59 GMT+09:00 Harikrishna Patnala harikrishna.patn...@citrix.com:
Hi Hiroki,
We should not override the ssh key pair provided in the deployVM API with the
ssh key in template metadata.
I just able to reproduce this. Please create a ticket for this issue.
Thanks
Harikrishna
On 06-Jun-2014, at 5:18 pm, Hiroki Ohashi hiroki.s...@gmail.com wrote:
Dear guys
I encountered a problem that a ssh public key of ssh_keypairs for a
newly created instance is overridden by another ssh key in template
meta data. I think this leads to security vulnerability because a
template owner can login to other user's instance created from the
template. So, could you fix this issue?
This behavior is caused by meta data import at commitUserVm method. A
ssh key value specified by an instance owner is set to a UserVmVO
object at line 2986-2988 of
server/src/com/cloud/vm/UserVmManagerImpl.java in 4.3 branch [1], but
this value is overridden at line 3035-3038 by template meta data.
Please note a database contains meta data entries related to a
template which you created from an instance in cloud.template_view
like [2]. 2nd row has detail_name and detail_value about ssh key and
CloudStack override the user specified ssh key value by the owner's
detail value in this situation. It results in delivery of the template
owner's ssh key for the instance created from the template to virtual
router inspite of specification of instance owner's ssh key.
You can reproduce this phenomenon like below.
1. Deploy an instance with a ssh key A by specifying 'keypair'
value.
2. Create a template from this instance.
3. Deploy an instance with another ssh key B by specifying
'keypair' value.
[1] server/src/com/cloud/vm/UserVmManagerImpl.java
2971 private UserVmVO commitUserVm(final DataCenter zone, final
VirtualMachineTemplate template, final String hostName, final String
displayName, final Account owner,
2972 final Long diskOfferingId, final Long diskSize, final
String userData, final HypervisorType hypervisor, final Account
caller, final Boolean isDisplayVmEnabled,
2973 final String keyboard, final long accountId, final
ServiceOfferingVO offering, final boolean isIso, final String
sshPublicKey,
2974 final LinkedHashMapString, NicProfile networkNicMap,
final long id, final String instanceName, final String uuidName, final
HypervisorType hypervisorType,
2975 final MapString, String customParameters) throws
InsufficientCapacityException {
2976 return Transaction.execute(new
TransactionCallbackWithExceptionUserVmVO,
InsufficientCapacityException() {
2977 @Override
2978 public UserVmVO doInTransaction(TransactionStatus
status) throws InsufficientCapacityException {
2979 UserVmVO vm = new UserVmVO(id, instanceName,
displayName,
2980 template.getId(), hypervisorType,
template.getGuestOSId(),
2981 offering.getOfferHA(),
offering.getLimitCpuUse(),
2982 owner.getDomainId(), owner.getId(),
offering.getId(), userData,
2983 hostName, diskOfferingId);
2984 vm.setUuid(uuidName);
2985
vm.setDynamicallyScalable(template.isDynamicallyScalable());
2986 if (sshPublicKey != null) {
2987 vm.setDetail(SSH.PublicKey, sshPublicKey);
2988 }
2989
2990 if (keyboard != null !keyboard.isEmpty())
2991 vm.setDetail(VmDetailConstants.KEYBOARD,
keyboard);
2992
2993 if (isIso) {
2994 vm.setIsoId(template.getId());
2995 }
2996
2997 if(isDisplayVmEnabled != null){
2998 if(!_accountMgr.isRootAdmin(caller.getType())){
2999 throw new PermissionDeniedException(
Cannot update parameter displayvm, only admin permitted );
3000 }
3001 vm.setDisplayVm(isDisplayVmEnabled);
3002 }else {
3003 vm.setDisplayVm(true);
3004 }
3005
3006 // If hypervisor is vSphere, check for clone
type setting.
3007 if (hypervisorType.equals(HypervisorType.VMware)) {
3008 // retrieve clone flag.
3009 UserVmCloneType cloneType =
UserVmCloneType.linked;
3010 String value =
_configDao.getValue(Config.VmwareCreateFullClone.key());
3011 if (value != null) {
3012 if (Boolean.parseBoolean(value) == true)
Re: [ACS44] 112 unpicked cherries in 4.4-forward. why?
On 09-Jun-2014, at 12:56 PM, sebgoa run...@gmail.com wrote: On Jun 8, 2014, at 7:25 PM, Mike Tutkowski mike.tutkow...@solidfire.com wrote: Yes, there appears to be at least two lines of thought on x.y-forward branches (specifically using 4.4-forward as an example here). 1) 4.4-forward and 4.4 should eventually be the same. Once the 4.4 release goes out the door, 4.4-forward should be removed and changes for 4.4.1, should such a release ever be made, should go into 4.4. That's what I tried to say. I you commit to 4.4-forward that means that you want it in 4.4.0, otherwise go to master. [Koushik] What if the fix is not that critical for 4.4 but would be good to have in 4.4.1 and above apart from master. I would be concerned if the fix is in 4.4-forward but not in master. 2) 4.4-forward contains changes that might go into 4.4 (if a cherry pick is requested) and changes that would go into 4.4.1, should such a release ever be made. In both cases: Most all changes that go into 4.4-forward would need to go into master (unless that part of the codebase in master has been modified in such a way as to no longer make this 4.4-forward change relevant for master). We should gain some consensus on what x.y-forward means. On Sun, Jun 8, 2014 at 10:51 AM, Daan Hoogland daan.hoogl...@gmail.com wrote: As I read your comments you actually don't agree with Sebastien, Sudha. 4.4-forward was cut of at code freeze. Since then fixes are committed there and cherry-pick are done by the RM. It seems that this is exactly what Sebastien meant it to be for. @Sebastien: in Shengs words I read that some of the things in 4.4-forward would only go in 4.4 after the release (thing with lower priority then critical) We all don't agree, let's agree on that.;} On Sun, Jun 8, 2014 at 2:27 PM, Sudha Ponnaganti sudha.ponnaga...@citrix.com wrote: Agree with Sebastien. 4.4 is cut too early. By cherry picking and missing some of the critical fixes in to 4.4, isn't it safe to baseline with 4.4-forward which has majority of the fixes. It is not like new features are added to 4.4 forward. Looks like most of them are fixes to existing functionality on 4.4. Thanks /Sudha -Original Message- From: sebgoa [mailto:run...@gmail.com] Sent: Sunday, June 08, 2014 4:52 AM To: dev@cloudstack.apache.org Subject: Re: [ACS44] 112 unpicked cherries in 4.4-forward. why? On Jun 8, 2014, at 5:31 AM, Sheng Yang sh...@yasker.org wrote: To my understanding 4.4-forward is a staging tree for 4.4.1 release currently, and only issues critical enough would go for 4.4 branch, I disagree. That makes things messy, 4.4-forward should only exist till 4.4 is out, really everything in 4.4-forward should go in 4.4.0. then folks commit to 4.4 for future bug fix releases... -sebastien and author would ask for pick up in that case. I think that's what's happened with 4.3-forward branch. --Sheng On Sat, Jun 7, 2014 at 3:05 PM, Sudha Ponnaganti sudha.ponnaga...@citrix.com wrote: Wondering why not baseline with 4.4-forward branch. -Original Message- From: Daan Hoogland [mailto:daan.hoogl...@gmail.com] Sent: Saturday, June 07, 2014 2:53 AM To: dev Subject: [ACS44] 112 unpicked cherries in 4.4-forward. why? H, To start with my personal finger pointing: especially test authors and UI devs seem to be to blame for the difference (and me of course). please take a moment to make sure your important work is not going to be lost for future generations. I am sure I did it way more complicated then needed but here it is: git config pretty.blame format:blame %H '%aN %aE' - '%cn %cE' - '%s' git cherry 4.4 4.4-forward | grep -e ^+ | cut -f 2 -d ' ' - | xargs -L 1 git show --pretty=blame | grep -e ^blame this gave me a nice list of commits that are not going into 4.4 as things are. There is some noise in there that might be filtered out by even more clever scripting on the output with ^- in the first grep command: 1. things that where picked the other way around for instance fc52e641d8f69d8c0b552119203b0a2bc58e488f 'Daan Hoogland d...@onecht.net' - 'Daan Hoogland d...@onecht.net' - 'try-with-resource to prevent resource leaks' 2. things that were reverted in the branch (8 occurs so that is a minus 16 dropping the number under 100) this leaves a lot of things though. Please have a look in the list below or execute the commands your self and decide what to do about them: blame 617826d16b4d5220bb3b51ed511b3c065d0e8926 'Koushik Das kous...@apache.org' - 'Daan Hoogland d...@onecht.net' - 'CLOUDSTACK-6445: Simulator enhancements Refer FS - https://cwiki.apache.org/confluence/display/CLOUDSTACK/Simulator+enha ncements ' blame 953167808c3e9a9c40a6f7be6e16ebf32c608a4c 'Jessica Wang jessicaw...@apache.org' - 'Jessica Wang jessicaw...@apache.org' - 'BUG-ID: CS-19795: UI Add LDAP Account - fix a bug that a LDAP account that
[ACS4.4] cherrypick c282bb3a1293fbbfdb306263ea52464862670fb3
Hi Daan, Cherry pick c282bb3a1293fbbfdb306263ea52464862670fb3 to resolve the blocker CLOUDSTACK-6603 [Upgrade]DB Exception while Autoscale monitoring after upgrading from 4.3 to 4.4 Thanks Rajesh Battala
Re: KVM + LXC on the same host
On 07.06.2014 03:38, John Kinsella wrote: Hey Ilya - So, for about a month now we’ve had a system running SmartOS[1], which gives a combination of containers and KVM, albeit on illumos instead of linux. In general I’m not impressed by SmartOS’s story, but we had one customer asking for it (and I sorta expect more, we’re not officially supporting it yet). It’s a little early for me to even ponder integrating it with ACS, but might but worth a glance if you’re looking for that KVM/container mix. Downsides: * Compared to a Linux hypervisor, much less feature support (missing Ceph is personally bugging me) * It’s…Solaris (the OS that won’t go away, c0t0d0s0 gives me flashbacks from 10 years ago). For us our infrastructure/automation is fairly tuned for linux, so really embracing this will require…effort. Check out the Proxmox project which provides both containers and VMs, in the form of OpenVZ (not ideal, I know) and KVM. Lucian -- Sent from the Delta quadrant using Borg technology! Nux! www.nux.ro
Re: ssh keypair value overridden by template meta data
Thanks Hiroki, I have added the fix versions, ’ll fix it asap.
-Harikrishna
On 09-Jun-2014, at 2:35 pm, Hiroki Ohashi hiroki.s...@gmail.com wrote:
Hi Harikrishna
Thank you very much for your reproduction work.
I created a ticket for this issue.
https://issues.apache.org/jira/browse/CLOUDSTACK-6869
Please confirm it.
Best Regards
2014-06-09 15:59 GMT+09:00 Harikrishna Patnala
harikrishna.patn...@citrix.com:
Hi Hiroki,
We should not override the ssh key pair provided in the deployVM API with
the ssh key in template metadata.
I just able to reproduce this. Please create a ticket for this issue.
Thanks
Harikrishna
On 06-Jun-2014, at 5:18 pm, Hiroki Ohashi hiroki.s...@gmail.com wrote:
Dear guys
I encountered a problem that a ssh public key of ssh_keypairs for a
newly created instance is overridden by another ssh key in template
meta data. I think this leads to security vulnerability because a
template owner can login to other user's instance created from the
template. So, could you fix this issue?
This behavior is caused by meta data import at commitUserVm method. A
ssh key value specified by an instance owner is set to a UserVmVO
object at line 2986-2988 of
server/src/com/cloud/vm/UserVmManagerImpl.java in 4.3 branch [1], but
this value is overridden at line 3035-3038 by template meta data.
Please note a database contains meta data entries related to a
template which you created from an instance in cloud.template_view
like [2]. 2nd row has detail_name and detail_value about ssh key and
CloudStack override the user specified ssh key value by the owner's
detail value in this situation. It results in delivery of the template
owner's ssh key for the instance created from the template to virtual
router inspite of specification of instance owner's ssh key.
You can reproduce this phenomenon like below.
1. Deploy an instance with a ssh key A by specifying 'keypair'
value.
2. Create a template from this instance.
3. Deploy an instance with another ssh key B by specifying
'keypair' value.
[1] server/src/com/cloud/vm/UserVmManagerImpl.java
2971 private UserVmVO commitUserVm(final DataCenter zone, final
VirtualMachineTemplate template, final String hostName, final String
displayName, final Account owner,
2972 final Long diskOfferingId, final Long diskSize, final
String userData, final HypervisorType hypervisor, final Account
caller, final Boolean isDisplayVmEnabled,
2973 final String keyboard, final long accountId, final
ServiceOfferingVO offering, final boolean isIso, final String
sshPublicKey,
2974 final LinkedHashMapString, NicProfile networkNicMap,
final long id, final String instanceName, final String uuidName, final
HypervisorType hypervisorType,
2975 final MapString, String customParameters) throws
InsufficientCapacityException {
2976 return Transaction.execute(new
TransactionCallbackWithExceptionUserVmVO,
InsufficientCapacityException() {
2977 @Override
2978 public UserVmVO doInTransaction(TransactionStatus
status) throws InsufficientCapacityException {
2979 UserVmVO vm = new UserVmVO(id, instanceName,
displayName,
2980 template.getId(), hypervisorType,
template.getGuestOSId(),
2981 offering.getOfferHA(),
offering.getLimitCpuUse(),
2982 owner.getDomainId(), owner.getId(),
offering.getId(), userData,
2983 hostName, diskOfferingId);
2984 vm.setUuid(uuidName);
2985
vm.setDynamicallyScalable(template.isDynamicallyScalable());
2986 if (sshPublicKey != null) {
2987 vm.setDetail(SSH.PublicKey, sshPublicKey);
2988 }
2989
2990 if (keyboard != null !keyboard.isEmpty())
2991 vm.setDetail(VmDetailConstants.KEYBOARD,
keyboard);
2992
2993 if (isIso) {
2994 vm.setIsoId(template.getId());
2995 }
2996
2997 if(isDisplayVmEnabled != null){
2998 if(!_accountMgr.isRootAdmin(caller.getType())){
2999 throw new PermissionDeniedException(
Cannot update parameter displayvm, only admin permitted );
3000 }
3001 vm.setDisplayVm(isDisplayVmEnabled);
3002 }else {
3003 vm.setDisplayVm(true);
3004 }
3005
3006 // If hypervisor is vSphere, check for clone
type setting.
3007 if (hypervisorType.equals(HypervisorType.VMware)) {
3008 // retrieve clone flag.
3009 UserVmCloneType cloneType =
UserVmCloneType.linked;
3010 String value =
Issue regarding cloudstack configuration.
Hello There. Kindly tell me where i can ask from apache support/community the issues which i am facing during configuration part. thanks. -- -- Warm Regards, Naval Saini Technical Support Executive L-2 - SARV Webs Pvt Ltd. Global Number : +44 141 4166 USA/Canada TollFree: 1-888 987 2698 India TollFree: 1-800 102 7278 visit us: sarvmail.com ,tld6.com --
Re: Issue regarding cloudstack configuration.
On Jun 9, 2014, at 8:27 AM, Naval Saini naval.sa...@snet.in wrote: Hello There. Kindly tell me where i can ask from apache support/community the issues which i am facing during configuration part. Hi, better to start on the users lists, us...@cloudstack.apache.org or on IRC #cloudstack on freenode. thanks. -- -- Warm Regards, Naval Saini Technical Support Executive L-2 - SARV Webs Pvt Ltd. Global Number : +44 141 4166 USA/Canada TollFree: 1-888 987 2698 India TollFree: 1-800 102 7278 visit us: sarvmail.com ,tld6.com --
Review Request 22376: tagging the test case test_09_delete_detached_volume with the bugId.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/22376/ --- Review request for cloudstack and Abhinandan Prateek. Bugs: CLOUDSTACK-6875 https://issues.apache.org/jira/browse/CLOUDSTACK-6875 Repository: cloudstack-git Description --- The test case test_09_delete_detached_volume is failing. we have logged a bug CLOUDSTACK-6875 to track this. this patch tags the test case with the above mentioned bugId. Diffs - test/integration/smoke/test_volumes.py 73c2722 Diff: https://reviews.apache.org/r/22376/diff/ Testing --- Thanks, bharat kumar
Review Request 22377: tagging the test case test_01_sys_vm_start with the bugId.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/22377/ --- Review request for cloudstack and Abhinandan Prateek. Bugs: CLOUDSTACK-6877 https://issues.apache.org/jira/browse/CLOUDSTACK-6877 Repository: cloudstack-git Description --- The test case test_01_sys_vm_start is failing. we have logged a bug CLOUDSTACK-6877 to track this. this patch tags the test case with the above mentioned bugId. Diffs - test/integration/smoke/test_secondary_storage.py 481d907 Diff: https://reviews.apache.org/r/22377/diff/ Testing --- Thanks, bharat kumar
Review Request 22378: tagging the test case test_01_snapshot_root_disk with the bugId.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/22378/ --- Review request for cloudstack and Abhinandan Prateek. Bugs: CLOUDSTACK-6878 https://issues.apache.org/jira/browse/CLOUDSTACK-6878 Repository: cloudstack-git Description --- The test case test_01_snapshot_root_disk is failing. we have logged a bug CLOUDSTACK-6876 to track this. this patch tags the test case with the above mentioned bugId. Diffs - test/integration/smoke/test_snapshots.py a960e70 Diff: https://reviews.apache.org/r/22378/diff/ Testing --- Thanks, bharat kumar
Review Request 22379: tagging the test case test_vpc_site2site_vpn with the bugId.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/22379/ --- Review request for cloudstack and Abhinandan Prateek. Bugs: CLOUDSTACK-6879 https://issues.apache.org/jira/browse/CLOUDSTACK-6879 Repository: cloudstack-git Description --- The test case test_vpc_site2site_vpn is failing. we have logged a bug CLOUDSTACK-6876 to track this. this patch tags the test case with the above mentioned bugId. Diffs - test/integration/smoke/test_vpc_vpn.py 03826e9 Diff: https://reviews.apache.org/r/22379/diff/ Testing --- Thanks, bharat kumar
Review Request 22375: tagging the test case test_deploy_vgpu_enabled_vm with the bugId.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/22375/ --- Review request for cloudstack and Abhinandan Prateek. Bugs: CLOUDSTACK-6876 https://issues.apache.org/jira/browse/CLOUDSTACK-6876 Repository: cloudstack-git Description --- The test case test_deploy_vgpu_enabled_vm is failing. we have logged a bug CLOUDSTACK-6876 to track this. this patch tags the test case with the above mentioned bugId. Diffs - test/integration/smoke/test_deploy_vgpu_enabled_vm.py 13fad61 Diff: https://reviews.apache.org/r/22375/diff/ Testing --- Thanks, bharat kumar
Re: Review Request 22354: CLOUDSTACK-6850: Return cpu cores, cpu speed and memory in listUsageRecords
On June 9, 2014, 9:55 a.m., Sebastien Goasguen wrote: Hi, is it a bug or a new feature ? I see that it alters the usage db schema, will this affects upgrades ? The jira entry says that it applies to 4.3 as well, does you patch applies to 4.3 ? Hi Sebastien, For me, it's a bug, because a CloudStack feature, usage tracking, is not working properly with dynamic compute offerings. Of course, I might be biased because I work on Amysta development and this bug prevents Amysta from tracking consumption with dynamic compute offerings. But this is a more general issue: there is currently no way to track VM instance usages with dynamic compute offerings using API only. No, the patch is for 4.4 (because I wrote the DB schema changes in schema-430to440.sql It would be great to fix that for ACS 4.3 too, but I'm annoyed with the schema update). Regarding the update process, I tested the patch both with a ACS 4.4 fresh install and with a ACS 4.3 update. There was no regression. Usage details are returned only for usages processed after the update. - Olivier --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/22354/#review45061 --- On June 8, 2014, 6:40 p.m., Olivier Lemasle wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/22354/ --- (Updated June 8, 2014, 6:40 p.m.) Review request for cloudstack and Kishan Kavala. Bugs: CLOUDSTACK-6850 https://issues.apache.org/jira/browse/CLOUDSTACK-6850 Repository: cloudstack-git Description --- This patch updates VMInstanceUsageParser to process usage details (cpu cores, cpu speed and memory) for dynamic compute offering usages, in order to save them in table cloud_usage.cloud_usage and return them with the API (listUsageRecords). See CLOUDSTACK-6850. A change in database model is required for this patch (three new columns in cloud_usage.cloud_usage). The patch should apply on master and 4.4. Diffs - api/src/org/apache/cloudstack/api/response/UsageRecordResponse.java 5e2e85d api/src/org/apache/cloudstack/usage/Usage.java 20dc189 engine/schema/src/com/cloud/usage/UsageVMInstanceVO.java 06d4876 engine/schema/src/com/cloud/usage/UsageVO.java 67014ef engine/schema/src/com/cloud/usage/dao/UsageVMInstanceDaoImpl.java b94e12f server/src/com/cloud/api/ApiResponseHelper.java 119f56d setup/db/db/schema-430to440.sql d149a65 usage/src/com/cloud/usage/parser/VMInstanceUsageParser.java ba0d4bf Diff: https://reviews.apache.org/r/22354/diff/ Testing --- Packaged in RPMs, installed and manually tested. 1. Create a VM with compute offering Small instance. 2. Stop the VM. 3. Change service offering to a custom offering, with cpu_number=1, cpu_speed=500 MHz, memory=512 MB 4. Start the VM, wait... Stop the VM. 6. Change service offering to an other custom offering, with cpu_number=2, cpu_speed=800 MHz, memory=1024 MB 7. Start the VM, wait... Stop the VM. 8. Change service offering to the first custom offering, with same details (cpu_number=2, cpu_speed=800 MHz, memory=1024 MB) 9. Start the VM, wait... Stop the VM. 10. Change service offering to Medium instance. Usages and details are reflected in database and in API response. There is no regression found for other usage types. Details are returned (and saved in database) only for RUNNING_VM and ALLOCATED_VM, only if the service offering is a custom one. Example of response: usagerecord accountadmin/account accountid184a6564-edab-11e3-b629-0050569ed71c/accountid domainidfbc7578a-edaa-11e3-b629-0050569ed71c/domainid zoneide5047089-c911-40ea-8251-4a76810ac1e0/zoneid descriptioncalifornia allocated (ServiceOffering: 12) (Template: 5)/description usage0.098078 Hrs/usage usagetype2/usagetype rawusage0.098078/rawusage virtualmachineide89c22af-65cf-42a2-8905-6f4bbd7d59a1/virtualmachineid namecalifornia/name offeringidb05b64e9-8e0f-4335-b0e0-acc5bad81938/offeringid templateidfbca2564-edaa-11e3-b629-0050569ed71c/templateid usageide89c22af-65cf-42a2-8905-6f4bbd7d59a1/usageid typeXenServer/type cpunumber1/cpunumber cpuspeed500/cpuspeed memory512/memory startdate2014-06-08'T'17:05:52+02:00/startdate enddate2014-06-08'T'17:15:52+02:00/enddate /usagerecord usagerecord accountadmin/account accountid184a6564-edab-11e3-b629-0050569ed71c/accountid domainidfbc7578a-edaa-11e3-b629-0050569ed71c/domainid zoneide5047089-c911-40ea-8251-4a76810ac1e0/zoneid descriptioncalifornia allocated (ServiceOffering: 1) (Template: 5)/description usage0.068594 Hrs/usage
feature : changing volume properties dynamically in 4.5
hi guys, since most of the third party storage providers have been implementing 1:1 mapping(managed storage) between a volume(dataset) and a vm disk(vdi/vmdk) for guaranteeing the Qos, i would like to propose a new feature to dynamically change the volume properties supported by storage vendors such as IOPS, Deduplication, Compression, Grace, Syncronization, Latency etc, depending on properties and features supported by respective storage vendors. hence providing more flexibility for users. in case of using default cloudstack storage provider, we can change the properties of the vdi/vmdk files apart from resizing the volume(vdi/vmdk). changes in management server include, new async web api ChangeVolumePropertiesCmd, new method in VolumeApiService for vo and dao validation implementations. new method in VolumeServiceManager for supporting callback and calling the respective storage provider driver's implementation. new method in PrimaryDataStoreDriver interface for implementing respective features according to their storage product. changes in UI include, new changing volume properties widget in volume section, showing different properties depending upon listed storage providers. any suggestions and feedbacks ? thanks -- regards, punith s cloudbyte.com
Re: [ACS44] 112 unpicked cherries in 4.4-forward. why?
Right - we need to decide as a community what a forward branch really means. On Monday, June 9, 2014, Koushik Das koushik@citrix.com wrote: On 09-Jun-2014, at 12:56 PM, sebgoa run...@gmail.com javascript:; wrote: On Jun 8, 2014, at 7:25 PM, Mike Tutkowski mike.tutkow...@solidfire.com javascript:; wrote: Yes, there appears to be at least two lines of thought on x.y-forward branches (specifically using 4.4-forward as an example here). 1) 4.4-forward and 4.4 should eventually be the same. Once the 4.4 release goes out the door, 4.4-forward should be removed and changes for 4.4.1, should such a release ever be made, should go into 4.4. That's what I tried to say. I you commit to 4.4-forward that means that you want it in 4.4.0, otherwise go to master. [Koushik] What if the fix is not that critical for 4.4 but would be good to have in 4.4.1 and above apart from master. I would be concerned if the fix is in 4.4-forward but not in master. 2) 4.4-forward contains changes that might go into 4.4 (if a cherry pick is requested) and changes that would go into 4.4.1, should such a release ever be made. In both cases: Most all changes that go into 4.4-forward would need to go into master (unless that part of the codebase in master has been modified in such a way as to no longer make this 4.4-forward change relevant for master). We should gain some consensus on what x.y-forward means. On Sun, Jun 8, 2014 at 10:51 AM, Daan Hoogland daan.hoogl...@gmail.com wrote: As I read your comments you actually don't agree with Sebastien, Sudha. 4.4-forward was cut of at code freeze. Since then fixes are committed there and cherry-pick are done by the RM. It seems that this is exactly what Sebastien meant it to be for. @Sebastien: in Shengs words I read that some of the things in 4.4-forward would only go in 4.4 after the release (thing with lower priority then critical) We all don't agree, let's agree on that.;} On Sun, Jun 8, 2014 at 2:27 PM, Sudha Ponnaganti sudha.ponnaga...@citrix.com wrote: Agree with Sebastien. 4.4 is cut too early. By cherry picking and missing some of the critical fixes in to 4.4, isn't it safe to baseline with 4.4-forward which has majority of the fixes. It is not like new features are added to 4.4 forward. Looks like most of them are fixes to existing functionality on 4.4. Thanks /Sudha -Original Message- From: sebgoa [mailto:run...@gmail.com] Sent: Sunday, June 08, 2014 4:52 AM To: dev@cloudstack.apache.org Subject: Re: [ACS44] 112 unpicked cherries in 4.4-forward. why? On Jun 8, 2014, at 5:31 AM, Sheng Yang sh...@yasker.org wrote: To my understanding 4.4-forward is a staging tree for 4.4.1 release currently, and only issues critical enough would go for 4.4 branch, I disagree. That makes things messy, 4.4-forward should only exist till 4.4 is out, really everything in 4.4-forward should go in 4.4.0. then folks commit to 4.4 for future bug fix releases... -sebastien and author would ask for pick up in that case. I think that's what's happened with 4.3-forward branch. --Sheng On Sat, Jun 7, 2014 at 3:05 PM, Sudha Ponnaganti sudha.ponnaga...@citrix.com wrote: Wondering why not baseline with 4.4-forward branch. -Original Message- From: Daan Hoogland [mailto:daan.hoogl...@gmail.com] Sent: Saturday, June 07, 2014 2:53 AM To: dev Subject: [ACS44] 112 unpicked cherries in 4.4-forward. why? H, To start with my personal finger pointing: especially test authors and UI devs seem to be to blame for the difference (and me of course). please take a moment to make sure your important work is not going to be lost for future generations. -- *Mike Tutkowski* *Senior CloudStack Developer, SolidFire Inc.* e: mike.tutkow...@solidfire.com o: 303.746.7302 Advancing the way the world uses the cloud http://solidfire.com/solution/overview/?video=play*™*
Re: Question about guaranteed IOPS feature
Mike, Marcus, Sorry for my belated reply. Thank you for clarifying the status of current implementation, and concerns we need to care. Yes, using hypervisor's rate limiting feature to guarantee IOPS seems challenging. Thanks, Noji 2014-06-02 14:52 GMT-04:00 Marcus shadow...@gmail.com: I think the current implementation, outside of custom vendors is to simply act as a tune-able rate-limiter for the admin to leverage how they see fit. I'm not sure it was intended to be a guarantee of service, just a cap, therefore the thought of oversubscribing wasn't an issue. It is an exercise for the admin, given the tool. We could potentially augment it by adding in the idea of 'iops per host' and/or 'iops per primary storage' parameters on each host/primary storage, whereby the allocator code can leverage this to deploy servers. On Mon, Jun 2, 2014 at 12:18 PM, Mike Tutkowski mike.tutkow...@solidfire.com wrote: Hi, 1 and 2) CloudStack now supports managed storage for the following hypervisor types: * Disk Offerings for XenServer and ESX as of CS 4.2 * Disk Offerings for KVM as of CS 4.3 * Compute Offerings for XenServer and ESX as of CS 4.4 The SolidFire plug-in is capable of supporting whatever CloudStack-managed configurations are available in the version of CloudStack that you are using. Rate limiting is supported for the KVM hypervisor only as of CS 4.2. Min knows the details of this KVM rate-limiting feature better than I do (he implemented it), so I have CCed him. I think this rate limiting is only from the hypervisor's point of view...without input into the storage system that actually provides the volume in question. In other words, the hypervisor doesn't actually know how many IOPS the primary storage is capable of delivering...it only knows to limit the IO it sends to the storage system for that volume. Rate limiting is not a guarantee on IOPS, though. It is simply a limit on the rate at which IO can be sent to the storage system. By the way, for the sake of simplicity, when I'm talking in terms of IOPS here, I'm assuming all of your IOPS are the same size...which is probably not the case in the real world, so you'd need to take that into account, as well. In theory, if you have primary storage that has, say, 10,000 IOPS available and you carve out 10 1,000 IOPS (rate limited) volumes on that primary storage, you may be OK. However, this depends. For many storage solutions, you still have to worry about drive groups, LUN assignments, failure scenarios, etc. when it comes to whether or not you'll actually get 1,000 IOPS for each of those 10 volumes on the storage side. In these environments, you also need to take into consideration write amplification from data protection (ex. RAID). If the combination of IOPS from all the SSDs on your primary storage is X, you will get less than X from the point of view of the clients of that storage system. All hypervisor rate limiting is doing is saying that at most you can get 1,000 IOPS for each of those 10 volumes...not that you actually will (a guarantee) get those IOPS. If performance can be guaranteed at the storage infrastructure, this becomes much easier. For example, with a SolidFire SAN, you simply specify a minimum (guaranteed) number of IOPS, a maximum number of IOPS, and a burst number of IOPS for each volume (LUN). The SAN will guarantee the minimum number of IOPS (from the point of view of the client) regardless of how the data is physically distributed on its SSDs (you don't have to worry about RAID groups or low-level details like that). This minimum number of IOPS is guaranteed even during failure scenarios. 3) Punith is referring to dynamically changing the number of IOPS available of the SAN volume and making those new IOPS available to the CloudStack volume or volumes that leverage that SAN volume. 4) Hypervisor rate limiting - by itself - does not guarantee network performance, that is true. This is true of storage QoS, as well, but rarely is the network the bottleneck (SolidFire requires a 10G Ethernet network for the storage traffic). I hope that helps, Mike On Mon, Jun 2, 2014 at 1:42 AM, Yoshikazu Nojima m...@ynojima.net wrote: Punith, Mike, Allow me to ask questions regarding guaranteed IOPS feature. (Since it may be off-topic to Hieu's original question, I changed the subject.) Q1) Does your plugins(Solidfire, CloudBytes) support KVM? From which version? In my understanding, Current CloudStack's implementation guarantees IOPS by two regulations: - Storage appliance (or Hypervisor) limits Volume's max IOPS and throughput. - Management Server ensures max IOPS performance of a storage will not be exceeded by the sum of volumes' IOPS in the storage by controlling volume provisioning to the storage. Q2) Is my understanding accurate? Q3) If we make volume IOPS configurable on fly, max IOPS of a storage can be exceeded
Re: [DISCUSS] extending the libvirt/KVM plugin to also support libvirt/Xen
Dave, Thanks for putting this up on the wiki. A few things jumped out at me... - Please change Xen to XenProject or Xen Project as appropriate. There's already a ton of confusion out there, and I'd like to see us get our terms correct from the outset where ever possible. - It would be good to see a UI mock up for how users would configure the Xen Project hypervisor option. I think that would go a long way to helping with the mixed hypervisor cluster concept and how it could be blocked. - It would also be good to see examples of how the APIs might need to be changed to support this. Minimally I'd expect to see things like supported disk/network/os types and that sort of thing. - I see you have a todo to document the supported Xen Project hypervisor and libvirt versions, but also dependencies on libxl changes. Are these critical dependencies, or if someone doesn't have latest upstream will things work in a reduced feature set? - C6.1 talks about exposing a config setting. Is that really required? Couldn't that be set correctly based on hypervisor type? - Would QCOW2 be used for the Xen Project disk type for all templates to keep with KVM consistency? I'm actually thinking about support for VMDK, but perhaps that's a different proposal? - Since we're talking about sharing a libvirt plugin, I'm not clear on if the shared work is done in a new libvirt plugin which is then exposed to a KVM and a XenProject plugin or if the existing KVM plugin is refactored to encompass both. -tim On Mon, Jun 9, 2014 at 2:35 AM, Wido den Hollander w...@widodh.nl wrote: On 06/08/2014 11:14 PM, Dave Scott wrote: Hi Wido, Thanks for your mail! On 8 Jun 2014, at 19:02, Wido den Hollander w...@widodh.nl wrote: On 06/08/2014 06:23 PM, Dave Scott wrote: Hi, Following on from the earlier [PROPOSAL] Support pure Xen as a hypervisor”, I’ve added a design doc to the wiki: https://cwiki.apache.org/confluence/display/CLOUDSTACK/Allow+hosts+running+the+Xen+hypervisor+to+be+managed+via+libvirt This design would allow people who want to manage their hypervisors purely through the libvirt tools to choose the Xen hypervisor. From the code point of view, I want to maximise sharing between the KVM and Xen code paths, partly to make QA easier and partly to maximise the chance that adding a feature for “Xen” causes it to work for “KVM” and vice-versa. In particular this means that, if a genuinely-useful capability is currently missing from the libvirt libxl driver, I want to implement it rather than work around it. Seems like a great route to me! You also want to support Xen+Qemu with this way? Yes, it should be possible to run fully virtualised VMs with Xen + Qemu. I think we’ll be able to choose whether to run VMs as PV or HVM. Ok, but those will be different code paths at some level. We have to be aware that there might be some storage differences between KVM and Xen like Ceph which is not fully supported yet by Xen. Ceph is an interesting one. Xen itself doesn’t know anything about storage— instead the dom0 takes care of it either via a kernel driver (blkback) or userspace program (qemu or tapdisk). When I tried to make Ceph work about a year ago[1] I hit a bug in libxl (the Xen control library). The good news is the fix made it into Xen 4.4, so with luck we can get it to work. When Xen runs with Qemu as full HVM it's Qemu which takes care of the Ceph storage, so in that case it's fixed. I haven't got a lot of experience with PV Xen. I heard stories of Ceph being integrated in blktap(2), but never tested it. If anything is missing in libvirt or the Java bindings we have to fix that indeed instead of hacking around it. Great :) Cheers, Dave [1] http://xenserver.org/discuss-virtualization/virtualization-blog/entry/tech-preview-of-xenserver-libvirt-ceph.html Wido Comments appreciated! Cheers, Dave [1] http://mail-archives.apache.org/mod_mbox/cloudstack-users/201403.mbox/%3ccajgxtbnbmqtq81ralgh2kma7v5wjyzkr3xnyasmkc_br+uk...@mail.gmail.com%3e
Re: [DISCUSS] Introducing Gerrit for quality? was: [PROPOSAL] Using continuous integration to maintain our code quality...
On Fri, Jun 6, 2014 at 7:26 PM, Sheng Yang sh...@yasker.org wrote: Hi all, Seems it's a good timing to bring back the discussion about the gerrit. We want to do CI, and improve our code quality. One obvious way of doing and reduce the workload of devs is introduce a tool to enforce the process. I've checked out quite a few projects using gerrit, which would force you to ask for review, and validation before the code can be committed to the repo. Looks it's really a easier way for devs according what I've heard. Even our competitor laid out a very detail workflow based on the use of gerrit( https://wiki.openstack.org/wiki/Gerrit_Workflow ). I guess it can make a good reference. Well, gerrit has been brought up a few times before. And now the new process we want to enforce just fits what gerrit(or other automation review/test/commit software) is for. Maybe it's the time for us to review the possibility of using a tool to enforce our commits and improve our code quality(as well as transfer knowledge) again? --Sheng ASF Infra has a very dour view on Gerrit. Don't read that as impossible; there are many projects at the ASF who are interested in Gerrit. That said; what about moving to using github pull requests instead of RB, and from their, having the jenkins pull request builder automatically process every pull request and list information. Here's an example: https://github.com/jclouds/jclouds-labs/pull/61 You'll see that every time the patch changes, the jenkins plugin pulled the patch - ran tests against it and reported back. That said; it almost seems like we have the cart before the horse; we need to finish figuring out the CI Infrastructure first. --David
Re: [ACS44] 112 unpicked cherries in 4.4-forward. why?
On Mon, Jun 9, 2014 at 10:39 AM, Mike Tutkowski mike.tutkow...@solidfire.com wrote: Right - we need to decide as a community what a forward branch really means. On Monday, June 9, 2014, Koushik Das koushik@citrix.com wrote: So a bit of background reading: http://markmail.org/message/aux2yjxpudotu7qu This is when we started with the -forward branches. --David
Re: [DISCUSS] Introducing Gerrit for quality? was: [PROPOSAL] Using continuous integration to maintain our code quality...
I like github pull request as well from past usage, it is very convenient for developers and reviewers to perform their tasks, compared to our current RB. Also agree with David, the pre-requisite for this enforcement is that we should have CI in place to make this happen. Thanks -min On 6/9/14 8:47 AM, David Nalley da...@gnsa.us wrote: On Fri, Jun 6, 2014 at 7:26 PM, Sheng Yang sh...@yasker.org wrote: Hi all, Seems it's a good timing to bring back the discussion about the gerrit. We want to do CI, and improve our code quality. One obvious way of doing and reduce the workload of devs is introduce a tool to enforce the process. I've checked out quite a few projects using gerrit, which would force you to ask for review, and validation before the code can be committed to the repo. Looks it's really a easier way for devs according what I've heard. Even our competitor laid out a very detail workflow based on the use of gerrit( https://wiki.openstack.org/wiki/Gerrit_Workflow ). I guess it can make a good reference. Well, gerrit has been brought up a few times before. And now the new process we want to enforce just fits what gerrit(or other automation review/test/commit software) is for. Maybe it's the time for us to review the possibility of using a tool to enforce our commits and improve our code quality(as well as transfer knowledge) again? --Sheng ASF Infra has a very dour view on Gerrit. Don't read that as impossible; there are many projects at the ASF who are interested in Gerrit. That said; what about moving to using github pull requests instead of RB, and from their, having the jenkins pull request builder automatically process every pull request and list information. Here's an example: https://github.com/jclouds/jclouds-labs/pull/61 You'll see that every time the patch changes, the jenkins plugin pulled the patch - ran tests against it and reported back. That said; it almost seems like we have the cart before the horse; we need to finish figuring out the CI Infrastructure first. --David
Re: [DISCUSS] extending the libvirt/KVM plugin to also support libvirt/Xen
Hi Tim, Thanks for your feedback! On 9 Jun 2014, at 16:44, Tim Mackey tmac...@gmail.com wrote: Dave, Thanks for putting this up on the wiki. A few things jumped out at me... - Please change Xen to XenProject or Xen Project as appropriate. There's already a ton of confusion out there, and I'd like to see us get our terms correct from the outset where ever possible. Sure — will do. - It would be good to see a UI mock up for how users would configure the Xen Project hypervisor option. I think that would go a long way to helping with the mixed hypervisor cluster concept and how it could be blocked. My comments about clusters were to emphasise that you shouldn’t expect to live migrate a VM between KVM and Xen, even if both are using libvirt underneath. Actually perhaps this is a misunderstanding of mine: is it possible today to mix hypervisors within a single CloudStack cluster? I’m not trying to change anything, but just point out the obvious. Maybe I got that wrong :-) - It would also be good to see examples of how the APIs might need to be changed to support this. Minimally I'd expect to see things like supported disk/network/os types and that sort of thing. I can talk about some of this more explicitly in the doc. Since Xen can use qemu for disks (for both PV and HVM guests) there should be no difference in supported disk formats between this and the existing KVM support. I’m not proposing to add anything to the Xen support which isn’t supported by KVM such as .vhd via tapdisk. Similarly, networking is handled by the regular Linux network stack so that should all work in the same way. - I see you have a todo to document the supported Xen Project hypervisor and libvirt versions, but also dependencies on libxl changes. Are these critical dependencies, or if someone doesn't have latest upstream will things work in a reduced feature set? In this proposal they would be critical dependencies (I’ll go make that clear). It is possible to make transitional arrangements but I didn’t want to overburden this proposal with backwards compat. - C6.1 talks about exposing a config setting. Is that really required? Couldn't that be set correctly based on hypervisor type? You’re right that this would be a hypervisor-specific thing. I’m still pondering the choice between PV and HVM. Using PV mode is convenient because it would allow the VMs to boot under Xen under virtualbox, like current devcloud. Using HVM might be more future-proof. - Would QCOW2 be used for the Xen Project disk type for all templates to keep with KVM consistency? I'm actually thinking about support for VMDK, but perhaps that's a different proposal? That sounds a separate proposal, but it shouldn’t conflict with this one (assuming the VMDK support is in qemu) - Since we're talking about sharing a libvirt plugin, I'm not clear on if the shared work is done in a new libvirt plugin which is then exposed to a KVM and a XenProject plugin or if the existing KVM plugin is refactored to encompass both. Not totally sure what the best thing to do is — I’ll have to play with the code a little more. Thanks, Dave -tim On Mon, Jun 9, 2014 at 2:35 AM, Wido den Hollander w...@widodh.nl wrote: On 06/08/2014 11:14 PM, Dave Scott wrote: Hi Wido, Thanks for your mail! On 8 Jun 2014, at 19:02, Wido den Hollander w...@widodh.nl wrote: On 06/08/2014 06:23 PM, Dave Scott wrote: Hi, Following on from the earlier [PROPOSAL] Support pure Xen as a hypervisor”, I’ve added a design doc to the wiki: https://cwiki.apache.org/confluence/display/CLOUDSTACK/Allow+hosts+running+the+Xen+hypervisor+to+be+managed+via+libvirt This design would allow people who want to manage their hypervisors purely through the libvirt tools to choose the Xen hypervisor. From the code point of view, I want to maximise sharing between the KVM and Xen code paths, partly to make QA easier and partly to maximise the chance that adding a feature for “Xen” causes it to work for “KVM” and vice-versa. In particular this means that, if a genuinely-useful capability is currently missing from the libvirt libxl driver, I want to implement it rather than work around it. Seems like a great route to me! You also want to support Xen+Qemu with this way? Yes, it should be possible to run fully virtualised VMs with Xen + Qemu. I think we’ll be able to choose whether to run VMs as PV or HVM. Ok, but those will be different code paths at some level. We have to be aware that there might be some storage differences between KVM and Xen like Ceph which is not fully supported yet by Xen. Ceph is an interesting one. Xen itself doesn’t know anything about storage— instead the dom0 takes care of it either via a kernel driver (blkback) or userspace program (qemu or tapdisk). When I tried to make Ceph work about a year ago[1] I hit a bug in libxl (the Xen control library). The good
Re: [DISCUSS] Introducing Gerrit for quality? was: [PROPOSAL] Using continuous integration to maintain our code quality...
On Mon, Jun 9, 2014 at 11:47 AM, David Nalley da...@gnsa.us wrote: On Fri, Jun 6, 2014 at 7:26 PM, Sheng Yang sh...@yasker.org wrote: Hi all, Seems it's a good timing to bring back the discussion about the gerrit. We want to do CI, and improve our code quality. One obvious way of doing and reduce the workload of devs is introduce a tool to enforce the process. I've checked out quite a few projects using gerrit, which would force you to ask for review, and validation before the code can be committed to the repo. Looks it's really a easier way for devs according what I've heard. Even our competitor laid out a very detail workflow based on the use of gerrit( https://wiki.openstack.org/wiki/Gerrit_Workflow ). I guess it can make a good reference. Well, gerrit has been brought up a few times before. And now the new process we want to enforce just fits what gerrit(or other automation review/test/commit software) is for. Maybe it's the time for us to review the possibility of using a tool to enforce our commits and improve our code quality(as well as transfer knowledge) again? --Sheng ASF Infra has a very dour view on Gerrit. Don't read that as impossible; there are many projects at the ASF who are interested in Gerrit. That said; what about moving to using github pull requests instead of RB, and from their, having the jenkins pull request builder automatically process every pull request and list information. Here's an example: https://github.com/jclouds/jclouds-labs/pull/61 You'll see that every time the patch changes, the jenkins plugin pulled the patch - ran tests against it and reported back. That said; it almost seems like we have the cart before the horse; we need to finish figuring out the CI Infrastructure first. --David Just as an additional comment. CF was using gerrit, and apparently dropped it in favor of GH Pull Requests plus automated testing with Travis-CI. I know folks like abayer and jfarrell are working on turning on automated builds via the github pull request plugin for jenkins. Might be something to consider. --David
Re: [DISCUSS] Introducing Gerrit for quality? was: [PROPOSAL] Using continuous integration to maintain our code quality...
On Jun 9, 2014, at 7:13 PM, David Nalley da...@gnsa.us wrote: On Mon, Jun 9, 2014 at 11:47 AM, David Nalley da...@gnsa.us wrote: On Fri, Jun 6, 2014 at 7:26 PM, Sheng Yang sh...@yasker.org wrote: Hi all, Seems it's a good timing to bring back the discussion about the gerrit. We want to do CI, and improve our code quality. One obvious way of doing and reduce the workload of devs is introduce a tool to enforce the process. I've checked out quite a few projects using gerrit, which would force you to ask for review, and validation before the code can be committed to the repo. Looks it's really a easier way for devs according what I've heard. Even our competitor laid out a very detail workflow based on the use of gerrit( https://wiki.openstack.org/wiki/Gerrit_Workflow ). I guess it can make a good reference. Well, gerrit has been brought up a few times before. And now the new process we want to enforce just fits what gerrit(or other automation review/test/commit software) is for. Maybe it's the time for us to review the possibility of using a tool to enforce our commits and improve our code quality(as well as transfer knowledge) again? --Sheng ASF Infra has a very dour view on Gerrit. Don't read that as impossible; there are many projects at the ASF who are interested in Gerrit. That said; what about moving to using github pull requests instead of RB, and from their, having the jenkins pull request builder automatically process every pull request and list information. Here's an example: https://github.com/jclouds/jclouds-labs/pull/61 You'll see that every time the patch changes, the jenkins plugin pulled the patch - ran tests against it and reported back. That said; it almost seems like we have the cart before the horse; we need to finish figuring out the CI Infrastructure first. --David Just as an additional comment. CF was using gerrit, and apparently dropped it in favor of GH Pull Requests plus automated testing with Travis-CI. I know folks like abayer and jfarrell are working on turning on automated builds via the github pull request plugin for jenkins. Might be something to consider. fwiw, I have my personal cloudstack fork on github with a simple travisCI setup. So when I make a commit it automatically triggers a build. I was working on running the integration tests on Travis as well but got distracted. If somebody wants to step in: https://github.com/runseb/cloudstack/blob/master/.travis.yml help welcome. :) --David
RE: [ACS44] 112 unpicked cherries in 4.4-forward. why?
-Original Message- From: David Nalley [mailto:da...@gnsa.us] Sent: Monday, June 09, 2014 9:24 AM To: dev@cloudstack.apache.org Subject: Re: [ACS44] 112 unpicked cherries in 4.4-forward. why? On Mon, Jun 9, 2014 at 10:39 AM, Mike Tutkowski mike.tutkow...@solidfire.com wrote: Right - we need to decide as a community what a forward branch really means. On Monday, June 9, 2014, Koushik Das koushik@citrix.com wrote: So a bit of background reading: http://markmail.org/message/aux2yjxpudotu7qu This is when we started with the -forward branches. --David [Animesh] I had started the forward branches right after first RC because we always have tons of fixes until we are ready with RC. IMHO starting it at code freeze is too early.
Re: [ACS44] 112 unpicked cherries in 4.4-forward. why?
Thanks for that reminder, David! OK, so, per David's e-mail reference, it looks like the intent of x.y-forward is what I referred to as option 2: 4.4-forward contains changes that might go into 4.4 (if a cherry pick is requested) and changes that would go into 4.4.1, should such a release ever be made. We can - of course - debate if this is still what we want to do; however, I would say - for 4.4 - we should treat 4.4-forward the way we described in this previous e-mail. In other words, do not assume 4.4-forward will eventually be equal to 4.4 before 4.4 is released. 4.4-forward should be used for fixes that need to make their way into 4.4 and for fixes that do not need to be in 4.4, but rather in 4.4.1, should such a release ever happen. On Mon, Jun 9, 2014 at 10:23 AM, David Nalley da...@gnsa.us wrote: On Mon, Jun 9, 2014 at 10:39 AM, Mike Tutkowski mike.tutkow...@solidfire.com wrote: Right - we need to decide as a community what a forward branch really means. On Monday, June 9, 2014, Koushik Das koushik@citrix.com wrote: So a bit of background reading: http://markmail.org/message/aux2yjxpudotu7qu This is when we started with the -forward branches. --David -- *Mike Tutkowski* *Senior CloudStack Developer, SolidFire Inc.* e: mike.tutkow...@solidfire.com o: 303.746.7302 Advancing the way the world uses the cloud http://solidfire.com/solution/overview/?video=play*™*
Re: redundant virtual routers for VPCs
Yes, I am in Pacific Time. --Sheng On Sat, Jun 7, 2014 at 10:33 PM, Daan Hoogland daan.hoogl...@gmail.com wrote: ok, I think your on pacific time, are you? On Sun, Jun 8, 2014 at 5:33 AM, Sheng Yang sh...@yasker.org wrote: I would be glad to join. --Sheng On Sat, Jun 7, 2014 at 3:01 AM, Daan Hoogland daan.hoogl...@gmail.com wrote: At Schuberg Philis, the urge to have virtual routers in a redundant way is getting to be pressing. It seems Citrix wants to move away from it entirely and Sungard is working on it but has to little resources for it. Withing our dev group we are now planning a sprint to experiment with this subject and then implement or fix the present routers to support this. However we would very much like input from the people who have already worked on this and are also very willing to let them in our sprint-team if they are willing. So: who want to join in a irc meeting (or conf call) on the subject coming week? Dutch people are not in the office on the day after ascension so tuesday is our intention. on behalf of some very impatient operators;) -- Daan -- Daan
Re: redundant virtual routers for VPCs
I will be happy to join! On Saturday, June 7, 2014, Daan Hoogland daan.hoogl...@gmail.com wrote: At Schuberg Philis, the urge to have virtual routers in a redundant way is getting to be pressing. It seems Citrix wants to move away from it entirely and Sungard is working on it but has to little resources for it. Withing our dev group we are now planning a sprint to experiment with this subject and then implement or fix the present routers to support this. However we would very much like input from the people who have already worked on this and are also very willing to let them in our sprint-team if they are willing. So: who want to join in a irc meeting (or conf call) on the subject coming week? Dutch people are not in the office on the day after ascension so tuesday is our intention. on behalf of some very impatient operators;) -- Daan
Re: feature : changing volume properties dynamically in 4.5
Hi Punith, This kind of a feature is something Chris Suich and I discussed a while back. We talked about leveraging arbitrary key/value pairs to make this happen (OpenStack does something similar). The key/value pairs would be vendor specific. If we take a key/value approach, we might be able to make this all work the way things work today when the user wants to change an existing Compute Offering and/or Disk Offering. For example, the user would pick a new Compute Offering (with presumably has different key/value pairs) and CloudStack could inform the applicable storage provider, who could update the volume in question. This way we don't need to introduce a new API command and the use model for the user doesn't really change. What are you thoughts on this? Thanks, Mike On Mon, Jun 9, 2014 at 8:08 AM, Punith S punit...@cloudbyte.com wrote: hi guys, since most of the third party storage providers have been implementing 1:1 mapping(managed storage) between a volume(dataset) and a vm disk(vdi/vmdk) for guaranteeing the Qos, i would like to propose a new feature to dynamically change the volume properties supported by storage vendors such as IOPS, Deduplication, Compression, Grace, Syncronization, Latency etc, depending on properties and features supported by respective storage vendors. hence providing more flexibility for users. in case of using default cloudstack storage provider, we can change the properties of the vdi/vmdk files apart from resizing the volume(vdi/vmdk). changes in management server include, new async web api ChangeVolumePropertiesCmd, new method in VolumeApiService for vo and dao validation implementations. new method in VolumeServiceManager for supporting callback and calling the respective storage provider driver's implementation. new method in PrimaryDataStoreDriver interface for implementing respective features according to their storage product. changes in UI include, new changing volume properties widget in volume section, showing different properties depending upon listed storage providers. any suggestions and feedbacks ? thanks -- regards, punith s cloudbyte.com -- *Mike Tutkowski* *Senior CloudStack Developer, SolidFire Inc.* e: mike.tutkow...@solidfire.com o: 303.746.7302 Advancing the way the world uses the cloud http://solidfire.com/solution/overview/?video=play*™*
Re: [DISCUSS] Introducing Gerrit for quality? was: [PROPOSAL] Using continuous integration to maintain our code quality...
On Mon, Jun 9, 2014 at 11:32 PM, sebgoa run...@gmail.com wrote: On Jun 9, 2014, at 7:13 PM, David Nalley da...@gnsa.us wrote: On Mon, Jun 9, 2014 at 11:47 AM, David Nalley da...@gnsa.us wrote: On Fri, Jun 6, 2014 at 7:26 PM, Sheng Yang sh...@yasker.org wrote: Hi all, Seems it's a good timing to bring back the discussion about the gerrit. We want to do CI, and improve our code quality. One obvious way of doing and reduce the workload of devs is introduce a tool to enforce the process. I've checked out quite a few projects using gerrit, which would force you to ask for review, and validation before the code can be committed to the repo. Looks it's really a easier way for devs according what I've heard. Even our competitor laid out a very detail workflow based on the use of gerrit( https://wiki.openstack.org/wiki/Gerrit_Workflow ). I guess it can make a good reference. Well, gerrit has been brought up a few times before. And now the new process we want to enforce just fits what gerrit(or other automation review/test/commit software) is for. Maybe it's the time for us to review the possibility of using a tool to enforce our commits and improve our code quality(as well as transfer knowledge) again? --Sheng ASF Infra has a very dour view on Gerrit. Don't read that as impossible; there are many projects at the ASF who are interested in Gerrit. That said; what about moving to using github pull requests instead of RB, and from their, having the jenkins pull request builder automatically process every pull request and list information. Here's an example: https://github.com/jclouds/jclouds-labs/pull/61 You'll see that every time the patch changes, the jenkins plugin pulled the patch - ran tests against it and reported back. That said; it almost seems like we have the cart before the horse; we need to finish figuring out the CI Infrastructure first. --David Just as an additional comment. CF was using gerrit, and apparently dropped it in favor of GH Pull Requests plus automated testing with Travis-CI. I know folks like abayer and jfarrell are working on turning on automated builds via the github pull request plugin for jenkins. Might be something to consider. fwiw, I have my personal cloudstack fork on github with a simple travisCI setup. So when I make a commit it automatically triggers a build. I was working on running the integration tests on Travis as well but got distracted. If somebody wants to step in: https://github.com/runseb/cloudstack/blob/master/.travis.yml This would be great if we could just use Github (pull requests, releases) and hook up Travis-CI. Both are free for opensource projects. Regards. help welcome. :) --David
Re: [ACS4.4] cherrypick c282bb3a1293fbbfdb306263ea52464862670fb3
On Mon, Jun 9, 2014 at 1:01 PM, Rajesh Battala rajesh.batt...@citrix.com wrote: c282bb3a1293fbbfdb306263ea52464862670fb3 is in -- Daan
Re: [DISCUSS] Introducing Gerrit for quality? was: [PROPOSAL] Using continuous integration to maintain our code quality...
That sounds good to me, as well. On Mon, Jun 9, 2014 at 1:14 PM, Rohit Yadav bhais...@apache.org wrote: On Mon, Jun 9, 2014 at 11:32 PM, sebgoa run...@gmail.com wrote: On Jun 9, 2014, at 7:13 PM, David Nalley da...@gnsa.us wrote: On Mon, Jun 9, 2014 at 11:47 AM, David Nalley da...@gnsa.us wrote: On Fri, Jun 6, 2014 at 7:26 PM, Sheng Yang sh...@yasker.org wrote: Hi all, Seems it's a good timing to bring back the discussion about the gerrit. We want to do CI, and improve our code quality. One obvious way of doing and reduce the workload of devs is introduce a tool to enforce the process. I've checked out quite a few projects using gerrit, which would force you to ask for review, and validation before the code can be committed to the repo. Looks it's really a easier way for devs according what I've heard. Even our competitor laid out a very detail workflow based on the use of gerrit( https://wiki.openstack.org/wiki/Gerrit_Workflow ). I guess it can make a good reference. Well, gerrit has been brought up a few times before. And now the new process we want to enforce just fits what gerrit(or other automation review/test/commit software) is for. Maybe it's the time for us to review the possibility of using a tool to enforce our commits and improve our code quality(as well as transfer knowledge) again? --Sheng ASF Infra has a very dour view on Gerrit. Don't read that as impossible; there are many projects at the ASF who are interested in Gerrit. That said; what about moving to using github pull requests instead of RB, and from their, having the jenkins pull request builder automatically process every pull request and list information. Here's an example: https://github.com/jclouds/jclouds-labs/pull/61 You'll see that every time the patch changes, the jenkins plugin pulled the patch - ran tests against it and reported back. That said; it almost seems like we have the cart before the horse; we need to finish figuring out the CI Infrastructure first. --David Just as an additional comment. CF was using gerrit, and apparently dropped it in favor of GH Pull Requests plus automated testing with Travis-CI. I know folks like abayer and jfarrell are working on turning on automated builds via the github pull request plugin for jenkins. Might be something to consider. fwiw, I have my personal cloudstack fork on github with a simple travisCI setup. So when I make a commit it automatically triggers a build. I was working on running the integration tests on Travis as well but got distracted. If somebody wants to step in: https://github.com/runseb/cloudstack/blob/master/.travis.yml This would be great if we could just use Github (pull requests, releases) and hook up Travis-CI. Both are free for opensource projects. Regards. help welcome. :) --David -- *Mike Tutkowski* *Senior CloudStack Developer, SolidFire Inc.* e: mike.tutkow...@solidfire.com o: 303.746.7302 Advancing the way the world uses the cloud http://solidfire.com/solution/overview/?video=play*™*
Re: redundant virtual routers for VPCs
nice, so given pacific time I can have a meeting at 15:00 utc or at 20:00 utc (8 am or 2 pm if I'm correct). Given office hours in Holland 17:00 (15:00 utc) would be best but if you're not in it could be later. As the person calling the meeting I would like to participate but if the highest attendance can be found only between 16 and 22 utc thats fine as well. On Mon, Jun 9, 2014 at 8:35 PM, Karl Harris karl.har...@sungardas.com wrote: I will be happy to join! On Saturday, June 7, 2014, Daan Hoogland daan.hoogl...@gmail.com wrote: At Schuberg Philis, the urge to have virtual routers in a redundant way is getting to be pressing. It seems Citrix wants to move away from it entirely and Sungard is working on it but has to little resources for it. Withing our dev group we are now planning a sprint to experiment with this subject and then implement or fix the present routers to support this. However we would very much like input from the people who have already worked on this and are also very willing to let them in our sprint-team if they are willing. So: who want to join in a irc meeting (or conf call) on the subject coming week? Dutch people are not in the office on the day after ascension so tuesday is our intention. on behalf of some very impatient operators;) -- Daan -- Daan
Re: feature : changing volume properties dynamically in 4.5
Allow me to follow this up with more detail (with regards to what Chris and I talked about): As you are aware, today the way you associate a Compute Offering (CO) or a Disk Offering (DO) with a Primary Storage (PS) is via storage tagging. This has some benefits and drawbacks. One benefit is being able to have some level of vendor independence from the point of view of the CO or DO. For example, if the storage tag of a DO is Fast, then this can be satisfied by PS that describes itself as Fast, regardless of vendor. A major drawback with the storage-tagging approach, however, is that you are not easily able to leverage vendor-specific features, which is often why you bought storage from the vendor in question to begin with. Ideally we do not want to add each vendor's features into the system as properties that can be seen by the admin regardless of whether or not the underlying storage he's actually using supports the feature in question. This coarse approach, however, was sort of business as usual when I started in with CloudStack 1.5 years ago. That being the case, when I added QoS options to CS, I did so in a way where the admin would see Min IOPS and Max IOPS options regardless of whether or not his storage actually supported those controls (to mitigate this a bit in the GUI, the admin has to explicitly select Storage QoS from a combobox). We leverage the same use model with Hypervisor QoS: The admin sees these options regardless of whether or not they actually apply on the hypervisor where the VM gets deployed. Going forward, we want to implement a more fine-grain and generic approach. We would like to have a storage provider field for the CO and DO windows (this equates to the name of one and only one storage provider). If the admin inputs a specific storage provider and does not use the storage tags field, he can enter in an arbitrary number of key/value pairs in another text field (perhaps we would provide a nice entry dialog to make this easier in the GUI). These key value pairs can be passed into the storage driver when it's asked to create (or update) a volume and the storage driver can decide what each and every key/value pair means. What do you think about this approach? Thanks On Mon, Jun 9, 2014 at 1:14 PM, Mike Tutkowski mike.tutkow...@solidfire.com wrote: Hi Punith, This kind of a feature is something Chris Suich and I discussed a while back. We talked about leveraging arbitrary key/value pairs to make this happen (OpenStack does something similar). The key/value pairs would be vendor specific. If we take a key/value approach, we might be able to make this all work the way things work today when the user wants to change an existing Compute Offering and/or Disk Offering. For example, the user would pick a new Compute Offering (with presumably has different key/value pairs) and CloudStack could inform the applicable storage provider, who could update the volume in question. This way we don't need to introduce a new API command and the use model for the user doesn't really change. What are you thoughts on this? Thanks, Mike On Mon, Jun 9, 2014 at 8:08 AM, Punith S punit...@cloudbyte.com wrote: hi guys, since most of the third party storage providers have been implementing 1:1 mapping(managed storage) between a volume(dataset) and a vm disk(vdi/vmdk) for guaranteeing the Qos, i would like to propose a new feature to dynamically change the volume properties supported by storage vendors such as IOPS, Deduplication, Compression, Grace, Syncronization, Latency etc, depending on properties and features supported by respective storage vendors. hence providing more flexibility for users. in case of using default cloudstack storage provider, we can change the properties of the vdi/vmdk files apart from resizing the volume(vdi/vmdk). changes in management server include, new async web api ChangeVolumePropertiesCmd, new method in VolumeApiService for vo and dao validation implementations. new method in VolumeServiceManager for supporting callback and calling the respective storage provider driver's implementation. new method in PrimaryDataStoreDriver interface for implementing respective features according to their storage product. changes in UI include, new changing volume properties widget in volume section, showing different properties depending upon listed storage providers. any suggestions and feedbacks ? thanks -- regards, punith s cloudbyte.com -- *Mike Tutkowski* *Senior CloudStack Developer, SolidFire Inc.* e: mike.tutkow...@solidfire.com o: 303.746.7302 Advancing the way the world uses the cloud http://solidfire.com/solution/overview/?video=play*™* -- *Mike Tutkowski* *Senior CloudStack Developer, SolidFire Inc.* e: mike.tutkow...@solidfire.com o: 303.746.7302 Advancing the way the world uses the cloud http://solidfire.com/solution/overview/?video=play*™*
Re: [DISCUSS] extending the libvirt/KVM plugin to also support libvirt/Xen
On Mon, Jun 9, 2014 at 1:02 PM, Dave Scott dave.sc...@citrix.com wrote: - It would be good to see a UI mock up for how users would configure the Xen Project hypervisor option. I think that would go a long way to helping with the mixed hypervisor cluster concept and how it could be blocked. My comments about clusters were to emphasise that you shouldn’t expect to live migrate a VM between KVM and Xen, even if both are using libvirt underneath. Actually perhaps this is a misunderstanding of mine: is it possible today to mix hypervisors within a single CloudStack cluster? I’m not trying to change anything, but just point out the obvious. Maybe I got that wrong :-) This comes down to your implementation. Today CS clusters need to have uniform hypervisor types, but if you're exposing a libvirt plugin, then those protections won't exist since CS will see things as just libvirt. If you create a KVM plugin and a separate XenProject one both of which implement a libvirt base class then you should get the protections for free. It should also allow you to define hypervisor specific details in a cleaner way. - It would also be good to see examples of how the APIs might need to be changed to support this. Minimally I'd expect to see things like supported disk/network/os types and that sort of thing. I can talk about some of this more explicitly in the doc. Since Xen can use qemu for disks (for both PV and HVM guests) there should be no difference in supported disk formats between this and the existing KVM support. I’m not proposing to add anything to the Xen support which isn’t supported by KVM such as .vhd via tapdisk. Similarly, networking is handled by the regular Linux network stack so that should all work in the same way. I wouldn't assume the existing KVM plugin is generic. For example, only QCOW2 is supported for disk images today. - I see you have a todo to document the supported Xen Project hypervisor and libvirt versions, but also dependencies on libxl changes. Are these critical dependencies, or if someone doesn't have latest upstream will things work in a reduced feature set? In this proposal they would be critical dependencies (I’ll go make that clear). It is possible to make transitional arrangements but I didn’t want to overburden this proposal with backwards compat. - C6.1 talks about exposing a config setting. Is that really required? Couldn't that be set correctly based on hypervisor type? You’re right that this would be a hypervisor-specific thing. I’m still pondering the choice between PV and HVM. Using PV mode is convenient because it would allow the VMs to boot under Xen under virtualbox, like current devcloud. Using HVM might be more future-proof. - Would QCOW2 be used for the Xen Project disk type for all templates to keep with KVM consistency? I'm actually thinking about support for VMDK, but perhaps that's a different proposal? That sounds a separate proposal, but it shouldn’t conflict with this one (assuming the VMDK support is in qemu) VMDK support is there, but the existing KVM plugin only supports QCOW2, so some of this is up to you for what you want to have supported with XenProject. - Since we're talking about sharing a libvirt plugin, I'm not clear on if the shared work is done in a new libvirt plugin which is then exposed to a KVM and a XenProject plugin or if the existing KVM plugin is refactored to encompass both. Not totally sure what the best thing to do is — I’ll have to play with the code a little more. I'd suggest looking at libvirt as a base class for KVM and XenProject and see if that cleans anything up. It might not, but worth looking into. Thanks, Dave -tim On Mon, Jun 9, 2014 at 2:35 AM, Wido den Hollander w...@widodh.nl wrote: On 06/08/2014 11:14 PM, Dave Scott wrote: Hi Wido, Thanks for your mail! On 8 Jun 2014, at 19:02, Wido den Hollander w...@widodh.nl wrote: On 06/08/2014 06:23 PM, Dave Scott wrote: Hi, Following on from the earlier [PROPOSAL] Support pure Xen as a hypervisor”, I’ve added a design doc to the wiki: https://cwiki.apache.org/confluence/display/CLOUDSTACK/Allow+hosts+running+the+Xen+hypervisor+to+be+managed+via+libvirt This design would allow people who want to manage their hypervisors purely through the libvirt tools to choose the Xen hypervisor. From the code point of view, I want to maximise sharing between the KVM and Xen code paths, partly to make QA easier and partly to maximise the chance that adding a feature for “Xen” causes it to work for “KVM” and vice-versa. In particular this means that, if a genuinely-useful capability is currently missing from the libvirt libxl driver, I want to implement it rather than work around it. Seems like a great route to me! You also want to support Xen+Qemu with this way? Yes, it should be possible to run fully
Re: Need help on the CS First class object hiding feature.
Answer inline On 08/06/14 6:45 PM, Girish Chaudhari girish.chaudh...@sungardas.com wrote: Hi Nitin, Can you please reply back to my previous mail thread. As well like to update you that I have even tried this feature with Networks object. Even in this case, I could set the display flag as '0', it get hided from normal users as well admin user similar to what I have mentioned above for virtual machine. Root admin can still list the hidden networks by using the display flag in the listNetworks. Thanks, Girish On Thu, Jun 5, 2014 at 9:18 AM, Girish Chaudhari girish.chaudh...@sungardas.com wrote: Thanks Nitin for immediate response. ... I see that some more first class entities have gotten added but the underlying concept remains the same. = Can you please name the newly added first class entities if possible, or what could be the best way to figure it out. Is there any criteria which is used to decided whether particular CS resource is first class candidate or not. Or every resource can be taken as first class object and hiding can be applied to it. Can you please let me know what steps/efforts will be involved in hiding new CS first class object using this feature? I would think that would be a bug. Please file it and I will try and look into it. = I will double check on it and try the same steps with another mentioned resource. Will file the bug accordingly. Also like to check whether Admin can update this feature flags or provide the meta data through UI? Yes, admin can update the display flag during creation time or using update apis. Check deployvm and updateVm apis. - You mean to say current CS UI(4.3 or upcoming 4.4) has ability to mark the existing VM as hidden or create new VM with hide flag? I could try it, using API programatically only. As well I have follow up extended questions as: Do this feature only applied to Root Admin vs the users Or even equally applied to Any Sub Domain Admin User vs its non admin users? Thanks, Girish On Wed, Jun 4, 2014 at 8:19 PM, Nitin Mehta nitin.me...@citrix.com wrote: On 04/06/14 4:52 AM, Girish Chaudhari girish.chaudh...@sungardas.com wrote: Hi Team, I am looking into the CS first class object hiding feature to verify how the Admin user has better control over the display of CS resources to end users. The only reference link I could find is https://cwiki.apache.org/confluence/display/CLOUDSTACK/Ability+to+have+ bet ter+control+over+first+class+objects+in+CS Is there any other documentation available around this feature providing more use cases and examples..etc. This is the functional spec and should provide you most of the information. I see that some more first class entities have gotten added but the underlying concept remains the same. Whereas I have tried to verify this feature by creating the resources(VM) by Admin and associated it with the account. In usual case Account user as well the admin user can retrieve the VM using the listVMs call. When I am trying to set the display_vm flag as false, as expected the account user don't see the VM. But even the Admin user can't see the VM in list. Not sure whether I am following the right steps or its buggy behavior. I would think that would be a bug. Please file it and I will try and look into it. Also like to check whether Admin can update this feature flags or provide the meta data through UI? Yes, admin can update the display flag during creation time or using update apis. Check deployvm and updateVm apis Curious to know whether anyone is using this feature in production and how? Thanks, Girish
Re: Need help on the CS First class object hiding feature.
Answers inline On 04/06/14 8:48 PM, Girish Chaudhari girish.chaudh...@sungardas.com wrote: Thanks Nitin for immediate response. ... I see that some more first class entities have gotten added but the underlying concept remains the same. = Can you please name the newly added first class entities if possible, or what could be the best way to figure it out. Is there any criteria which is used to decided whether particular CS resource is first class candidate or not. Or every resource can be taken as first class object and hiding can be applied to it. Can you please let me know what steps/efforts will be involved in hiding new CS first class object using this feature? You can check the create/update/list apis and they should have display flag as a parameter. I would think that would be a bug. Please file it and I will try and look into it. = I will double check on it and try the same steps with another mentioned resource. Will file the bug accordingly. Also like to check whether Admin can update this feature flags or provide the meta data through UI? Yes, admin can update the display flag during creation time or using update apis. Check deployvm and updateVm apis. - You mean to say current CS UI(4.3 or upcoming 4.4) has ability to mark the existing VM as hidden or create new VM with hide flag? I could try it, using API programatically only. API doesn't have the ability currently. You can file a bug for that if it doesn't already exist. As well I have follow up extended questions as: Do this feature only applied to Root Admin vs the users Or even equally applied to Any Sub Domain Admin User vs its non admin users? Only for ROOT admins as far as hiding/exposing the resources are concerned. The reason being the ROOT admin is the owner of the cloud and should have this ability. If you have a use case other than this please bring it forward and we can discuss. Thanks, Girish On Wed, Jun 4, 2014 at 8:19 PM, Nitin Mehta nitin.me...@citrix.com wrote: On 04/06/14 4:52 AM, Girish Chaudhari girish.chaudh...@sungardas.com wrote: Hi Team, I am looking into the CS first class object hiding feature to verify how the Admin user has better control over the display of CS resources to end users. The only reference link I could find is https://cwiki.apache.org/confluence/display/CLOUDSTACK/Ability+to+have+b et ter+control+over+first+class+objects+in+CS Is there any other documentation available around this feature providing more use cases and examples..etc. This is the functional spec and should provide you most of the information. I see that some more first class entities have gotten added but the underlying concept remains the same. Whereas I have tried to verify this feature by creating the resources(VM) by Admin and associated it with the account. In usual case Account user as well the admin user can retrieve the VM using the listVMs call. When I am trying to set the display_vm flag as false, as expected the account user don't see the VM. But even the Admin user can't see the VM in list. Not sure whether I am following the right steps or its buggy behavior. I would think that would be a bug. Please file it and I will try and look into it. Also like to check whether Admin can update this feature flags or provide the meta data through UI? Yes, admin can update the display flag during creation time or using update apis. Check deployvm and updateVm apis Curious to know whether anyone is using this feature in production and how? Thanks, Girish
[GSoC] [CLOUDSTACK-6114] Progress update
Hi All, Started making some fast progress on this. I uploaded the xenserver box[1] to vagrant cloud. This means people can easily get a xenserver VM by executing vagrant init duffy/xenserver vagrant up. I adjusted the configuration on the box to allow for multiple xenserver boxes to be brought up by one vagrant file. In order to allow this I had to remove the host-only network configuration from the packaged box into an external script [2]. It can be used as follows: config.vm.define xenserver do |xenserver| xenserver.vm.box = duffy/xenserver xenserver.vm.provision shell do |s| s.path = xenserver/reset-network.sh s.args = [eth1, 192.168.56.10, 255.255.255.0] end end I created a skeleton for the chef cookbook [3]. At the moment it include calls to the MySQL, NFS and IPTables gateway recipes with default attributes specified for the included devcloud.cfg. Along with this I wrote a systemvm downloading recipe which reads an array of systemvms from the cookbooks attributes file and downloads/installed them accordingly. It uses the -t switch to on the cloud-install-sys-tmplt script to avoid querying the MySQL db for information. I added some brief usage documentation too [4] [1] https://github.com/imduffy15/packer-xenserver [2] https://github.com/imduffy15/GSoC-2014/blob/master/MySql_NFS_XenServer/xenserver/reset-network.sh [3] https://github.com/imduffy15/cookbook_cloudstack [4] https://github.com/imduffy15/GSoC-2014/blob/master/README.md
Re: [DISCUSS] Increasing VM IOPS by separating golden image in high IOPS partition in Xen Server ?
Thanks, Hieu! I have reviewed your design (making only minor changes to your Wiki). Please feel free to have me review your code when you are ready. Also, do you have a plan for integration testing? It would be great if you could update your Wiki page to include what your plans on in this regard. Thanks! Mike On Mon, Jun 9, 2014 at 4:24 AM, Hieu LE hieul...@gmail.com wrote: Hi guys, I have updated this proposal wiki[1], included diagram for VM migrate, volume migrate and snapshot. Please review and give feedback. [1]: https://cwiki.apache.org/confluence/display/CLOUDSTACK/Golden+Primary+Storage On Fri, Jun 6, 2014 at 7:14 PM, Todd Pigram t...@toddpigram.com wrote: Sorry, thought you were based off the link you provided in this reply. In our case, we are using CloudStack integrated in VDI solution to provived pooled VM type[1]. So may be my approach can bring better UX for user with lower bootime ... A short change in design are followings - VM will be deployed with golden primary storage if primary storage is marked golden and this VM template is also marked as golden. - Choosing the best deploy destionation for both golden primary storage and normal root volume primary storage. Chosen host can also access both storage pools. - New Xen Server plug-in for modifying VHD parent id. Is there some place for me to submit my design and code. Can I write a new proposal in CS wiki ? [1]: http://support.citrix.com/proddocs/topic/xendesktop-rho/cds-choose-scheme-type-rho.html On Thu, Jun 5, 2014 at 11:55 PM, Hieu LE hieul...@gmail.com wrote: Hi Todd, On Fri, Jun 6, 2014 at 9:17 AM, Todd Pigram t...@toddpigram.com wrote: Hieu, I assume you are using MCS for you golden image? What version of XD? Given you are using pooled desktops, have you thought about using a PVS BDM iso and mount it with in your 1000 VMs? This way you can stagger reboots via PVS console or Studio. This would require a change to your delivery group. Sorry but I did not use MCS or XenDesktop in my company :-) On Thu, Jun 5, 2014 at 9:28 PM, Mike Tutkowski mike.tutkow...@solidfire.com wrote: 6) The copy_vhd_from_secondarystorage XenServer plug-in is not used when you're using XenServer + XS62ESP1 + XS62ESP1004. In that case, please refer to copyTemplateToPrimaryStorage(CopyCommand) method in the Xenserver625StorageProcessor class. Thank Mike, I will take note of that. On Thu, Jun 5, 2014 at 1:56 PM, Mike Tutkowski mike.tutkow...@solidfire.com wrote: Other than going through a for loop and deploying VM after VM, I don't think CloudStack currently supports a bulk-VM-deploy operation. It would be nice if CS did so at some point in the future; however, that is probably a separate proposal from Hieu's. On Thu, Jun 5, 2014 at 12:13 AM, Amit Das amit@cloudbyte.com wrote: Hi Hieu, Will it be good to include bulk operation of this feature? In addition, does Xen support parallel execution of these operations ? Regards, Amit *CloudByte Inc.* http://www.cloudbyte.com/ On Thu, Jun 5, 2014 at 8:59 AM, Hieu LE hieul...@gmail.com wrote: Mike, Punith, Please review Golden Primary Storage proposal. [1] Thank you. [1]: https://cwiki.apache.org/confluence/display/CLOUDSTACK/Golden+Primary+Storage On Wed, Jun 4, 2014 at 10:32 PM, Mike Tutkowski mike.tutkow...@solidfire.com wrote: Daan helped out with this. You should be good to go now. On Tue, Jun 3, 2014 at 8:50 PM, Hieu LE hieul...@gmail.com wrote: Hi Mike, Could you please give edit/create permission on ASF Jira/Wiki confluence ? I can not add a new Wiki page. My Jira ID: hieulq Wiki: hieulq89 Review Board: hieulq Thanks ! On Wed, Jun 4, 2014 at 9:17 AM, Mike Tutkowski mike.tutkow...@solidfire.com wrote: Hi, Yes, please feel free to add a new Wiki page for your design. Here is a link to applicable design info: https://cwiki.apache.org/confluence/display/CLOUDSTACK/Design Also, feel free to ask more questions and have me review your design. Thanks! Mike On Tue, Jun 3, 2014 at 7:29 PM, Hieu LE hieul...@gmail.com wrote: Hi Mike, You are right, performance will be decreased over time because writes IOPS will always end up on
Re: deleting or cancelling broken ACS jobs
have you seem this thread? http://mail-archives.apache.org/mod_mbox/cloudstack-users/201404.mbox/%3cblu176-w37d2e5fc141028f7d1937db4...@phx.gbl%3E I think this might work to delete the broken tasks. On Mon, Jun 9, 2014 at 5:35 PM, Andrei Mikhailovsky and...@arhont.com wrote: Hello guys, was wondering if anyone have come across an issue where acs would get stuck on several jobs and keeps trying to do them over and over again? I've come across an issue a few days ago. For some reason I have about 5 or 6 XenServer cluster jobs which have gone crazy. These jobs are of different nature, like template creation, vm start and enable host maintenance. They keep on repeating in the logs about 20-30 times a second, causing overfilling of logs. I get about 20GB of management server logs each day and it seems that these stuck jobs are causing the overflow. I am also not able to perform any activity on the XenServer cluster which has those stuck jobs. I am unable to start or stop jobs or pretty much do anything with it. I've tried restarting both the management server and the xenserver hosts, but that didn't help. After a short while following a restart the same thing starts to happen. Is there a way for ACS to cancel / remove these jobs? I've looked at the async_job and async_job_view db tables and I can see 28 entries there amongst which are these stuck jobs gone crazy. Is it safe for me to simply remove them from the database and restart the management server? Are there any other db tables that I should look at? Many thanks Andrei -- Rafael Weingärtner
Re: [DISCUSS] Increasing VM IOPS by separating golden image in high IOPS partition in Xen Server ?
Hieu, I made a couple of minor edits to your design to ensure everything is XenServer based. If you haven't done so already, please also fetch the most recent master and base off of that. I refactored the old Xen plugin into a XenServer specific one since Xen Project isn't currently supported, and files have moved. Also please ensure you don't use the term Xen in your code/docs to avoid any future confusion when the Xen Project work starts to materialize. Looking forward to seeing your work!! -tim On Mon, Jun 9, 2014 at 4:31 PM, Mike Tutkowski mike.tutkow...@solidfire.com wrote: Thanks, Hieu! I have reviewed your design (making only minor changes to your Wiki). Please feel free to have me review your code when you are ready. Also, do you have a plan for integration testing? It would be great if you could update your Wiki page to include what your plans on in this regard. Thanks! Mike On Mon, Jun 9, 2014 at 4:24 AM, Hieu LE hieul...@gmail.com wrote: Hi guys, I have updated this proposal wiki[1], included diagram for VM migrate, volume migrate and snapshot. Please review and give feedback. [1]: https://cwiki.apache.org/confluence/display/CLOUDSTACK/Golden+Primary+Storage On Fri, Jun 6, 2014 at 7:14 PM, Todd Pigram t...@toddpigram.com wrote: Sorry, thought you were based off the link you provided in this reply. In our case, we are using CloudStack integrated in VDI solution to provived pooled VM type[1]. So may be my approach can bring better UX for user with lower bootime ... A short change in design are followings - VM will be deployed with golden primary storage if primary storage is marked golden and this VM template is also marked as golden. - Choosing the best deploy destionation for both golden primary storage and normal root volume primary storage. Chosen host can also access both storage pools. - New Xen Server plug-in for modifying VHD parent id. Is there some place for me to submit my design and code. Can I write a new proposal in CS wiki ? [1]: http://support.citrix.com/proddocs/topic/xendesktop-rho/cds-choose-scheme-type-rho.html On Thu, Jun 5, 2014 at 11:55 PM, Hieu LE hieul...@gmail.com wrote: Hi Todd, On Fri, Jun 6, 2014 at 9:17 AM, Todd Pigram t...@toddpigram.com wrote: Hieu, I assume you are using MCS for you golden image? What version of XD? Given you are using pooled desktops, have you thought about using a PVS BDM iso and mount it with in your 1000 VMs? This way you can stagger reboots via PVS console or Studio. This would require a change to your delivery group. Sorry but I did not use MCS or XenDesktop in my company :-) On Thu, Jun 5, 2014 at 9:28 PM, Mike Tutkowski mike.tutkow...@solidfire.com wrote: 6) The copy_vhd_from_secondarystorage XenServer plug-in is not used when you're using XenServer + XS62ESP1 + XS62ESP1004. In that case, please refer to copyTemplateToPrimaryStorage(CopyCommand) method in the Xenserver625StorageProcessor class. Thank Mike, I will take note of that. On Thu, Jun 5, 2014 at 1:56 PM, Mike Tutkowski mike.tutkow...@solidfire.com wrote: Other than going through a for loop and deploying VM after VM, I don't think CloudStack currently supports a bulk-VM-deploy operation. It would be nice if CS did so at some point in the future; however, that is probably a separate proposal from Hieu's. On Thu, Jun 5, 2014 at 12:13 AM, Amit Das amit@cloudbyte.com wrote: Hi Hieu, Will it be good to include bulk operation of this feature? In addition, does Xen support parallel execution of these operations ? Regards, Amit *CloudByte Inc.* http://www.cloudbyte.com/ On Thu, Jun 5, 2014 at 8:59 AM, Hieu LE hieul...@gmail.com wrote: Mike, Punith, Please review Golden Primary Storage proposal. [1] Thank you. [1]: https://cwiki.apache.org/confluence/display/CLOUDSTACK/Golden+Primary+Storage On Wed, Jun 4, 2014 at 10:32 PM, Mike Tutkowski mike.tutkow...@solidfire.com wrote: Daan helped out with this. You should be good to go now. On Tue, Jun 3, 2014 at 8:50 PM, Hieu LE hieul...@gmail.com wrote: Hi Mike, Could you please give edit/create permission on ASF Jira/Wiki confluence ? I can not add a new Wiki page. My Jira ID: hieulq Wiki: hieulq89 Review Board: hieulq Thanks !
Re: [DISCUSS] Increasing VM IOPS by separating golden image in high IOPS partition in Xen Server ?
Yes, I was going to mention what Tim said about using the term XenServer instead of Xen as Tim has done a bunch of work recently to separate the two. I made a few changes in your Wiki when I saw a reference to Xen instead of to XenServer. On Mon, Jun 9, 2014 at 2:53 PM, Tim Mackey tmac...@gmail.com wrote: Hieu, I made a couple of minor edits to your design to ensure everything is XenServer based. If you haven't done so already, please also fetch the most recent master and base off of that. I refactored the old Xen plugin into a XenServer specific one since Xen Project isn't currently supported, and files have moved. Also please ensure you don't use the term Xen in your code/docs to avoid any future confusion when the Xen Project work starts to materialize. Looking forward to seeing your work!! -tim On Mon, Jun 9, 2014 at 4:31 PM, Mike Tutkowski mike.tutkow...@solidfire.com wrote: Thanks, Hieu! I have reviewed your design (making only minor changes to your Wiki). Please feel free to have me review your code when you are ready. Also, do you have a plan for integration testing? It would be great if you could update your Wiki page to include what your plans on in this regard. Thanks! Mike On Mon, Jun 9, 2014 at 4:24 AM, Hieu LE hieul...@gmail.com wrote: Hi guys, I have updated this proposal wiki[1], included diagram for VM migrate, volume migrate and snapshot. Please review and give feedback. [1]: https://cwiki.apache.org/confluence/display/CLOUDSTACK/Golden+Primary+Storage On Fri, Jun 6, 2014 at 7:14 PM, Todd Pigram t...@toddpigram.com wrote: Sorry, thought you were based off the link you provided in this reply. In our case, we are using CloudStack integrated in VDI solution to provived pooled VM type[1]. So may be my approach can bring better UX for user with lower bootime ... A short change in design are followings - VM will be deployed with golden primary storage if primary storage is marked golden and this VM template is also marked as golden. - Choosing the best deploy destionation for both golden primary storage and normal root volume primary storage. Chosen host can also access both storage pools. - New Xen Server plug-in for modifying VHD parent id. Is there some place for me to submit my design and code. Can I write a new proposal in CS wiki ? [1]: http://support.citrix.com/proddocs/topic/xendesktop-rho/cds-choose-scheme-type-rho.html On Thu, Jun 5, 2014 at 11:55 PM, Hieu LE hieul...@gmail.com wrote: Hi Todd, On Fri, Jun 6, 2014 at 9:17 AM, Todd Pigram t...@toddpigram.com wrote: Hieu, I assume you are using MCS for you golden image? What version of XD? Given you are using pooled desktops, have you thought about using a PVS BDM iso and mount it with in your 1000 VMs? This way you can stagger reboots via PVS console or Studio. This would require a change to your delivery group. Sorry but I did not use MCS or XenDesktop in my company :-) On Thu, Jun 5, 2014 at 9:28 PM, Mike Tutkowski mike.tutkow...@solidfire.com wrote: 6) The copy_vhd_from_secondarystorage XenServer plug-in is not used when you're using XenServer + XS62ESP1 + XS62ESP1004. In that case, please refer to copyTemplateToPrimaryStorage(CopyCommand) method in the Xenserver625StorageProcessor class. Thank Mike, I will take note of that. On Thu, Jun 5, 2014 at 1:56 PM, Mike Tutkowski mike.tutkow...@solidfire.com wrote: Other than going through a for loop and deploying VM after VM, I don't think CloudStack currently supports a bulk-VM-deploy operation. It would be nice if CS did so at some point in the future; however, that is probably a separate proposal from Hieu's. On Thu, Jun 5, 2014 at 12:13 AM, Amit Das amit@cloudbyte.com wrote: Hi Hieu, Will it be good to include bulk operation of this feature? In addition, does Xen support parallel execution of these operations ? Regards, Amit *CloudByte Inc.* http://www.cloudbyte.com/ On Thu, Jun 5, 2014 at 8:59 AM, Hieu LE hieul...@gmail.com wrote: Mike, Punith, Please review Golden Primary Storage proposal. [1] Thank you. [1]: https://cwiki.apache.org/confluence/display/CLOUDSTACK/Golden+Primary+Storage On Wed, Jun 4, 2014 at 10:32 PM, Mike Tutkowski
Re: deleting or cancelling broken ACS jobs
Raf, Not sure if this thread is related to my problem. I have a single management server with a few messed up tasks. Is it safe to remove everything from the async_job and async_job_view tables (apart from any current jobs)? Thanks Andrei - Original Message - From: Rafael Weingartner rafaelweingart...@gmail.com To: dev@cloudstack.apache.org Sent: Monday, 9 June, 2014 9:42:24 PM Subject: Re: deleting or cancelling broken ACS jobs have you seem this thread? http://mail-archives.apache.org/mod_mbox/cloudstack-users/201404.mbox/%3cblu176-w37d2e5fc141028f7d1937db4...@phx.gbl%3E I think this might work to delete the broken tasks. On Mon, Jun 9, 2014 at 5:35 PM, Andrei Mikhailovsky and...@arhont.com wrote: Hello guys, was wondering if anyone have come across an issue where acs would get stuck on several jobs and keeps trying to do them over and over again? I've come across an issue a few days ago. For some reason I have about 5 or 6 XenServer cluster jobs which have gone crazy. These jobs are of different nature, like template creation, vm start and enable host maintenance. They keep on repeating in the logs about 20-30 times a second, causing overfilling of logs. I get about 20GB of management server logs each day and it seems that these stuck jobs are causing the overflow. I am also not able to perform any activity on the XenServer cluster which has those stuck jobs. I am unable to start or stop jobs or pretty much do anything with it. I've tried restarting both the management server and the xenserver hosts, but that didn't help. After a short while following a restart the same thing starts to happen. Is there a way for ACS to cancel / remove these jobs? I've looked at the async_job and async_job_view db tables and I can see 28 entries there amongst which are these stuck jobs gone crazy. Is it safe for me to simply remove them from the database and restart the management server? Are there any other db tables that I should look at? Many thanks Andrei -- Rafael Weingärtner
Re: redundant virtual routers for VPCs
Hi all I can make 15:00. Cannon join at 20:00. Cheers Sander Verstuurd vanaf mijn iPhone Op 9 jun. 2014 om 21:31 heeft Daan Hoogland daan.hoogl...@gmail.com het volgende geschreven: nice, so given pacific time I can have a meeting at 15:00 utc or at 20:00 utc (8 am or 2 pm if I'm correct). Given office hours in Holland 17:00 (15:00 utc) would be best but if you're not in it could be later. As the person calling the meeting I would like to participate but if the highest attendance can be found only between 16 and 22 utc thats fine as well. On Mon, Jun 9, 2014 at 8:35 PM, Karl Harris karl.har...@sungardas.com wrote: I will be happy to join! On Saturday, June 7, 2014, Daan Hoogland daan.hoogl...@gmail.com wrote: At Schuberg Philis, the urge to have virtual routers in a redundant way is getting to be pressing. It seems Citrix wants to move away from it entirely and Sungard is working on it but has to little resources for it. Withing our dev group we are now planning a sprint to experiment with this subject and then implement or fix the present routers to support this. However we would very much like input from the people who have already worked on this and are also very willing to let them in our sprint-team if they are willing. So: who want to join in a irc meeting (or conf call) on the subject coming week? Dutch people are not in the office on the day after ascension so tuesday is our intention. on behalf of some very impatient operators;) -- Daan -- Daan
Re: deleting or cancelling broken ACS jobs
Well, I think it is safe, but before you do that, put disabe your zones, shutdown the MS and then remove the tasks. Or, you could simple look for some task ids that are giving you trouble and then delete them. On Mon, Jun 9, 2014 at 6:37 PM, Andrei Mikhailovsky and...@arhont.com wrote: Raf, Not sure if this thread is related to my problem. I have a single management server with a few messed up tasks. Is it safe to remove everything from the async_job and async_job_view tables (apart from any current jobs)? Thanks Andrei - Original Message - From: Rafael Weingartner rafaelweingart...@gmail.com To: dev@cloudstack.apache.org Sent: Monday, 9 June, 2014 9:42:24 PM Subject: Re: deleting or cancelling broken ACS jobs have you seem this thread? http://mail-archives.apache.org/mod_mbox/cloudstack-users/201404.mbox/%3cblu176-w37d2e5fc141028f7d1937db4...@phx.gbl%3E I think this might work to delete the broken tasks. On Mon, Jun 9, 2014 at 5:35 PM, Andrei Mikhailovsky and...@arhont.com wrote: Hello guys, was wondering if anyone have come across an issue where acs would get stuck on several jobs and keeps trying to do them over and over again? I've come across an issue a few days ago. For some reason I have about 5 or 6 XenServer cluster jobs which have gone crazy. These jobs are of different nature, like template creation, vm start and enable host maintenance. They keep on repeating in the logs about 20-30 times a second, causing overfilling of logs. I get about 20GB of management server logs each day and it seems that these stuck jobs are causing the overflow. I am also not able to perform any activity on the XenServer cluster which has those stuck jobs. I am unable to start or stop jobs or pretty much do anything with it. I've tried restarting both the management server and the xenserver hosts, but that didn't help. After a short while following a restart the same thing starts to happen. Is there a way for ACS to cancel / remove these jobs? I've looked at the async_job and async_job_view db tables and I can see 28 entries there amongst which are these stuck jobs gone crazy. Is it safe for me to simply remove them from the database and restart the management server? Are there any other db tables that I should look at? Many thanks Andrei -- Rafael Weingärtner -- Rafael Weingärtner
Re: [DISCUSS] Introducing Gerrit for quality? was: [PROPOSAL] Using continuous integration to maintain our code quality...
On Mon, Jun 9, 2014 at 3:14 PM, Rohit Yadav bhais...@apache.org wrote: On Mon, Jun 9, 2014 at 11:32 PM, sebgoa run...@gmail.com wrote: On Jun 9, 2014, at 7:13 PM, David Nalley da...@gnsa.us wrote: On Mon, Jun 9, 2014 at 11:47 AM, David Nalley da...@gnsa.us wrote: On Fri, Jun 6, 2014 at 7:26 PM, Sheng Yang sh...@yasker.org wrote: Hi all, Seems it's a good timing to bring back the discussion about the gerrit. We want to do CI, and improve our code quality. One obvious way of doing and reduce the workload of devs is introduce a tool to enforce the process. I've checked out quite a few projects using gerrit, which would force you to ask for review, and validation before the code can be committed to the repo. Looks it's really a easier way for devs according what I've heard. Even our competitor laid out a very detail workflow based on the use of gerrit( https://wiki.openstack.org/wiki/Gerrit_Workflow ). I guess it can make a good reference. Well, gerrit has been brought up a few times before. And now the new process we want to enforce just fits what gerrit(or other automation review/test/commit software) is for. Maybe it's the time for us to review the possibility of using a tool to enforce our commits and improve our code quality(as well as transfer knowledge) again? --Sheng ASF Infra has a very dour view on Gerrit. Don't read that as impossible; there are many projects at the ASF who are interested in Gerrit. That said; what about moving to using github pull requests instead of RB, and from their, having the jenkins pull request builder automatically process every pull request and list information. Here's an example: https://github.com/jclouds/jclouds-labs/pull/61 You'll see that every time the patch changes, the jenkins plugin pulled the patch - ran tests against it and reported back. That said; it almost seems like we have the cart before the horse; we need to finish figuring out the CI Infrastructure first. --David Just as an additional comment. CF was using gerrit, and apparently dropped it in favor of GH Pull Requests plus automated testing with Travis-CI. I know folks like abayer and jfarrell are working on turning on automated builds via the github pull request plugin for jenkins. Might be something to consider. fwiw, I have my personal cloudstack fork on github with a simple travisCI setup. So when I make a commit it automatically triggers a build. I was working on running the integration tests on Travis as well but got distracted. If somebody wants to step in: https://github.com/runseb/cloudstack/blob/master/.travis.yml This would be great if we could just use Github (pull requests, releases) and hook up Travis-CI. Both are free for opensource projects. To stop us from going down a bad road. We have to use the ASF repo as the canonical repo. We can use the github mirrors. Also - I am not sure if travis-ci would give us free services when the CI stuff we are talking about needing is well into 6 figures. --David
Re: SNAT iptable entry on VirtualRouter
correctVif is not null would means the network you want to is already
created.
Then ip would get device id later from:
ip.setNicDevId(Integer.valueOf(correctVif.getDevice(conn)));
So seems your correctVif is wrong with public network.
I am not very familiar with hypervisor part of find the correct network.
Anthony, could you help to take a look?
And yes, the command would also executed when you have any additional
public IPs to be associate with VR, like static nat and port forwarding.
The new interface would be only created if the public IPs are in the
different vlans.
--Sheng
On Sun, Jun 8, 2014 at 6:47 AM, Sachchidanand Vaidya vaidy...@juniper.net
wrote:
Hi Sheng,
Thanks. We are using XenServer 6.2 SP1. After adding debug in the code,
I see that addVif never gets called in my case.
That's why nic deviceId is zero. Code falls thru the case where vif is
already present.If I dump correctVif.getDevice(conn)
it returns zero.
Does this command handler also gets called when Public-ip is associated
with VM's private ip (StaticNAT) ?
Do we create a new interface in DomainRouter when staticNAT entry is
created?
Thanks,
Sachin
From: Sheng Yang sh...@yasker.org
Date: Friday, June 6, 2014 4:12 PM
To: dev@cloudstack.apache.org dev@cloudstack.apache.org,
Sachchidanand Vaidya vaidy...@juniper.net
Subject: Re: SNAT iptable entry on VirtualRouter
Hi Sachin,
The nicDevId() you see is coming from
prepareNetworkElementCommand(IpAssocCommand cmd) in CitrixResourceBase in
case of Xen.
You would see this:
if (addVif) {
// Add a new VIF to DomR
String vifDeviceNum =
getLowestAvailableVIFDeviceNum(conn, router);
if (vifDeviceNum == null) {
throw new InternalErrorException(There were no
more available slots for a new VIF on router: +
router.getNameLabel(conn));
}
nic.setDeviceId(Integer.valueOf(vifDeviceNum));
correctVif = createVif(conn, routerName, router,
null, nic);
correctVif.plug(conn);
// Add iptables rule for network usage
networkUsage(conn, routerIp, addVif, eth +
correctVif.getDevice(conn));
}
And nic.setDeviceId() should set the public nic id(which should be 2 in
your case) to it.
And what's the XenServer version you're using? Could you help to debug
it further more? Sadly we cannot reproduce it in our lab...
Thanks!
--Sheng
On Fri, Jun 6, 2014 at 12:29 AM, Sachchidanand Vaidya
vaidy...@juniper.net wrote:
Hi,
I'm seeing the same issue with 4.4 code. After further debug, I see
that CS mgmt server is sending
following command to XenHost,
xensource.log: script /opt/cloud/bin/ipassoc.sh -A -s -f -l
10.84.59.131/24 -c eth0 -g 10.84.59.254 /script
VirtualRouter's public interface is eth2. Also as per dump of VIF list on
XenHost, deviceid for public interface
of domainRouter is 2.
As part of VirtualRoutingResource.java:generateConfig(), CS mgmt server
generates this command.
It generates publicNic = eth + ip.getNicDevId()?
Which deviceId does it refer to? Shouldn't it be the deviceid as per the
XenHost dump?
Does anyone have input on what could he happening here ?
Thanks,
Sachin
Hi,
I have an isolated network (192.168.3.x/24) being served by
VirtualRouter, where 10.84.59.131 is SourceNAT address
and eth0 is VN interface of VirtualRouter eth2 is the public interface
of VirtualRouter.
When I look at the nat table entries on the VirtualRouter, it shows
following :
root@r-6-VM:~# iptables -L -t nat -n -v
..
..
Chain POSTROUTING (policy ACCEPT 330 packets, 22113 bytes)
pkts bytes target prot opt in out source
destination
0 0 SNAT all -- * eth00.0.0.0/0
0.0.0.0/0to:10.84.59.131
-- Why the out interface for the SNAT entry is VN interface (eth0)
instead of Public interface (eth2) ?
I'm using Cloudstack Release 4.3.0 (64-bit) Thu Apr 10 20:27:11 UTC
2014 cloudstack-release template.
Thanks,
Sachin
---
root@r-6-VM:~# ifconfig
eth0 Link encap:Ethernet HWaddr 02:13:87:88:e6:dd
inet addr:192.168.3.226 Bcast:192.168.3.255
Mask:255.255.255.0
inet6 addr: fe80::13:87ff:fe88:e6dd/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:350 errors:0 dropped:0 overruns:0 frame:0
TX packets:7 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:29400 (28.7 KiB) TX bytes:602 (602.0 B)
Interrupt:25
eth1 Link encap:Ethernet HWaddr 0e:00:a9:fe:02:6b
inet addr:169.254.2.107 Bcast:169.254.255.255
Mask:255.255.0.0
inet6 addr: fe80::c00:a9ff:fefe:26b/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500
Re: redundant virtual routers for VPCs
BTW, do we have an agenda or some topics we want to discuss for the meeting? I just want to make sure it's efficient. --Sheng On Mon, Jun 9, 2014 at 1:14 PM, Sander Botman sbot...@schubergphilis.com wrote: Hi all I can make 15:00. Cannon join at 20:00. Cheers Sander Verstuurd vanaf mijn iPhone Op 9 jun. 2014 om 21:31 heeft Daan Hoogland daan.hoogl...@gmail.com het volgende geschreven: nice, so given pacific time I can have a meeting at 15:00 utc or at 20:00 utc (8 am or 2 pm if I'm correct). Given office hours in Holland 17:00 (15:00 utc) would be best but if you're not in it could be later. As the person calling the meeting I would like to participate but if the highest attendance can be found only between 16 and 22 utc thats fine as well. On Mon, Jun 9, 2014 at 8:35 PM, Karl Harris karl.har...@sungardas.com wrote: I will be happy to join! On Saturday, June 7, 2014, Daan Hoogland daan.hoogl...@gmail.com wrote: At Schuberg Philis, the urge to have virtual routers in a redundant way is getting to be pressing. It seems Citrix wants to move away from it entirely and Sungard is working on it but has to little resources for it. Withing our dev group we are now planning a sprint to experiment with this subject and then implement or fix the present routers to support this. However we would very much like input from the people who have already worked on this and are also very willing to let them in our sprint-team if they are willing. So: who want to join in a irc meeting (or conf call) on the subject coming week? Dutch people are not in the office on the day after ascension so tuesday is our intention. on behalf of some very impatient operators;) -- Daan -- Daan
[ACS44] Cherry pick request
Hi Daan, Request you to please cherry-pick the following two commits to 4.4 branch : 1. ac92b3690304ff224e7e2530ea7d8e39f28a05c3 for CLOUDSTACK-6710 2. a4b401f29f83f2f0b467a9d05b509f951b5a3bca for CLOUDSTACK-6358 Thanks, Amogh
Re: Review Request 22364: Fixed 26 issues reported by coverity
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/22364/ --- (Updated June 10, 2014, 4:06 a.m.) Review request for cloudstack, daan Hoogland, Kelven Yang, Koushik Das, and Santhosh Edukulla. Repository: cloudstack-git Description --- NPEs, unused code or dead code, unwritten field access and self assignment Diffs - engine/orchestration/src/com/cloud/vm/VirtualMachineManagerImpl.java 25c67db Diff: https://reviews.apache.org/r/22364/diff/ Testing --- Thanks, Rajani Karuturi
FW: New Defects reported by Coverity Scan for cloudstack
Folks please check on the issues reported. -Original Message- From: scan-ad...@coverity.com [mailto:scan-ad...@coverity.com] Sent: Monday, June 09, 2014 7:37 PM Subject: New Defects reported by Coverity Scan for cloudstack Hi, Please find the latest report on new defect(s) introduced to cloudstack found with Coverity Scan. Defect(s) Reported-by: Coverity Scan Showing 20 of 67 defect(s) ** CID 1222156: Value not atomically updated (ATOMICITY) /plugins/hypervisors/xenserver/src/com/cloud/hypervisor/xenserver/resou rce/CitrixResourceBase.java: 5272 in com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.execute(com.cl oud.agent.api.OvsDestroyBridgeCommand)() ** CID 1222157: Dereference after null check (FORWARD_NULL) /plugins/hypervisors/xenserver/src/com/cloud/hypervisor/xenserver/resou rce/CitrixResourceBase.java: 1097 in com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.getNetwork(co m.xensource.xenapi.Connection, com.cloud.agent.api.to.NicTO)() /plugins/hypervisors/xenserver/src/com/cloud/hypervisor/xenserver/resou rce/CitrixResourceBase.java: 1088 in com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.getNetwork(co m.xensource.xenapi.Connection, com.cloud.agent.api.to.NicTO)() /plugins/hypervisors/xenserver/src/com/cloud/hypervisor/xenserver/resou rce/CitrixResourceBase.java: 1088 in com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.getNetwork(co m.xensource.xenapi.Connection, com.cloud.agent.api.to.NicTO)() ** CID 1222175: Dereference null return value (NULL_RETURNS) /plugins/hypervisors/xenserver/src/com/cloud/hypervisor/xenserver/resou rce/CitrixResourceBase.java: 3025 in com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.execute(com.cl oud.agent.api.MigrateCommand)() ** CID 1222174: Dereference null return value (NULL_RETURNS) /plugins/hypervisors/xenserver/src/com/cloud/hypervisor/xenserver/resou rce/CitrixResourceBase.java: 6347 in com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.execute(com.cl oud.agent.api.AttachVolumeCommand)() ** CID 1222173: Dereference null return value (NULL_RETURNS) /plugins/hypervisors/xenserver/src/com/cloud/hypervisor/xenserver/resou rce/CitrixResourceBase.java: 6285 in com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.execute(com.cl oud.agent.api.AttachVolumeCommand)() ** CID 1222172: Dereference null return value (NULL_RETURNS) /plugins/hypervisors/xenserver/src/com/cloud/hypervisor/xenserver/resou rce/CitrixResourceBase.java: 7200 in com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.clusterVMMet aDataSync(com.xensource.xenapi.Connection)() ** CID 1222171: Dereference null return value (NULL_RETURNS) /plugins/hypervisors/xenserver/src/com/cloud/hypervisor/xenserver/resou rce/CitrixResourceBase.java: 6131 in com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.getNfsSR(com. xensource.xenapi.Connection, com.cloud.agent.api.to.StorageFilerTO)() ** CID 1222170: Dereference null return value (NULL_RETURNS) /plugins/hypervisors/xenserver/src/com/cloud/hypervisor/xenserver/resou rce/CitrixResourceBase.java: 6412 in com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.getVMSnapsh otChainSize(com.xensource.xenapi.Connection, org.apache.cloudstack.storage.to.VolumeObjectTO, java.lang.String)() ** CID 1222169: Dereference null return value (NULL_RETURNS) /plugins/hypervisors/xenserver/src/com/cloud/hypervisor/xenserver/resou rce/CitrixResourceBase.java: 6098 in com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.getIscsiSR(com .xensource.xenapi.Connection, java.lang.String, java.lang.String, java.lang.String, java.lang.String, java.lang.String, boolean)() ** CID 1222168: Dereference null return value (NULL_RETURNS) /plugins/hypervisors/xenserver/src/com/cloud/hypervisor/xenserver/resou rce/CitrixResourceBase.java: 1944 in com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.prepareManag edStorage(com.xensource.xenapi.Connection, java.util.Map, java.lang.String, java.lang.String)() ** CID 1222167: Dereference null return value (NULL_RETURNS) /plugins/hypervisors/xenserver/src/com/cloud/hypervisor/xenserver/resou rce/CitrixResourceBase.java: 7519 in com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.prepareNetwo rkElementCommand(com.cloud.agent.api.routing.SetNetworkACLCommand )() ** CID 1222166: Dereference null return value (NULL_RETURNS) /plugins/hypervisors/xenserver/src/com/cloud/hypervisor/xenserver/resou rce/CitrixResourceBase.java: 7499 in com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.prepareNetwo rkElementCommand(com.cloud.agent.api.routing.SetSourceNatCommand)() ** CID 1222165: Dereference null return value (NULL_RETURNS) /plugins/hypervisors/xenserver/src/com/cloud/hypervisor/xenserver/resou rce/CitrixResourceBase.java: 4166 in com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.enableVlanNet
Re: [DISCUSS] Introducing Gerrit for quality? was: [PROPOSAL] Using continuous integration to maintain our code quality...
+1 for github pull requests. They are much better and cleaner than review board. ~Rajani On 09-Jun-2014, at 9:17 pm, David Nalley da...@gnsa.usmailto:da...@gnsa.us wrote: On Fri, Jun 6, 2014 at 7:26 PM, Sheng Yang sh...@yasker.orgmailto:sh...@yasker.org wrote: Hi all, Seems it's a good timing to bring back the discussion about the gerrit. We want to do CI, and improve our code quality. One obvious way of doing and reduce the workload of devs is introduce a tool to enforce the process. I've checked out quite a few projects using gerrit, which would force you to ask for review, and validation before the code can be committed to the repo. Looks it's really a easier way for devs according what I've heard. Even our competitor laid out a very detail workflow based on the use of gerrit( https://wiki.openstack.org/wiki/Gerrit_Workflow ). I guess it can make a good reference. Well, gerrit has been brought up a few times before. And now the new process we want to enforce just fits what gerrit(or other automation review/test/commit software) is for. Maybe it's the time for us to review the possibility of using a tool to enforce our commits and improve our code quality(as well as transfer knowledge) again? --Sheng ASF Infra has a very dour view on Gerrit. Don't read that as impossible; there are many projects at the ASF who are interested in Gerrit. That said; what about moving to using github pull requests instead of RB, and from their, having the jenkins pull request builder automatically process every pull request and list information. Here's an example: https://github.com/jclouds/jclouds-labs/pull/61 You'll see that every time the patch changes, the jenkins plugin pulled the patch - ran tests against it and reported back. That said; it almost seems like we have the cart before the horse; we need to finish figuring out the CI Infrastructure first. --David
Re: Review Request 22356: Fixed few coverity issues reported
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/22356/ --- (Updated June 10, 2014, 4:43 a.m.) Review request for cloudstack and daan Hoogland. Changes --- Added Latest one Repository: cloudstack-git Description --- Fixed few coverity issues reported for resource leak, value comparison, invalid loop check for result set. Diffs (updated) - engine/schema/src/com/cloud/upgrade/DatabaseCreator.java 91ef318 engine/schema/src/com/cloud/upgrade/DatabaseIntegrityChecker.java c20a418 engine/schema/src/com/cloud/upgrade/DatabaseUpgradeChecker.java 0761c9f framework/db/src/com/cloud/utils/crypt/EncryptionSecretKeyChanger.java 58584f9 framework/db/src/com/cloud/utils/db/Merovingian2.java 6eeea9f framework/db/src/com/cloud/utils/db/ScriptRunner.java 6614527 framework/db/src/com/cloud/utils/db/TransactionLegacy.java ac0ea21 server/src/com/cloud/test/IPRangeConfig.java 1d56471 usage/src/com/cloud/usage/UsageSanityChecker.java 5e6123b utils/src/com/cloud/utils/crypt/EncryptionSecretKeySender.java 086e8a8 Diff: https://reviews.apache.org/r/22356/diff/ Testing --- 1.Built the code and found no issues. 2.Built the simulator and ran a deploy datacenter with the changes. Thanks, Santhosh Edukulla
Re: Review Request 19270: Added service-instance delete command to Juniper Contrail Plugin and related fixes
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/19270/#review45197 --- Looks good to me. I didn't fine unittests for the new commands and functionality added. community wont push new features without UnitTests/Marvin tests. what about documentation of these features? - Rajesh Battala On May 7, 2014, 10:44 p.m., Sachchidanand Vaidya wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/19270/ --- (Updated May 7, 2014, 10:44 p.m.) Review request for cloudstack and Rajesh Battala. Repository: cloudstack-git Description --- This patch has following changes: - Added service-instance delete command to Juniper Contrail Plugin. - Fixes to plugin code to handle instance-delete operation properly. - Fix related to service-instance persistence and re-creation during Mgmt server reboot. Diffs - api/src/com/cloud/event/EventTypes.java 075b1c8 client/tomcatconf/commands.properties.in d3a24f6 plugins/network-elements/juniper-contrail/src/org/apache/cloudstack/network/contrail/api/command/CreateServiceInstanceCmd.java 59c4e61 plugins/network-elements/juniper-contrail/src/org/apache/cloudstack/network/contrail/api/command/DeleteServiceInstanceCmd.java PRE-CREATION plugins/network-elements/juniper-contrail/src/org/apache/cloudstack/network/contrail/management/ContrailElementImpl.java 05723b0 plugins/network-elements/juniper-contrail/src/org/apache/cloudstack/network/contrail/management/ContrailGuru.java f81c0d0 plugins/network-elements/juniper-contrail/src/org/apache/cloudstack/network/contrail/management/ServerDBSyncImpl.java b9a3ed3 plugins/network-elements/juniper-contrail/src/org/apache/cloudstack/network/contrail/management/ServiceManager.java e4ef26d plugins/network-elements/juniper-contrail/src/org/apache/cloudstack/network/contrail/management/ServiceManagerImpl.java f34eacc plugins/network-elements/juniper-contrail/src/org/apache/cloudstack/network/contrail/model/ServiceInstanceModel.java e79053c Diff: https://reviews.apache.org/r/19270/diff/ Testing --- Unit tests passed and additional testing done locally. Thanks, Sachchidanand Vaidya
