Re: NPE while adding a host
Yes. I checked in a fix yesterday with both code and schema update. If you updated code, you need to update schema as well. Thanks -min Sent from my iPhone On Dec 4, 2014, at 5:20 AM, Kuang-Ching Wang kuangching.w...@gmail.com wrote: Ah - I know what happened - need to flush the DB. KC On Dec 3, 2014, at 1:17 PM, Kuang-Ching Wang kuangching.w...@gmail.com wrote: I noticed the latest commit that reverts this. After pulling the change, Add Host works correctly. However, when creating a VM, the Instances page will throw the following ERROR and show nothing. 2014-12-03 13:15:22,720 ERROR [c.c.a.ApiServer] (catalina-exec-3:ctx-23e1ec5b ctx-baa51695) unhandled exception executing api command: [Ljava.lang.String;@2bf86dad com.cloud.utils.exception.CloudRuntimeException: DB Exception on: com.mysql.jdbc.JDBC4PreparedStatement@29a37b5e: SELECT user_vm_view.id, user_vm_view.name, user_vm_view.display_name, user_vm_view.account_id, user_vm_view.account_uuid, user_vm_view.account_name, user_vm_view.account_type, user_vm_view.domain_id, user_vm_view.domain_uuid, user_vm_view.domain_name, user_vm_view.domain_path, user_vm_view.instance_group_id, user_vm_view.instance_group_uuid, user_vm_view.instance_group_name, user_vm_view.vm_type, user_vm_view.state, user_vm_view.created, user_vm_view.removed, user_vm_view.instance_name, user_vm_view.guest_os_id, user_vm_view.guest_os_uuid, user_vm_view.hypervisor_type, user_vm_view.ha_enabled, user_vm_view.limit_cpu_use, user_vm_view.display_vm, user_vm_view.last_host_id, user_vm_view.private_ip_address, user_vm_view.private_mac_address, user_vm_view.pod_id, user_vm_view.pod_uuid, user_vm_view.data_center_id, user_vm_view.data_center_uuid, user_vm_view.data_center_name, user_vm_view.security_group_enabled, user_vm_view.host_id, user_vm_view.host_uuid, user_vm_view.host_name, user_vm_view.template_id, user_vm_view.template_uuid, user_vm_view.template_name, user_vm_view.template_display_text, user_vm_view.password_enabled, user_vm_view.iso_id, user_vm_view.iso_uuid, user_vm_view.iso_name, user_vm_view.iso_display_text, user_vm_view.disk_offering_id, user_vm_view.disk_offering_uuid, user_vm_view.disk_offering_name, user_vm_view.service_offering_id, user_vm_view.service_offering_uuid, user_vm_view.service_offering_name, user_vm_view.cpu, user_vm_view.speed, user_vm_view.ram_size, user_vm_view.pool_id, user_vm_view.pool_uuid, user_vm_view.pool_type, user_vm_view.volume_id, user_vm_view.volume_uuid, user_vm_view.volume_device_id, user_vm_view.volume_type, user_vm_view.security_group_id, user_vm_view.security_group_uuid, user_vm_view.security_group_name, user_vm_view.security_group_description, user_vm_view.vpc_id, user_vm_view.vpc_uuid, user_vm_view.nic_id, user_vm_view.nic_uuid, user_vm_view.is_default_nic, user_vm_view.ip_address, user_vm_view.gateway, user_vm_view.netmask, user_vm_view.ip6_address, user_vm_view.ip6_gateway, user_vm_view.ip6_cidr, user_vm_view.mac_address, user_vm_view.broadcast_uri, user_vm_view.isolation_uri, user_vm_view.network_id, user_vm_view.network_uuid, user_vm_view.network_name, user_vm_view.traffic_type, user_vm_view.guest_type, user_vm_view.public_ip_id, user_vm_view.public_ip_uuid, user_vm_view.public_ip_address, user_vm_view.user_data, user_vm_view.project_id, user_vm_view.project_uuid, user_vm_view.project_name, user_vm_view.keypair_name, user_vm_view.job_id, user_vm_view.job_uuid, user_vm_view.job_status, user_vm_view.tag_id, user_vm_view.tag_uuid, user_vm_view.tag_key, user_vm_view.tag_value, user_vm_view.tag_domain_id, user_vm_view.tag_account_id, user_vm_view.tag_resource_id, user_vm_view.tag_resource_uuid, user_vm_view.tag_resource_type, user_vm_view.tag_customer, user_vm_view.affinity_group_id, user_vm_view.affinity_group_uuid, user_vm_view.affinity_group_name, user_vm_view.affinity_group_description, user_vm_view.uuid, user_vm_view.dynamically_scalable FROM user_vm_view WHERE user_vm_view.id=3 at com.cloud.utils.db.GenericDaoBase.searchIncludingRemoved(GenericDaoBase.java:425) at com.cloud.utils.db.GenericDaoBase.searchIncludingRemoved(GenericDaoBase.java:361) at com.cloud.api.query.dao.UserVmJoinDaoImpl.searchByIds(UserVmJoinDaoImpl.java:410) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:317) at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:183) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150) at
Re: [QUESTION] @ReflectionUse
If I understand this clearly, this annotation was introduced by Kelven to prevent people from mistakenly removing those annotated methods if they find from IDE that those methods are not explicitly called anywhere. These methods are actually invoked through reflection. Thanks -min On Nov 27, 2014, at 3:21 AM, Daan Hoogland daan.hoogl...@gmail.com wrote: H Kelven (or others), What are the plans with this annotation, ReflectionUse. Is there to be an implementation or folow up or is it maybe just there to ignore? -- Daan
Re: [DISCUSS] Major business logic refactoring: Move from Account to UserAccount
If I understood correctly, (account, domainId) passed into deployVMCmd is used for impersonation-like behavior, that is, caller is deploying a VM on behalf of an account. Personally I don't like this kind of putting so many parameters in one API to perform several different functionalities, impersonation should be done through IAM separately. Too many parameters will just make our API semantics very hard to understand and maintain. Along this line, I will not like to see this user_id added here. Thanks -min On 11/21/14 5:20 AM, Rohit Yadav rohit.ya...@shapeblue.com wrote: Hi Prachi, Since we¹re already allowing users to specific account and list VMs by account, following the same pattern I added the case so as to allow users to specify user_id in both list/deploy VM commands. In case the userid is not specified, in that case the logged in user¹s ID will be used. It¹s open for discussion of course, let me know if it¹s a good idea to follow the same pattern or strictly use the logged-in user¹s ID? On 21-Nov-2014, at 1:41 am, Prachi Damle prachi.da...@citrix.com wrote: Rohit, I checked the code here https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;a=shortlog;h=ref s/heads/useraccount-refactoring and I don't understand why we need to expose the userId parameter in the deployVm API. I think we should be using the userId of the logged in user always. Exposing the parameter at the API allows it to be set by a user to the ID of another user . Also we need validation around it to make sure it belongs to the passed account etc. +//Owner userId +@Parameter(name = ApiConstants.USER_ID, type = CommandType.UUID, entityType = UserResponse.class, required = true, description = the user ID of the owner, optional to use with account and domainId. If not provided logged in user's ID is used.) +private Long userId; Prachi -Original Message- From: Rohit Yadav [mailto:rohit.ya...@shapeblue.com] Sent: Sunday, November 16, 2014 6:06 AM To: dev@cloudstack.apache.org Subject: Re: [DISCUSS] Major business logic refactoring: Move from Account to UserAccount Only one table will be affected. On 16-Nov-2014, at 3:14 am, Amogh Vasekar amogh.vase...@citrix.com wrote: Question - What happens to the already existing VMs with entries in the DB? Do we keep it NULL? NULL will be and not useful. I think it should be okay to have a db migration path that sets user_id to the first user in account_id (which usually has the same name as account) for existing VMs. The amount of code change will be minimal. Checkout some code in this branch (has the db migration code and API layer changes); https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;a=shortlog;h=ref s/heads/useraccount-refactoring Regards, Rohit Yadav Software Architect, ShapeBlue M. +91 88 262 30892 | rohit.ya...@shapeblue.com Blog: bhaisaab.org | Twitter: @_bhaisaab Find out more about ShapeBlue and our range of CloudStack related services IaaS Cloud Design Buildhttp://shapeblue.com/iaas-cloud-design-and-build// CSForge - rapid IaaS deployment frameworkhttp://shapeblue.com/csforge/ CloudStack Consultinghttp://shapeblue.com/cloudstack-consultancy/ CloudStack Software Engineeringhttp://shapeblue.com/cloudstack-software-engineering/ CloudStack Infrastructure Supporthttp://shapeblue.com/cloudstack-infrastructure-support/ CloudStack Bootcamp Training Courseshttp://shapeblue.com/cloudstack-training/ This email and any attachments to it may be confidential and are intended solely for the use of the individual to whom it is addressed. Any views or opinions expressed are solely those of the author and do not necessarily represent those of Shape Blue Ltd or related companies. If you are not the intended recipient of this email, you must neither take any action based upon its contents, nor copy or show it to anyone. Please contact the sender if you believe you have received this email in error. Shape Blue Ltd is a company incorporated in England Wales. ShapeBlue Services India LLP is a company incorporated in India and is operated under license from Shape Blue Ltd. Shape Blue Brasil Consultoria Ltda is a company incorporated in Brasil and is operated under license from Shape Blue Ltd. ShapeBlue SA Pty Ltd is a company registered by The Republic of South Africa and is traded under license from Shape Blue Ltd. ShapeBlue is a registered trademark. Regards, Rohit Yadav Software Architect, ShapeBlue M. +91 88 262 30892 | rohit.ya...@shapeblue.com Blog: bhaisaab.org | Twitter: @_bhaisaab Find out more about ShapeBlue and our range of CloudStack related services IaaS Cloud Design Buildhttp://shapeblue.com/iaas-cloud-design-and-build// CSForge rapid IaaS deployment frameworkhttp://shapeblue.com/csforge/ CloudStack Consultinghttp://shapeblue.com/cloudstack-consultancy/ CloudStack Software Engineeringhttp://shapeblue.com/cloudstack-software-engineering/ CloudStack Infrastructure
Re: No event publish can be wrapped within db transaction...why?
Hi Leo, NO EVENT PUBLISH CAN BE WRAPPED WITHIN DB TRANSACTION! is along the same line as NO AGENT COMMAND CAN BE WRAPPED WITHIN DB TRANSACTION!. The rationale behind this is simple: event subscriber execution or agent command handling at resource layer may take too long, and we don't want to have that long transaction window to hold DB for too long. As for your questions about why we bother to use message bus to communicate between two java component, there is a reason for it: loose coupling. IAMApiServiceImpl is a class in IAM plugin service, which can be deployed as a totally different service from CloudStack management server and ideally with future 3rd-party authentication/authorization integration, they may use a totally different database from cloud database we are currently using just for simplicity. In this deployment architecture, we have to make sure that this IAM service and CloudStack MS components are loosely coupled. Message bus provided us a very good approach to achieve that. As you said, ideally we would like to achieve a prefect transaction related to account creation in both CloudStack main component and its plugin services, but in reality, this may not work always and big transaction will be error-prone for large scale distributed systems, especially for this loosely coupled components that are crossing different DBs. The plugin architecture in CloudStack is designed to easily enable/disable each plugin component without impacting too much on main CloudStack components. So in this case, I would personally prefer that we should make sure of data integrity in the scope of CloudStack main components first and handle potential message handling failure in plugin module separately through application level logic. Thanks -min On 11/18/14 5:52 AM, Leo Simons lsim...@schubergphilis.com wrote: Hi Min, hi Koushik, Cloudstack is shouting at me: NO EVENT PUBLISH CAN BE WRAPPED WITHIN DB TRANSACTION! (full stack trace below). I've learned this is happening on our systemvm-persistent-config feature branch because it has commit ffaabdc13fde0f0f7b2667a483006e2a4b805f63 but it does not have commit f585dd266188a134a9c8b911376b066b9d3806e8 yet. I'm now trying to understand what's happening here -- the transaction / concurrency / messaging logic gave me significant headache with its triple negatives, nested transaction scoping and home-grown gates, but I think I got it now. As I understand it, in the olde world, creating an account: * opens a database transaction * creates an account in the db * creates the first user in that account in the db * publishes an event * which is listened to by 0 subscribers * commmits the database transaction * check the user is there * opens a database transaction * find the created user in the database * (auto)closes transaction * returns success if the user is in the db this, err, works, but in some other cases, apparently, there are concerns that the db transaction is open too long while message handling happens. So that's why the warning was added, and follow up on, and so now, creating an account: * opens a database transaction * creates an account in the db * creates the first user in that account in the db * commmits the database transaction * publishes an event * which is still listened to by on average 0 subscribers, but there could be an IAM subscriber * check the user is there * opens a database transaction * find the created user in the database * (auto)closes transaction * returns success if the user is in the db The one possible subscriber for account creation is IAMApiServiceImpl, which when receiving the event * opens a database transaction * adds the account to acl_group_account_map * commits the database transaction * finds the domain for the account * opens a database transaction * finds the domain for the account * (auto)closes transaction * finds the domain groups for the domain * opens a database transaction * finds the domain groups for the domain * (auto)closes transaction * for each domain group * opens a database transaction * adds the account to acl_group_account_map * commits the database transaction in other words, if there's 1 domain group and an enabled IAM thingie, this spreads out make an account over 6 transactions. Without IAM thingie its 2 two transactions with a no-op message bus thingie in the middle. Is that correct? If so, I don't understand this at all. The pre-November code doesn't make that much sense to me (why query the database? If you don't trust your database its ACID guarantees...why use transactions? Why do we ever need a message bus between two java components in the same classloader?), but the new code scares me. In the case of errors in between transactions, you can end up with accounts that are not in all the groups they should be in. I imagine I
Re: No event publish can be wrapped within db transaction...why?
For IAM feature design goal, you can take a look at our FS at https://cwiki.apache.org/confluence/display/CLOUDSTACK/CloudStack+Identity+ and+Access+Management+%28IAM%29+Plugin. Also Prachi and I presented this work at Denver Apache CloudStack Collab conference, you can also view our slideshare at http://events.linuxfoundation.org/sites/events/files/slides/ApachIAM.pdf and view the recorded presentation at https://www.youtube.com/watch?v=iUThjMl2yl8list=PLU2OcwpQkYCyPx_cwJxyOK0YK SM86Mj9nindex=24. Hope that those pointers can provide some help. -min On 11/18/14 11:26 AM, Leo Simons lsim...@schubergphilis.com wrote: Hi Min, Thanks for a very clear answer! However, I'm afraid I still don't get it :-). So... ...do you have any specific example or use case of an external IAM service to integrate with? Is there some kind of design document for me to understand the goals? I ask because all the ones that I'm familiar with tend to assume that the owner of identity information (and grouping, and possibly other kinds of AAA assertions) is externalized from systems like cloudstack to the identity system, i.e. integration is the other way around. So i.e. you would have AD or other LDAP or an SSO server or a SAML implementation (or all of those...), where systems like cloudstack then delegate AAA questions/assertions to those systems, rather than propagating local identities to that central system. I imagine if you have an external identity provider, you plug in a different implementation of AccountManager (LDAPAccountManager? etc.), and then a CreateAccountCmd would fail with an error saying the server is configured to use external thing so account creation is unsupported. cheers! Leo On Nov 18, 2014, at 7:50 PM, Min Chen min.c...@citrix.com wrote: Hi Leo, NO EVENT PUBLISH CAN BE WRAPPED WITHIN DB TRANSACTION! is along the same line as NO AGENT COMMAND CAN BE WRAPPED WITHIN DB TRANSACTION!. The rationale behind this is simple: event subscriber execution or agent command handling at resource layer may take too long, and we don't want to have that long transaction window to hold DB for too long. As for your questions about why we bother to use message bus to communicate between two java component, there is a reason for it: loose coupling. IAMApiServiceImpl is a class in IAM plugin service, which can be deployed as a totally different service from CloudStack management server and ideally with future 3rd-party authentication/authorization integration, they may use a totally different database from cloud database we are currently using just for simplicity. In this deployment architecture, we have to make sure that this IAM service and CloudStack MS components are loosely coupled. Message bus provided us a very good approach to achieve that. As you said, ideally we would like to achieve a prefect transaction related to account creation in both CloudStack main component and its plugin services, but in reality, this may not work always and big transaction will be error-prone for large scale distributed systems, especially for this loosely coupled components that are crossing different DBs. The plugin architecture in CloudStack is designed to easily enable/disable each plugin component without impacting too much on main CloudStack components. So in this case, I would personally prefer that we should make sure of data integrity in the scope of CloudStack main components first and handle potential message handling failure in plugin module separately through application level logic. Thanks -min On 11/18/14 5:52 AM, Leo Simons lsim...@schubergphilis.com wrote: Hi Min, hi Koushik, Cloudstack is shouting at me: NO EVENT PUBLISH CAN BE WRAPPED WITHIN DB TRANSACTION! (full stack trace below). I've learned this is happening on our systemvm-persistent-config feature branch because it has commit ffaabdc13fde0f0f7b2667a483006e2a4b805f63 but it does not have commit f585dd266188a134a9c8b911376b066b9d3806e8 yet. I'm now trying to understand what's happening here -- the transaction / concurrency / messaging logic gave me significant headache with its triple negatives, nested transaction scoping and home-grown gates, but I think I got it now. As I understand it, in the olde world, creating an account: * opens a database transaction * creates an account in the db * creates the first user in that account in the db * publishes an event * which is listened to by 0 subscribers * commmits the database transaction * check the user is there * opens a database transaction * find the created user in the database * (auto)closes transaction * returns success if the user is in the db this, err, works, but in some other cases, apparently, there are concerns that the db transaction is open too long while message handling happens. So that's why the warning was added, and follow up on, and so now, creating an account
Re: [DISCUSS] Major business logic refactoring: Move from Account to UserAccount
Rohit, I think that the historic reason for this is that CloudStack is only doing IAM access permission check on account level, user is only login authentication purpose. That is why we will see that all our CloudStack resource owner field is an account, since that is the only information used for controlling whether you have some permissions to the resource. Thanks -min On 11/14/14 12:53 AM, Rohit Yadav rohit.ya...@shapeblue.com wrote: Hi, All CloudStack DB entities (VM, storage, network etc.) have an owner field which is mostly the account. An account can have multiple users so just by looking at the resource (say VM) it¹s not possible to make out which user in the account (owner or account_id field in the db row of the entity) created it. CloudStack users may want to know this information for at least entities such as VMs and Volumes. Historically, why is the account owner of an entity and not a user? If user were the owner, we could easily get the account Id using the user Id. One solution to fix this problem is to refactor and replace Account (interface) usage with UserAccount (interface) usage, fix the DAO and resource layer, and add columns in the schema. This gets us all the information we need to determine domainId, AccountId and Id (the user ID). Should we do it for all entities or just keep status quo (use account as owners), or just fix it on-demand basis for specific entities such as for user VMs [1]. [1] https://issues.apache.org/jira/browse/CLOUDSTACK-7908 Regards, Rohit Yadav Software Architect, ShapeBlue M. +91 88 262 30892 | rohit.ya...@shapeblue.com Blog: bhaisaab.org | Twitter: @_bhaisaab Find out more about ShapeBlue and our range of CloudStack related services IaaS Cloud Design Buildhttp://shapeblue.com/iaas-cloud-design-and-build// CSForge rapid IaaS deployment frameworkhttp://shapeblue.com/csforge/ CloudStack Consultinghttp://shapeblue.com/cloudstack-consultancy/ CloudStack Software Engineeringhttp://shapeblue.com/cloudstack-software-engineering/ CloudStack Infrastructure Supporthttp://shapeblue.com/cloudstack-infrastructure-support/ CloudStack Bootcamp Training Courseshttp://shapeblue.com/cloudstack-training/ This email and any attachments to it may be confidential and are intended solely for the use of the individual to whom it is addressed. Any views or opinions expressed are solely those of the author and do not necessarily represent those of Shape Blue Ltd or related companies. If you are not the intended recipient of this email, you must neither take any action based upon its contents, nor copy or show it to anyone. Please contact the sender if you believe you have received this email in error. Shape Blue Ltd is a company incorporated in England Wales. ShapeBlue Services India LLP is a company incorporated in India and is operated under license from Shape Blue Ltd. Shape Blue Brasil Consultoria Ltda is a company incorporated in Brasil and is operated under license from Shape Blue Ltd. ShapeBlue SA Pty Ltd is a company registered by The Republic of South Africa and is traded under license from Shape Blue Ltd. ShapeBlue is a registered trademark.
Re: [DISCUSS] Major business logic refactoring: Move from Account to UserAccount
Rohit, If I understood you correctly, the user_id column is only used for listing resources to indicate which user is the real owner/creator of the resource, but you don't want to change CloudStack account-level permission model to user-level permission model, right? If so, the change will be smaller, maybe some Response classes, which should not involve too many business layer change. I will hesitate to really change CloudStack IAM model though. Thanks -min On 11/14/14 10:35 AM, Rohit Yadav rohit.ya...@shapeblue.com wrote: Hi Min, Good to know. What do you propose we do moving forward. Do a refactoring run to fix it or leave it as it is and perhaps add user_id columns to few resources that are more useful for sysadmins such as vm_instance table. On 14-Nov-2014, at 11:49 pm, Min Chen min.c...@citrix.com wrote: Rohit, I think that the historic reason for this is that CloudStack is only doing IAM access permission check on account level, user is only login authentication purpose. That is why we will see that all our CloudStack resource owner field is an account, since that is the only information used for controlling whether you have some permissions to the resource. Thanks -min On 11/14/14 12:53 AM, Rohit Yadav rohit.ya...@shapeblue.com wrote: Hi, All CloudStack DB entities (VM, storage, network etc.) have an owner field which is mostly the account. An account can have multiple users so just by looking at the resource (say VM) it¹s not possible to make out which user in the account (owner or account_id field in the db row of the entity) created it. CloudStack users may want to know this information for at least entities such as VMs and Volumes. Historically, why is the account owner of an entity and not a user? If user were the owner, we could easily get the account Id using the user Id. One solution to fix this problem is to refactor and replace Account (interface) usage with UserAccount (interface) usage, fix the DAO and resource layer, and add columns in the schema. This gets us all the information we need to determine domainId, AccountId and Id (the user ID). Should we do it for all entities or just keep status quo (use account as owners), or just fix it on-demand basis for specific entities such as for user VMs [1]. [1] https://issues.apache.org/jira/browse/CLOUDSTACK-7908 Regards, Rohit Yadav Software Architect, ShapeBlue M. +91 88 262 30892 | rohit.ya...@shapeblue.com Blog: bhaisaab.org | Twitter: @_bhaisaab Find out more about ShapeBlue and our range of CloudStack related services IaaS Cloud Design Buildhttp://shapeblue.com/iaas-cloud-design-and-build// CSForge rapid IaaS deployment frameworkhttp://shapeblue.com/csforge/ CloudStack Consultinghttp://shapeblue.com/cloudstack-consultancy/ CloudStack Software Engineeringhttp://shapeblue.com/cloudstack-software-engineering/ CloudStack Infrastructure Supporthttp://shapeblue.com/cloudstack-infrastructure-support/ CloudStack Bootcamp Training Courseshttp://shapeblue.com/cloudstack-training/ This email and any attachments to it may be confidential and are intended solely for the use of the individual to whom it is addressed. Any views or opinions expressed are solely those of the author and do not necessarily represent those of Shape Blue Ltd or related companies. If you are not the intended recipient of this email, you must neither take any action based upon its contents, nor copy or show it to anyone. Please contact the sender if you believe you have received this email in error. Shape Blue Ltd is a company incorporated in England Wales. ShapeBlue Services India LLP is a company incorporated in India and is operated under license from Shape Blue Ltd. Shape Blue Brasil Consultoria Ltda is a company incorporated in Brasil and is operated under license from Shape Blue Ltd. ShapeBlue SA Pty Ltd is a company registered by The Republic of South Africa and is traded under license from Shape Blue Ltd. ShapeBlue is a registered trademark. Regards, Rohit Yadav Software Architect, ShapeBlue M. +91 88 262 30892 | rohit.ya...@shapeblue.com Blog: bhaisaab.org | Twitter: @_bhaisaab Find out more about ShapeBlue and our range of CloudStack related services IaaS Cloud Design Buildhttp://shapeblue.com/iaas-cloud-design-and-build// CSForge rapid IaaS deployment frameworkhttp://shapeblue.com/csforge/ CloudStack Consultinghttp://shapeblue.com/cloudstack-consultancy/ CloudStack Software Engineeringhttp://shapeblue.com/cloudstack-software-engineering/ CloudStack Infrastructure Supporthttp://shapeblue.com/cloudstack-infrastructure-support/ CloudStack Bootcamp Training Courseshttp://shapeblue.com/cloudstack-training/ This email and any attachments to it may be confidential and are intended solely for the use of the individual to whom it is addressed. Any views or opinions expressed are solely those of the author and do not necessarily represent those of Shape Blue Ltd or related companies
Re: [DISCUSS] Major business logic refactoring: Move from Account to UserAccount
Yes, we support going both directions, account - user. Thanks -min On 11/14/14 10:59 AM, Mike Tutkowski mike.tutkow...@solidfire.com wrote: I didn't have access to the VM with my CS DB on it for a while, but I do now and checked the DB structure. I see the user table has a reference to the account table, so that's what I was expecting and hoped to see. On Fri, Nov 14, 2014 at 11:48 AM, Mike Tutkowski mike.tutkow...@solidfire.com wrote: I haven't looked at the DB tables for this, but presumably there is a user table like we have an account table and you can figure out what account a given user is in? That would be OK then. I just wasn't sure if we only allowed you to go from account to user, but not user to account in the DB. On Fri, Nov 14, 2014 at 11:39 AM, Mike Tutkowski mike.tutkow...@solidfire.com wrote: Can a username like mike be re-used in multiple accounts? For example: Acct1\mike Acct2\mike If so, the name mike would be insufficient to determine ownership of the resource in some situations (unless it was fully qualified with its account). On Fri, Nov 14, 2014 at 11:35 AM, Rohit Yadav rohit.ya...@shapeblue.com wrote: Hi Min, Good to know. What do you propose we do moving forward. Do a refactoring run to fix it or leave it as it is and perhaps add user_id columns to few resources that are more useful for sysadmins such as vm_instance table. On 14-Nov-2014, at 11:49 pm, Min Chen min.c...@citrix.com wrote: Rohit, I think that the historic reason for this is that CloudStack is only doing IAM access permission check on account level, user is only login authentication purpose. That is why we will see that all our CloudStack resource owner field is an account, since that is the only information used for controlling whether you have some permissions to the resource. Thanks -min On 11/14/14 12:53 AM, Rohit Yadav rohit.ya...@shapeblue.com wrote: Hi, All CloudStack DB entities (VM, storage, network etc.) have an owner field which is mostly the account. An account can have multiple users so just by looking at the resource (say VM) it¹s not possible to make out which user in the account (owner or account_id field in the db row of the entity) created it. CloudStack users may want to know this information for at least entities such as VMs and Volumes. Historically, why is the account owner of an entity and not a user? If user were the owner, we could easily get the account Id using the user Id. One solution to fix this problem is to refactor and replace Account (interface) usage with UserAccount (interface) usage, fix the DAO and resource layer, and add columns in the schema. This gets us all the information we need to determine domainId, AccountId and Id (the user ID). Should we do it for all entities or just keep status quo (use account as owners), or just fix it on-demand basis for specific entities such as for user VMs [1]. [1] https://issues.apache.org/jira/browse/CLOUDSTACK-7908 Regards, Rohit Yadav Software Architect, ShapeBlue M. +91 88 262 30892 | rohit.ya...@shapeblue.com Blog: bhaisaab.org | Twitter: @_bhaisaab Find out more about ShapeBlue and our range of CloudStack related services IaaS Cloud Design Buildhttp://shapeblue.com/iaas-cloud-design-and-build// CSForge rapid IaaS deployment framework http://shapeblue.com/csforge/ CloudStack Consultinghttp://shapeblue.com/cloudstack-consultancy/ CloudStack Software Engineeringhttp://shapeblue.com/cloudstack-software-engineering/ CloudStack Infrastructure Supporthttp://shapeblue.com/cloudstack-infrastructure-support/ CloudStack Bootcamp Training Courseshttp://shapeblue.com/cloudstack-training/ This email and any attachments to it may be confidential and are intended solely for the use of the individual to whom it is addressed. Any views or opinions expressed are solely those of the author and do not necessarily represent those of Shape Blue Ltd or related companies. If you are not the intended recipient of this email, you must neither take any action based upon its contents, nor copy or show it to anyone. Please contact the sender if you believe you have received this email in error. Shape Blue Ltd is a company incorporated in England Wales. ShapeBlue Services India LLP is a company incorporated in India and is operated under license from Shape Blue Ltd. Shape Blue Brasil Consultoria Ltda is a company incorporated in Brasil and is operated under license from Shape Blue Ltd. ShapeBlue SA Pty Ltd is a company registered by The Republic of South Africa and is traded under license from Shape Blue Ltd. ShapeBlue is a registered trademark. Regards, Rohit Yadav Software Architect, ShapeBlue M. +91 88 262 30892 | rohit.ya...@shapeblue.com Blog: bhaisaab.org | Twitter: @_bhaisaab Find out more about ShapeBlue and our range of CloudStack related services IaaS
Re: Cloudstack 4.3.1 - NullPointerException when deleting VM
This is a bug https://issues.apache.org/jira/browse/CLOUDSTACK-6934, which has been fixed in 4.5.0. The VM to be expunged did not allocate volume on primary storage. Thanks -min On 11/6/14 4:57 AM, Rafael Weingartner rafaelweingart...@gmail.com wrote: HI, Have already someone seen a null pointer when deleting a VM? I have and environment running for a while with CS 4.3.1, and today I tried to delete a VM that was in expunging state but it keeps getting an error: 2014-11-06 10:54:23,473 DEBUG [c.c.v.VirtualMachineManagerImpl] (Job-Executor-37:ctx-0dd66018 ctx-086a8303) Cleaning up hypervisor data structures (ex. SRs in XenServer) for managed storage 2014-11-06 10:54:23,491 ERROR [c.c.a.ApiAsyncJobDispatcher] (Job-Executor-37:ctx-0dd66018) Unexpected exception while executing org.apache.cloudstack.api.command.admin.vm.ExpungeVMCmd java.lang.NullPointerException at org.apache.cloudstack.engine.orchestration.VolumeOrchestrator.disconnectVo lumesFromHost(VolumeOrchestrator.java:885) at com.cloud.vm.VirtualMachineManagerImpl.advanceExpunge(VirtualMachineManage rImpl.java:513) at com.cloud.vm.VirtualMachineManagerImpl.advanceExpunge(VirtualMachineManage rImpl.java:449) at com.cloud.vm.UserVmManagerImpl.expunge(UserVmManagerImpl.java:1709) at com.cloud.vm.UserVmManagerImpl.expungeVm(UserVmManagerImpl.java:3815) at com.cloud.vm.UserVmManagerImpl.expungeVm(UserVmManagerImpl.java:2104) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java: 57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorIm pl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(Ao pUtils.java:317) at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoi nt(ReflectiveMethodInvocation.java:183) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(Refle ctiveMethodInvocation.java:150) We are using, Xen hypervisor (4.1) with XCP packages (1.6). The CS version is 4.3.1 The storage are volumes exported with NFS. -- Rafael Weingärtner
Re: Cloudstack 4.3.1 - NullPointerException when deleting VM
Yes, that is the workaround, just set pool_id to some dummy pool. On 11/6/14 10:01 AM, Rafael Weingartner rafaelweingart...@gmail.com wrote: Thanks, After I looked at the CS source code. I worked around it by setting a pool_id in the database in the volume table. On Thu, Nov 6, 2014 at 3:58 PM, Min Chen min.c...@citrix.com wrote: This is a bug https://issues.apache.org/jira/browse/CLOUDSTACK-6934, which has been fixed in 4.5.0. The VM to be expunged did not allocate volume on primary storage. Thanks -min On 11/6/14 4:57 AM, Rafael Weingartner rafaelweingart...@gmail.com wrote: HI, Have already someone seen a null pointer when deleting a VM? I have and environment running for a while with CS 4.3.1, and today I tried to delete a VM that was in expunging state but it keeps getting an error: 2014-11-06 10:54:23,473 DEBUG [c.c.v.VirtualMachineManagerImpl] (Job-Executor-37:ctx-0dd66018 ctx-086a8303) Cleaning up hypervisor data structures (ex. SRs in XenServer) for managed storage 2014-11-06 10:54:23,491 ERROR [c.c.a.ApiAsyncJobDispatcher] (Job-Executor-37:ctx-0dd66018) Unexpected exception while executing org.apache.cloudstack.api.command.admin.vm.ExpungeVMCmd java.lang.NullPointerException at org.apache.cloudstack.engine.orchestration.VolumeOrchestrator.disconnect Vo lumesFromHost(VolumeOrchestrator.java:885) at com.cloud.vm.VirtualMachineManagerImpl.advanceExpunge(VirtualMachineMana ge rImpl.java:513) at com.cloud.vm.VirtualMachineManagerImpl.advanceExpunge(VirtualMachineMana ge rImpl.java:449) at com.cloud.vm.UserVmManagerImpl.expunge(UserVmManagerImpl.java:1709) at com.cloud.vm.UserVmManagerImpl.expungeVm(UserVmManagerImpl.java:3815) at com.cloud.vm.UserVmManagerImpl.expungeVm(UserVmManagerImpl.java:2104) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.jav a: 57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessor Im pl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection( Ao pUtils.java:317) at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinp oi nt(ReflectiveMethodInvocation.java:183) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(Ref le ctiveMethodInvocation.java:150) We are using, Xen hypervisor (4.1) with XCP packages (1.6). The CS version is 4.3.1 The storage are volumes exported with NFS. -- Rafael Weingärtner -- Rafael Weingärtner
Re: Cloudstack 4.3.1 - NullPointerException when deleting VM
This is for VMs that are not properly deployed. Thanks -min On 11/6/14 10:12 AM, Mike Tutkowski mike.tutkow...@solidfire.com wrote: Hey everyone, If a VM has been running on a host at some point, how can any of the volumes it's associated with have a null for pool_id? Thanks, Mike On Thu, Nov 6, 2014 at 11:08 AM, Min Chen min.c...@citrix.com wrote: Yes, that is the workaround, just set pool_id to some dummy pool. On 11/6/14 10:01 AM, Rafael Weingartner rafaelweingart...@gmail.com wrote: Thanks, After I looked at the CS source code. I worked around it by setting a pool_id in the database in the volume table. On Thu, Nov 6, 2014 at 3:58 PM, Min Chen min.c...@citrix.com wrote: This is a bug https://issues.apache.org/jira/browse/CLOUDSTACK-6934, which has been fixed in 4.5.0. The VM to be expunged did not allocate volume on primary storage. Thanks -min On 11/6/14 4:57 AM, Rafael Weingartner rafaelweingart...@gmail.com wrote: HI, Have already someone seen a null pointer when deleting a VM? I have and environment running for a while with CS 4.3.1, and today I tried to delete a VM that was in expunging state but it keeps getting an error: 2014-11-06 10:54:23,473 DEBUG [c.c.v.VirtualMachineManagerImpl] (Job-Executor-37:ctx-0dd66018 ctx-086a8303) Cleaning up hypervisor data structures (ex. SRs in XenServer) for managed storage 2014-11-06 10:54:23,491 ERROR [c.c.a.ApiAsyncJobDispatcher] (Job-Executor-37:ctx-0dd66018) Unexpected exception while executing org.apache.cloudstack.api.command.admin.vm.ExpungeVMCmd java.lang.NullPointerException at org.apache.cloudstack.engine.orchestration.VolumeOrchestrator.disconne ct Vo lumesFromHost(VolumeOrchestrator.java:885) at com.cloud.vm.VirtualMachineManagerImpl.advanceExpunge(VirtualMachineMa na ge rImpl.java:513) at com.cloud.vm.VirtualMachineManagerImpl.advanceExpunge(VirtualMachineMa na ge rImpl.java:449) at com.cloud.vm.UserVmManagerImpl.expunge(UserVmManagerImpl.java:1709) at com.cloud.vm.UserVmManagerImpl.expungeVm(UserVmManagerImpl.java:3815) at com.cloud.vm.UserVmManagerImpl.expungeVm(UserVmManagerImpl.java:2104) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.j av a: 57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccess or Im pl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflectio n( Ao pUtils.java:317) at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoi np oi nt(ReflectiveMethodInvocation.java:183) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(R ef le ctiveMethodInvocation.java:150) We are using, Xen hypervisor (4.1) with XCP packages (1.6). The CS version is 4.3.1 The storage are volumes exported with NFS. -- Rafael Weingärtner -- Rafael Weingärtner -- *Mike Tutkowski* *Senior CloudStack Developer, SolidFire Inc.* e: mike.tutkow...@solidfire.com o: 303.746.7302 Advancing the way the world uses the cloud http://solidfire.com/solution/overview/?video=play**
Re: Heart Beat messages...
This is not error, that is why it is printed as DEBUG. There is job heartbeat thread that runs regularly trying to pick up jobs from each VM queue. When there is already a currently active job running for that queue, this message will be printed. Thanks -min On 10/15/14 12:58 PM, Mo m...@daoenix.com wrote: My management log is being flooded with errors: 2014-10-15 15:57:47,790 DEBUG [o.a.c.f.j.i.SyncQueueManagerImpl] (AsyncJobMgr-Heartbeat-1:ctx-257021c2) Queue (queue id, sync type, sync id) - (1,VmWorkJobQueue, 1) is reaching concurrency limit 1 2014-10-15 15:57:47,792 DEBUG [o.a.c.f.j.i.SyncQueueManagerImpl] (AsyncJobMgr-Heartbeat-1:ctx-257021c2) Queue (queue id, sync type, sync id) - (43,VmWorkJobQueue, 11) is reaching concurrency limit 1 2014-10-15 15:57:49,790 DEBUG [o.a.c.f.j.i.SyncQueueManagerImpl] (AsyncJobMgr-Heartbeat-1:ctx-eaf952ad) Queue (queue id, sync type, sync id) - (1,VmWorkJobQueue, 1) is reaching concurrency limit 1 2014-10-15 15:57:49,792 DEBUG [o.a.c.f.j.i.SyncQueueManagerImpl] (AsyncJobMgr-Heartbeat-1:ctx-eaf952ad) Queue (queue id, sync type, sync id) - (43,VmWorkJobQueue, 11) is reaching concurrency limit 1 2014-10-15 15:57:51,789 DEBUG [o.a.c.f.j.i.SyncQueueManagerImpl] (AsyncJobMgr-Heartbeat-1:ctx-e5444b6a) Queue (queue id, sync type, sync id) - (1,VmWorkJobQueue, 1) is reaching concurrency limit 1 2014-10-15 15:57:51,790 DEBUG [o.a.c.f.j.i.SyncQueueManagerImpl] (AsyncJobMgr-Heartbeat-1:ctx-e5444b6a) Queue (queue id, sync type, sync id) - (43,VmWorkJobQueue, 11) is reaching concurrency limit 1 2014-10-15 15:57:53,538 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl] (RouterStatusMonitor-1:ctx-3e5c7227) Found 0 routers to update status. 2014-10-15 15:57:53,540 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl] (RouterStatusMonitor-1:ctx-3e5c7227) Found 0 networks to update RvR status. 2014-10-15 15:57:53,789 DEBUG [o.a.c.f.j.i.SyncQueueManagerImpl] (AsyncJobMgr-Heartbeat-1:ctx-e2b67d52) Queue (queue id, sync type, sync id) - (1,VmWorkJobQueue, 1) is reaching concurrency limit 1 2014-10-15 15:57:53,791 DEBUG [o.a.c.f.j.i.SyncQueueManagerImpl] (AsyncJobMgr-Heartbeat-1:ctx-e2b67d52) Queue (queue id, sync type, sync id) - (43,VmWorkJobQueue, 11) is reaching concurrency limit 1 2014-10-15 15:57:55,790 DEBUG [o.a.c.f.j.i.SyncQueueManagerImpl] (AsyncJobMgr-Heartbeat-1:ctx-237104ea) Queue (queue id, sync type, sync id) - (1,VmWorkJobQueue, 1) is reaching concurrency limit 1 2014-10-15 15:57:55,792 DEBUG [o.a.c.f.j.i.SyncQueueManagerImpl] (AsyncJobMgr-Heartbeat-1:ctx-237104ea) Queue (queue id, sync type, sync id) - (43,VmWorkJobQueue, 11) is reaching concurrency limit 1 2014-10-15 15:57:57,605 DEBUG [c.c.a.m.AgentManagerImpl] (AgentManager-Handler-9:null) Ping from 1 2014-10-15 15:57:57,606 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl] (AgentManager-Handler-9:null) Process host VM state report from ping process. host: 1 2014-10-15 15:57:57,606 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl] (AgentManager-Handler-9:null) Process VM state report. host: 1, number of records in report: 0 2014-10-15 15:57:57,607 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl] (AgentManager-Handler-9:null) Done with process of VM state report. host: 1 2014-10-15 15:57:57,789 DEBUG [o.a.c.f.j.i.SyncQueueManagerImpl] (AsyncJobMgr-Heartbeat-1:ctx-7eec8962) Queue (queue id, sync type, sync id) - (1,VmWorkJobQueue, 1) is reaching concurrency limit 1 2014-10-15 15:57:57,791 DEBUG [o.a.c.f.j.i.SyncQueueManagerImpl] (AsyncJobMgr-Heartbeat-1:ctx-7eec8962) Queue (queue id, sync type, sync id) - (43,VmWorkJobQueue, 11) is reaching concurrency limit 1 2014-10-15 15:57:59,790 DEBUG [o.a.c.f.j.i.SyncQueueManagerImpl] (AsyncJobMgr-Heartbeat-1:ctx-51b59d97) Queue (queue id, sync type, sync id) - (1,VmWorkJobQueue, 1) is reaching concurrency limit 1 2014-10-15 15:57:59,791 DEBUG [o.a.c.f.j.i.SyncQueueManagerImpl] (AsyncJobMgr-Heartbeat-1:ctx-51b59d97) Queue (queue id, sync type, sync id) - (43,VmWorkJobQueue, 11) is reaching concurrency limit 1 What exactly is it trying to tell me? From what I gather, it seems there is a queue it¹s unable to process? // mo
Re: [ANNOUNCE] New PMC Member: Ian Duffy
Congratulations, Ian. -min On 9/30/14 10:42 PM, Prasanna Santhanam t...@apache.org wrote: Congratulations Ian! On Tue, Sep 30, 2014 at 7:20 PM, Chip Childers chipchild...@apache.org wrote: The Project Management Committee (PMC) for Apache CloudStack has asked Ian Duffy to join the PMC and we are pleased to announce that he has accepted. Join me in congratulating Ian! -chip On behalf of the Apache CloudStack PMC
Re: [ANNOUNCE] New PMC Member: Pierre-Luc Dion
Congrats, Pierre-Luc! -min On 9/30/14 10:43 PM, Prasanna Santhanam t...@apache.org wrote: Congratulations Pierre-Luc! On Tue, Sep 30, 2014 at 7:20 PM, Chip Childers chipchild...@apache.org wrote: The Project Management Committee (PMC) for Apache CloudStack has asked Pierre-Luc Dion to join the PMC and we are pleased to announce that he has accepted. Join me in congratulating Pierre-Luc! -chip On behalf of the Apache CloudStack PMC
Re: Review Request 25722: CLOUDSTACK-7567 - Automate ACL test cases relating to depoying VM in shared network with different scopes - All/Domain/Domain with subdomain/Account for Admin, domain admin a
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/25722/#review53633 --- Ship it! Ship It! - Min Chen On Sept. 17, 2014, 12:42 a.m., sangeetha hariharan wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/25722/ --- (Updated Sept. 17, 2014, 12:42 a.m.) Review request for cloudstack, edison su, Min Chen, Prachi Damle, and Santhosh Edukulla. Repository: cloudstack-git Description --- CLOUDSTACK-7567 - Automate ACL test cases relating to depoying VM in shared network with different scopes - All/Domain/Domain with subdomain/Account for Admin, domain admin and regular users. This test suite contains test cases that validates access checks relating to depoying VM in shared network with different scopes - All/Domain/Domain with subdomain/Account for Admin, domain admin and regular users. Diffs - test/integration/component/test_acl_sharednetwork.py PRE-CREATION Diff: https://reviews.apache.org/r/25722/diff/ Testing --- Tested on simulator build from master branch: Validate that admin user in ROOT domain is NOT allowed to deploy VM in a shared network created with scope=account for an account ... === TestName: test_deployVM_in_sharedNetwork_scope_account_ROOTadmin | Status : SUCCESS === ok Validate that user in ROOT domain is NOT allowed to deploy VM in a shared network created with scope=account for an account ... === TestName: test_deployVM_in_sharedNetwork_scope_account_ROOTuser | Status : SUCCESS === ok Validate that regular user from a domain different from that of the account is NOT allowed to deploy VM in a shared network created with scope=account for an account ... === TestName: test_deployVM_in_sharedNetwork_scope_account_differentdomain | Status : SUCCESS === ok Validate that an admin user under the same domain but belonging to a different account is allowed to deploy VM in a shared network created with scope=account for an account ... === TestName: test_deployVM_in_sharedNetwork_scope_account_domainadminuser | Status : SUCCESS === ok Validate that any other user in same domain is NOT allowed to deploy VM in a shared network created with scope=account for an account ... === TestName: test_deployVM_in_sharedNetwork_scope_account_domainuser | Status : SUCCESS === ok Validate that regular user in the account is allowed to deploy VM in a shared network created with scope=account for an account ... === TestName: test_deployVM_in_sharedNetwork_scope_account_user | Status : SUCCESS === ok Validate that admin user in ROOT domain is allowed to deploy VM in a shared network created with scope=all ... === TestName: test_deployVM_in_sharedNetwork_scope_all_ROOTadmin | Status : SUCCESS === ok Validate that regular user in ROOT domain is allowed to deploy VM in a shared network created with scope=all ... === TestName: test_deployVM_in_sharedNetwork_scope_all_ROOTuser | Status : SUCCESS === ok Validate that regular user in ROOT domain is allowed to deploy VM in a shared network created with scope=all ... === TestName: test_deployVM_in_sharedNetwork_scope_all_domainadminuser | Status : SUCCESS === ok Validate that regular user in a domain is allowed to deploy VM in a shared network created with scope=all ... === TestName: test_deployVM_in_sharedNetwork_scope_all_domainuser | Status : SUCCESS === ok Validate that regular user in a subdomain under ROOT is allowed to deploy VM in a shared network created with scope=all ... === TestName: test_deployVM_in_sharedNetwork_scope_all_subdomainadminuser | Status : SUCCESS === ok Validate that regular user in any subdomain is allowed to deploy VM in a shared network created with scope=all ... === TestName: test_deployVM_in_sharedNetwork_scope_all_subdomainuser | Status : SUCCESS === ok Validate that admin in ROOT domain is NOT allowed to deploy VM in a shared network created with scope=domain and no subdomain access ... === TestName: test_deployVM_in_sharedNetwork_scope_domain_nosubdomainaccess_ROOTadmin | Status : SUCCESS === ok Validate that user in ROOT domain is NOT allowed to deploy VM in a shared network created with scope=domain and no subdomain access ... === TestName: test_deployVM_in_sharedNetwork_scope_domain_nosubdomainaccess_ROOTuser | Status : SUCCESS === ok Validate that admin user in a domain is allowed to deploy VM in a shared network created with scope=domain and no subdomain access ... === TestName: test_deployVM_in_sharedNetwork_scope_domain_nosubdomainaccess_domainadminuser | Status : SUCCESS === ok Validate that regular user in a domain is allowed to deploy
Re: Review Request 22707: Test suite contains test cases relating to access checks for listSnapshot() with parameters - id, listall, isrecursive, account and domainid executed as ROOT admin, domain ad
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/22707/#review52756 --- Can we change to use consistent docstring (triple quotes style) for your method comments? - Min Chen On Sept. 9, 2014, 12:27 a.m., sangeetha hariharan wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/22707/ --- (Updated Sept. 9, 2014, 12:27 a.m.) Review request for cloudstack, edison su, Min Chen, Prachi Damle, and Santhosh Edukulla. Repository: cloudstack-git Description --- This test suite contains test cases relating to access checks for listSnapshot() with parameters - id,listall,isrecursive,account and domainid executed as ROOT admin,domain admin and regular users. Diffs - test/integration/component/test_acl_listsnapshot.py PRE-CREATION Diff: https://reviews.apache.org/r/22707/diff/ Testing --- test_listSnapshot_as_domainadmin (integration.component.test_acl_listsnapshot.TestSnapshotList) ... === TestName: test_listSnapshot_as_domainadmin | Status : SUCCESS === ok test_listSnapshot_as_domainadmin_cross_domainid (integration.component.test_acl_listsnapshot.TestSnapshotList) ... === TestName: test_listSnapshot_as_domainadmin_cross_domainid | Status : SUCCESS === ok test_listSnapshot_as_domainadmin_cross_domainid_accountid (integration.component.test_acl_listsnapshot.TestSnapshotList) ... === TestName: test_listSnapshot_as_domainadmin_cross_domainid_accountid | Status : SUCCESS === ok test_listSnapshot_as_domainadmin_domainid (integration.component.test_acl_listsnapshot.TestSnapshotList) ... === TestName: test_listSnapshot_as_domainadmin_domainid | Status : SUCCESS === ok test_listSnapshot_as_domainadmin_domainid_accountid (integration.component.test_acl_listsnapshot.TestSnapshotList) ... === TestName: test_listSnapshot_as_domainadmin_domainid_accountid | Status : SUCCESS === ok test_listSnapshot_as_domainadmin_domainid_accountid_listall_false (integration.component.test_acl_listsnapshot.TestSnapshotList) ... === TestName: test_listSnapshot_as_domainadmin_domainid_accountid_listall_false | Status : SUCCESS === ok test_listSnapshot_as_domainadmin_domainid_accountid_listall_false_rec_false (integration.component.test_acl_listsnapshot.TestSnapshotList) ... === TestName: test_listSnapshot_as_domainadmin_domainid_accountid_listall_false_rec_false | Status : SUCCESS === ok test_listSnapshot_as_domainadmin_domainid_accountid_listall_false_rec_true (integration.component.test_acl_listsnapshot.TestSnapshotList) ... === TestName: test_listSnapshot_as_domainadmin_domainid_accountid_listall_false_rec_true | Status : SUCCESS === ok test_listSnapshot_as_domainadmin_domainid_accountid_listall_true (integration.component.test_acl_listsnapshot.TestSnapshotList) ... === TestName: test_listSnapshot_as_domainadmin_domainid_accountid_listall_true | Status : SUCCESS === ok test_listSnapshot_as_domainadmin_domainid_accountid_listall_true_rec_false (integration.component.test_acl_listsnapshot.TestSnapshotList) ... === TestName: test_listSnapshot_as_domainadmin_domainid_accountid_listall_true_rec_false | Status : SUCCESS === ok test_listSnapshot_as_domainadmin_domainid_accountid_listall_true_rec_true (integration.component.test_acl_listsnapshot.TestSnapshotList) ... === TestName: test_listSnapshot_as_domainadmin_domainid_accountid_listall_true_rec_true | Status : SUCCESS === ok test_listSnapshot_as_domainadmin_domainid_accountid_rec_false (integration.component.test_acl_listsnapshot.TestSnapshotList) ... === TestName: test_listSnapshot_as_domainadmin_domainid_accountid_rec_false | Status : SUCCESS === ok test_listSnapshot_as_domainadmin_domainid_accountid_rec_true (integration.component.test_acl_listsnapshot.TestSnapshotList) ... === TestName: test_listSnapshot_as_domainadmin_domainid_accountid_rec_true | Status : SUCCESS === ok test_listSnapshot_as_domainadmin_domainid_listall_false (integration.component.test_acl_listsnapshot.TestSnapshotList) ... === TestName: test_listSnapshot_as_domainadmin_domainid_listall_false | Status : SUCCESS === ok test_listSnapshot_as_domainadmin_domainid_listall_false_rec_false (integration.component.test_acl_listsnapshot.TestSnapshotList) ... === TestName: test_listSnapshot_as_domainadmin_domainid_listall_false_rec_false | Status : SUCCESS === ok test_listSnapshot_as_domainadmin_domainid_listall_false_rec_true (integration.component.test_acl_listsnapshot.TestSnapshotList) ... === TestName: test_listSnapshot_as_domainadmin_domainid_listall_false_rec_true | Status : SUCCESS === ok
Re: Review Request 22712: This Test suite has test cases relating to acess checks for deleteNetwork() for Admin, domain admin and regular users
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/22712/#review52787 --- Ship it! Ship It! - Min Chen On Sept. 9, 2014, 8:39 p.m., sangeetha hariharan wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/22712/ --- (Updated Sept. 9, 2014, 8:39 p.m.) Review request for cloudstack, Min Chen, Prachi Damle, and Santhosh Edukulla. Repository: cloudstack-git Description --- This Test suite has test cases relating to acess checks for deleteNetwork() for Admin, domain admin and regular users Diffs - test/integration/component/test_acl_isolatednetwork_delete.py PRE-CREATION Diff: https://reviews.apache.org/r/22712/diff/ Testing --- Test Suite was executed against a management server built from 4.4-forward branch using a simulator set up: test_deleteNetwork_admin (integration.component.test_acl_isolatednetwork_delete.TestIsolatedNetworkDelete) ... === TestName: test_deleteNetwork_admin | Status : SUCCESS === ok test_deleteNetwork_admin_foruserinotherdomain (integration.component.test_acl_isolatednetwork_delete.TestIsolatedNetworkDelete) ... === TestName: test_deleteNetwork_admin_foruserinotherdomain | Status : SUCCESS === ok test_deleteNetwork_admin_foruserinsamedomain (integration.component.test_acl_isolatednetwork_delete.TestIsolatedNetworkDelete) ... === TestName: test_deleteNetwork_admin_foruserinsamedomain | Status : SUCCESS === ok test_deleteNetwork_domaindmin (integration.component.test_acl_isolatednetwork_delete.TestIsolatedNetworkDelete) ... === TestName: test_deleteNetwork_domaindmin | Status : SUCCESS === ok test_deleteNetwork_domaindmin_forcrossdomainuser (integration.component.test_acl_isolatednetwork_delete.TestIsolatedNetworkDelete) ... === TestName: test_deleteNetwork_domaindmin_forcrossdomainuser | Status : SUCCESS === ok test_deleteNetwork_domaindmin_foruserinsamedomain (integration.component.test_acl_isolatednetwork_delete.TestIsolatedNetworkDelete) ... === TestName: test_deleteNetwork_domaindmin_foruserinsamedomain | Status : SUCCESS === ok test_deleteNetwork_domaindmin_foruserinsubdomain (integration.component.test_acl_isolatednetwork_delete.TestIsolatedNetworkDelete) ... === TestName: test_deleteNetwork_domaindmin_foruserinsubdomain | Status : SUCCESS === ok test_deleteNetwork_user (integration.component.test_acl_isolatednetwork_delete.TestIsolatedNetworkDelete) ... === TestName: test_deleteNetwork_user | Status : SUCCESS === ok test_deleteNetwork_user_foruserinotherdomain (integration.component.test_acl_isolatednetwork_delete.TestIsolatedNetworkDelete) ... === TestName: test_deleteNetwork_user_foruserinotherdomain | Status : SUCCESS === ok test_deleteNetwork_user_foruserinsamedomain (integration.component.test_acl_isolatednetwork_delete.TestIsolatedNetworkDelete) ... === TestName: test_deleteNetwork_user_foruserinsamedomain | Status : SUCCESS === ok -- Ran 10 tests in 61.766s OK Thanks, sangeetha hariharan
Re: Commit cd8af6a
Thanks Hugo for pointing that out. That is a bug caused by copy-and-paste,
the code should be in decrRefCnt routine like other SnapshotDataStoreVO
and TemplateDataStoreVO.
I will fix this in a minute.
-min
On 9/3/14 12:30 AM, Hugo Trippaers h...@trippaers.nl wrote:
Min,
I¹m checking some findbugs reports and found the following piece of code
in VolumeDataStoreVO (line 352):
public void setRefCnt(Long refCnt) {
if (refCnt 0) {
refCnt--;
}
else {
s_logger.warn(We should not try to decrement a zero
reference count even though our code has guarded);
}
}
This doesn¹t seem to make a lot of sense to me as the refCnt field
doesn¹t get set in this function. Could you have a look as you committed
this and might know what the intended behavior is?
Cheers,
Hugo
Re: Commit cd8af6a
Fix is checked into master branch. Can I back port it to 4.4?
Thanks
-min
On 9/3/14 9:42 AM, Min Chen min.c...@citrix.com wrote:
Thanks Hugo for pointing that out. That is a bug caused by copy-and-paste,
the code should be in decrRefCnt routine like other SnapshotDataStoreVO
and TemplateDataStoreVO.
I will fix this in a minute.
-min
On 9/3/14 12:30 AM, Hugo Trippaers h...@trippaers.nl wrote:
Min,
I¹m checking some findbugs reports and found the following piece of code
in VolumeDataStoreVO (line 352):
public void setRefCnt(Long refCnt) {
if (refCnt 0) {
refCnt--;
}
else {
s_logger.warn(We should not try to decrement a zero
reference count even though our code has guarded);
}
}
This doesn¹t seem to make a lot of sense to me as the refCnt field
doesn¹t get set in this function. Could you have a look as you committed
this and might know what the intended behavior is?
Cheers,
Hugo
Re: Commit cd8af6a
This is a bug that will impact reference counting only for old volume on
NFS to S3 object store migration.
Thanks
-min
On 9/3/14 11:25 AM, Daan Hoogland daan.hoogl...@gmail.com wrote:
H Min,
what is this fix for?
On Wed, Sep 3, 2014 at 8:20 PM, Min Chen min.c...@citrix.com wrote:
Fix is checked into master branch. Can I back port it to 4.4?
Thanks
-min
On 9/3/14 9:42 AM, Min Chen min.c...@citrix.com wrote:
Thanks Hugo for pointing that out. That is a bug caused by
copy-and-paste,
the code should be in decrRefCnt routine like other SnapshotDataStoreVO
and TemplateDataStoreVO.
I will fix this in a minute.
-min
On 9/3/14 12:30 AM, Hugo Trippaers h...@trippaers.nl wrote:
Min,
I¹m checking some findbugs reports and found the following piece of
code
in VolumeDataStoreVO (line 352):
public void setRefCnt(Long refCnt) {
if (refCnt 0) {
refCnt--;
}
else {
s_logger.warn(We should not try to decrement a zero
reference count even though our code has guarded);
}
}
This doesn¹t seem to make a lot of sense to me as the refCnt field
doesn¹t get set in this function. Could you have a look as you
committed
this and might know what the intended behavior is?
Cheers,
Hugo
--
Daan
Re: Commit cd8af6a
Backported to 4.4 branch.
-min
From: Daan Hoogland daan.hoogl...@gmail.commailto:daan.hoogl...@gmail.com
Date: Wednesday, September 3, 2014 11:52 AM
To: dev dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org, Min Chen
min.c...@citrix.commailto:min.c...@citrix.com
Subject: Re: Commit cd8af6a
guess you should go ahead and backport.
On Wed, Sep 3, 2014 at 8:28 PM, Min Chen
min.c...@citrix.commailto:min.c...@citrix.com wrote:
This is a bug that will impact reference counting only for old volume on
NFS to S3 object store migration.
Thanks
-min
On 9/3/14 11:25 AM, Daan Hoogland
daan.hoogl...@gmail.commailto:daan.hoogl...@gmail.com wrote:
H Min,
what is this fix for?
On Wed, Sep 3, 2014 at 8:20 PM, Min Chen
min.c...@citrix.commailto:min.c...@citrix.com wrote:
Fix is checked into master branch. Can I back port it to 4.4?
Thanks
-min
On 9/3/14 9:42 AM, Min Chen
min.c...@citrix.commailto:min.c...@citrix.com wrote:
Thanks Hugo for pointing that out. That is a bug caused by
copy-and-paste,
the code should be in decrRefCnt routine like other SnapshotDataStoreVO
and TemplateDataStoreVO.
I will fix this in a minute.
-min
On 9/3/14 12:30 AM, Hugo Trippaers
h...@trippaers.nlmailto:h...@trippaers.nl wrote:
Min,
I¹m checking some findbugs reports and found the following piece of
code
in VolumeDataStoreVO (line 352):
public void setRefCnt(Long refCnt) {
if (refCnt 0) {
refCnt--;
}
else {
s_logger.warn(We should not try to decrement a zero
reference count even though our code has guarded);
}
}
This doesn¹t seem to make a lot of sense to me as the refCnt field
doesn¹t get set in this function. Could you have a look as you
committed
this and might know what the intended behavior is?
Cheers,
Hugo
--
Daan
--
Daan
[BLOCKED] Unable to connect to management server on current master builds
CC Rohit here in case he didn't see this email. Rohit, can you fix this? Thanks -min On 8/29/14 9:21 AM, Alex Brett alex.br...@citrix.com wrote: Hello all, On current master builds (such as http://jenkins.buildacloud.org/job/package-rhel63-master/3202/), I can't connect to the management server, either via the API or UI. The major changes since the last working build I had seems to be the SAML2 merge (there are a couple of other things, but looking at those commits there doesn't appear to be much chance of them being the cause of the problem), so the SAML2 code would be where my suspicion lies. I've filed https://issues.apache.org/jira/browse/CLOUDSTACK-7455 for this with a log etc, but if someone familiar with the code (Rohit?) would mind looking at this and seeing if they could reproduce/fix, that would be good! Thanks, Alex
Re: S3/Swift Problem around Virtual Size
No. For S3/Swift, register template will directly upload templates to S3 without going through staging NFS. It will only be copied to staging NFS when we first use that template to provision a VM. Thanks -min On 8/22/14 2:25 PM, Francois Gaudreault fgaudrea...@cloudops.com wrote: Edison, Isnt the templates downloaded to the Staging NFS first? FG On Aug 22, 2014 5:20 PM, Edison Su edison...@citrix.com wrote: I know the reason why the size of template doesn¹t have correct virtual size if it¹s registered in S3/Swift: In case of s3/swift, the template is directly stored into s3/swift through swift/s3 api, there is no place for cloudstack to look into template, to find out the virtual size during template registration. While, if secondary storage is NFS, the template is first stored on NFS(which has file system), cloudstack can unzip the template(if it¹s a zipped file), and read virtual size from the file, then report back to mgt server. In order to fix it, we can add some code as: all the templates registered on Swift/S3, need to be downloaded to a NFS intermediate storage before it can be consumed by primary storage. After the download finished, then we check virtual size of template, and report back to mgt server/update DB etc. From: Mike Tutkowski [mailto:mike.tutkow...@solidfire.com] Sent: Friday, August 22, 2014 1:38 PM To: dev@cloudstack.apache.org Cc: Edison Su Subject: S3/Swift Problem around Virtual Size Hi, This was brought up in a different e-mail thread, but I wanted to make it more clear that it's related to CloudStack's download code around S3/Swift, so I'm opening up a new thread. Francois (from CloudOps) noticed today that when he downloaded a template (VHD format) to Swift (but it looks like the same applies for S3) that the physical and virtual sizes are set to be the same. This appears to have the following consequence: You can download a template with a physical size of, say, 3 GB and a root disk that's supposed to be, say, 20 GB. Instead of the virtual size showing as 20 GB, it shows as 3 GB. This is not an issue with NFS. In that situation, the two sizes are correctly accounted for. What later can happen is the template is downloaded from Swift and takes up an unexpected amount of space on the XenServer storage repository (SR). If there is enough space on the SR, this isn't too big of a deal. However, for so-called managed storage plug-ins (examples are SolidFire and CloudByte), this will lead to them dynamically creating a SAN volume of the wrong size. Francois opened up the following ticket: https://issues.apache.org/jira/browse/CLOUDSTACK-7406 Thanks! -- Mike Tutkowski Senior CloudStack Developer, SolidFire Inc. e: mike.tutkow...@solidfire.commailto:mike.tutkow...@solidfire.com o: 303.746.7302 Advancing the way the world uses the cloud http://solidfire.com/solution/overview/?video=play
Hotfix for CLOUDSTACK-7260 is back ported to 4.3
Hi Ilya, Per your request and instruction, I have back ported hot fix for CLOUDSTACK-7260 to ACS 4.3 branch. Thanks -min
Re: [VOTE] Adapting git workflow for release branches
I would rather CI be considered together with this thread, since this thread needs to decide at what condition RM can merge a hotfix/bugfic branch to release branch. Thanks -min Sent from my iPhone On Aug 19, 2014, at 8:09 AM, Pierre-Luc Dion pd...@cloudops.com wrote: +1 , CI shouldn't be another topic? What is required or missing to have CI in place? *Pierre-Luc DION* Architecte de Solution Cloud | Cloud Solutions Architect t 855.652.5683 *CloudOps* Votre partenaire infonuagique* | *Cloud Solutions Experts 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w cloudops.com *|* tw @CloudOps_ On Tue, Aug 19, 2014 at 5:50 AM, Rohit Yadav rohit.ya...@shapeblue.com wrote: On 19-Aug-2014, at 11:29 am, Sebastien Goasguen run...@gmail.com wrote: Say you grab a patch from review board and stick it in a hotfix branch, test that …call for merge on release branch. Do we *merge* to master or can we apply the patch directly to master (git am -s…) ? Once the hotfix branch is merged on release branch, we would merge the release branch to master, that will bring the hotfix on master as well. We don’t want to encourage working on master for fixes that qualify for release branches directly so ideally we should not git am -s the patch on master. But there is scope for non-strictness for a situation needing git am -s patch on master directly, it would be at the discretion of the RM and committers. Regards, Rohit Yadav Software Architect, ShapeBlue M. +41 779015219 | rohit.ya...@shapeblue.com Blog: bhaisaab.org | Twitter: @_bhaisaab Find out more about ShapeBlue and our range of CloudStack related services IaaS Cloud Design Build http://shapeblue.com/iaas-cloud-design-and-build// CSForge – rapid IaaS deployment frameworkhttp://shapeblue.com/csforge/ CloudStack Consultinghttp://shapeblue.com/cloudstack-consultancy/ CloudStack Infrastructure Support http://shapeblue.com/cloudstack-infrastructure-support/ CloudStack Bootcamp Training Courses http://shapeblue.com/cloudstack-training/ This email and any attachments to it may be confidential and are intended solely for the use of the individual to whom it is addressed. Any views or opinions expressed are solely those of the author and do not necessarily represent those of Shape Blue Ltd or related companies. If you are not the intended recipient of this email, you must neither take any action based upon its contents, nor copy or show it to anyone. Please contact the sender if you believe you have received this email in error. Shape Blue Ltd is a company incorporated in England Wales. ShapeBlue Services India LLP is a company incorporated in India and is operated under license from Shape Blue Ltd. Shape Blue Brasil Consultoria Ltda is a company incorporated in Brasil and is operated under license from Shape Blue Ltd. ShapeBlue SA Pty Ltd is a company registered by The Republic of South Africa and is traded under license from Shape Blue Ltd. ShapeBlue is a registered trademark.
Re: [VOTE] Adapting git workflow for release branches
In that case, we should call out this procedure
If (you¹re a committer) {
Go create a hotfix branch and ask RM to pick it up
} else {
Go upload your patch and get RM to review your request from reviewboard
}
in your proposal. I don't want people to have a misunderstanding that with
this proposal, RM is not needed anymore. Actually, RM is MORE IMPORTANT
with this proposal.
Also, we should also call out the enforcement plan for this procedure.
What happens if somebody still directly commits to release branch after it
is cut? Ideally, based on this proposal, after RC is cut, we should only
see branch merge/cherry-pick done by RM. If not, RM should revert it to
enforce the flow.
Thanks
-min
On 8/19/14 9:19 AM, Rohit Yadav rohit.ya...@shapeblue.com wrote:
Hey,
On 19-Aug-2014, at 5:34 pm, Pierre-Luc Dion pd...@cloudops.com wrote:
Thanks Min for the comment, make sense.
Rohit, how do we plan to managed merge request or submit one? I don't
think using the mailing list to keep track of merge request is good,
does
https://reviews.apache.org/account/login/ is keep up to date and all
merge
request should go there ?
If (you¹re a committer) {
Go create a hotfix branch and ask RM to pick it up
} else {
Go upload your patch and get RM to review your request from reviewboard
}
What about using Jira to follow merge request ? maybe by having a
'merge-request' issue type as sub-task?
You may do that as long as RMs are okay with that. We don¹t want people
to attack RMs from too many of channels such as reviewboard, jira,
twitter, fb, linkedin and whatnot; sticking to just using email is
recommended.
Also I'm a bit confuse for some commit cases:
Let say that I want to fix the release version display in the API doc,
it
is not code related right not it show as 4.2.0, it's not a bugfix or a
new
feature, so should I create branch + merge request or this type of
commit
could be push directly in the release branch (ie: 4.4) ?
Such cases will ³depend on your chemistry with the RM, if they¹re cool
you go ahead alongwith them and fix doc/build fixes directly on release
(4.4 in the example) branch.
This is a reason as to why this proposal is flexible, and it does not
introduce any policing but gives a guideline for people to follow.
Lastly, checking out branches and working on them using git is not
expensive at all, just few keyboard strokes maybe so just don¹t be afraid.
Also, for multiple fixes feel free to do several bugfixes and ask the RM
to pick the fixes from that (hot/bug) fix branch.
HTH, cheers.
Sorry if I add confusion...
Pierre-Luc
On Tue, Aug 19, 2014 at 11:16 AM, Min Chen min.c...@citrix.com wrote:
I would rather CI be considered together with this thread, since this
thread needs to decide at what condition RM can merge a hotfix/bugfic
branch to release branch.
Thanks
-min
Sent from my iPhone
On Aug 19, 2014, at 8:09 AM, Pierre-Luc Dion pd...@cloudops.com
wrote:
+1 , CI shouldn't be another topic?
What is required or missing to have CI in place?
*Pierre-Luc DION*
Architecte de Solution Cloud | Cloud Solutions Architect
t 855.652.5683
*CloudOps* Votre partenaire infonuagique* | *Cloud Solutions Experts
420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6
w cloudops.com *|* tw @CloudOps_
On Tue, Aug 19, 2014 at 5:50 AM, Rohit Yadav
rohit.ya...@shapeblue.com
wrote:
On 19-Aug-2014, at 11:29 am, Sebastien Goasguen run...@gmail.com
wrote:
Say you grab a patch from review board and stick it in a hotfix
branch,
test that Šcall for merge on release branch.
Do we *merge* to master or can we apply the patch directly to master
(git am -sŠ) ?
Once the hotfix branch is merged on release branch, we would merge
the
release branch to master, that will bring the hotfix on master as
well.
We don¹t want to encourage working on master for fixes that qualify
for
release branches directly so ideally we should not git am -s the
patch
on
master. But there is scope for non-strictness for a situation needing
git
am -s patch on master directly, it would be at the discretion of
the
RM
and committers.
Regards,
Rohit Yadav
Software Architect, ShapeBlue
M. +41 779015219 | rohit.ya...@shapeblue.com
Blog: bhaisaab.org | Twitter: @_bhaisaab
Find out more about ShapeBlue and our range of CloudStack related
services
IaaS Cloud Design Build
http://shapeblue.com/iaas-cloud-design-and-build//
CSForge rapid IaaS deployment
frameworkhttp://shapeblue.com/csforge/
CloudStack Consultinghttp://shapeblue.com/cloudstack-consultancy/
CloudStack Infrastructure Support
http://shapeblue.com/cloudstack-infrastructure-support/
CloudStack Bootcamp Training Courses
http://shapeblue.com/cloudstack-training/
This email and any attachments to it may be confidential and are
intended
solely for the use of the individual to whom it is addressed. Any
views
or
opinions expressed are solely those of the author and do not
necessarily
represent those of Shape Blue Ltd
Re: [VOTE] Adapting git workflow for release branches
I will hesitate on this No enforcement approach in the flow. I bet that
without some kind of enforcement, based on past experience, after one
release, we will come together to discuss flaw in our flow again:) Sorry
if I am too pessimistic on this.
Thanks
-min
On 8/19/14 10:00 AM, Rohit Yadav rohit.ya...@shapeblue.com wrote:
Hi,
On 19-Aug-2014, at 6:48 pm, Min Chen min.c...@citrix.com wrote:
In that case, we should call out this procedure
If (you¹re a committer) {
Go create a hotfix branch and ask RM to pick it up
} else {
Go upload your patch and get RM to review your request from reviewboard
}
in your proposal. I don't want people to have a misunderstanding that
with
this proposal, RM is not needed anymore. Actually, RM is MORE IMPORTANT
with this proposal.
Yes.
Also, we should also call out the enforcement plan for this procedure.
Subjective. All committers have privilege to commit so enforcement will
be unnecessary, instead if you find an issue with anyone/anything you
raise it privately or on public dev ML just like we do it now.
What happens if somebody still directly commits to release branch after
it
is cut? Ideally, based on this proposal, after RC is cut, we should only
see branch merge/cherry-pick done by RM. If not, RM should revert it to
enforce the flow.
At RM’s discretion.
Cheers.
Thanks
-min
On 8/19/14 9:19 AM, Rohit Yadav rohit.ya...@shapeblue.com wrote:
Hey,
On 19-Aug-2014, at 5:34 pm, Pierre-Luc Dion pd...@cloudops.com wrote:
Thanks Min for the comment, make sense.
Rohit, how do we plan to managed merge request or submit one? I
don't
think using the mailing list to keep track of merge request is good,
does
https://reviews.apache.org/account/login/ is keep up to date and all
merge
request should go there ?
If (you¹re a committer) {
Go create a hotfix branch and ask RM to pick it up
} else {
Go upload your patch and get RM to review your request from reviewboard
}
What about using Jira to follow merge request ? maybe by having a
'merge-request' issue type as sub-task?
You may do that as long as RMs are okay with that. We don¹t want people
to attack RMs from too many of channels such as reviewboard, jira,
twitter, fb, linkedin and whatnot; sticking to just using email is
recommended.
Also I'm a bit confuse for some commit cases:
Let say that I want to fix the release version display in the API doc,
it
is not code related right not it show as 4.2.0, it's not a bugfix or a
new
feature, so should I create branch + merge request or this type of
commit
could be push directly in the release branch (ie: 4.4) ?
Such cases will ³depend on your chemistry with the RM, if they¹re cool
you go ahead alongwith them and fix doc/build fixes directly on release
(4.4 in the example) branch.
This is a reason as to why this proposal is flexible, and it does not
introduce any policing but gives a guideline for people to follow.
Lastly, checking out branches and working on them using git is not
expensive at all, just few keyboard strokes maybe so just don¹t be
afraid.
Also, for multiple fixes feel free to do several bugfixes and ask the
RM
to pick the fixes from that (hot/bug) fix branch.
HTH, cheers.
Sorry if I add confusion...
Pierre-Luc
On Tue, Aug 19, 2014 at 11:16 AM, Min Chen min.c...@citrix.com
wrote:
I would rather CI be considered together with this thread, since this
thread needs to decide at what condition RM can merge a hotfix/bugfic
branch to release branch.
Thanks
-min
Sent from my iPhone
On Aug 19, 2014, at 8:09 AM, Pierre-Luc Dion pd...@cloudops.com
wrote:
+1 , CI shouldn't be another topic?
What is required or missing to have CI in place?
*Pierre-Luc DION*
Architecte de Solution Cloud | Cloud Solutions Architect
t 855.652.5683
*CloudOps* Votre partenaire infonuagique* | *Cloud Solutions Experts
420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6
w cloudops.com *|* tw @CloudOps_
On Tue, Aug 19, 2014 at 5:50 AM, Rohit Yadav
rohit.ya...@shapeblue.com
wrote:
On 19-Aug-2014, at 11:29 am, Sebastien Goasguen run...@gmail.com
wrote:
Say you grab a patch from review board and stick it in a hotfix
branch,
test that Šcall for merge on release branch.
Do we *merge* to master or can we apply the patch directly to
master
(git am -sŠ) ?
Once the hotfix branch is merged on release branch, we would merge
the
release branch to master, that will bring the hotfix on master as
well.
We don¹t want to encourage working on master for fixes that qualify
for
release branches directly so ideally we should not git am -s the
patch
on
master. But there is scope for non-strictness for a situation
needing
git
am -s patch on master directly, it would be at the discretion of
the
RM
and committers.
Regards,
Rohit Yadav
Software Architect, ShapeBlue
M. +41 779015219 | rohit.ya...@shapeblue.com
Blog: bhaisaab.org | Twitter: @_bhaisaab
Find out more about ShapeBlue and our
Re: 4.5 RM
+1 on what David said. Shared the same feelings as him, we really need to focus on the high priority issue facing CloudStack. Gerrit/Github pull request will definitely help in this regards. Thanks -min On 8/19/14 10:15 AM, David Nalley da...@gnsa.us wrote: IMHO we should not even release 4.5 until we have a agreed upon: -what our issues are and why we released 4.4 and 4.3 late. -taken action to resolve those issues -guarantees that 4.5 will be on time If we don't do that, I don't even know why we are putting ourselves through the pain of a release schedule. So I've been trying to give this some thought. Here's my current line of thinking. The issues with late releases are not a function of our release process per se; but are instead a function of our development process. CloudStack is a relatively large codebase. It has a lots of points that interact with each other, and it's moderately complex. Development moves forward and at least happy-path testing is done for new features, but the range of options is so large that testing everything is a bit difficult. When someone makes a merge request; I suspect few people do much looking. Understandable, it's a boring task; and really looking doesn't tell us much except for style and egregious errors. We've rarely done mandatory testing of feature branches before they are merged in. If you want to ship on time, you must ensure that we are vociferously guarding the quality of the master and release branches; that we can verify programmatically that a commit or merge doesn't break things. We must insist on automated testing being added. So I've said all of that to say that I think that ship has sailed for 4.5. We are well past feature freeze; and we didn't really have any gating functionality. We frankly have very little idea of quality of whats in master right now. It's certainly worse than 4.4. So now we'll enter code freeze, we'll try and play catch up and fix all of the things we discover that are broken. And invariably, we'll be late again. If you want to solve this problem; my personal belief is that its really is tied to CI. Efforts around Travis are interesting and perhaps are a piece of that puzzle. Discussions around running CI are important as well, but I truly believe that we need a gating function that prohibits commits that increase our level of untested code or code that fails to pass testing. I've seen some other projects using pull requests in github, and then using the github pull request builder[1] for jenkins to verify that every PR works. I know we've talked about gerrit previously, and perhaps that will work as well. [1] https://wiki.cloudbees.com/bin/view/DEV/Github+Pull+Request+Validation
Re: [VOTE] Adapting git workflow for release branches
I personally don't think introducing CI or gerrit or github pull request
will take away a committer's privilege to commit. If you are a stakeholder
who really care about your product, you will not be scared away by this
extra enforcement, this will actually make our cloudstack ecosystem better
and healthier, and will also benefit other contributors in the same
community. To me, this is not a punishment per se, your code is just not
directly committed in, and you can enhance it and commit later. We are not
even requesting for bug-free code, I know, that is not possible in
software world, but at least we should have some stableness in our code to
not block other peers. As far as I know, Linux has much better control
about what patch to be pulled in, which makes it much more stabler than
us. As for your traffic example, at least there we have DMV road test,
traffic light, traffic patrol infrastructure to have basic guard.
CI/gerrit are just like those basic enforcement infrastructure for
CloudStack development process.
I don't want this to become a two-person argument forever. I already
clearly explained my ideas and takes on this. Let's hear others.
Thanks
-min
On 8/19/14 10:55 AM, Rohit Yadav rohit.ya...@shapeblue.com wrote:
tl;dr? hope you read it;
On 19-Aug-2014, at 7:10 pm, Min Chen min.c...@citrix.com wrote:
I will hesitate on this No enforcement approach in the flow. I bet
that
without some kind of enforcement, based on past experience, after one
Stakeholder will always care. The system should be optimistic to allow
things in and not to punish in advance.
That’s how many scalable opensource project such as Linux, Firefox and
our own Apache works (people over code that is).
If you’ve any ideas please share. We’ve to find a “non-strict”
enforcement to get best of all things and I don’t have a solution right
now to suggest.
“Strict enforcement” won’t be scalable anyway, hackers always find way
around gaming the system.
Unicorns, Santas and bug-less code perhaps don’t exist IMHO so may never
guarantee code quality but only improve over time. Even things like
pacemakers, realtime systems used in mining, surgery, aerospace, mars
rover etc; openssl (heartbleed) have bugs and issues. This is not to say
that we should not have code reviews, other processes and automated
build/CI/smoke-tests are “not” needed, they are needed but they “should
come only gradually. I also think improvements are better severed in
small nibbles than a one big main course.
In my past startup, last year, I helped them introduce a non-strict code
reviewing process and what it did was not to scare anyone but slowly and
gradually everyone started using code reviews and that became a process
now. What I learnt was that people like slow changes (don’t put the frog
in hot water), processes should be effortless and invisible (you don’t
have to think much on how to do it, like say driving a car or touch
typing on keyboard), and a cultural change is best implemented without
enforcement. For example, in a many countries people don’t honk i.e. the
culture even if no one gets a ticket for honking which is my point we
need an understanding, a guideline, a protocol to implement a culture not
an “enforcement” or team of police.
release, we will come together to discuss flaw in our flow again:) Sorry
if I am too pessimistic on this.
To introduce an enforcement would mean take away a committer’s privilege
to commit. If you have some ideas start a thread.
IMO “pessimistic approach is not a good approach — you don't ban or
restrict people driving cars just because of accidents or ban the
Internet just because there are things you don’t like on it, or put
people in jail just because they can commit a crime in future. I would
want an open and transparent workflow/protocol/guideline that is
agreeable to all/most of us and that does not impose any restrictions.
Cheers.
Thanks
-min
On 8/19/14 10:00 AM, Rohit Yadav rohit.ya...@shapeblue.com wrote:
Hi,
On 19-Aug-2014, at 6:48 pm, Min Chen min.c...@citrix.com wrote:
In that case, we should call out this procedure
If (you¹re a committer) {
Go create a hotfix branch and ask RM to pick it up
} else {
Go upload your patch and get RM to review your request from
reviewboard
}
in your proposal. I don't want people to have a misunderstanding that
with
this proposal, RM is not needed anymore. Actually, RM is MORE
IMPORTANT
with this proposal.
Yes.
Also, we should also call out the enforcement plan for this procedure.
Subjective. All committers have privilege to commit so enforcement will
be unnecessary, instead if you find an issue with anyone/anything you
raise it privately or on public dev ML just like we do it now.
What happens if somebody still directly commits to release branch
after
it
is cut? Ideally, based on this proposal, after RC is cut, we should
only
see branch merge/cherry-pick done by RM. If not, RM should revert it
to
enforce the flow.
At RM’s
Re: [VOTE] Adapting git workflow for release branches
Rohit, I think that Edison and I have clearly indicated our objection reason in our previous email. Based on current cloudstack quality, RM needs to have control over what commits to be in release branch to get a release at least having some quality. With this proposed model, how can you guarantee the quality of a release? We cannot just talk about changing a process without resolving this important concern. What is your solution to this concern? Thanks -min On 8/18/14 10:59 AM, Rohit Yadav rohit.ya...@shapeblue.com wrote: Hi, @Jessica ‹ Can you please suggest what¹s wrong with the ³things² that were proposed here as I could not figure out your or Min¹s or Edison's individual opinion and reason behind the vote. We have three -1s (1 binding) but none of them share valid reasons or concerns that would point out issues and challenges with adopting the proposed items so we¹ll continue with the voting. Min, Jessica, Edison ‹ I would love to know what¹s wrong in the proposed things so we don¹t make mistake. @Rajani ‹ Thanks, but when we should cut a release branch is a different topic and IMO is per the RM¹s discretion so if you¹ve any ideas or proposals please go ahead and start a thread on that. Cheers. On 18-Aug-2014, at 6:52 pm, Jessica Wang jessica.w...@citrix.com wrote: I agree with Edison. I am -1 on this as well. -Original Message- From: Edison Su [mailto:edison...@citrix.com] Sent: Saturday, August 16, 2014 12:11 PM To: dev Subject: RE: [VOTE] Adapting git workflow for release branches I agree with what Min said on thread: http://markmail.org/message/dqdlqu7phgijfelc, and not satisfied with your answer: Currently we don't have a CI running continuously, no code review, it's too risky to let developer check in whatever commit they want into release branch. RM needs to have to control over what commit should be put into release branch and what should not, otherwise, we could get into a stage where no control on the quality. How RM will do the control, that's something we could discuss. I know, current model is not scale, as RM needs to manually cherry pick commits into release branch. The way I thinking about, is all the commits put into release branch, must be put into review board, or gerrit, must be passed by CI and reviewers, then the commits can be put into release branch. For above reason, I am -1(binding) on your proposal for now until we solve the quality control problem. -Original Message- From: Rohit Yadav [mailto:rohit.ya...@shapeblue.com] Sent: Friday, August 15, 2014 3:25 AM To: dev Subject: [VOTE] Adapting git workflow for release branches Hello everyone, With reference to my proposal on changing our release-master commit flow [1], I would like to start a voting thread to decide on the adoption starting 4.5 release. Any opinion, ideas, modifications is welcome to help reach a consensus and improve our present situation. Today's Friday so it will be only fair to extend the voting window to more than our usual 72 hours window. Therefore, we'll end this voting on Wednesday, 20 August 2014 at 18:00H UTC giving about 5 days of time for people to share what works and what does not. We'll stop anytime we've three -1s (binding). Short summary: - Base line protocol: Continuous changes from release/stable branches to master/unable branches - Get contributors more engaged with release branches by working (fixing bugs, docs etc.) on release branches first (and not on master) - Fixes on release branches are recommended (non strict enforcement) go via a hotfix/bugfix branch that get automatically tested by Jenkins, when they are green RMs get the changes to release branch Long Summary of what we'll adopt: (I'm skipping writing them on wiki, as this may change/modify in this thread) - Continuous flow of changes from stable branches to un-stables ones i.e. from release branches to master and from master to features etc. Use of merge -fast-forward is encourages over cherry-picking and -no-ff (no ff will create merge commit). This happens couple of times a day to ensure we get solid/robust changes from release branches (such as bugfixes etc.) on master, any conflicts will be resolved. If we do it continuously we'll also save ourselves from a big conflict at the end of the release cycle and we'll also avoid missing/misplacing any commit when cherry-picking. - After code freeze is declared and release branch is cut out, contributors work on fixing bugs and other changes (such as documentation, build/packaging fixes etc.) first on the release branch (and not master). This is not to restrict anyone working on master, features and other changes can keep landing on master as well. This is to encourage contributors to give more attention to release branches by at least fixing bugs on release branches first and not our current way where we fix it on master and ask RMs to cherry pick it to release
Re: [VOTE] Adapting git workflow for release branches
Rohit, I read your proposal, but maybe I mistook your idea: - After code freeze is declared and release branch is cut out, contributors work on fixing bugs and other changes (such as documentation, build/packaging fixes etc.) first on the release branch (and not master). This is not to restrict anyone working on master, features and other changes can keep landing on master as well. This is to encourage contributors to give more attention to release branches by at least fixing bugs on release branches first and not our current way where we fix it on master and ask RMs to cherry pick it to release branch. What do you mean by this? So after release branch is cut, and contributors need to fix a bug (as we experienced so many times in past releases), what should they do? Based on your sentence above, I understood that they can just directly commit to release branch instead of currently checking into some forward branch and RM then cherry-pick it up to release branch to control quality. What is your new proposed approach then? Please clarify. Thanks -min On 8/18/14 12:06 PM, Rohit Yadav rohit.ya...@shapeblue.com wrote: Min, On 18-Aug-2014, at 8:25 pm, Min Chen min.c...@citrix.com wrote: Rohit, I think that Edison and I have clearly indicated our objection reason in our previous email. Based on current cloudstack quality, RM needs to have control over what commits to be in release branch to get a release at least having some quality. With this proposed model, how can you guarantee the quality of a release? We cannot just talk about changing a process without resolving this important concern. What is your solution to this concern? In my proposal we’re not saying people “can commit directly to release branches, I suggest you re-read the proposal. I cannot emphasis this enough that this does not try to solve the issue you’re raising (which deserves a thread of its own), so the expectation from everyone is to stick to the agenda and comment on it. Min, I’ve said this at least four times now I feel like people are just skimming emails :P If they are, may I deserve their attention to read my email with full attention like I do when I read theirs? We’re not giving power to everyone commit directly on release branches, so we’re not changing the status quo around this issue so there is no point of questioning “release quality”. This sort of workflow is something used at several companies such as Google and Facebook which has turned out to work for them. If you find any issues or challenges with this I would love to hear from you. At the end of the day as an individual wearing your Apache hat it’s your call and right to votes and opinions so we respect your votes but it would be only encouraging if they are backed by a good reason. Lastly, I don’t have the “unicorn solution that will guarantee quality of a release and I think perhaps it does not exist. This proposal does not aim to solve the “release quality issue” but to: - encourage involvement of contributors during release: My personal opinion is that we’ve a major problem that unless a commercial distribution’s releases is based on ACS release, many of the “sponsored” developers won’t participate much in opensource ACS releasess. How do we solve it? I guess we need some way to increase participation, by increasing participation we’ll have much better release quality than perhaps that will less involvement. - a guideline to reduce conflicts and make sure no commit is missed or misplaced - give a flow of change (baseline protocol) on how to maintain multiple release branches Min and others, I would welcome if you’ve any issues or challenges you can find with “what” the above will try to implement. Cheers. Thanks -min On 8/18/14 10:59 AM, Rohit Yadav rohit.ya...@shapeblue.com wrote: Hi, @Jessica ‹ Can you please suggest what¹s wrong with the ³things² that were proposed here as I could not figure out your or Min¹s or Edison's individual opinion and reason behind the vote. We have three -1s (1 binding) but none of them share valid reasons or concerns that would point out issues and challenges with adopting the proposed items so we¹ll continue with the voting. Min, Jessica, Edison ‹ I would love to know what¹s wrong in the proposed things so we don¹t make mistake. @Rajani ‹ Thanks, but when we should cut a release branch is a different topic and IMO is per the RM¹s discretion so if you¹ve any ideas or proposals please go ahead and start a thread on that. Cheers. On 18-Aug-2014, at 6:52 pm, Jessica Wang jessica.w...@citrix.com wrote: I agree with Edison. I am -1 on this as well. -Original Message- From: Edison Su [mailto:edison...@citrix.com] Sent: Saturday, August 16, 2014 12:11 PM To: dev Subject: RE: [VOTE] Adapting git workflow for release branches I agree with what Min said on thread: http://markmail.org/message/dqdlqu7phgijfelc, and not satisfied with your answer
Re: [VOTE] Adapting git workflow for release branches
See my comments inline. Thanks -min On 8/18/14 3:22 PM, Rohit Yadav rohit.ya...@shapeblue.com wrote: Hey, in-line; On 18-Aug-2014, at 11:09 pm, Min Chen min.c...@citrix.com wrote: Rohit, I read your proposal, but maybe I mistook your idea: - After code freeze is declared and release branch is cut out, contributors work on fixing bugs and other changes (such as documentation, build/packaging fixes etc.) first on the release branch (and not master). This is not to restrict anyone working on master, features and other changes can keep landing on master as well. This is to encourage contributors to give more attention to release branches by at least fixing bugs on release branches first and not our current way where we fix it on master and ask RMs to cherry pick it to release branch”. So, the next point to the above in the proposal tell you how we should do it: “”” - Changes to release branches can be done by pushing a bugfix/change branch and asking the RM to pick it up if they are tested. Our automated systems can perform checks on such branches too (starting with a suffix that can automatically trigger such builds/tests) and if everything is fine, RMs to land the changes to release branches. “”” What do you mean by this? So after release branch is cut, and contributors need to fix a bug (as we experienced so many times in past releases), what should they do? Based on your sentence above, I understood that they can just directly commit to release branch instead of currently checking into some forward branch and RM then cherry-pick it up to release branch to control quality. What is your new proposed approach then? Please clarify. Sorry if that confused you, this is what a contributor should do: (everything in parenthesis below is just a guideline or recommendation but not a rule) 1. They should create a branch check’d out from release branch to fix an issue for the release branch and push with a bugfix or hotfix prefix (possibly with a JIRA bug ID in the branch name) and then ask RM to pick it up. In case the contributor is not a committer they can contribute their work to be applied on release branch via reviewboard. 2. Once their hotfix/bugfix branch is merged (-ff preferably to avoid having a merge commit) by the RM or the contributor is asked to rework their fix and resubmit 3. Once contributor’s work lands/merges on release branch, this branch is merged by either any committer or the RM to master (perhaps several times a day using —fast-forward to avoid merge commits). This way changes land to master as well. [Min] Thanks for clarification on this. In this case, this is required after release branch is cut. For each bug fix, contributor needs to create a personal branch and fixes there, then ask RM to manually merge branch, right? So we still need a RM to do such manual things. If so, what is the advantage of this new model compared to currently contributors committing into release-forward branch and then ask RM to cherry-pick? I am not against your #3 above. For supporting multiple release branches, for example fixing a bug on multiple branches such as on 4.2, 4.3, 4.4 branches, the contributor starts with 4.2 and follows the above and go on until 4.4 and finally their fix lands on master. For the long run, if we use the above if a bugfix is accepted and applied on 4.2, we could have merged 4.2 fast-forward to 4.3 branch, and then 4.3 on 4.4 and 4.4 on master. This operation will be valid and won’t cause any conflicts etc because if we take an example -- the 4.3 branch contains the whole history of 4.2 (think them as link lists) so merge -ff will result in landing the original bugfix to the next branch. This is the flow of change we could benefit from and it’s a guideline and not a rule and one will be still free to cherry-pick or fixing something manually per release branch. [Min] I don't quite understand why you are saying that it won't cause conflicts when we do 4.2 fast-forward to 4.3 branch. What if the codebase in 4.3 has been changed a lot? With our (already) releases we don’t backport or fix bugs, I hope such a workflow can be helpful for doing a LTS or long term maintenance release. The proposal also consists of scope of non-strictness and changeability, quoting: “”” - Nothing is written in stones, this should be change-able. And, this can only work if we all agree to follow this with 4.5 “” Cheers. Thanks -min On 8/18/14 12:06 PM, Rohit Yadav rohit.ya...@shapeblue.com wrote: Min, On 18-Aug-2014, at 8:25 pm, Min Chen min.c...@citrix.com wrote: Rohit, I think that Edison and I have clearly indicated our objection reason in our previous email. Based on current cloudstack quality, RM needs to have control over what commits to be in release branch to get a release at least having some quality. With this proposed model, how can you guarantee the quality of a release? We cannot just talk about changing a process without resolving
Re: [VOTE] Adapting git workflow for release branches
I am -1 on this as well. Any git process change needs to be considered together with the quality problem we are facing now, which i believe that it is the root cause for RM manual cherry pick issue. We cannot just adopt a process by ignoring its root quality problem. Sorry that your answer here still cannot satisfy me. Thanks -min Sent from my iPhone On Aug 16, 2014, at 2:32 PM, Rohit Yadav rohit.ya...@shapeblue.com wrote: Hi Edison, Thank you for your email, I suggest you read my reply to Min’s email as well: http://markmail.org/message/vytkbqhdjmhewonl Let me begin by saying that perhaps I was unable to capture everyone’s imagination with my proposal, I’ll work on my writing skills and try to keep them short and objective. I think I’ve an answer to satisfy your worries but if at the end of reading this email you’re not satisfied, instead of going back and forth on this thread we could talk on phone, Skype, GTM etc. (whatever is convenient for you, my contact details can be found in my signature). Now, this voting thread (and also in the proposal thread) has nothing to do with enforcing quality, but has: 1. Change of flow: Get people more involved with release branches: get their stuff to release branches first via a hotfix/bugfix branch and then merge -ff (or cherry-pick) to master, feature branches etc. 2. Baseline protocol: Give a guideline for contributors on how to fix something that spans multiple releases and branches, for example starting with oldest version/release (firm) to latest and then to master (soft) etc. Since, I’m not trying to solve any “quality control” issue, I cannot take the responsibility of trying to fix anything around it as well. Therefore, it will be only fair if you could re-read what we’re voting on and return back with your unbiased, individual and objective opinion. On 16-Aug-2014, at 9:10 pm, Edison Su edison...@citrix.com wrote: How RM will do the control, that's something we could discuss. I know, current model is not scale, as RM needs to manually cherry pick commits into release branch. The way I thinking about, is all the commits put into release branch, must be put into review board, or gerrit, must be passed by CI and reviewers, then the commits can be put into release branch. Good ideas, I think you should start a proposal thread and help with actionable items. At present, I’m not trying to address the aforementioned challenges because it will be tricky and I don’t have a solution. This will be important for us but will be challenging, IMHO will be time taking and call for: - getting everyone’s agreement - a change of culture - requirement of infrastructure - expecting everyone learning to use the new system and workflow - workflow enforcement and policing I think we do have (some) CI solutions, I may be wrong but I recall Hugo made some build job to trigger on all branches starting with “hotfix” or some prefix, so we do have such things. This week, Ian and Sebastien found a cheap way of having a CI for testing with simulator on TravisCI which is free (as in cost). For above reason, I am -1(binding) on your proposal for now until we solve the quality control problem. Thank you for your vote, it’s important that we don’t make a mistake. But since the reason mentioned had nothing to do with the voting proposal I would welcome your reconsideration. Cheers, Rohit Yadav Software Architect, ShapeBlue M. +41 779015219 | rohit.ya...@shapeblue.com Blog: bhaisaab.org | Twitter: @_bhaisaab Find out more about ShapeBlue and our range of CloudStack related services IaaS Cloud Design Buildhttp://shapeblue.com/iaas-cloud-design-and-build// CSForge – rapid IaaS deployment frameworkhttp://shapeblue.com/csforge/ CloudStack Consultinghttp://shapeblue.com/cloudstack-consultancy/ CloudStack Infrastructure Supporthttp://shapeblue.com/cloudstack-infrastructure-support/ CloudStack Bootcamp Training Courseshttp://shapeblue.com/cloudstack-training/ This email and any attachments to it may be confidential and are intended solely for the use of the individual to whom it is addressed. Any views or opinions expressed are solely those of the author and do not necessarily represent those of Shape Blue Ltd or related companies. If you are not the intended recipient of this email, you must neither take any action based upon its contents, nor copy or show it to anyone. Please contact the sender if you believe you have received this email in error. Shape Blue Ltd is a company incorporated in England Wales. ShapeBlue Services India LLP is a company incorporated in India and is operated under license from Shape Blue Ltd. Shape Blue Brasil Consultoria Ltda is a company incorporated in Brasil and is operated under license from Shape Blue Ltd. ShapeBlue SA Pty Ltd is a company registered by The Republic of South Africa and is traded under
Re: Cancel VM migration when cluster is facing issues
There should be a global configuration for this: enable.ha.storage.migration. You can turn that off. Thanks -min From: Amin Samir a...@opencloud.net.aumailto:a...@opencloud.net.au Reply-To: dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org Date: Thursday, August 14, 2014 7:39 PM To: dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org Subject: Cancel VM migration when cluster is facing issues Hello, Has anyone know how to cancel the process of migrating VM from cluster to another, this happens automatically when a cluster does not have the adequate capacity or storage has issues, it is also attempts migrating the storage of VM to the destination cluster primary storage. Thank in advance for your help Kind Regards Kind Regards Amin Samir Open Cloud Pty. Sales Support Mobile: +61414356558 E-mail: a...@opencloud.net.aumailto:a...@opencloud.net.au [image002]http://www.linkedin.com/company/backonline?trk=company_name [LikeUs on facebook] http://www.facebook.com/plugins/likebox.php?href=https%3A%2F%2Fwww.facebook.com%2Fpages%2FOpenCloud-Pty-Ltd%2F281961941864174widthheight=290colorscheme=lightshow_faces=trueheader=truestream=falseshow_border=true [OpenCloud Signature]
Re: [SHOW] Authentication refactoring
Hi Rohit, Any reason why you didn't implement response class for login and logout like any other API cmd? I think that will be useful as mentioned in your FS. Thanks -min On 8/12/14 2:10 AM, Rohit Yadav rohit.ya...@shapeblue.com wrote: This was done: https://cwiki.apache.org/confluence/display/CLOUDSTACK/Authentication+Refa ctoring This is the branch: https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;a=shortlog;h=refs /heads/auth-refactor Updates: - Every auth mechanism now implements as a APICommand but these are special APIs are not allowed to execute, i.e. the execute() method returns with an error - Existing tests were fixed - We no longer need to hardcode login/logout for doc generation etc. - Api discovery now has login/logout docs etc as well - Since these APIs are tightly coupled with cloud-server artifact, except for responses all the interface definitions etc are within cloud-server - This allows for implementation of other login mechanisms such as saml, oauth, something-custom etc. though implementing it as a plugin is still tricky now I¹ve tested UI and cloudmonkey on port 8080 and 8096, with apikey/secret keys but would welcome help around this area from anyone. I¹ll merge the branch later this week if no one objects. Cheers. On 12-Aug-2014, at 5:50 am, Rohit Yadav rohit.ya...@shapeblue.com wrote: Hi, The way we handle login and logout is hardcoded and since there is no APICommand/BaseCmd implementation the apidoc, apidiscovery and other don¹t discover these apis. For supporting SAML as an authentication mechanism, I¹ve refactored the Auth mechanism as a pluggable service that loads with api-server artifact and both login and logout are now implemented as a pseduo BaseCmd classes. I call them pseudo because their execute() is never called, the authentication guards in ApiServlet class make sure we call an authenticate method of such classes. Since, they are tightly coupled with cloud-server¹s ApiServlet it only made sense to have the interface definition and implementation within the same package/artifact as well. This also solves the apidoc issue for login/logout and saml related auth apis. I¹ll merge them after sometime and continue working on saml stuff. Will push the code in the branch ³auth-refactor² in an hour for review/testing now. This does not break anything and should not cause any auth related issues for all existing clients. Any suggestions, feedback welcome! Refactoring was pretty straight forward but I¹ll make sure to write a wiki page on this before merging to master. Regards, Rohit Yadav Software Architect, ShapeBlue M. +41 779015219 | rohit.ya...@shapeblue.com Blog: bhaisaab.org | Twitter: @_bhaisaab Find out more about ShapeBlue and our range of CloudStack related services IaaS Cloud Design Buildhttp://shapeblue.com/iaas-cloud-design-and-build// CSForge rapid IaaS deployment frameworkhttp://shapeblue.com/csforge/ CloudStack Consultinghttp://shapeblue.com/cloudstack-consultancy/ CloudStack Infrastructure Supporthttp://shapeblue.com/cloudstack-infrastructure-support/ CloudStack Bootcamp Training Courseshttp://shapeblue.com/cloudstack-training/ This email and any attachments to it may be confidential and are intended solely for the use of the individual to whom it is addressed. Any views or opinions expressed are solely those of the author and do not necessarily represent those of Shape Blue Ltd or related companies. If you are not the intended recipient of this email, you must neither take any action based upon its contents, nor copy or show it to anyone. Please contact the sender if you believe you have received this email in error. Shape Blue Ltd is a company incorporated in England Wales. ShapeBlue Services India LLP is a company incorporated in India and is operated under license from Shape Blue Ltd. Shape Blue Brasil Consultoria Ltda is a company incorporated in Brasil and is operated under license from Shape Blue Ltd. ShapeBlue SA Pty Ltd is a company registered by The Republic of South Africa and is traded under license from Shape Blue Ltd. ShapeBlue is a registered trademark. Regards, Rohit Yadav Software Architect, ShapeBlue M. +41 779015219 | rohit.ya...@shapeblue.com Blog: bhaisaab.org | Twitter: @_bhaisaab Find out more about ShapeBlue and our range of CloudStack related services IaaS Cloud Design Buildhttp://shapeblue.com/iaas-cloud-design-and-build// CSForge rapid IaaS deployment frameworkhttp://shapeblue.com/csforge/ CloudStack Consultinghttp://shapeblue.com/cloudstack-consultancy/ CloudStack Infrastructure Supporthttp://shapeblue.com/cloudstack-infrastructure-support/ CloudStack Bootcamp Training Courseshttp://shapeblue.com/cloudstack-training/ This email and any attachments to it may be confidential and are intended solely for the use of the individual to whom it is addressed. Any views or opinions expressed are solely those of the author and do not necessarily represent those
Re: [SHOW] Authentication refactoring
Sorry, I might just get part of your commit yesterday when I looked. Now I
saw them. Thanks for your clarification.
-min
On 8/14/14 10:01 AM, Rohit Yadav rohit.ya...@shapeblue.com wrote:
On 14-Aug-2014, at 6:54 pm, Min Chen min.c...@citrix.com wrote:
Hi Rohit,
Any reason why you didn't implement response class for login and logout
like any other API cmd? I think that will be useful as mentioned in your
FS.
In cloud-api, checkout
org.apache.cloudstack.api.response.{LoginCmdResponse, LogoutCmdResponse}.
These are special response classes used by only authentication apis and
they extend org.apache.cloudstack.api.response.AuthenticationCmdResponse.
The serialized output of all the
org.apache.cloudstack.api.response.AuthenticationCmdResponse classes (and
children) are not boxed to have object name, it follows: {
“classresponse”: {response object json here} }.
Cheers.
Thanks
-min
On 8/12/14 2:10 AM, Rohit Yadav rohit.ya...@shapeblue.com wrote:
This was done:
https://cwiki.apache.org/confluence/display/CLOUDSTACK/Authentication+Re
fa
ctoring
This is the branch:
https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;a=shortlog;h=re
fs
/heads/auth-refactor
Updates:
- Every auth mechanism now implements as a APICommand but these are
special APIs are not allowed to execute, i.e. the execute() method
returns with an error
- Existing tests were fixed
- We no longer need to hardcode login/logout for doc generation etc.
- Api discovery now has login/logout docs etc as well
- Since these APIs are tightly coupled with cloud-server artifact,
except
for responses all the interface definitions etc are within cloud-server
- This allows for implementation of other login mechanisms such as
saml,
oauth, something-custom etc. though implementing it as a plugin is
still
tricky now
I¹ve tested UI and cloudmonkey on port 8080 and 8096, with
apikey/secret
keys but would welcome help around this area from anyone. I¹ll merge
the
branch later this week if no one objects.
Cheers.
On 12-Aug-2014, at 5:50 am, Rohit Yadav rohit.ya...@shapeblue.com
wrote:
Hi,
The way we handle login and logout is hardcoded and since there is no
APICommand/BaseCmd implementation the apidoc, apidiscovery and other
don¹t discover these apis. For supporting SAML as an authentication
mechanism, I¹ve refactored the Auth mechanism as a pluggable service
that loads with api-server artifact and both login and logout are now
implemented as a pseduo BaseCmd classes.
I call them pseudo because their execute() is never called, the
authentication guards in ApiServlet class make sure we call an
authenticate method of such classes. Since, they are tightly coupled
with cloud-server¹s ApiServlet it only made sense to have the
interface
definition and implementation within the same package/artifact as
well.
This also solves the apidoc issue for login/logout and saml related
auth
apis.
I¹ll merge them after sometime and continue working on saml stuff.
Will
push the code in the branch ³auth-refactor² in an hour for
review/testing now. This does not break anything and should not cause
any auth related issues for all existing clients.
Any suggestions, feedback welcome! Refactoring was pretty straight
forward but I¹ll make sure to write a wiki page on this before merging
to master.
Regards,
Rohit Yadav
Software Architect, ShapeBlue
M. +41 779015219 | rohit.ya...@shapeblue.com
Blog: bhaisaab.org | Twitter: @_bhaisaab
Find out more about ShapeBlue and our range of CloudStack related
services
IaaS Cloud Design
Buildhttp://shapeblue.com/iaas-cloud-design-and-build//
CSForge rapid IaaS deployment
frameworkhttp://shapeblue.com/csforge/
CloudStack Consultinghttp://shapeblue.com/cloudstack-consultancy/
CloudStack Infrastructure
Supporthttp://shapeblue.com/cloudstack-infrastructure-support/
CloudStack Bootcamp Training
Courseshttp://shapeblue.com/cloudstack-training/
This email and any attachments to it may be confidential and are
intended solely for the use of the individual to whom it is addressed.
Any views or opinions expressed are solely those of the author and do
not necessarily represent those of Shape Blue Ltd or related
companies.
If you are not the intended recipient of this email, you must neither
take any action based upon its contents, nor copy or show it to
anyone.
Please contact the sender if you believe you have received this email
in
error. Shape Blue Ltd is a company incorporated in England Wales.
ShapeBlue Services India LLP is a company incorporated in India and is
operated under license from Shape Blue Ltd. Shape Blue Brasil
Consultoria Ltda is a company incorporated in Brasil and is operated
under license from Shape Blue Ltd. ShapeBlue SA Pty Ltd is a company
registered by The Republic of South Africa and is traded under license
from Shape Blue Ltd. ShapeBlue is a registered trademark.
Regards,
Rohit Yadav
Software Architect, ShapeBlue
M. +41
Re: [DISCUSS] Removing template URL format checking logic
A better fix to address this issue has been committed to master: https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;a=commit;h=e356465 8befaa72cbe5fd510bea2a25b40f108f5. This should address CLOUDSTACK-6940, CLOUDSTACK-7312, CLOUDSTACK-5512 altogether, same logic to handle registerTemplate, registerIso and uploadVolume. Thanks -min On 8/11/14 5:41 PM, Nitin Mehta nitin.me...@citrix.com wrote: Yes, its important to fail fast and suggest the user that he/she didn't register the correct format URI. Also we shouldn’t have boiler plate code for checking the url format for register iso/template/volume. It should be one util method. But would you know all the valid formats a URL can be - or will you just check for say 'vhd' in the URL ? Thanks, -Nitin On 11/08/14 5:26 PM, Min Chen min.c...@citrix.com wrote: Yes. But on second thought, I think that purely removing template url format checking is too brute force and aggressive, and will cause hard time in debugging issues thrown by later orchestration flow that relies on template/volume in specific file format. A better fix to address the issue raised in CLOUDSTACK-5512 is to fix the code to parse download or upload URI properly to get correct path. I will check in such a fix soon. Thanks -min On 8/11/14 5:16 PM, Nitin Mehta nitin.me...@citrix.com wrote: Hi Min, We need to do this for Upload Volume as well - right ? Thanks, -Nitin On 11/08/14 3:01 PM, Min Chen min.c...@citrix.com wrote: Just checked in a fix to remove similar format check for ISO for https://issues.apache.org/jira/browse/CLOUDSTACK-7312. Thanks -min On 8/11/14 9:59 AM, Rohit Yadav rohit.ya...@shapeblue.com wrote: Thanks for fixing that Min! I personally did not check the current state of code other than discussing it with Marcus on the issue. Cheers. On 11-Aug-2014, at 6:54 pm, Min Chen min.c...@citrix.com wrote: Yes, I have filed and fixed that bug a month ago. Thanks -min On 8/10/14 10:39 PM, Harikrishna Patnala harikrishna.patn...@citrix.com wrote: Hi Rohit, This was already fixed for templates https://issues.apache.org/jira/browse/CLOUDSTACK-6940 We have to do the same for registering ISOs as well. Thanks, Harikrishna On 08-Aug-2014, at 11:20 pm, Rohit Yadav rohit.ya...@shapeblue.com wrote: Hi, With reference to https://issues.apache.org/jira/browse/CLOUDSTACK-5512 Marcus and I think we should remove the template URL format checking logic because: - It does not handle pre-signed URL (say something that does not end with .vhd etc, but has bunch of http params) - One can game the system by say renaming any file to respective format - We dumb down, take whatever URL user gives and use the format they specify in their register template API call Marcus also notes that TemplateUtils utility would validate selected format. Please discuss if you¹ve any use-case that can get affected by this? Regards, Rohit Yadav Software Architect, ShapeBlue M. +41 779015219 | rohit.ya...@shapeblue.com Blog: bhaisaab.org | Twitter: @_bhaisaab Find out more about ShapeBlue and our range of CloudStack related services IaaS Cloud Design Buildhttp://shapeblue.com/iaas-cloud-design-and-build// CSForge rapid IaaS deployment frameworkhttp://shapeblue.com/csforge/ CloudStack Consultinghttp://shapeblue.com/cloudstack-consultancy/ CloudStack Infrastructure Supporthttp://shapeblue.com/cloudstack-infrastructure-support/ CloudStack Bootcamp Training Courseshttp://shapeblue.com/cloudstack-training/ This email and any attachments to it may be confidential and are intended solely for the use of the individual to whom it is addressed. Any views or opinions expressed are solely those of the author and do not necessarily represent those of Shape Blue Ltd or related companies. If you are not the intended recipient of this email, you must neither take any action based upon its contents, nor copy or show it to anyone. Please contact the sender if you believe you have received this email in error. Shape Blue Ltd is a company incorporated in England Wales. ShapeBlue Services India LLP is a company incorporated in India and is operated under license from Shape Blue Ltd. Shape Blue Brasil Consultoria Ltda is a company incorporated in Brasil and is operated under license from Shape Blue Ltd. ShapeBlue SA Pty Ltd is a company registered by The Republic of South Africa and is traded under license from Shape Blue Ltd. ShapeBlue is a registered trademark. Regards, Rohit Yadav Software Architect, ShapeBlue M. +41 779015219 | rohit.ya...@shapeblue.com Blog: bhaisaab.org | Twitter: @_bhaisaab Find out more about ShapeBlue and our range of CloudStack related services IaaS Cloud Design Buildhttp://shapeblue.com/iaas-cloud-design-and-build// CSForge rapid IaaS deployment frameworkhttp://shapeblue.com/csforge/ CloudStack Consultinghttp://shapeblue.com/cloudstack-consultancy/ CloudStack Infrastructure Supporthttp://shapeblue.com/cloudstack
Re: [SHOW] Authentication refactoring
Hi Rohit, My understanding is that you will do this on your feature branch auth-refactor, then merge them after passing at least some CI automation tests. Today, I saw all these commits already in master: 10 hours agoRohit Yadav DefaultLoginAPIAuthenticatorCmd: return userId as UUID commit | commitdiff | tree | snapshot 10 hours agoRohit Yadav utils: fix pom.xml to have references for javax.servlet...commit | commitdiff | tree | snapshot 10 hours agoRohit Yadav ApiServer: take UTF_8 and other static vars from HttpUtils commit | commitdiff | tree | snapshot 10 hours agoRohit Yadav ApiServlet: use HttpUtils instead of class specific... commit | commitdiff | tree | snapshot 10 hours agoRohit Yadav ApiResponseSerializer: Use HttpUtils instead of BaseCmd commit | commitdiff | tree | snapshot 10 hours agoRohit Yadav BaseCmd: Use HttpUtils to have single source of static... commit | commitdiff | tree | snapshot 10 hours agoRohit Yadav utils: refactor HTTP transport stuff to HttpUtils commit | commitdiff | tree | snapshot 10 hours agoRohit Yadav ApiServletTest: Fix test, now login/logout have their...commit | commitdiff | tree | snapshot 10 hours agoRohit Yadav APIAuthenticator: refactor signature of APIAuthenticato... commit | commitdiff | tree | snapshot 10 hours agoRohit Yadav ApiServlet: move setting of response type up in the... commit | commitdiff | tree | snapshot 10 hours agoRohit Yadav ApiXmlDocWriter: get rid of hardcoded login/logout... commit | commitdiff | tree | snapshot 10 hours agoRohit Yadav ApiServlet: use the new and refactored authentication... commit | commitdiff | tree | snapshot 10 hours agoRohit Yadav ApiXmlDocWriter: remove hardcoded login and logout... commit | commitdiff | tree | snapshot 10 hours agoRohit Yadav ApiResponseSerializer: Skip extra boxing for Auth responses commit | commitdiff | tree | snapshot 10 hours agoRohit Yadav response: add command response for login and logout... commit | commitdiff | tree | snapshot 10 hours agoRohit Yadav APIAuthenticationManagerImpl: add the auth manager... commit | commitdiff | tree | snapshot 10 hours agoRohit Yadav DefaultLoginAPIAuthenticatorCmd: Refactor and implement...commit | commitdiff | tree | snapshot 10 hours agoRohit Yadav DefaultLogoutAPIAuthenticatorCmd: Refactor and implemen... commit | commitdiff | tree | snapshot 10 hours agoRohit Yadav APIAuthenticationManager: Add Auth manager definition commit | commitdiff | tree | snapshot 10 hours agoRohit Yadav APIAuthenticationType: Add auth enum type, login or... commit | commitdiff | tree | snapshot 10 hours agoRohit Yadav APIAuthenticator: Add interface definition for the... commit | commitdiff | tree | snapshot 10 hours agoRohit Yadav saml2: add opensaml as dependency commit | commitdiff | tree | snapshot 10 hours agoRohit Yadav commands.properties: add login,logout,samlsso,samlslo... commit | commitdiff | tree | snapshot 10 hours agoRohit Yadav ApiErrorCode: Add API error code 401, 405 commit | commitdiff | tree | snapshot 10 hours agoRohit Yadav ApiConstants: add Api constant registered commit | commitdiff | tree | snapshot 10 hours agoRohit Yadav saml2: add spring security saml2 extension 1.0.0.RELEASE commit | commitdiff | tree | snapshot 10 hours agoRohit Yadav client: add saml2 plugin dependency on client artifactcommit | commitdiff | tree | snapshot 10 hours agoRohit Yadav CLOUDSTACK-7083: Add SAML2 SSO plugin skeleton and... commit | commitdiff | tree | snapshot Are these commits related to the refactor you are talking about here? Why are they not going through some merge request? Thanks -min On 8/12/14 2:10 AM, Rohit Yadav rohit.ya...@shapeblue.com wrote: This was done: https://cwiki.apache.org/confluence/display/CLOUDSTACK/Authentication+Refa ctoring This is the branch: https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;a=shortlog;h=refs /heads/auth-refactor Updates: - Every auth mechanism now implements as a APICommand but these are special APIs are not allowed to execute, i.e. the execute() method returns with an error - Existing tests were fixed - We no longer need to hardcode login/logout for doc generation etc. - Api discovery now has login/logout docs etc as well - Since these APIs are tightly coupled with cloud-server artifact, except for responses all the interface definitions etc are within cloud-server - This allows for implementation of other login mechanisms such as saml, oauth, something-custom etc. though implementing it as a plugin is still tricky now I¹ve tested UI and cloudmonkey on port 8080 and 8096, with apikey/secret keys but would
Re: [DISCUSS] Removing template URL format checking logic
Completely remove the checkFormat() will cause trouble for debugging issues later if the template format is wrong. I would prefer fail fast approach instead of spending hours later to deal with wrong file format issues. Thanks -min On 8/12/14 2:37 PM, Rohit Yadav rohit.ya...@shapeblue.com wrote: Hey, On 12-Aug-2014, at 8:53 pm, Min Chen min.c...@citrix.com wrote: A better fix to address this issue has been committed to master: https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;a=commit;h=e3564 65 8befaa72cbe5fd510bea2a25b40f108f5. This should address CLOUDSTACK-6940, CLOUDSTACK-7312, CLOUDSTACK-5512 altogether, same logic to handle registerTemplate, registerIso and uploadVolume. Thanks for the fix Min, I saw that. I was hoping that we completely remove the checkFormat() method (do we need it, let’s get rid of it?). Lastly, Daan can comment if we want that in 4.4? It looks more like a bugfix though :) Cheers. Thanks -min On 8/11/14 5:41 PM, Nitin Mehta nitin.me...@citrix.com wrote: Yes, its important to fail fast and suggest the user that he/she didn't register the correct format URI. Also we shouldn’t have boiler plate code for checking the url format for register iso/template/volume. It should be one util method. But would you know all the valid formats a URL can be - or will you just check for say 'vhd' in the URL ? Thanks, -Nitin On 11/08/14 5:26 PM, Min Chen min.c...@citrix.com wrote: Yes. But on second thought, I think that purely removing template url format checking is too brute force and aggressive, and will cause hard time in debugging issues thrown by later orchestration flow that relies on template/volume in specific file format. A better fix to address the issue raised in CLOUDSTACK-5512 is to fix the code to parse download or upload URI properly to get correct path. I will check in such a fix soon. Thanks -min On 8/11/14 5:16 PM, Nitin Mehta nitin.me...@citrix.com wrote: Hi Min, We need to do this for Upload Volume as well - right ? Thanks, -Nitin On 11/08/14 3:01 PM, Min Chen min.c...@citrix.com wrote: Just checked in a fix to remove similar format check for ISO for https://issues.apache.org/jira/browse/CLOUDSTACK-7312. Thanks -min On 8/11/14 9:59 AM, Rohit Yadav rohit.ya...@shapeblue.com wrote: Thanks for fixing that Min! I personally did not check the current state of code other than discussing it with Marcus on the issue. Cheers. On 11-Aug-2014, at 6:54 pm, Min Chen min.c...@citrix.com wrote: Yes, I have filed and fixed that bug a month ago. Thanks -min On 8/10/14 10:39 PM, Harikrishna Patnala harikrishna.patn...@citrix.com wrote: Hi Rohit, This was already fixed for templates https://issues.apache.org/jira/browse/CLOUDSTACK-6940 We have to do the same for registering ISOs as well. Thanks, Harikrishna On 08-Aug-2014, at 11:20 pm, Rohit Yadav rohit.ya...@shapeblue.com wrote: Hi, With reference to https://issues.apache.org/jira/browse/CLOUDSTACK-5512 Marcus and I think we should remove the template URL format checking logic because: - It does not handle pre-signed URL (say something that does not end with .vhd etc, but has bunch of http params) - One can game the system by say renaming any file to respective format - We dumb down, take whatever URL user gives and use the format they specify in their register template API call Marcus also notes that TemplateUtils utility would validate selected format. Please discuss if you¹ve any use-case that can get affected by this? Regards, Rohit Yadav Software Architect, ShapeBlue M. +41 779015219 | rohit.ya...@shapeblue.com Blog: bhaisaab.org | Twitter: @_bhaisaab Find out more about ShapeBlue and our range of CloudStack related services IaaS Cloud Design Buildhttp://shapeblue.com/iaas-cloud-design-and-build// CSForge rapid IaaS deployment frameworkhttp://shapeblue.com/csforge/ CloudStack Consultinghttp://shapeblue.com/cloudstack-consultancy/ CloudStack Infrastructure Supporthttp://shapeblue.com/cloudstack-infrastructure-support/ CloudStack Bootcamp Training Courseshttp://shapeblue.com/cloudstack-training/ This email and any attachments to it may be confidential and are intended solely for the use of the individual to whom it is addressed. Any views or opinions expressed are solely those of the author and do not necessarily represent those of Shape Blue Ltd or related companies. If you are not the intended recipient of this email, you must neither take any action based upon its contents, nor copy or show it to anyone. Please contact the sender if you believe you have received this email in error. Shape Blue Ltd is a company incorporated in England Wales. ShapeBlue Services India LLP is a company incorporated in India and is operated under license from Shape Blue Ltd. Shape Blue Brasil Consultoria Ltda is a company incorporated in Brasil and is operated under license from Shape
Re: [DISCUSS] Removing template URL format checking logic
Yes, I have filed and fixed that bug a month ago. Thanks -min On 8/10/14 10:39 PM, Harikrishna Patnala harikrishna.patn...@citrix.com wrote: Hi Rohit, This was already fixed for templates https://issues.apache.org/jira/browse/CLOUDSTACK-6940 We have to do the same for registering ISOs as well. Thanks, Harikrishna On 08-Aug-2014, at 11:20 pm, Rohit Yadav rohit.ya...@shapeblue.com wrote: Hi, With reference to https://issues.apache.org/jira/browse/CLOUDSTACK-5512 Marcus and I think we should remove the template URL format checking logic because: - It does not handle pre-signed URL (say something that does not end with .vhd etc, but has bunch of http params) - One can game the system by say renaming any file to respective format - We dumb down, take whatever URL user gives and use the format they specify in their register template API call Marcus also notes that TemplateUtils utility would validate selected format. Please discuss if you¹ve any use-case that can get affected by this? Regards, Rohit Yadav Software Architect, ShapeBlue M. +41 779015219 | rohit.ya...@shapeblue.com Blog: bhaisaab.org | Twitter: @_bhaisaab Find out more about ShapeBlue and our range of CloudStack related services IaaS Cloud Design Buildhttp://shapeblue.com/iaas-cloud-design-and-build// CSForge rapid IaaS deployment frameworkhttp://shapeblue.com/csforge/ CloudStack Consultinghttp://shapeblue.com/cloudstack-consultancy/ CloudStack Infrastructure Supporthttp://shapeblue.com/cloudstack-infrastructure-support/ CloudStack Bootcamp Training Courseshttp://shapeblue.com/cloudstack-training/ This email and any attachments to it may be confidential and are intended solely for the use of the individual to whom it is addressed. Any views or opinions expressed are solely those of the author and do not necessarily represent those of Shape Blue Ltd or related companies. If you are not the intended recipient of this email, you must neither take any action based upon its contents, nor copy or show it to anyone. Please contact the sender if you believe you have received this email in error. Shape Blue Ltd is a company incorporated in England Wales. ShapeBlue Services India LLP is a company incorporated in India and is operated under license from Shape Blue Ltd. Shape Blue Brasil Consultoria Ltda is a company incorporated in Brasil and is operated under license from Shape Blue Ltd. ShapeBlue SA Pty Ltd is a company registered by The Republic of South Africa and is traded under license from Shape Blue Ltd. ShapeBlue is a registered trademark.
Re: [DISCUSS] Removing template URL format checking logic
Just checked in a fix to remove similar format check for ISO for https://issues.apache.org/jira/browse/CLOUDSTACK-7312. Thanks -min On 8/11/14 9:59 AM, Rohit Yadav rohit.ya...@shapeblue.com wrote: Thanks for fixing that Min! I personally did not check the current state of code other than discussing it with Marcus on the issue. Cheers. On 11-Aug-2014, at 6:54 pm, Min Chen min.c...@citrix.com wrote: Yes, I have filed and fixed that bug a month ago. Thanks -min On 8/10/14 10:39 PM, Harikrishna Patnala harikrishna.patn...@citrix.com wrote: Hi Rohit, This was already fixed for templates https://issues.apache.org/jira/browse/CLOUDSTACK-6940 We have to do the same for registering ISOs as well. Thanks, Harikrishna On 08-Aug-2014, at 11:20 pm, Rohit Yadav rohit.ya...@shapeblue.com wrote: Hi, With reference to https://issues.apache.org/jira/browse/CLOUDSTACK-5512 Marcus and I think we should remove the template URL format checking logic because: - It does not handle pre-signed URL (say something that does not end with .vhd etc, but has bunch of http params) - One can game the system by say renaming any file to respective format - We dumb down, take whatever URL user gives and use the format they specify in their register template API call Marcus also notes that TemplateUtils utility would validate selected format. Please discuss if you¹ve any use-case that can get affected by this? Regards, Rohit Yadav Software Architect, ShapeBlue M. +41 779015219 | rohit.ya...@shapeblue.com Blog: bhaisaab.org | Twitter: @_bhaisaab Find out more about ShapeBlue and our range of CloudStack related services IaaS Cloud Design Buildhttp://shapeblue.com/iaas-cloud-design-and-build// CSForge rapid IaaS deployment frameworkhttp://shapeblue.com/csforge/ CloudStack Consultinghttp://shapeblue.com/cloudstack-consultancy/ CloudStack Infrastructure Supporthttp://shapeblue.com/cloudstack-infrastructure-support/ CloudStack Bootcamp Training Courseshttp://shapeblue.com/cloudstack-training/ This email and any attachments to it may be confidential and are intended solely for the use of the individual to whom it is addressed. Any views or opinions expressed are solely those of the author and do not necessarily represent those of Shape Blue Ltd or related companies. If you are not the intended recipient of this email, you must neither take any action based upon its contents, nor copy or show it to anyone. Please contact the sender if you believe you have received this email in error. Shape Blue Ltd is a company incorporated in England Wales. ShapeBlue Services India LLP is a company incorporated in India and is operated under license from Shape Blue Ltd. Shape Blue Brasil Consultoria Ltda is a company incorporated in Brasil and is operated under license from Shape Blue Ltd. ShapeBlue SA Pty Ltd is a company registered by The Republic of South Africa and is traded under license from Shape Blue Ltd. ShapeBlue is a registered trademark. Regards, Rohit Yadav Software Architect, ShapeBlue M. +41 779015219 | rohit.ya...@shapeblue.com Blog: bhaisaab.org | Twitter: @_bhaisaab Find out more about ShapeBlue and our range of CloudStack related services IaaS Cloud Design Buildhttp://shapeblue.com/iaas-cloud-design-and-build// CSForge rapid IaaS deployment frameworkhttp://shapeblue.com/csforge/ CloudStack Consultinghttp://shapeblue.com/cloudstack-consultancy/ CloudStack Infrastructure Supporthttp://shapeblue.com/cloudstack-infrastructure-support/ CloudStack Bootcamp Training Courseshttp://shapeblue.com/cloudstack-training/ This email and any attachments to it may be confidential and are intended solely for the use of the individual to whom it is addressed. Any views or opinions expressed are solely those of the author and do not necessarily represent those of Shape Blue Ltd or related companies. If you are not the intended recipient of this email, you must neither take any action based upon its contents, nor copy or show it to anyone. Please contact the sender if you believe you have received this email in error. Shape Blue Ltd is a company incorporated in England Wales. ShapeBlue Services India LLP is a company incorporated in India and is operated under license from Shape Blue Ltd. Shape Blue Brasil Consultoria Ltda is a company incorporated in Brasil and is operated under license from Shape Blue Ltd. ShapeBlue SA Pty Ltd is a company registered by The Republic of South Africa and is traded under license from Shape Blue Ltd. ShapeBlue is a registered trademark.
Re: [DISCUSS] Removing template URL format checking logic
Yes. But on second thought, I think that purely removing template url format checking is too brute force and aggressive, and will cause hard time in debugging issues thrown by later orchestration flow that relies on template/volume in specific file format. A better fix to address the issue raised in CLOUDSTACK-5512 is to fix the code to parse download or upload URI properly to get correct path. I will check in such a fix soon. Thanks -min On 8/11/14 5:16 PM, Nitin Mehta nitin.me...@citrix.com wrote: Hi Min, We need to do this for Upload Volume as well - right ? Thanks, -Nitin On 11/08/14 3:01 PM, Min Chen min.c...@citrix.com wrote: Just checked in a fix to remove similar format check for ISO for https://issues.apache.org/jira/browse/CLOUDSTACK-7312. Thanks -min On 8/11/14 9:59 AM, Rohit Yadav rohit.ya...@shapeblue.com wrote: Thanks for fixing that Min! I personally did not check the current state of code other than discussing it with Marcus on the issue. Cheers. On 11-Aug-2014, at 6:54 pm, Min Chen min.c...@citrix.com wrote: Yes, I have filed and fixed that bug a month ago. Thanks -min On 8/10/14 10:39 PM, Harikrishna Patnala harikrishna.patn...@citrix.com wrote: Hi Rohit, This was already fixed for templates https://issues.apache.org/jira/browse/CLOUDSTACK-6940 We have to do the same for registering ISOs as well. Thanks, Harikrishna On 08-Aug-2014, at 11:20 pm, Rohit Yadav rohit.ya...@shapeblue.com wrote: Hi, With reference to https://issues.apache.org/jira/browse/CLOUDSTACK-5512 Marcus and I think we should remove the template URL format checking logic because: - It does not handle pre-signed URL (say something that does not end with .vhd etc, but has bunch of http params) - One can game the system by say renaming any file to respective format - We dumb down, take whatever URL user gives and use the format they specify in their register template API call Marcus also notes that TemplateUtils utility would validate selected format. Please discuss if you¹ve any use-case that can get affected by this? Regards, Rohit Yadav Software Architect, ShapeBlue M. +41 779015219 | rohit.ya...@shapeblue.com Blog: bhaisaab.org | Twitter: @_bhaisaab Find out more about ShapeBlue and our range of CloudStack related services IaaS Cloud Design Buildhttp://shapeblue.com/iaas-cloud-design-and-build// CSForge rapid IaaS deployment frameworkhttp://shapeblue.com/csforge/ CloudStack Consultinghttp://shapeblue.com/cloudstack-consultancy/ CloudStack Infrastructure Supporthttp://shapeblue.com/cloudstack-infrastructure-support/ CloudStack Bootcamp Training Courseshttp://shapeblue.com/cloudstack-training/ This email and any attachments to it may be confidential and are intended solely for the use of the individual to whom it is addressed. Any views or opinions expressed are solely those of the author and do not necessarily represent those of Shape Blue Ltd or related companies. If you are not the intended recipient of this email, you must neither take any action based upon its contents, nor copy or show it to anyone. Please contact the sender if you believe you have received this email in error. Shape Blue Ltd is a company incorporated in England Wales. ShapeBlue Services India LLP is a company incorporated in India and is operated under license from Shape Blue Ltd. Shape Blue Brasil Consultoria Ltda is a company incorporated in Brasil and is operated under license from Shape Blue Ltd. ShapeBlue SA Pty Ltd is a company registered by The Republic of South Africa and is traded under license from Shape Blue Ltd. ShapeBlue is a registered trademark. Regards, Rohit Yadav Software Architect, ShapeBlue M. +41 779015219 | rohit.ya...@shapeblue.com Blog: bhaisaab.org | Twitter: @_bhaisaab Find out more about ShapeBlue and our range of CloudStack related services IaaS Cloud Design Buildhttp://shapeblue.com/iaas-cloud-design-and-build// CSForge rapid IaaS deployment frameworkhttp://shapeblue.com/csforge/ CloudStack Consultinghttp://shapeblue.com/cloudstack-consultancy/ CloudStack Infrastructure Supporthttp://shapeblue.com/cloudstack-infrastructure-support/ CloudStack Bootcamp Training Courseshttp://shapeblue.com/cloudstack-training/ This email and any attachments to it may be confidential and are intended solely for the use of the individual to whom it is addressed. Any views or opinions expressed are solely those of the author and do not necessarily represent those of Shape Blue Ltd or related companies. If you are not the intended recipient of this email, you must neither take any action based upon its contents, nor copy or show it to anyone. Please contact the sender if you believe you have received this email in error. Shape Blue Ltd is a company incorporated in England Wales. ShapeBlue Services India LLP is a company incorporated in India and is operated under license from Shape Blue Ltd. Shape Blue Brasil Consultoria Ltda is a company
Re: [PROPOSAL] Solving the cherry-picking problem
Hi Rohit, IMHO, the root cause for RM cherry-pick problem is code quality. Without solving that first based on some kind of enforcement, this will not help much. The reason to use forward branch and RM cherry-picking is to control what can go to release branch. Your proposal removed that part, then I am not sure how you can guarantee a quality release. Thanks -min On 8/7/14 6:06 AM, Rohit Yadav rohit.ya...@shapeblue.com wrote: Hi Hari, You¹ve a valid concern, but on master when we¹re pushing bugfixes for multiple issues the RM eventually picks them to release branch anyway. At the end of the day, usage of automated tests, build/unit tests will ensure some quality control. This proposal will solve issues for RM (the cherry-pick and losing commits ones) and it does not do much about code quality or control. To get it working: - During codefreeze, a contributor should not slip in half baked features and use bugfix as an excuse to finish/fix the feature - On the release branch you work first, fix/commit only bug/fixes and release specific enhancements (such as docs, config files, scripts, packaging etc.) - Master branch is free and you can do wild development, merge your feature branch that did not went in release etc. but you merge ‹fast-forward the release branch on it often (couple of times a day is recommended), fix conflicts and carry on Cheers. On 07-Aug-2014, at 2:51 pm, Harikrishna Patnala harikrishna.patn...@citrix.com wrote: Hi Rohit, Thanks for the proposal. I¹ve some concerns. If we work directly on release branch only (with out forward branch) I¹m not sure how we control regressions in the release time. In case of forward branch cut from the release branch RMs will merge only critical bug fixes to release branch, where do the non-critical bug fixes go into ? according to your 2nd statement minor/major bug fixes remain in forward branch only. Thanks, Harikrishna On 07-Aug-2014, at 2:09 pm, Rohit Yadav rohit.ya...@shapeblue.com wrote: Hi, I think the following can solve the cherry-picking problem but it needs everyone¹s support to work: - Once a release branch is cut out, all the committers and contributors ³should² only work on the release branch. It can be discussed if we want to work on it directly or branch out on it and work in that branch and have RMs to merge that branch on the release branch. IMO if we work directly on the release branch we potentially reduce a lot of RM¹s work. - Only (new) feature development and related enhancements/bugfixes can land on master directly or merged from their respective branches. - The RMs or anyone would keep merging the release branch with fast forward only on regular basis: git checkout master git merge --ff release-branch fix any conflicts and git commit -as etc. This way Œmaster' gets all the good stuff from release branch and the release branch gets ³more attention². If we somehow can reduce the release cycle timeline/length, the divergence between master and release branches can be potentially less causing less conflicts/issues when following the above. Thoughts, flames? Regards. Regards, Rohit Yadav Software Architect, ShapeBlue M. +41 779015219 | rohit.ya...@shapeblue.com Blog: bhaisaab.org | Twitter: @_bhaisaab Find out more about ShapeBlue and our range of CloudStack related services IaaS Cloud Design Buildhttp://shapeblue.com/iaas-cloud-design-and-build// CSForge rapid IaaS deployment frameworkhttp://shapeblue.com/csforge/ CloudStack Consultinghttp://shapeblue.com/cloudstack-consultancy/ CloudStack Infrastructure Supporthttp://shapeblue.com/cloudstack-infrastructure-support/ CloudStack Bootcamp Training Courseshttp://shapeblue.com/cloudstack-training/ This email and any attachments to it may be confidential and are intended solely for the use of the individual to whom it is addressed. Any views or opinions expressed are solely those of the author and do not necessarily represent those of Shape Blue Ltd or related companies. If you are not the intended recipient of this email, you must neither take any action based upon its contents, nor copy or show it to anyone. Please contact the sender if you believe you have received this email in error. Shape Blue Ltd is a company incorporated in England Wales. ShapeBlue Services India LLP is a company incorporated in India and is operated under license from Shape Blue Ltd. Shape Blue Brasil Consultoria Ltda is a company incorporated in Brasil and is operated under license from Shape Blue Ltd. ShapeBlue SA Pty Ltd is a company registered by The Republic of South Africa and is traded under license from Shape Blue Ltd. ShapeBlue is a registered trademark. Regards, Rohit Yadav Software Architect, ShapeBlue M. +41 779015219 | rohit.ya...@shapeblue.com Blog: bhaisaab.org | Twitter: @_bhaisaab Find out more about ShapeBlue and our range of CloudStack related services IaaS Cloud Design
Re: [VOTE] git workflow
Agree with Animesh. Didn't see any gains from this, we just shift pain from one branch to another, so vote -1. -min On 8/2/14 9:50 PM, Animesh Chaturvedi animesh.chaturv...@citrix.com wrote: +0 While this protects master with only commits which are merges from release branch and keeps it clean but the issues that we have shift to develop branch. -Original Message- From: Rajani Karuturi [mailto:rajani.karut...@citrix.com] Sent: Thursday, July 31, 2014 3:28 AM To: dev Subject: [VOTE] git workflow Hi All, We had long discussions on the git flow. I tried to capture the summary of it @ http://markmail.org/message/j5z7dxjcqxfkfhpj This is updated on wiki @ https://cwiki.apache.org/confluence/display/CLOUDSTACK/Git#Git- ProposedGitflowbasedCheck-inProcess and is up for a vote: Can you share your opinion on the proposal? [ ] +1 approve [ ] +0 no opinion [ ] -1 disapprove (and reason why) Thanks, ~Rajani
Re: [VOTE] git workflow
Agree with Prachi and Alena here. As a developer, I am more towards having developer branch as a staging area to periodically merge into master when CI pass on it, this is the simplest way to make master branch stable, if that is the original intention of this proposal. Also, before we figure out all the details of the proposal, we should not start implementing and adopting the process. -min On 8/5/14 3:55 PM, Prachi Damle prachi.da...@citrix.com wrote: I fail to understand how will this model help us with the maintenance releases? For CloudStack we also keep working on prior releases and ship out maintenance releases. I suppose we will be cutting the maintenance releases from the release branches? So we will have to keep around the release branches for that purpose. In that case isn't master branch a redundant copy of the release branches? I think what we really need is having a staging branch where CI runs and pushes code to master only if CI passes in turn keeping master stable. I think 'develop' branch should serve such use. Thanks, Prachi -Original Message- From: Sebastien Goasguen [mailto:run...@gmail.com] Sent: Tuesday, August 05, 2014 2:56 PM To: dev@cloudstack.apache.org Subject: Re: [VOTE] git workflow On Aug 5, 2014, at 2:33 PM, Jessica Wang jessica.w...@citrix.com wrote: Exactly. This just shifts pain from one branch to another. I don't see any gains from this, either. I vote -1. It is much more than shifting pains, the wiki page is not discussion the workflow quite extensively, with several pointers that we should all take the time to view/read: https://cwiki.apache.org/confluence/display/CLOUDSTACK/Git Jessica -Original Message- From: Min Chen [mailto:min.c...@citrix.com] Sent: Tuesday, August 05, 2014 11:27 AM To: dev@cloudstack.apache.org Subject: Re: [VOTE] git workflow Agree with Animesh. Didn't see any gains from this, we just shift pain from one branch to another, so vote -1. -min On 8/2/14 9:50 PM, Animesh Chaturvedi animesh.chaturv...@citrix.com wrote: +0 While this protects master with only commits which are merges from release branch and keeps it clean but the issues that we have shift to develop branch. -Original Message- From: Rajani Karuturi [mailto:rajani.karut...@citrix.com] Sent: Thursday, July 31, 2014 3:28 AM To: dev Subject: [VOTE] git workflow Hi All, We had long discussions on the git flow. I tried to capture the summary of it @ http://markmail.org/message/j5z7dxjcqxfkfhpj This is updated on wiki @ https://cwiki.apache.org/confluence/display/CLOUDSTACK/Git#Git- ProposedGitflowbasedCheck-inProcess and is up for a vote: Can you share your opinion on the proposal? [ ] +1 approve [ ] +0 no opinion [ ] -1 disapprove (and reason why) Thanks, ~Rajani
Re: 4.4-forward picked empty
Daan, That commit is already there in 4.4 branch. See https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;a=commit;h=6ba541a fb73475758a62a17dae1ec1be088da810. Thanks -min On 7/30/14 5:33 AM, Daan Hoogland daan.hoogl...@gmail.com wrote: thanks Rajani On Wed, Jul 30, 2014 at 2:20 PM, Rajani Karuturi rajani.karut...@citrix.com wrote: Hi Daan, commit 2ab7bcade2f37cb17e071a6154fdae1d80e4ff99 Author: Rajani Karuturi rajanikarut...@gmail.com Date: Fri May 23 15:50:06 2014 +0530 Fixed CLOUDSTACK-6756: usage id is not being returned for an ip in deleted ip range This is same as df42ce903d399cf30055e55bc24b84fbc0b563a9 on 4.4 commit d511847cfedad5478d1b4087c8f97be2c5bf3cc8 Author: Rajani Karuturi rajanikarut...@gmail.com Date: Tue Jun 3 16:11:01 2014 +0530 Fixed Resource leak (RESOURCE_LEAK) 11. overwrite_var: Overwriting pstmt in pstmt = conn.prepareStatement(INSERT INTO `cloud`.`ldap_configuration`(hostname, po Signed-off-by: Koushik Das kous...@apache.org Can be ignored. Fixes are already there. ~ Rajani -- Daan
Re: download template - delete vhd
In current ACS master, Template is not deleted from secondary storage when extractTemplate is called, just its symlink is deleted. Thanks -min On 7/23/14 4:15 AM, Tomasz Zięba t.a.zi...@gmail.com wrote: Hello, Could someone confirm that download template deletes the vhd file from secondary storage. We are testing on the ACS version 4.2.1 but the code responsible for removing is the same in version 4.4 https://github.com/apache/cloudstack/blob/8b6dc7ce2f0058b9cf29bd9c72e4e0db 9162fe6e/services/secondary-storage/server/src/org/apache/cloudstack/stora ge/template/UploadManagerImpl.java funkcja: handleDeleteEntityDownloadURLCommand -- Regards, Tomasz Zięba Twitter: @TZieba LinkedIn: pl.linkedin.com/pub/tomasz-zięba-ph-d/3b/7a8/ab6/ http://pl.linkedin.com/pub/tomasz-zi%C4%99ba-ph-d/3b/7a8/ab6/
Re: Replace primary secondary storage
I am not sure if we have any document for ACS 4.3. But you may be able to reference the old document, and just remember that we have replaced the following old 3 tables: template_host_ref - template_store_ref volume_host_ref- volume_store_ref snapshot_host_ref - snapshot_store_ref And image_store table stores the secondary storage information. Hope that this can help. Thanks -min On 7/21/14 10:34 PM, Tejas Gadaria refond.g...@gmail.com wrote: Hi Min, Thanks for clarification, template_store_ref provided me install_path secondary storage id information. it was quite helpful. I found this blog http://stankirdey.com/content/cloudstack-merging-secondary-storages but he is using `template_host_ref`for ACS 4.2. Is there any documented procedure to replace secondary storage for ACS 4.3 ? Regards, Tejas On Mon, Jul 21, 2014 at 10:18 PM, Min Chen min.c...@citrix.com wrote: That article only applied to ACS 4.1 and older versions. In ACS 4.2 storage refactor, db tables are changed. template_host_ref has been deprecated and replaced with template_store_ref. Thanks -min On 7/21/14 3:47 AM, Tejas Gadaria refond.g...@gmail.com wrote: Hi, I found this article http://support.citrix.com/article/CTX135229 I was just going through this but could not get some of the points. Currently I have no snapshots and all templates are public another secondary storage is not added yet. 1) In above article 2nd point says Copy the snapshots and templates from Secondary Storage host S2 to S1. 6th point in article says You must copy only private templates on Secondary storage host S2 to S1. Here I got confused a little. 2) currently both tables are showing empty as shown below. Am I doing anything wrong or it is normal? mysql select sechost_id from snapshots; Empty set (0.00 sec) mysql select host_id from template_host_ref; Empty set (0.00 sec) Regards, Tejas On Mon, Jul 21, 2014 at 1:02 PM, Tejas Gadaria refond.g...@gmail.com wrote: Hi, I have production vms running on ACS 4.3 with xen 6.2 SP1. I have requirement to change primary Secondary storage. I am planning live storage migration for all running vms, after adding new storage as primary storage storage in same cluster. ( correct me if i am missing anything)..but how can i replace secondary storage? Regards, Tejas
Re: Replace primary secondary storage
That article only applied to ACS 4.1 and older versions. In ACS 4.2 storage refactor, db tables are changed. template_host_ref has been deprecated and replaced with template_store_ref. Thanks -min On 7/21/14 3:47 AM, Tejas Gadaria refond.g...@gmail.com wrote: Hi, I found this article http://support.citrix.com/article/CTX135229 I was just going through this but could not get some of the points. Currently I have no snapshots and all templates are public another secondary storage is not added yet. 1) In above article 2nd point says Copy the snapshots and templates from Secondary Storage host S2 to S1. 6th point in article says You must copy only private templates on Secondary storage host S2 to S1. Here I got confused a little. 2) currently both tables are showing empty as shown below. Am I doing anything wrong or it is normal? mysql select sechost_id from snapshots; Empty set (0.00 sec) mysql select host_id from template_host_ref; Empty set (0.00 sec) Regards, Tejas On Mon, Jul 21, 2014 at 1:02 PM, Tejas Gadaria refond.g...@gmail.com wrote: Hi, I have production vms running on ACS 4.3 with xen 6.2 SP1. I have requirement to change primary Secondary storage. I am planning live storage migration for all running vms, after adding new storage as primary storage storage in same cluster. ( correct me if i am missing anything)..but how can i replace secondary storage? Regards, Tejas
Re: [DISCUSS] [PROPOSAL] SAML2 plugin for SSO/SLO in CloudStack
+1. Very well-written FS and email, Rohit. Those open questions are very valid, I added a little comment in your FS regarding the flow. Thanks -min On 7/20/14 8:35 AM, Rohit Yadav rohit.ya...@shapeblue.com wrote: Hi, I'm assuming no one objects the proposal and the spec, I'll move forward with the first implementation starting next week but will be mostly offline till 28th July. Regards. Rohit Yadav wrote: Hi guys, There has been a lot of interest [4] around auth related problems in CloudStach such as -- SSO/SLO (single sign on / log out), 2-factor authentication, role based network/IP/CIDR checking etc. A lot of challenge in implementing them in CloudStack is because of two divergent authentication mechanisms (one that is username/password/cookie based, other which is api/secret keys or hmac/signature based). This thread tries to kickstart a project in that direction which will in short term try to implement a SAML2 plugin and in long term have a much better authentication framework. Let me start by briefly explaining what SAML2 [1] is -- it's an XML based authentication and authorization protocol widely used to implement single sign on service. Having a SAML plugin in ACS will give users and organization a new mode of authentication who already have such an infrastructure in place. A SAML based SSO infrastructure consists of three entities - user-agent (UA), service provider (SP) and identity provider (IdP). The UA is the user/browser, the SP is the application that the UA is accessing (i.e. Apache CloudStack UI) and the IdP is the identity service and does authentication and authorization, management of users among other things. IdP could be backed by LDAP, AD etc. For the scope of this feature, we only need to implement SAML SP plugin in CloudStack and use any free SAML 2.0 compliant IdP server [5] for testing. For this I researched and explored ways of implementing that and have a first draft which needs to be discussed and iterated in the ACS dev community. After comparing many opensource SAML 2.0 implementations, their security and stability, we'll use OpenSAML [2] which is the most stable and widely used Java implementation. Since within CloudStack, we've been using Spring (for DI etc.) I explored and found Spring security SAML extension [3] which fits perfectly and it too uses OpenSAML. I also have a working proof-of-concept general implementation using the above based on which I've put together a design document draft on this feature for your review: https://cwiki.apache.org/confluence/display/CLOUDSTACK/SAML+2.0+Plugin There are some complex stories/cases around security and user management in CloudStack, some of which are listed under 'open ended questions' in the draft above most of which I'm not sure how to address. After first round of discussion, I'll go ahead with a basic implementation of this feature. The second phase will address broader use cases. Comments, questions, suggestions? References: [1] http://en.wikipedia.org/wiki/SAML_2.0 [2] https://wiki.shibboleth.net/confluence/display/OpenSAML/Home [3] http://projects.spring.io/spring-security-saml [4] John Burwell's talk on SSO in CloudStack: https://www.youtube.com/watch?v=kCR0TzrfCOM [5] https://idp.ssocircle.com/sso/UI/Login Regards, Rohit Yadav Software Architect, ShapeBlue M. +91 88 262 30892 | rohit.ya...@shapeblue.com Blog: bhaisaab.org | Twitter: @_bhaisaab Find out more about ShapeBlue and our range of CloudStack related services IaaS Cloud Design Buildhttp://shapeblue.com/iaas-cloud-design-and-build// CSForge rapid IaaS deployment frameworkhttp://shapeblue.com/csforge/ CloudStack Consultinghttp://shapeblue.com/cloudstack-consultancy/ CloudStack Infrastructure Supporthttp://shapeblue.com/cloudstack-infrastructure-support/ CloudStack Bootcamp Training Courseshttp://shapeblue.com/cloudstack-training/ This email and any attachments to it may be confidential and are intended solely for the use of the individual to whom it is addressed. Any views or opinions expressed are solely those of the author and do not necessarily represent those of Shape Blue Ltd or related companies. If you are not the intended recipient of this email, you must neither take any action based upon its contents, nor copy or show it to anyone. Please contact the sender if you believe you have received this email in error. Shape Blue Ltd is a company incorporated in England Wales. ShapeBlue Services India LLP is a company incorporated in India and is operated under license from Shape Blue Ltd. Shape Blue Brasil Consultoria Ltda is a company incorporated in Brasil and is operated under license from Shape Blue Ltd. ShapeBlue SA Pty Ltd is a company registered by The Republic of South Africa and is traded under license from Shape Blue Ltd. ShapeBlue is a registered trademark. -- Rohit Yadav Software Architect, ShapeBlue M. +41 779015219 |
Re: Documentation on cloud-engine and cloudstack-framework
Hi Rohit, Regarding IAM, here is the FS for the work Prachi and I have done: https://cwiki.apache.org/confluence/display/CLOUDSTACK/CloudStack+Identity+ and+Access+Management+%28IAM%29+Plugin, but we disable it on ACS 4.4 due to API and service layer gap mentioned in https://cwiki.apache.org/confluence/display/CLOUDSTACK/API+changes, specifically, API Gap with IAM section. Thanks -min On 7/14/14 7:09 AM, Rohit Yadav rohit.ya...@shapeblue.com wrote: Hi Daan, Thanks for replying. Good to know about the work, but this does not answer my questions. Daan Hoogland wrote: see inline On Mon, Jul 14, 2014 at 2:48 PM, Rohit Yadavrohit.ya...@shapeblue.com wrote: Ping? Rohit Yadav wrote: Hi, I'm trying to read the code and follow architectural and design changes. I see most of the code and various layers have the same workflow and architecture, same as last year except for few places. Can anyone point me to documentations and goals of cloud-engine and cloudstack-framework modules, where we are heading with those and what's the current status, and any on-going re-architecture or refactoring work? Not sure if this answers any of your questions: At SBP we are refactoring (Vpc)VirtualNetworkApplianceManager and related resources. Santhosh is working on fixing coverity report issues. You could consider those as guidelines Also, if we're going to implement any authentication framework (or have already?) that was discussed last year [1], current state docs on IAM [2] which works or is still under development? This was restracted for 4.4 due to semver issues Good to know. I would still like to know the development status. Regards, Rohit Yadav Software Architect, ShapeBlue M. +91 88 262 30892 | rohit.ya...@shapeblue.com Blog: bhaisaab.org | Twitter: @_bhaisaab Find out more about ShapeBlue and our range of CloudStack related services IaaS Cloud Design Buildhttp://shapeblue.com/iaas-cloud-design-and-build// CSForge rapid IaaS deployment frameworkhttp://shapeblue.com/csforge/ CloudStack Consultinghttp://shapeblue.com/cloudstack-consultancy/ CloudStack Infrastructure Supporthttp://shapeblue.com/cloudstack-infrastructure-support/ CloudStack Bootcamp Training Courseshttp://shapeblue.com/cloudstack-training/ This email and any attachments to it may be confidential and are intended solely for the use of the individual to whom it is addressed. Any views or opinions expressed are solely those of the author and do not necessarily represent those of Shape Blue Ltd or related companies. If you are not the intended recipient of this email, you must neither take any action based upon its contents, nor copy or show it to anyone. Please contact the sender if you believe you have received this email in error. Shape Blue Ltd is a company incorporated in England Wales. ShapeBlue Services India LLP is a company incorporated in India and is operated under license from Shape Blue Ltd. Shape Blue Brasil Consultoria Ltda is a company incorporated in Brasil and is operated under license from Shape Blue Ltd. ShapeBlue is a registered trademark.
Re: Why to have API Commands for Admin actions?
Hi Wilder, #3 in your list is definitely something we need to do to reuse command code and avoid the mistake of changing one place and forgetting the other one. But I don't quite get how to avoid passing response view parameter in ApiResponseGenerator in the execute() method using your method annotation idea. It will be great if you can use CreateVpcCmd as an example to give some code snippet to illustrate your idea, as well as your proposed flyweight pattern. Thanks -min On 6/25/14 4:27 AM, Wilder Rodrigues wrodrig...@schubergphilis.com wrote: Hi there, Thanks to Min, Rohit and Prachi for the details about the current implementation. On the if-else thing, that's not a problem to have them. At some point in time you have to have if-else in the code. The point is just to not abuse of it. But no worries, because there is no abuse of if-else in the current IAM implementation. IMO it is just there because we use the enum in the parameter. We could perhaps also have the enum associated with a type in a lightweight fly-weight-ish implementation, which would get rid of it. I will get back to that one in detail... I now understand that there might be more command types in the future, which could actually bring more response views as well. Concerning changes on the API, I don't think there would be any. Thinking a bit more about the approach I had in mind yesterday - which I did not made clear in the email - we could try something like this: 1. change the annotation from type to method, so we can annotate the execute method 2. in the annotation we inform the response view (as it is already done) 3. move the common code to the base command (e.g. CreateVpcCmd has the common code and CreateVpcAsAdminCmd has some specialized code, if needed) 4. The response view won't be passed as parameter, it will be in the method annotation 5. The methods down the pipe that need the response view, can extract it from the annotation 6. have patterns in place that will keep track of ResponseView and algorithm related to it (i.e. implement flyweight and template patterns to accomplish this). This approach will kill the ifs and also increase extensibility and maintainability of the code. Imho that would make clear and easier to implement commands and would avoid copy/paste code from a base command into an as admin command, for example. What do you guys think? We are currently busy with the implementation of redundant VPC feature. But I think I will keep an I on that one because I would like to contribute more. :) Cheers, Wilder -Original Message- From: Prachi Damle [mailto:prachi.da...@citrix.com] Sent: woensdag 25 juni 2014 12:43 To: dev@cloudstack.apache.org Cc: int-toolkit Subject: RE: Why to have API Commands for Admin actions? To elaborate further, I would like to add that even if this is adding some if-else code around checking the enum value when generating the response, it is replacing several other if-else's that were present in the code earlier that used to check if the user is an Admin/Domain Admin/regular user against the Db. With IAM, we cannot have such if-else conditions around hardcoded roles. The design should work with custom roles and custom 'response views' allocated to the user. This change is a first step in achieving this. Prachi -Original Message- From: Min Chen [mailto:min.c...@citrix.com] Sent: Tuesday, June 24, 2014 10:04 AM To: dev@cloudstack.apache.org Cc: int-toolkit Subject: Re: Why to have API Commands for Admin actions? Hi Wilder, This is a recent change introduced by IAM feature, see FS here https://cwiki.apache.org/confluence/display/CLOUDSTACK/CloudStack+Identity + and+Access+Management+%28IAM%29+Plugin, particularly see details in Response View section. The intention of this is to eventually provide custom response view for different custom IAM roles. As the first step, we only provided two static response view: Full and Restricted, and just map Full view to current admin commands. In the future, we should allow admin to define custom response view through column filter, and associate each custom response view with IAM policy. Hope that this can give you some context on this part of code. Thanks. -min On 6/24/14 4:55 AM, Wilder Rodrigues wrodrig...@schubergphilis.com wrote: Hi all, We are currently working on the redundant VPC implementation. In order to take the right steps from the beginning, we started analyzing the existing code base, from the API commands into the VPCVirtualNetworkAppliancaManagerImpl. Although it's not related to the feature itself, we found out that the current way of using the APICommand annotation and the CreateVPCCmd class (and its derived) is not really clear. For example, there are 2 command classes to create a VPC. The difference between them is: one has ResponseView.Full parameter in the @APICommand (ie. CreateVPCCmdByAdmin); and the other has ResponseView.Rstricted parameter
Re: Why to have API Commands for Admin actions?
Hi Wilder, This is a recent change introduced by IAM feature, see FS here https://cwiki.apache.org/confluence/display/CLOUDSTACK/CloudStack+Identity+ and+Access+Management+%28IAM%29+Plugin, particularly see details in Response View section. The intention of this is to eventually provide custom response view for different custom IAM roles. As the first step, we only provided two static response view: Full and Restricted, and just map Full view to current admin commands. In the future, we should allow admin to define custom response view through column filter, and associate each custom response view with IAM policy. Hope that this can give you some context on this part of code. Thanks. -min On 6/24/14 4:55 AM, Wilder Rodrigues wrodrig...@schubergphilis.com wrote: Hi all, We are currently working on the redundant VPC implementation. In order to take the right steps from the beginning, we started analyzing the existing code base, from the API commands into the VPCVirtualNetworkAppliancaManagerImpl. Although it's not related to the feature itself, we found out that the current way of using the APICommand annotation and the CreateVPCCmd class (and its derived) is not really clear. For example, there are 2 command classes to create a VPC. The difference between them is: one has ResponseView.Full parameter in the @APICommand (ie. CreateVPCCmdByAdmin); and the other has ResponseView.Rstricted parameter in the @APICommand (i.e. CreateVPCCmd). Moreover, the call to responseGenerator.createVpcresponde() method uses a ResponseView enum according to what has been specified in the annotation parameter. We understand that having a different enum in the responseGenerator.createVpcresponde() method will affect many things, because it goes deep into the code until reaches the APIDBUtils and the database. It is also checked in the ApiServer class, when the command is evaluated based on a string passed to the getCmdClass() method. Since we can identify the user in the account manager, also checking the kind of access the user has, what is the point in having the Annotation + the Enum in the code? Keep in mind that the latter is passed several times as parameters to other methods, which adds many ifs and unnecessary complexity. We could also make possible to use the Annotation in the method itself, which could avoid having to pass the Enum to the method. That means that the Enum would be removed from the annotation and we would use the annotation only to identify the API name, response object and entity type. In order to check the user's credentials, we would use the checkAccess() from the account manager, as it is already being used in the ApiDispatcher class. I know those are huge changes, if we actually agree in going for anything like this in the future. But the Admin commands are not doing much except for change the enum which is passed to the create response method. Most of the content of the execute() method is a copy/paste from the extended Command class. Just trying to start some chat towards the subject. Thanks for your time. Cheers, Wilder
[ACS4.4] Cherry pick CLOUDSTACK-6971
Hi Daan, Would you please cherry pick the following commit from 4.4-forward to 4.4 branch? Commit: 31e250a9d2adbf9ee59da66073497e38c02ded86 CLOUDSTACK-6971: createAutoScaleVmProfile failed with NPE due to lack of bean Injection. Thanks -min
Re: S3 use with simulator
Hi Sebastien, I am curious why you are using updateCloudToUseObjectStore api to add S3 as a secondary store in your data center? To set up a data center using S3, you need to use addImageStore api. Thanks -min On 6/20/14 7:47 AM, Sebastien Goasguen run...@gmail.com wrote: I switch to provider=S3 but I get: 530: Failed to add data store: DataCenter id is null, and simulator image store has to be associated with a data center On Jun 20, 2014, at 7:07 AM, Koushik Das koushik@citrix.com wrote: As per the code the following are the supported image store provider types // constants for provider names static final String NFS_IMAGE = NFS; static final String S3_IMAGE = S3; static final String SWIFT_IMAGE = Swift; static final String SAMPLE_IMAGE = Sample; static final String SMB = NFS; static final String DEFAULT_PRIMARY = DefaultPrimary; Try passing S3 in the provider. BTW when you setup simulator the image provider is NFS. -Original Message- From: Sebastien Goasguen [mailto:run...@gmail.com] Sent: Thursday, 19 June 2014 5:08 PM To: dev@cloudstack.apache.org Subject: S3 use with simulator Hi, I am using the simulator and started a basic zone. I have an S3 object store locally (riakCS), and I am trying to add it to the infra using the 'cloudtouseobjectstore' api with cloudmonkey. I tried with: update cloudtouseobjectstore url=http://localhost:9081/riak-cs name=riak provider=riakcs details[0].key=accesskeydetails[0].value=STU6Z-ZMK1TPMDAXL9I1details[1 ].key=secretkeydetails[1].value=8OuY3mHDXihu0Tdb2aVJ4vuYZLBAl5Z7NiWKsg= = 530: Failed to add data store: DataCenter id is null, and simulator image store has to be associated with a data center I am not sure if the arguments are right, has anyone done this before ? thanks, -sebastien
Re: ID instead of name in deployVirtualMachine
That seems a convention in all of our APIs. For APIs with support sort of on-behalf-of semantics, currently we are always using (account, domainId) parameter pair, where account requires accountName. You can see similar patterns in list APIs. Of course, we can change this, but may need to be in next API release. Thanks -min On 6/19/14 6:25 AM, Nux! n...@li.nux.ro wrote: Hi, We need to deploy VMs on behalf of certain users and surprisingly it's not possible to use accountid with deployvirtualmachine, the actual name of the account needs to be used. Anyone knows why this is and if it can be changed? It's quite awkward since everything else in the API seems to rely heavily on IDs. Lucian -- Sent from the Delta quadrant using Borg technology! Nux! www.nux.ro
Re: S3 use with simulator
Got it. The error seems from simulator, maybe Koushik has better idea on that. I didn't use that on simulator env though. Thanks -min On 6/20/14 11:05 AM, Sebastien Goasguen run...@gmail.com wrote: I already have an Nsf secondary storage. I understood that i could not add a second secondary storage that is s3 based, and that the only way was to use the updatecloud Api -Sebastien On 20 Jun 2014, at 18:49, Min Chen min.c...@citrix.com wrote: Hi Sebastien, I am curious why you are using updateCloudToUseObjectStore api to add S3 as a secondary store in your data center? To set up a data center using S3, you need to use addImageStore api. Thanks -min On 6/20/14 7:47 AM, Sebastien Goasguen run...@gmail.com wrote: I switch to provider=S3 but I get: 530: Failed to add data store: DataCenter id is null, and simulator image store has to be associated with a data center On Jun 20, 2014, at 7:07 AM, Koushik Das koushik@citrix.com wrote: As per the code the following are the supported image store provider types // constants for provider names static final String NFS_IMAGE = NFS; static final String S3_IMAGE = S3; static final String SWIFT_IMAGE = Swift; static final String SAMPLE_IMAGE = Sample; static final String SMB = NFS; static final String DEFAULT_PRIMARY = DefaultPrimary; Try passing S3 in the provider. BTW when you setup simulator the image provider is NFS. -Original Message- From: Sebastien Goasguen [mailto:run...@gmail.com] Sent: Thursday, 19 June 2014 5:08 PM To: dev@cloudstack.apache.org Subject: S3 use with simulator Hi, I am using the simulator and started a basic zone. I have an S3 object store locally (riakCS), and I am trying to add it to the infra using the 'cloudtouseobjectstore' api with cloudmonkey. I tried with: update cloudtouseobjectstore url=http://localhost:9081/riak-cs name=riak provider=riakcs details[0].key=accesskeydetails[0].value=STU6Z-ZMK1TPMDAXL9I1details [1 ].key=secretkeydetails[1].value=8OuY3mHDXihu0Tdb2aVJ4vuYZLBAl5Z7NiWKs g= = 530: Failed to add data store: DataCenter id is null, and simulator image store has to be associated with a data center I am not sure if the arguments are right, has anyone done this before ? thanks, -sebastien
Re: Can we delete a template after a VM is instantiated from it?
This seems a bug to me, start vm should not rely on the template not deleted. Please file a bug for that. Thanks -min On 6/16/14 10:33 AM, Rafael Weingartner rafaelweingart...@gmail.com wrote: This way, it seems that you have a bug in CS 4.3.0 when starting a machine that was created from a template that has been deleted. There will happen a null pointer exception in: ³858 -if (volTemplateId != null volTemplateId.longValue() != template.getId())² The object, ³template² is going to be null, because in: ³811 - VirtualMachineTemplate template = _entityMgr.findById(VirtualMachineTemplate.class, vm.getTemplateId());² The findById, will add a where clause, looking for template that have the column removed that is null, therefore It will return a null object. On Mon, Jun 16, 2014 at 4:41 AM, Nux! n...@li.nux.ro wrote: Hi, You can remove it from the UI, but not directly from the disk as it is used (in many cases) as a backing file for the VMs spawned from it. HTH Lucian -- Sent from the Delta quadrant using Borg technology! Nux! www.nux.ro - Original Message - From: Sanjeev Neelarapu sanjeev.neelar...@citrix.com To: dev@cloudstack.apache.org Sent: Monday, 16 June, 2014 5:18:04 AM Subject: RE: Can we delete a template after a VM is instantiated from it? We can delete template without any issues. -Sanjeev -Original Message- From: Rafael Weingartner [mailto:rafaelweingart...@gmail.com] Sent: Sunday, June 15, 2014 8:48 PM To: dev@cloudstack.apache.org Subject: Can we delete a template after a VM is instantiated from it? Hi, I was wondering if we can delete a template that has already been used to instantiate some VMs. Would that cause any trouble? -- Rafael Weingärtner -- Rafael Weingärtner
Can you please check resizeVolume bug CLOUSTACK-6797?
Hi Marcus, Would you mind checking this resizeVolume bug CLOUSTACK-6797? It is about resizeVolume not checking storage pool capacity. Thanks -min
Re: [DISCUSS] Release 4.4
I have assigned CLOUDSTACK-6825 to me. But from the stack trace, it seems failing in doing check access on the snapshot owner that is not active anymore, feel like a racing condition happening on automation setup. Rayees, several points to clarify: 1. Is this still happening on recent automation run on KVM? Based on the error, it should be hypervisor independent. 2. Is the owner of the snapshot passed in createTemplate call already deleted? Thanks -min On 6/11/14 10:58 AM, Alena Prokharchyk alena.prokharc...@citrix.com wrote: CLOUDSTACK-6825 happens while doing object access check in ParamProcessWorker. Min/Prachi, can you take a look to see if its related to RBAC feature? -Alena. On 6/11/14, 10:45 AM, Amogh Vasekar amogh.vase...@citrix.com wrote: Hi, AFAIK a couple of Automation blockers [1] [2], which had no owner assigned, were moved to critical. [1] CLOUDSTACK-6842 [2] CLOUDSTACK-6825 Thanks, Amogh On 6/11/14 10:33 AM, Mike Tutkowski mike.tutkow...@solidfire.com wrote: we have four blockers remaining...all network oriented. Murali Reddy has two. All four have an owner and presumably progress is being made.
Re: [DISCUSS] Release 4.4
Just looked at automation setup with Rayees, CLOUDSTACK-6825 is not really an issue of createTemplateFromSnapshot, the real issue is that the snapshot passed to createTemplate command has its owner removed in DB, thus causing checkAccess failure. As for why the snapshot is still not removed when the account is removed on automation setup, that should be a different issue. As for CLOUDSTACK-6825, we should be able to close it as not an issue. Thanks -min On 6/11/14 1:37 PM, Daan Hoogland daan.hoogl...@gmail.com wrote: On Wed, Jun 11, 2014 at 10:31 PM, Alena Prokharchyk alena.prokharc...@citrix.com wrote: If we confirm that its a race condition, then the bug should be punted to 4.5 or solve it? -- Daan
[ACS4.4]Cherry-pick 09a357fb90b48ed6e2725ea60e632a2ad5529f79
Hi Daan, Would you please cherry-pick the following commit from 4.4-forward to 4.4? Commit: 09a357fb90b48ed6e2725ea60e632a2ad5529f79 CLOUDSTACK-6890:createVPC invoked by admin does not observe start flag. Thanks -min
Re: [DISCUSS] Introducing Gerrit for quality? was: [PROPOSAL] Using continuous integration to maintain our code quality...
I like github pull request as well from past usage, it is very convenient for developers and reviewers to perform their tasks, compared to our current RB. Also agree with David, the pre-requisite for this enforcement is that we should have CI in place to make this happen. Thanks -min On 6/9/14 8:47 AM, David Nalley da...@gnsa.us wrote: On Fri, Jun 6, 2014 at 7:26 PM, Sheng Yang sh...@yasker.org wrote: Hi all, Seems it's a good timing to bring back the discussion about the gerrit. We want to do CI, and improve our code quality. One obvious way of doing and reduce the workload of devs is introduce a tool to enforce the process. I've checked out quite a few projects using gerrit, which would force you to ask for review, and validation before the code can be committed to the repo. Looks it's really a easier way for devs according what I've heard. Even our competitor laid out a very detail workflow based on the use of gerrit( https://wiki.openstack.org/wiki/Gerrit_Workflow ). I guess it can make a good reference. Well, gerrit has been brought up a few times before. And now the new process we want to enforce just fits what gerrit(or other automation review/test/commit software) is for. Maybe it's the time for us to review the possibility of using a tool to enforce our commits and improve our code quality(as well as transfer knowledge) again? --Sheng ASF Infra has a very dour view on Gerrit. Don't read that as impossible; there are many projects at the ASF who are interested in Gerrit. That said; what about moving to using github pull requests instead of RB, and from their, having the jenkins pull request builder automatically process every pull request and list information. Here's an example: https://github.com/jclouds/jclouds-labs/pull/61 You'll see that every time the patch changes, the jenkins plugin pulled the patch - ran tests against it and reported back. That said; it almost seems like we have the cart before the horse; we need to finish figuring out the CI Infrastructure first. --David
[ACS4.4]Cherry-pick CLOUDSTACK-6859
Hi Daan, Would you please cherry-pick the following commit from 4.4-forward to 4.4? Commit: 961cd7657e7dbb69d497123071db3156bd15d8e9 CLOUDSTACK-6859:Management Server PermGen run out of memory after some time due to class leak. Thanks -min
Re: [ACS4.4] [Issue] Unable to create a resource tag on ISO and Template resource
Santhosh,
The API command to create a resource tag is createTagsCmd. From just
ResourceObjectType enum listed in ResourceTag interface, Template/ISO is
supported.
Thanks
-min
On 6/4/14 7:57 AM, Santhosh Edukulla santhosh.eduku...@citrix.com
wrote:
Namitha,
Actually, my point is mainly to say that there is no provision available
to tag an iso or template from CS. So, is the reason to inquire the below
point 1. You may wanted to confirm the below point on normal CS, with
out simulator?
1. Is this feature of tagging iso or template available\supported in CS?
What API we are using for it?( check on normal CS with out simulator)
Santhosh
From: Namita Chaudhari [namita.chaudh...@sungardas.com]
Sent: Wednesday, June 04, 2014 10:14 AM
To: dev@cloudstack.apache.org
Subject: Re: [ACS4.4] [Issue] Unable to create a resource tag on ISO and
Template resource
Hi Santhosh,
I am unable to register a Template or ISO from UI on simulator. I get an
exception Unable to register a template due to some missing parameters.
So I cannot proceed further and try to manually create tags on them.
Thanks and Regards,
On Tue, Jun 3, 2014 at 6:35 PM, Santhosh Edukulla
santhosh.eduku...@citrix.commailto:santhosh.eduku...@citrix.com wrote:
Two things:
1. Is this feature of tagging iso or template available\supported in CS?
What API we are using for it?
2. If this is supported feature, can you manually do it from UI on
simulator and check if its working there?
Thanks!
Santhosh
From: Namita Chaudhari
[namita.chaudh...@sungardas.commailto:namita.chaudh...@sungardas.com]
Sent: Tuesday, June 03, 2014 8:51 AM
To: dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org
Subject: Re: [ACS4.4] [Issue] Unable to create a resource tag on ISO and
Template resource
Hi Santhosh,
These are new test cases. On simulator and in same test case, I have
created tags on various resources like volume, project, snapshot etc and
they work perfectly fine.
I face this issue only for ISO and Template resource tag creation.
Thanks,
Namita
On Tue, Jun 3, 2014 at 6:16 PM, Santhosh Edukulla
santhosh.eduku...@citrix.commailto:santhosh.eduku...@citrix.commailto:
santhosh.eduku...@citrix.commailto:santhosh.eduku...@citrix.com wrote:
Namita,
1. Just to separate this issue as simulator vs test code, can we just
check from UI whether the objective of creating tags for simulator is
possible there? If yes, then we can look in to test code.
2. As well, are these new tests or existing tests?
Thanks!
Santhosh
From: Namita Chaudhari
[namita.chaudh...@sungardas.commailto:namita.chaudh...@sungardas.commai
lto:namita.chaudh...@sungardas.commailto:namita.chaudh...@sungardas.com
]
Sent: Tuesday, June 03, 2014 8:27 AM
To:
dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.orgmailto:dev@clo
udstack.apache.orgmailto:dev@cloudstack.apache.org
Subject: [ACS4.4] [Issue] Unable to create a resource tag on ISO and
Template resource
Hi All,
I was trying to create a resource tag on ISO and Template with a sample
data for a test
case. Can anyone help me toknow is there anything I'm missing in the input
parameters ?
For both resources, I get a db exception Out of range value for column
'domain_id' at row 1 where the domainid gets value as -1.
I'm running this test on simulator.
1] For ISO:
a) ISO in json
iso1A: {
displaytext: Dummy ISO,
name: Dummy ISO,
url: http://people.apache.org/~tsp/dummy.iso;,
zoneid: 9ecf9d8b-cf18-4322-a641-a1c0aced5857,
# Source URL where ISO is located
isextractable: True,
isfeatured: True,
ispublic: False,
ostype: 'CentOS 5.3 (64-bit)',
mode: 'HTTP_DOWNLOAD',
# Used in Extract template, value must be HTTP_DOWNLOAD
},
b) Creating an iso with its tag
self.account_1A = Account.create(
self.apiclient,
self.services[account1A],
admin=False,
domainid=self.domain_1.idhttp://self.domain_1.idhttp://self.domain_1.id
http://self.domain_1.id/
)
self.userapiclient_1A =
self.testClient.getUserApiClient(self.user_1A.username,
self.domain_1.namehttp://self.domain_1.namehttp://self.domain_1.nameh
ttp://self.domain_1.name/)
self.iso1A = Iso.create(
self.apiclient,
self.services[iso],
account=self.account_1A.namehttp://self.account_1A.namehttp://self.acco
unt_1A.namehttp://self.account_1A.name,
domainid=self.account_1A.domainid
)
self.debug(ISO created with ID: %s %
self.iso1A.idhttp://self.iso1A.idhttp://self.iso1A.idhttp://self.iso1
Re: [ACS5.0] IAM feature postponed from 4.4 to 5.0?
As mentioned in our FS doc in wiki, In phase I, all the permissions attached to any policy are by default explicit 'Allow' permissions. As of now 'Deny' permissions cannot be added. For your use cases, you can have two options: 1. Assign the two accounts into 2 different groups, and attach different policy for the group. 2. Directly attach an Allow policy to account 2 instead of assigning both accounts into the Allow All group. Thanks -min On 6/3/14 5:03 AM, Meghna Kale meghna.k...@sungardas.com wrote: Hi Min, With reference to the wiki doc, I had a query. In case of a customized role with deny permissions how will the listAll, isrecursive ..etc. input parameters values will be ? For example, there are two accounts and they belong to a group with Allow all permissions. If I have to remove some permissions for only account 1 but keep them for account 2 is it possible? Thanks Meghna. On Thu, May 22, 2014 at 10:22 PM, Min Chen min.c...@citrix.com wrote: Added API issues we found through IAM feature in the wiki page created by Demetrius: https://cwiki.apache.org/confluence/display/CLOUDSTACK/API+changes Thanks -min On 5/14/14 9:34 AM, Min Chen min.c...@citrix.com wrote: Thanks Daan. Yes, I saw that there is another thread about putting an API request for 5.0 api. Once we are done with this disabling, we will put the issues we have found with current API in that wiki page to take into consideration when we design the new API. -min On 5/14/14 12:12 AM, Daan Hoogland daan.hoogl...@gmail.com wrote: Min, I think everybody knows I am all for less features per release. I don't think you are making a bad call, per se. I do think we should consider if we can come up with a total picture of what 5.x would require af the api, though. Can you add to the discussion what it is that is keeping you from implementing. And what requirements you have for the 5.0 api so we can start devising the architectural guidelines for the new api. more and more calls for a 5.0 are coming up lately so let's move forward. (changing title) On Wed, May 14, 2014 at 1:53 AM, Min Chen min.c...@citrix.com wrote: Hi All, In the past several weeks, QA has done some testing on IAM feature and found several backward-compatibility issues. Even though Prachi and I have tried our best to fix bugs to maintain backward compatibility, we realized that in order to support true IAM model documented in our FS https://cwiki.apache.org/confluence/display/CLOUDSTACK/CloudStack+Identi t y+and+Access+Management+%28IAM%29+Plugin, we will have to make several API changes that will require us to increment CloudStack major version. Therefore we think that IAM feature is not ready for ACS 4.4 release, and we would like to propose to disable it in 4.4 branch and re-enable it later when community decides to go for 5.x. Thanks -min -- Daan
Re: [OFFLINE] Going away for some time
Welcome back, Rohit. -min On 6/2/14 9:56 AM, Rohit Yadav rohityada...@gmail.com wrote: Hi, I've found a way to sponsor myself to work full-time on the project again in upcoming months, and will slowly and gradually start contributing to the project. Please do help me in catching up about the changes and developments since last year. Cheers. On Tue, Aug 27, 2013 at 2:19 PM, Rohit Yadav rohityada...@gmail.com wrote: Hi folks, I'm not able to participate on the MLs and contribute code to the project for past couple of months now. So, I reached out to the pmc and some friends on the ML on this issue. Right now the project bylaws has no information on committer moving to emeritus status so I just want to tell you that I'll going away for quite some time and will try to contribute from time to time. I may not follow the MLs so feel free to email me on rohityadav89 AT gmail.com to include me in some discussion thread and whatnot. Regards.
Re: [ANNOUNCE] Amogh Vasekar as committer
Congratulations, Amogh. -min On 6/2/14 11:53 AM, Ahmad Emneina aemne...@gmail.com wrote: oh dang! good stuff Amogh! On Mon, Jun 2, 2014 at 11:29 AM, Rayees Namathponnan rayees.namathpon...@citrix.com wrote: Congrats Amogh. Regards, Rayees -Original Message- From: Nitin Mehta [mailto:nitin.me...@citrix.com] Sent: Monday, June 02, 2014 11:17 AM To: dev@cloudstack.apache.org Subject: Re: [ANNOUNCE] Amogh Vasekar as committer Great news. Congrats Amogh !!! On 02/06/14 11:14 AM, John Kinsella j...@stratosec.co wrote: The Project Management Committee (PMC) for Apache CloudStack has asked Amogh Vasekar to become a committer and we are pleased to announce that he has accepted. Being a committer allows many contributors to contribute more autonomously. For developers, it makes it easier to submit changes and eliminates the need to have contributions reviewed via the patch submission process. Whether contributions are development-related or otherwise, it is a recognition of a contributor's participation in the project and commitment to the project and the Apache Way. Please join me in congratulating Amogh! -John, on behalf of the CloudStack PMC
Re: Erro: No remote endpoint to send command, check if host or ssvm is down?
Check if agent is running your ssvm. Thanks -min On 5/28/14 7:26 AM, Rafael Weingartner rafaelweingart...@gmail.com wrote: Hi all, sorry, to bother you again with this thread. Could at least someone that has the CS 4.3.0 deployed try to download a volume and/or a template? Could someone also check the database, table ³cloud.host² and look if there is an entry there for the running SSVM and console proxy VM? On Mon, May 26, 2014 at 4:42 PM, Rafael Weingartner rafaelweingart...@gmail.com wrote: Hi folks, I am sending this to the dev list as well, hence I am not sure if it is or not a bug. Sorry if I am mistaken. So, after I upgraded CS from 4.1.1 to 4.3.0, everything seemed to be working just fine, however, whenever I try to download a template or volume I am getting the following error: ERROR [o.a.c.s.d.d.CloudStackImageStoreDriverImpl] (Job-Executor-1:ctx-1cb86faa ctx-61f7e4f9) No remote endpoint to send command, check if host or ssvm is down? ERROR [o.a.c.s.d.d.CloudStackImageStoreDriverImpl] (Job-Executor-1:ctx-1cb86faa ctx-61f7e4f9) Unable to create a link for entity at ... on ssvm,No remote endpoint to send command, check if host or ssvm is down I have already destroyed the system VMs in order to let CS recreate them with the new system VM template. I logged into the SSVM to check whether or not it was created using the new template and it indeed. Therefore, I have ruled out that problem. Then, I took a look into the source code, more specifically at org.apache.cloudstack.storage.datastore.driver.CloudStackImageStoreDrive rImpl.createEntityExtractUrl(DataStore, String, ImageFormat, DataObject) and org.apache.cloudstack.storage.endpoint.DefaultEndPointSelector.findEndpo intForImageStorage(DataStore). I noticed that it looks for an entry at table Cloud.host that has the type ³SecondaryStorageVM². I took a look into the DB, but, there are no entries either for SSVM or Console proxy in ³Cloud.host² table. I mean no entries for the VMs that are running. The older VMs that were destroyed have an entry there. Have anyone seem a problem like this before? Is there something I am missing? The VMs are running, should not them be registered there? -- Rafael Weingärtner -- Rafael Weingärtner
Re: Erro: No remote endpoint to send command, check if host or ssvm is down?
Your UI indicates that agent is not running on your SSVM, it should show as 'Running', that is why your cloud.host does not have an entry for your new SSVM, thus we cannot find a remote host to send command to. Agent is a java code running inside your SSVM, you should see what error you encountered in starting agent in your ssvm from cloud.out file in SSVM log. Thanks -min On 5/28/14 10:20 AM, Rafael Weingartner rafaelweingart...@gmail.com wrote: What agent are you talking about? Is it an app that run into the SSVM? I am using xenserver and XCP as the hypervisor. I looked into the CS UI, infrastructureSystem VMs and the column that has some values about the agent-state has a -- in it. Is it normal? At the end, should the SSVM have an entry into the host table? On Wed, May 28, 2014 at 2:05 PM, Min Chen min.c...@citrix.com wrote: Check if agent is running your ssvm. Thanks -min On 5/28/14 7:26 AM, Rafael Weingartner rafaelweingart...@gmail.com wrote: Hi all, sorry, to bother you again with this thread. Could at least someone that has the CS 4.3.0 deployed try to download a volume and/or a template? Could someone also check the database, table ³cloud.host² and look if there is an entry there for the running SSVM and console proxy VM? On Mon, May 26, 2014 at 4:42 PM, Rafael Weingartner rafaelweingart...@gmail.com wrote: Hi folks, I am sending this to the dev list as well, hence I am not sure if it is or not a bug. Sorry if I am mistaken. So, after I upgraded CS from 4.1.1 to 4.3.0, everything seemed to be working just fine, however, whenever I try to download a template or volume I am getting the following error: ERROR [o.a.c.s.d.d.CloudStackImageStoreDriverImpl] (Job-Executor-1:ctx-1cb86faa ctx-61f7e4f9) No remote endpoint to send command, check if host or ssvm is down? ERROR [o.a.c.s.d.d.CloudStackImageStoreDriverImpl] (Job-Executor-1:ctx-1cb86faa ctx-61f7e4f9) Unable to create a link for entity at ... on ssvm,No remote endpoint to send command, check if host or ssvm is down I have already destroyed the system VMs in order to let CS recreate them with the new system VM template. I logged into the SSVM to check whether or not it was created using the new template and it indeed. Therefore, I have ruled out that problem. Then, I took a look into the source code, more specifically at org.apache.cloudstack.storage.datastore.driver.CloudStackImageStoreDri ve rImpl.createEntityExtractUrl(DataStore, String, ImageFormat, DataObject) and org.apache.cloudstack.storage.endpoint.DefaultEndPointSelector.findEnd po intForImageStorage(DataStore). I noticed that it looks for an entry at table Cloud.host that has the type ³SecondaryStorageVM². I took a look into the DB, but, there are no entries either for SSVM or Console proxy in ³Cloud.host² table. I mean no entries for the VMs that are running. The older VMs that were destroyed have an entry there. Have anyone seem a problem like this before? Is there something I am missing? The VMs are running, should not them be registered there? -- Rafael Weingärtner -- Rafael Weingärtner -- Rafael Weingärtner
Re: Erro: No remote endpoint to send command, check if host or ssvm is down?
You should clear old systemvm.iso from xen server so that new systemvm.iso built can be deployed there. Thanks -min On 5/28/14 12:42 PM, Rafael Weingartner rafaelweingart...@gmail.com wrote: Another interesting fact: All the jars, cloud-**.jar are label as if they were 4.1.1 version. Is that right? On Wed, May 28, 2014 at 4:01 PM, Rafael Weingartner rafaelweingart...@gmail.com wrote: Just an update. I have just checked my console proxy VM, which is also with -- in agent state at insfrastructureSystem VM page. The agente is running there, however its state is --. I also looked into the source code, and the class org.apache.cloudstack.storage.resource.NfsSecondaryStorageResource is located in a jar called cloud-secondary-storage which is not in the directory /usr/local/cloud/systemvm that contains the jars used to run the agent. Neither in my SSVM nor in Console proxy VM that jar exist. I do not get, I cannot be the only one with this problem. It seems that this jar is missing into the system vm template. On Wed, May 28, 2014 at 2:45 PM, Rafael Weingartner rafaelweingart...@gmail.com wrote: thanks, I checked the /var/log/cloud/cloud.out, And I found this error: ERROR AgentShell:607 - Unable to start agent: Resource class not found: org.apache.cloudstack.storage.resource.NfsSecondaryStorageResource due to: java.lang.ClassNotFoundException: org.apache.cloudstack.storage.resource.NfsSecondaryStorageResource Unable to start agent: Resource class not found: org.apache.cloudstack.storage.resource.NfsSecondaryStorageResource due to: java.lang.ClassNotFoundException: org.apache.cloudstack.storage.resource.NfsSecondaryStorageResource It seems that there is a jar dependency missing. On Wed, May 28, 2014 at 2:31 PM, Min Chen min.c...@citrix.com wrote: Your UI indicates that agent is not running on your SSVM, it should show as 'Running', that is why your cloud.host does not have an entry for your new SSVM, thus we cannot find a remote host to send command to. Agent is a java code running inside your SSVM, you should see what error you encountered in starting agent in your ssvm from cloud.out file in SSVM log. Thanks -min On 5/28/14 10:20 AM, Rafael Weingartner rafaelweingart...@gmail.com wrote: What agent are you talking about? Is it an app that run into the SSVM? I am using xenserver and XCP as the hypervisor. I looked into the CS UI, infrastructureSystem VMs and the column that has some values about the agent-state has a -- in it. Is it normal? At the end, should the SSVM have an entry into the host table? On Wed, May 28, 2014 at 2:05 PM, Min Chen min.c...@citrix.com wrote: Check if agent is running your ssvm. Thanks -min On 5/28/14 7:26 AM, Rafael Weingartner rafaelweingart...@gmail.com wrote: Hi all, sorry, to bother you again with this thread. Could at least someone that has the CS 4.3.0 deployed try to download a volume and/or a template? Could someone also check the database, table ³cloud.host² and look if there is an entry there for the running SSVM and console proxy VM? On Mon, May 26, 2014 at 4:42 PM, Rafael Weingartner rafaelweingart...@gmail.com wrote: Hi folks, I am sending this to the dev list as well, hence I am not sure if it is or not a bug. Sorry if I am mistaken. So, after I upgraded CS from 4.1.1 to 4.3.0, everything seemed to be working just fine, however, whenever I try to download a template or volume I am getting the following error: ERROR [o.a.c.s.d.d.CloudStackImageStoreDriverImpl] (Job-Executor-1:ctx-1cb86faa ctx-61f7e4f9) No remote endpoint to send command, check if host or ssvm is down? ERROR [o.a.c.s.d.d.CloudStackImageStoreDriverImpl] (Job-Executor-1:ctx-1cb86faa ctx-61f7e4f9) Unable to create a link for entity at ... on ssvm,No remote endpoint to send command, check if host or ssvm is down I have already destroyed the system VMs in order to let CS recreate them with the new system VM template. I logged into the SSVM to check whether or not it was created using the new template and it indeed. Therefore, I have ruled out that problem. Then, I took a look into the source code, more specifically at org.apache.cloudstack.storage.datastore.driver.CloudStackImageStor eDri ve rImpl.createEntityExtractUrl(DataStore, String, ImageFormat, DataObject) and org.apache.cloudstack.storage.endpoint.DefaultEndPointSelector.fin dEnd po intForImageStorage(DataStore). I noticed that it looks for an entry at table Cloud.host that has the type ³SecondaryStorageVM². I took a look into the DB, but, there are no entries either for SSVM or Console proxy in ³Cloud.host² table. I mean no entries for the VMs that are running. The older VMs that were destroyed have an entry there. Have anyone seem a problem like this before? Is there something I am
Re: [ACS 4.4] Blocker and Critical Defects 5/27
Done. Thanks -min On 5/28/14 2:01 PM, Sudha Ponnaganti sudha.ponnaga...@citrix.com wrote: Can you resolve it so QA can pick it up. It is still in open state -Original Message- From: Min Chen [mailto:min.c...@citrix.com] Sent: Tuesday, May 27, 2014 4:15 PM To: dev@cloudstack.apache.org Subject: Re: [ACS 4.4] Blocker and Critical Defects 5/27 CLOUDSTACK-6644 was filed before I disabled IAM feature on 4.4. I have updated the bug to retest with latest code. Thanks -min On 5/27/14 3:21 PM, Sudha Ponnaganti sudha.ponnaga...@citrix.com wrote: There are 8 blockers and 77 critical defects outstanding Dashboard https://issues.apache.org/jira/secure/Dashboard.jspa?selectPageId=12323 265 Blocker Defect List: KeySummary Assignee CLOUDSTACK-6779 [OVS] Expunging VM (deleting vif) deletes all the rules from ovs bridge flow tableMurali Reddy CLOUDSTACK-6755 [OVS] Can't create more than 7 GRE tunnel networks in xen cluster Murali Reddy CLOUDSTACK-6710 [Automation] VM snapshot failing with NPE in vmware Likitha Shetty CLOUDSTACK-6644 Unable to attach Volume to a VM as a System User Min Chen CLOUDSTACK-6623 Register template does not work as expected, when deploying simulator and xen zones simultaneously on a single management server. edison su CLOUDSTACK-6603 [Upgrade]DB Exception while Autoscale monitoring after upgrading from 4.3 to 4.4 Rajesh Battala CLOUDSTACK-6602 [UI] createNetworkACL API action param value passed incorrectly Jessica Wang CLOUDSTACK-6599 Template/Volume URLs expiration functionality not working Nitin Mehta
Re: Erro: No remote endpoint to send command, check if host or ssvm is down?
I don't know the exact cmd line to clear that, and I have always used XenCenter UI to do that, where clear tags in General tabs. Thanks -min On 5/28/14 2:11 PM, Rafael Weingartner rafaelweingart...@gmail.com wrote: by clear, you mean xe vdi-destroy uuid=vdi of systemvm.iso? So, I just need to stop the management servers, run that in every cluster that I have and everything should be fine? On Wed, May 28, 2014 at 5:42 PM, Min Chen min.c...@citrix.com wrote: You should clear old systemvm.iso from xen server so that new systemvm.iso built can be deployed there. Thanks -min On 5/28/14 12:42 PM, Rafael Weingartner rafaelweingart...@gmail.com wrote: Another interesting fact: All the jars, cloud-**.jar are label as if they were 4.1.1 version. Is that right? On Wed, May 28, 2014 at 4:01 PM, Rafael Weingartner rafaelweingart...@gmail.com wrote: Just an update. I have just checked my console proxy VM, which is also with -- in agent state at insfrastructureSystem VM page. The agente is running there, however its state is --. I also looked into the source code, and the class org.apache.cloudstack.storage.resource.NfsSecondaryStorageResource is located in a jar called cloud-secondary-storage which is not in the directory /usr/local/cloud/systemvm that contains the jars used to run the agent. Neither in my SSVM nor in Console proxy VM that jar exist. I do not get, I cannot be the only one with this problem. It seems that this jar is missing into the system vm template. On Wed, May 28, 2014 at 2:45 PM, Rafael Weingartner rafaelweingart...@gmail.com wrote: thanks, I checked the /var/log/cloud/cloud.out, And I found this error: ERROR AgentShell:607 - Unable to start agent: Resource class not found: org.apache.cloudstack.storage.resource.NfsSecondaryStorageResource due to: java.lang.ClassNotFoundException: org.apache.cloudstack.storage.resource.NfsSecondaryStorageResource Unable to start agent: Resource class not found: org.apache.cloudstack.storage.resource.NfsSecondaryStorageResource due to: java.lang.ClassNotFoundException: org.apache.cloudstack.storage.resource.NfsSecondaryStorageResource It seems that there is a jar dependency missing. On Wed, May 28, 2014 at 2:31 PM, Min Chen min.c...@citrix.com wrote: Your UI indicates that agent is not running on your SSVM, it should show as 'Running', that is why your cloud.host does not have an entry for your new SSVM, thus we cannot find a remote host to send command to. Agent is a java code running inside your SSVM, you should see what error you encountered in starting agent in your ssvm from cloud.out file in SSVM log. Thanks -min On 5/28/14 10:20 AM, Rafael Weingartner rafaelweingart...@gmail.com wrote: What agent are you talking about? Is it an app that run into the SSVM? I am using xenserver and XCP as the hypervisor. I looked into the CS UI, infrastructureSystem VMs and the column that has some values about the agent-state has a -- in it. Is it normal? At the end, should the SSVM have an entry into the host table? On Wed, May 28, 2014 at 2:05 PM, Min Chen min.c...@citrix.com wrote: Check if agent is running your ssvm. Thanks -min On 5/28/14 7:26 AM, Rafael Weingartner rafaelweingart...@gmail.com wrote: Hi all, sorry, to bother you again with this thread. Could at least someone that has the CS 4.3.0 deployed try to download a volume and/or a template? Could someone also check the database, table ³cloud.host² and look if there is an entry there for the running SSVM and console proxy VM? On Mon, May 26, 2014 at 4:42 PM, Rafael Weingartner rafaelweingart...@gmail.com wrote: Hi folks, I am sending this to the dev list as well, hence I am not sure if it is or not a bug. Sorry if I am mistaken. So, after I upgraded CS from 4.1.1 to 4.3.0, everything seemed to be working just fine, however, whenever I try to download a template or volume I am getting the following error: ERROR [o.a.c.s.d.d.CloudStackImageStoreDriverImpl] (Job-Executor-1:ctx-1cb86faa ctx-61f7e4f9) No remote endpoint to send command, check if host or ssvm is down? ERROR [o.a.c.s.d.d.CloudStackImageStoreDriverImpl] (Job-Executor-1:ctx-1cb86faa ctx-61f7e4f9) Unable to create a link for entity at ... on ssvm,No remote endpoint to send command, check if host or ssvm is down I have already destroyed the system VMs in order to let CS recreate them with the new system VM template. I logged into the SSVM to check whether or not it was created using the new template and it indeed. Therefore, I have ruled out that problem. Then, I took a look into the source code, more specifically at org.apache.cloudstack.storage.datastore.driver.CloudStackImageSt or eDri
Re: [ACS 4.4] Blocker and Critical Defects 5/27
CLOUDSTACK-6644 was filed before I disabled IAM feature on 4.4. I have updated the bug to retest with latest code. Thanks -min On 5/27/14 3:21 PM, Sudha Ponnaganti sudha.ponnaga...@citrix.com wrote: There are 8 blockers and 77 critical defects outstanding Dashboard https://issues.apache.org/jira/secure/Dashboard.jspa?selectPageId=12323265 Blocker Defect List: KeySummary Assignee CLOUDSTACK-6779 [OVS] Expunging VM (deleting vif) deletes all the rules from ovs bridge flow tableMurali Reddy CLOUDSTACK-6755 [OVS] Can't create more than 7 GRE tunnel networks in xen cluster Murali Reddy CLOUDSTACK-6710 [Automation] VM snapshot failing with NPE in vmware Likitha Shetty CLOUDSTACK-6644 Unable to attach Volume to a VM as a System User Min Chen CLOUDSTACK-6623 Register template does not work as expected, when deploying simulator and xen zones simultaneously on a single management server. edison su CLOUDSTACK-6603 [Upgrade]DB Exception while Autoscale monitoring after upgrading from 4.3 to 4.4 Rajesh Battala CLOUDSTACK-6602 [UI] createNetworkACL API action param value passed incorrectly Jessica Wang CLOUDSTACK-6599 Template/Volume URLs expiration functionality not working Nitin Mehta
[ACS4.4] Cherry pick d5fbcafc2ff04e665c2ec8079c759fe4cabf9d1a
Hi Daan, Can you please cherry-pick commit d5fbcafc2ff04e665c2ec8079c759fe4cabf9d1a from 4.4-forward to 4.4 branch? This commit removes windows line ending from IAMService.java. Thanks -min
Re: [ACS5.0] IAM feature postponed from 4.4 to 5.0?
Added API issues we found through IAM feature in the wiki page created by Demetrius: https://cwiki.apache.org/confluence/display/CLOUDSTACK/API+changes Thanks -min On 5/14/14 9:34 AM, Min Chen min.c...@citrix.com wrote: Thanks Daan. Yes, I saw that there is another thread about putting an API request for 5.0 api. Once we are done with this disabling, we will put the issues we have found with current API in that wiki page to take into consideration when we design the new API. -min On 5/14/14 12:12 AM, Daan Hoogland daan.hoogl...@gmail.com wrote: Min, I think everybody knows I am all for less features per release. I don't think you are making a bad call, per se. I do think we should consider if we can come up with a total picture of what 5.x would require af the api, though. Can you add to the discussion what it is that is keeping you from implementing. And what requirements you have for the 5.0 api so we can start devising the architectural guidelines for the new api. more and more calls for a 5.0 are coming up lately so let's move forward. (changing title) On Wed, May 14, 2014 at 1:53 AM, Min Chen min.c...@citrix.com wrote: Hi All, In the past several weeks, QA has done some testing on IAM feature and found several backward-compatibility issues. Even though Prachi and I have tried our best to fix bugs to maintain backward compatibility, we realized that in order to support true IAM model documented in our FS https://cwiki.apache.org/confluence/display/CLOUDSTACK/CloudStack+Identi t y+and+Access+Management+%28IAM%29+Plugin, we will have to make several API changes that will require us to increment CloudStack major version. Therefore we think that IAM feature is not ready for ACS 4.4 release, and we would like to propose to disable it in 4.4 branch and re-enable it later when community decides to go for 5.x. Thanks -min -- Daan
[ACS4.4] Cherry-pick da5ad74d5f4388c4aa1df2f2e5f9053bfb70d83d
Hi Daan, Would you please cherry-pick the following commit from 4.4-forward to 4.4? Commit: da5ad74d5f4388c4aa1df2f2e5f9053bfb70d83d CLOUDSTACK-6752: IAM command class separation caused ApiDoc warning of duplicated cmd class for the same api name. Thanks -min
[ACS4.4] Cherry-pick 356f6121a78d147d72136044c90472234f667730
Hi Daan, Can you please cherry-pick this commit from 4.4-forward to 4.4? Commit: 356f6121a78d147d72136044c90472234f667730 CLOUDSTACK-6742: listVolumes - As regularuser , able to list Vms and volumes of other users. Thanks -min
Re: [ACS4.4] Cherry-pick f748a552e9546e91e18c574b375f3ea6c8d7e043
Hi Daan, What is the conflict? Do you need me to resolve the conflict? Sorry, just realized that the URL is internal to citrix, not visible to outside. Thanks -min On 5/20/14 1:16 AM, Daan Hoogland daan.hoogl...@gmail.com wrote: Min, On Tue, May 20, 2014 at 1:30 AM, Min Chen min.c...@citrix.com wrote: Hi Daan, As we proposed earlier in community, since IAM feature is not ready for 4.4 release due to backwards compatibility issue, we need to disable IAM on 4.4. I have checked in commit f748a552e9546e91e18c574b375f3ea6c8d7e043 on 4.4-forward to disable IAM, would you please cherry-pick this commit from 4.4-forward to 4.4 branch? It does not cherry-pick cleanly. To avoid causing some regressions due to this disabling, we have run BVT on this commit and got results (http://jenkins-ccp.citrix.com/view/SC_Automation/job/Adv_KVM_BVT_Report/ 370/) is this a public url? I can not reach it. compared to 4.3 BVT baseline. Thanks -min -- Daan
Re: Patch missing in 4.4-forward
I ran into same issues in analyzing automation test. The related integration tests are merged into 4.4-forward along with marvin test framework merge, but this patch does not exist in 4.4-forward, so we should either disable these two integration tests, or pull that patch code into 4.4-forward. Thanks -min On 5/20/14 1:22 AM, Daan Hoogland daan.hoogl...@gmail.com wrote: Gaurav, to avoid confusion: these look like integration tests. Is that what you mean? On Tue, May 20, 2014 at 7:17 AM, Gaurav Aradhye gaurav.arad...@clogeny.com wrote: Hello Yoshikazu, Please take a look at this patch (https://reviews.apache.org/r/21682/). These are the unit tests I am talking about. These should not be present in 4.4-forward right? Regards, Gaurav On Mon, May 19, 2014 at 8:28 PM, Yoshikazu Nojima m...@ynojima.net wrote: Gaurav, This patch is intended to be released in 4.5 release, because when I wrote up it, 4.4 feature freeze date was already passed. I'm not sure which unit test are you talking about? Regards, Noji 2014-05-19 8:20 GMT-06:00 Gaurav Aradhye gaurav.arad...@clogeny.com: Hi, The patch related to feature Volume Provisioning (https://reviews.apache.org/r/19446/) is not present in 4.4-forward branch, it is committed to only master. Is this intended? If not can you push it to 4.4-forward too? Don't know how but the unit tests related to this feature are present in 4.4-forward branch and are failing in the daily run. I think until this patch comes into forward branch, we should disable the related tests. Regards, Gaurav -- Daan
Re: [ACS4.4] Cherry-pick f748a552e9546e91e18c574b375f3ea6c8d7e043
Ok. Let me try. Can I cherry pick this into 4.4 after resolving the conflict? Thanks -min On 5/20/14 1:44 PM, Daan Hoogland daan.hoogl...@gmail.com wrote: Sorry Min, I didn't check the conflict, just aborted. On Tue, May 20, 2014 at 6:20 PM, Min Chen min.c...@citrix.com wrote: Hi Daan, What is the conflict? Do you need me to resolve the conflict? Sorry, just realized that the URL is internal to citrix, not visible to outside. Thanks -min On 5/20/14 1:16 AM, Daan Hoogland daan.hoogl...@gmail.com wrote: Min, On Tue, May 20, 2014 at 1:30 AM, Min Chen min.c...@citrix.com wrote: Hi Daan, As we proposed earlier in community, since IAM feature is not ready for 4.4 release due to backwards compatibility issue, we need to disable IAM on 4.4. I have checked in commit f748a552e9546e91e18c574b375f3ea6c8d7e043 on 4.4-forward to disable IAM, would you please cherry-pick this commit from 4.4-forward to 4.4 branch? It does not cherry-pick cleanly. To avoid causing some regressions due to this disabling, we have run BVT on this commit and got results (http://jenkins-ccp.citrix.com/view/SC_Automation/job/Adv_KVM_BVT_Repor t/ 370/) is this a public url? I can not reach it. compared to 4.3 BVT baseline. Thanks -min -- Daan -- Daan
Re: [ACS4.4] Cherry-pick f748a552e9546e91e18c574b375f3ea6c8d7e043
Hi Daan, The cherry pick error is due to a deleted iam test since we disabled IAM feature: # Unmerged paths: # (use git add/rm file... as appropriate to mark resolution) # # deleted by them:test/integration/smoke/test_vm_iam.py So the fix is to run git rm test/integration/smoke/test_vm_iam.py. I can do that if you allow me to check into 4.4. Thanks. -min On 5/20/14 2:12 PM, Min Chen min.c...@citrix.com wrote: Ok. Let me try. Can I cherry pick this into 4.4 after resolving the conflict? Thanks -min On 5/20/14 1:44 PM, Daan Hoogland daan.hoogl...@gmail.com wrote: Sorry Min, I didn't check the conflict, just aborted. On Tue, May 20, 2014 at 6:20 PM, Min Chen min.c...@citrix.com wrote: Hi Daan, What is the conflict? Do you need me to resolve the conflict? Sorry, just realized that the URL is internal to citrix, not visible to outside. Thanks -min On 5/20/14 1:16 AM, Daan Hoogland daan.hoogl...@gmail.com wrote: Min, On Tue, May 20, 2014 at 1:30 AM, Min Chen min.c...@citrix.com wrote: Hi Daan, As we proposed earlier in community, since IAM feature is not ready for 4.4 release due to backwards compatibility issue, we need to disable IAM on 4.4. I have checked in commit f748a552e9546e91e18c574b375f3ea6c8d7e043 on 4.4-forward to disable IAM, would you please cherry-pick this commit from 4.4-forward to 4.4 branch? It does not cherry-pick cleanly. To avoid causing some regressions due to this disabling, we have run BVT on this commit and got results (http://jenkins-ccp.citrix.com/view/SC_Automation/job/Adv_KVM_BVT_Repo r t/ 370/) is this a public url? I can not reach it. compared to 4.3 BVT baseline. Thanks -min -- Daan -- Daan
Re: [PROPOSAL] Add getPathSeparator() to DataStoreTO.java
The proposal looks ok to me. But it may involve several changes in our
code base, some places in our code we even used hard-coded separator
instead of using constants. With this change, some global cleanup needs to
be done to make sure that it will not bring regressions.
Thanks
-min
On 5/13/14 10:24 AM, Damoder Reddy damoder.re...@citrix.com wrote:
Hi,
While working on windowsfication of management server, I found the issues
when creating a template from a ROOT disk snapshot.
It is not able to create template out of that snapshot due to the
snapshot path we are constructing before creating template is using
³File.Separator² and on windows it would be ³\² instead of ³/². Due to
this the template creation is failing from snapshot when management
server is running on windows.
Code snippet example is
String snapshotInstallPath = snapshot.getPath();
int index = snapshotInstallPath.lastIndexOf(File.separator);
String snapshotName = snapshotInstallPath.substring(index +
1);
if (!snapshotName.startsWith(VHD-)
!snapshotName.endsWith(.vhd)) {
snapshotInstallPath = snapshotInstallPath + .vhd;
}
URI snapshotURI = new URI(secondaryStorageUrl +
File.separator + snapshotInstallPath);
String snapshotPath = snapshotURI.getHost() + : +
snapshotURI.getPath();
To fix this problem I am proposing the following solution.
The path separator is specific to the file system(NFS, samba etc..) we
are using as a secondary storage on SSVM. Instead of using File.separator
in the path, we can add a method say ³getPathSeparator()² to
³DataStoreTO.java² which is implemented by several file systems. And we
will replace ³File.separator² with the above method call at all
respective places.
Anybody see any problems or any other better approach can think of here?
Is this the only place we need to do changes or any other places also
changes are required to achieve this?
Thanks Regards
Damodar/
[ACS4.4] Cherry-pick f748a552e9546e91e18c574b375f3ea6c8d7e043
Hi Daan, As we proposed earlier in community, since IAM feature is not ready for 4.4 release due to backwards compatibility issue, we need to disable IAM on 4.4. I have checked in commit f748a552e9546e91e18c574b375f3ea6c8d7e043 on 4.4-forward to disable IAM, would you please cherry-pick this commit from 4.4-forward to 4.4 branch? To avoid causing some regressions due to this disabling, we have run BVT on this commit and got results (http://jenkins-ccp.citrix.com/view/SC_Automation/job/Adv_KVM_BVT_Report/370/) compared to 4.3 BVT baseline. Thanks -min
[ACS4.4] Cherry-pick 3 bugs
Hi Daan, Would you please cherry-pick the following 3 bug fixes from 4.4-forward to 4.4 branch? Commit: 500c99eef7f7a04d95ed95697a2678fef283e61d CLOUDSTACK-6600:IAM Security checker needs to have cache to improve checkAccess performance. Commit: b9bdaf9d31803cd080475e27da56695c3293c4ca CLOUDSTACK-6598:IAM - listAccount() retrurns Caller cannot be passed as NULL to IAM! when domain deletion is in progress. Commit: e8e0449ed4eaa489d53b43c0137833a660b6d6cf CLOUDSTACK-6613:IAM: authorizeSecurityGroupIngress fails when SG Name is Passed. Thanks -min
Re: [ACS4.4] Cherry-pick CLOUDSTACK-6600
Daan, this commit is already pulled into 4.4 by you. I didn't see trailing white space on my eclipse? Which file are you talking about? Thanks -min On 5/16/14 12:54 AM, Daan Hoogland daan.hoogl...@gmail.com wrote: On Thu, May 8, 2014 at 2:19 AM, Min Chen min.c...@citrix.com wrote: 500c99eef7f7a04d95ed95697a2678fef283e61d Not pulled. Min can you confirm this, it is 8 days old, has trailing white spaces and windows line endings. It seems like something I shouldn't pull in. -- Daan
Re: [ACS4.4] Cherry-pick CLOUDSTACK-6600
Daan, we found the file with windows line endings. It will be fixed in our disabling IAM commit to be checked in next week. Thanks -min On 5/16/14 3:26 PM, Min Chen min.c...@citrix.com wrote: Daan, this commit is already pulled into 4.4 by you. I didn't see trailing white space on my eclipse? Which file are you talking about? Thanks -min On 5/16/14 12:54 AM, Daan Hoogland daan.hoogl...@gmail.com wrote: On Thu, May 8, 2014 at 2:19 AM, Min Chen min.c...@citrix.com wrote: 500c99eef7f7a04d95ed95697a2678fef283e61d Not pulled. Min can you confirm this, it is 8 days old, has trailing white spaces and windows line endings. It seems like something I shouldn't pull in. -- Daan
Re: [ACS5.0] IAM feature postponed from 4.4 to 5.0?
Thanks Daan. Yes, I saw that there is another thread about putting an API request for 5.0 api. Once we are done with this disabling, we will put the issues we have found with current API in that wiki page to take into consideration when we design the new API. -min On 5/14/14 12:12 AM, Daan Hoogland daan.hoogl...@gmail.com wrote: Min, I think everybody knows I am all for less features per release. I don't think you are making a bad call, per se. I do think we should consider if we can come up with a total picture of what 5.x would require af the api, though. Can you add to the discussion what it is that is keeping you from implementing. And what requirements you have for the 5.0 api so we can start devising the architectural guidelines for the new api. more and more calls for a 5.0 are coming up lately so let's move forward. (changing title) On Wed, May 14, 2014 at 1:53 AM, Min Chen min.c...@citrix.com wrote: Hi All, In the past several weeks, QA has done some testing on IAM feature and found several backward-compatibility issues. Even though Prachi and I have tried our best to fix bugs to maintain backward compatibility, we realized that in order to support true IAM model documented in our FS https://cwiki.apache.org/confluence/display/CLOUDSTACK/CloudStack+Identit y+and+Access+Management+%28IAM%29+Plugin, we will have to make several API changes that will require us to increment CloudStack major version. Therefore we think that IAM feature is not ready for ACS 4.4 release, and we would like to propose to disable it in 4.4 branch and re-enable it later when community decides to go for 5.x. Thanks -min -- Daan
[ACS 4.4] Proposal to Disable IAM feature in 4.4
Hi All, In the past several weeks, QA has done some testing on IAM feature and found several backward-compatibility issues. Even though Prachi and I have tried our best to fix bugs to maintain backward compatibility, we realized that in order to support true IAM model documented in our FS https://cwiki.apache.org/confluence/display/CLOUDSTACK/CloudStack+Identity+and+Access+Management+%28IAM%29+Plugin, we will have to make several API changes that will require us to increment CloudStack major version. Therefore we think that IAM feature is not ready for ACS 4.4 release, and we would like to propose to disable it in 4.4 branch and re-enable it later when community decides to go for 5.x. Thanks -min
[ACS 4.4] Proposal to Disable IAM feature in 4.4
Hi All, In the past several weeks, QA has done some testing on IAM feature and found several backward-compatibility issues. Even though Prachi and I have tried our best to fix bugs to maintain backward compatibility, we realized that in order to support true IAM model documented in our FS https://cwiki.apache.org/confluence/display/CLOUDSTACK/CloudStack+Identity+and+Access+Management+%28IAM%29+Plugin, we will have to make several API changes that will require us to increment CloudStack major version. Therefore we think that IAM feature is not ready for ACS 4.4 release, and we would like to propose to disable it in 4.4 branch and re-enable it later when community decides to go for 5.x. Thanks -min
Re: Problem with mailing list....
I had the same issue, my email sent a while back is still not sent to community yet. -min On 5/13/14 4:57 PM, Alex Huang alex.hu...@citrix.com wrote: I've find some of my emails did not make it to the mailing list. Does anyone else have this problem? --Alex
[ACS44] Cherry pick requests
Hi Daan, Can you please cherry pick the following commits from 4.4-forward to 4.4 branch? Some of them are resending previous requests. Commit: 500c99eef7f7a04d95ed95697a2678fef283e61d CLOUDSTACK-6600:IAM Security checker needs to have cache to improve checkAccess performance. Commit:b9bdaf9d31803cd080475e27da56695c3293c4ca CLOUDSTACK-6598:IAM - listAccount() retrurns Caller cannot be passed as NULL to IAM! when domain deletion is in progress. Commit: e8e0449ed4eaa489d53b43c0137833a660b6d6cf CLOUDSTACK-6613:IAM: authorizeSecurityGroupIngress fails when SG Name is passed. Commit: 4889031492e543c318fcd639d7491cfb2013eb76 CLOUDSTACK-6600: fix a bug in IAM cache in constructing cache key. Commit: f447a2c38c15ad5b7844237e72ab551149f46bfb CLOUDSTACK-6617: [Automation] detach / resize volume test cases failing with permission error. Commit: a7dd5aae55c1f6059aea324760f2800537f5d294 CLOUDSTACK-6628:[Automation] Create PF rulw API failing with error database id can only provided by VO objects. Commit: be73571c74084a31819c150dfca2ca28bd13e64b CLOUDSTACK-6628: Fix IpAddress import typo. Thanks. -min
Re: [QUERY] [IAM TEST CASE] Vritual Machine IAM test scenarios
Yes, that is a good point. We can fix test cases to revoke grant at tear down of each test case. Thanks -min On 5/12/14 9:21 PM, Rajani Karuturi rajani.karut...@citrix.com wrote: Assuming the order in which test cases should run is not a good practice. Every test case should do the setup required for it and also the teardown once its done so that it leaves the system clean in the same state it started with. This also helps in easy understanding of what is being tested. I think we should file a bug for this and fix it. ~Rajani On 12-May-2014, at 10:16 pm, Min Chen min.c...@citrix.commailto:min.c...@citrix.com wrote: Hi Sonal, That is because test_03_grant_account_vm is running after test_02_grant_domain_vm, where we have granted domain2 VM to account_1B. At the end of test_02_grant_domain_vm, we didn't revoke this granting, so it will still take effect in test_03_grant_account_vm. Therefore, in test_03, account_1B should have two grants effective, and thus he can see 3 VMs. Thanks -min From: Sonal Ojha sonal.o...@sungardas.commailto:sonal.o...@sungardas.com Date: Monday, May 12, 2014 5:04 AM To: Prachi Damle prachi.da...@citrix.commailto:prachi.da...@citrix.com, Min Chen min.c...@citrix.commailto:min.c...@citrix.com Cc: dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org Subject: [QUERY] [IAM TEST CASE] Vritual Machine IAM test scenarios Hello, I have a small query regarding the sample test case written for virtual machine entity type for the IAM plugin. The test case test_03_grant_account_vm which is to validate the below mentioned scenarios is returning vms for all the three accounts (account_1A, account_1B and account_2A) as the list_vm_response. # Validate the following # 1. Grant account_1A VM access to account_1B # 2. listVM command should return account_1A and account_1B VMs. Shouldn't it be returning vms for the two accounts (account_1A and account_1B)? Kindly help to understand this scenario. -- Thanks and Regards, Sonal Ojha ▪ Sr.Engineer - Product Development ▪ Sungard Availability Services, India 2nd Floor, Wing 4, Cluster D, MIDC Kharadi Knowledge Park, Pune - 411 014 ▪ Office: 267-234-9014 ▪ Mobile: +91 9922412645 ▪ sonal.o...@sungardas.commailto:sonal.o...@sungardas.com ▪ www.sungardas.http://www.sungardas.com/in http://www.sungardas.com/ http://blog.sungardas.com/ http://www.youtube.com/user/SunGardAS https://plus.google.com/u/0/102459878242108588663/posts https://www.facebook.com/SunGardAS http://www.linkedin.com/company/sungard-availability-services https://twitter.com/SunGardAS CONFIDENTIALITY: This e-mail (including any attachments) may contain confidential, proprietary and privileged information, and unauthorized disclosure or use is prohibited. If you received this e-mail in error, please notify the sender and delete this e-mail from your system.
Re: [ACS5.0][DISCUSS][API] revising call structure and other backward incompatabilities
+1 In working IAM feature, we realized that we have very inconsistent interpretations on the same parameters (i.e. listAll, isRecursive, account, domainId, etc) in different list and operational APIs, which render granting almost impossible without breaking some backward compatibility. We really need to start thinking about new APIs. Thanks -min On 5/2/14 11:22 AM, Daan Hoogland daan.hoogl...@gmail.com wrote: People, There are several features in the API that different people want to revise. 1 the calls that use sensitive data using a http-get. When security is needed this hurts and should be disabled (being by configuration or altogether) 2 list* API calls interpret the name parameter in a non-consistent way. some use name = name others use name like '%name%' 3 the way maps and lists are passed is not standard and has different implementations. these are just the examples from the top of my head. There are bound to be others. It seems to me it is time to start work on a definition of how the 5.0 API should look. any takers? -- Daan
[ACS4.4] cherry-pick for CLOUDSTACK-6569
Hi Hugo/Daan, Can you please cherry-pick the following commit from 4.4-forward to 4.4? Commit: 03f6188c1c478222914904e3d7abac2740e98655 CLOUDSTACK-6569: IAM - Regular user is able to listNetworks of another user in the same domain , by passing account and domainId. Thanks -min
[ACS4.4] cherry-pick for CLOUDSTACK-6568
Hi Hugo/Daan, Can you please cherry-pick the following commit from 4.4-forward to 4.4? Commit: ffec3d02844fe644d40f35c55a83719b8d6102b6 CLOUDSTACK-6568:API:createVolume: Volume gets created with wrong domain ID. Thanks -min
Re: [ACS4.4] Cherry pick several IAM bug fixes
Looks good to me, thanks, Daan -min On 5/1/14 1:41 AM, Daan Hoogland daan.hoogl...@gmail.com wrote: min, I put what I did inline. It seems strange to me so please check if the result what you want. On Wed, Apr 30, 2014 at 7:41 PM, Min Chen min.c...@citrix.com wrote: Hi Daan, I have fixed those two commits. Could you please cherry-pick the following commits from 4.4-forward to 4.4 branch/ 1. CLOUDSTACK-6502:IAMGroup.list and IAMPolicy.list in marvin base.py are not working. Commit: 4f2a20f7b35c2a250614b7cf32879d7bb87b7a62 does not apply. not applied. Commit: 42b803dd4b6e01107cbcf08d87f17c47934e8ae7 that one is applied (without the previous). 2. CLOUDSTACK-6513: templateFilter=shared should not show self-owned template. Commit: 3af2f6d1418c66533067b5050d2fe3fd4200beb9 applied Commit: 412793f9b41bb03eda2b8474f9f22a12b5097327 does not apply without the previous. applied regards, -- Daan
[ACS44]cherry pick CLOUDSTACK-6535 and CLOUDSTACK-6556
Hi Hugo/Daan, Would you please cherry pick the following two bug fixes from 4.4-forward to 4.4 branch? 1. CLOUDSTACK-6535: IAM:MS:API createVMSnapshot doesn't preserve access rights. Commit: f0edfc7ac5a8c3c22cea3e90a2a77835967234a2 2. CLOUDSTACK-6556: [Automation] Deploy VM failing with error does not have permission to access resource Ntwk. This includes two commits: Commit: 4415aee1e9f7b0b788ff1e0a6a7fe744fd37921f Commit: 6435b261deb0160f774a8815f0024e38f49d45e9 Thanks -min
Re: [ACS4.4] Cherry pick several IAM bug fixes
Thanks Daan. I am addressing those two commits now. -min On 4/30/14 1:42 AM, Daan Hoogland daan.hoogl...@gmail.com wrote: Min, see inline On Wed, Apr 30, 2014 at 7:15 AM, Min Chen min.c...@citrix.com wrote: Hi Hugo/Daan, Would you please cherry-pick the following IAM bug fixes from 4.4-forward branch to 4.4 branch? Commit : 4f2a20f7b35c2a250614b7cf32879d7bb87b7a62 this contains some trailing spaces in the altered lines, can you please amend? CLOUDSTACK-6502:IAMGroup.list and IAMPolicy.list in marvin base.py are not working. Commit: 092b4be8d91e2e63aea12e1d40aa264e144e6d84 done CLOUDSTACK-6512:IAM - Not able to list shared networks in the Vm deployment flow Commit: a4cb8d11462cb404eba067541639d46e509d717b done CLOUDSTACK-6513: IAM - Templates - When templates are listed with templatefilter=shared is used , we see public templates also being included in the list. Commit: b2b59ed83a566762c960371717b7998b4719ba70 done CLOUDSTACK-6533: IAM - Templates - Public templates do not have permissions to be used by ROOT group. Commit: 0767060b1f4cf96db721bd625971345fbda4c48c done CLOUDSTACK-6532:Affinity Groups - As admin user, not able to list all affinity groups available for regular users by passing account and domainId paramater. Commit:7309146f9f3f57b7f7cad40945e11e34a6ff6fab done CLOUDSTACK-6513: Optimize code by removing deprecated utility to QueryManagerImpl as private method just used for listTemplates and listAffinityGroups to avoid misuse by new list APIs. Commit: 3af2f6d1418c66533067b5050d2fe3fd4200beb9 contains code in comment, can you please delete? CLOUDSTACK-6513: templateFilter=shared should not show self-owned template. Thanks. -min regards -- Daan
Re: [ACS4.4] Cherry pick several IAM bug fixes
Hi Daan, I have fixed those two commits. Could you please cherry-pick the following commits from 4.4-forward to 4.4 branch/ 1. CLOUDSTACK-6502:IAMGroup.list and IAMPolicy.list in marvin base.py are not working. Commit: 4f2a20f7b35c2a250614b7cf32879d7bb87b7a62 Commit: 42b803dd4b6e01107cbcf08d87f17c47934e8ae7 2. CLOUDSTACK-6513: templateFilter=shared should not show self-owned template. Commit: 3af2f6d1418c66533067b5050d2fe3fd4200beb9 Commit: 412793f9b41bb03eda2b8474f9f22a12b5097327 Thanks -min On 4/30/14 9:39 AM, Min Chen min.c...@citrix.com wrote: Thanks Daan. I am addressing those two commits now. -min On 4/30/14 1:42 AM, Daan Hoogland daan.hoogl...@gmail.com wrote: Min, see inline On Wed, Apr 30, 2014 at 7:15 AM, Min Chen min.c...@citrix.com wrote: Hi Hugo/Daan, Would you please cherry-pick the following IAM bug fixes from 4.4-forward branch to 4.4 branch? Commit : 4f2a20f7b35c2a250614b7cf32879d7bb87b7a62 this contains some trailing spaces in the altered lines, can you please amend? CLOUDSTACK-6502:IAMGroup.list and IAMPolicy.list in marvin base.py are not working. Commit: 092b4be8d91e2e63aea12e1d40aa264e144e6d84 done CLOUDSTACK-6512:IAM - Not able to list shared networks in the Vm deployment flow Commit: a4cb8d11462cb404eba067541639d46e509d717b done CLOUDSTACK-6513: IAM - Templates - When templates are listed with templatefilter=shared is used , we see public templates also being included in the list. Commit: b2b59ed83a566762c960371717b7998b4719ba70 done CLOUDSTACK-6533: IAM - Templates - Public templates do not have permissions to be used by ROOT group. Commit: 0767060b1f4cf96db721bd625971345fbda4c48c done CLOUDSTACK-6532:Affinity Groups - As admin user, not able to list all affinity groups available for regular users by passing account and domainId paramater. Commit:7309146f9f3f57b7f7cad40945e11e34a6ff6fab done CLOUDSTACK-6513: Optimize code by removing deprecated utility to QueryManagerImpl as private method just used for listTemplates and listAffinityGroups to avoid misuse by new list APIs. Commit: 3af2f6d1418c66533067b5050d2fe3fd4200beb9 contains code in comment, can you please delete? CLOUDSTACK-6513: templateFilter=shared should not show self-owned template. Thanks. -min regards -- Daan
