Re: NFS Cache storage query
Edison, As I understand Chip's concern, if we can't safely disassociate a staging area from a zone, we will break zone deletion. My understanding of your 4.2 proposal is that the system administrator/operation can place the staging area in maintenance mod How would this address disassociating from the zone to allow deletion? It feels like the shortest path would be to support detaching a staging area from a zone. It also seems like behavior we would want to support going forward. Also, what would it mean to put a secondary store in maintenance mode? My understanding is that we haven't worked out the functionality of secondary storage maintenance mode. Also, why don't we define a detach methods adjoining each of the attach methods? Thanks, -John On Jun 19, 2013, at 3:11 PM, Edison Su edison...@citrix.com wrote: -Original Message- From: John Burwell [mailto:jburw...@basho.com] Sent: Wednesday, June 19, 2013 11:43 AM To: Edison Su Cc: dev@cloudstack.apache.org Subject: Re: NFS Cache storage query Edison, Based on the stance you have outlined, the usage of NFS in the object_store branch and 4.1 are not comparable. In 4.1, NFS is the store of record for data. Therefore, presence of the file in the NFS volume indicates that the data is permanently stored. However, in object_store, presence in NFS in the object_store branch means that the data may be awaiting permanent stage (dependent on the type of pending transfer operation). As such, I think we will need to provide insurance that in-flight operations are completed before a staging/temporary area is destroyed. Another option I can see is to change the way these staging/temporary areas are associated with zones. If we approached them as standalone entities that are attached or detached from a zone, we could define the detach operation as only disassociation from a zone without impacting in-flight operations. This solution would allow zones to be deleted without impacting in-flight operations. The interface is there: https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;a=blob;f=engine/api/src/org/apache/cloudstack/engine/subsystem/api/storage/DataStoreLifeCycle.java;h=1e893db6bb5b1dbae0142e8a26019ae34d44320a;hb=refs/heads/object_store Admin should be able to put the data store into maintenance mode, and then delete it, but the implementation is not there yet for both NFS secondary storage and staging area. For NFS, S3 secondary storage, we should at least implement maintenance/cancelMaintain in 4.2 For NFS staging area, we should at least implement maintenance/cancelMaintain in 4.2, the delete method in 4.3. How do you think? Thanks, -John On Jun 19, 2013, at 2:15 PM, Edison Su edison...@citrix.com wrote: Yes and No:) Yes, as all the hypervisors(KVM/Vmware/Xenserver) still need NFS in 4.2, even S3 is used as the place to store templates. No, we make NFS is optional, if you don't want to use NFS. E.g the HyperV implementation will not depend on NFS. In 4.3, we can start work on the hypervisor side refactor, to eliminate the dependence on NFS as much as possible, then we may can truly make the statement that, S3 will be the secondary storage. -Original Message- From: John Burwell [mailto:jburw...@basho.com] Sent: Wednesday, June 19, 2013 11:00 AM To: Edison Su Cc: dev@cloudstack.apache.org Subject: Re: NFS Cache storage query Edison, For 4.2, are we going to state that the object store is just a backup of NFS (i.e. the same as 4.1)? Thanks, -John On Jun 19, 2013, at 1:58 PM, Edison Su edison...@citrix.com wrote: -Original Message- From: John Burwell [mailto:jburw...@basho.com] Sent: Wednesday, June 19, 2013 10:42 AM To: dev@cloudstack.apache.org Cc: Edison Su Subject: Re: NFS Cache storage query Chip, Your concern had not occurred to me -- making me realize that either destroy or a zone attach/detach operation for the staging/temporary area mechanism in 4.2. Thinking through it, there are two types of operations occurring with the staging/temporary area. The first is data being pulled from the object store to support some activity (e.g. copying a template down to create a VM). From a data integrity perspective, it is safe to kill these operations since the data has already been persisted to the object store. The second is data being pushed to the object store which are much more problematic. Of particular concern would be snapshots that are in the staging/temporary area, but not yet transferred to the object store. Edison/Min: Does the current implementation provide a destroy or attach/detach behavior? If so, how are in-flight operations handled to ensure there is no data loss? The current mater branch, there is no such operation for secondary storage yet, so does the object_store branch. Maybe we can discuss/implement a better life cycle management of both nfs
RE: NFS Cache storage query - an UI change where you can create a NFS Cache Storage independently
Sanjeev, I've made an UI change where you can create a NFS Cache Storage independently. After getting latest code from master branch, you'll see a new dropdown Select view in Infrastructure menu Secondary Storage. Change Select view to Cache Storage, then you'll see Add NFS Cache Storage button on right top corner. Jessica -Original Message- From: Sanjeev Neelarapu [mailto:sanjeev.neelar...@citrix.com] Sent: Friday, June 14, 2013 6:07 AM To: dev@cloudstack.apache.org Subject: NFS Cache storage query Hi, I have a query on how to add NFS Cache storage. Before creating a zone if we create a secondary storage with s3 as the storage provider and don't select NFS Cache Storage then we treat it as S3 at region level. Later I create a zone and at add secondary storage creation wizard in UI if I select NFS as secondary storage provider will it be treated as NFS Cache Storage? If not is there a way to add NFS cache storage for that zone? Thanks, Sanjeev
Re: NFS Cache storage query
On Wed, Jun 19, 2013 at 07:11:19PM +, Edison Su wrote: -Original Message- From: John Burwell [mailto:jburw...@basho.com] Sent: Wednesday, June 19, 2013 11:43 AM To: Edison Su Cc: dev@cloudstack.apache.org Subject: Re: NFS Cache storage query Edison, Based on the stance you have outlined, the usage of NFS in the object_store branch and 4.1 are not comparable. In 4.1, NFS is the store of record for data. Therefore, presence of the file in the NFS volume indicates that the data is permanently stored. However, in object_store, presence in NFS in the object_store branch means that the data may be awaiting permanent stage (dependent on the type of pending transfer operation). As such, I think we will need to provide insurance that in-flight operations are completed before a staging/temporary area is destroyed. Another option I can see is to change the way these staging/temporary areas are associated with zones. If we approached them as standalone entities that are attached or detached from a zone, we could define the detach operation as only disassociation from a zone without impacting in-flight operations. This solution would allow zones to be deleted without impacting in-flight operations. The interface is there: https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;a=blob;f=engine/api/src/org/apache/cloudstack/engine/subsystem/api/storage/DataStoreLifeCycle.java;h=1e893db6bb5b1dbae0142e8a26019ae34d44320a;hb=refs/heads/object_store Admin should be able to put the data store into maintenance mode, and then delete it, but the implementation is not there yet for both NFS secondary storage and staging area. For NFS, S3 secondary storage, we should at least implement maintenance/cancelMaintain in 4.2 For NFS staging area, we should at least implement maintenance/cancelMaintain in 4.2, the delete method in 4.3. How do you think? Seems reasonable to me.
Re: NFS Cache storage query
Agree with Min. How about CRUD operations on staging - all apis available ? Thanks, -Nitin On 19/06/13 6:44 AM, Min Chen min.c...@citrix.com wrote: Edison, we can just add a parameter in listImageStoreCmd to show cache store. Thanks -min On 6/18/13 6:06 PM, Edison Su edison...@citrix.com wrote: Currently, no, I can add an API for it. -Original Message- From: Jessica Wang [mailto:jessica.w...@citrix.com] Sent: Tuesday, June 18, 2013 4:24 PM To: Min Chen Cc: dev@cloudstack.apache.org Subject: RE: NFS Cache storage query Min, We may need to provide a way from UI to allow users to configure and display their NFS cache. Is there an API that list NFS cache? Jessica -Original Message- From: Min Chen Sent: Friday, June 14, 2013 9:26 AM To: dev@cloudstack.apache.org Cc: Jessica Wang Subject: Re: NFS Cache storage query Hi Sanjeev, In 4.2 release, we require that a NFS cache storage has to be added if you choose S3 as the storage provider since we haven't refactored hypervisor side code to handle s3 directly by bypassing NFS caching, which is the goal for 4.3 release. I see an issue with current UI, where user can only add cache storage when he/she adds a S3 storage. We may need to provide a way from UI to allow users to configure and display their NFS cache. You can file a JIRA ticket for this UI enhancement. Thanks -min On 6/14/13 6:35 AM, Chip Childers chip.child...@sungard.com wrote: On Fri, Jun 14, 2013 at 01:06:30PM +, Sanjeev Neelarapu wrote: Hi, I have a query on how to add NFS Cache storage. Before creating a zone if we create a secondary storage with s3 as the storage provider and don't select NFS Cache Storage then we treat it as S3 at region level. Later I create a zone and at add secondary storage creation wizard in UI if I select NFS as secondary storage provider will it be treated as NFS Cache Storage? If not is there a way to add NFS cache storage for that zone? Thanks, Sanjeev Based on the thread talking about this [1], I'm not sure that it will be implemented this way anymore. -chip [1] http://markmail.org/message/c73nagj45q6iktfh
RE: NFS Cache storage query
Min, Edison, Either way is fine with me. Jessica -Original Message- From: Min Chen [mailto:min.c...@citrix.com] Sent: Tuesday, June 18, 2013 6:07 PM To: Edison Su; dev@cloudstack.apache.org Subject: Re: NFS Cache storage query Edison, we can just add a parameter in listImageStoreCmd to show cache store. Thanks -min On 6/18/13 6:06 PM, Edison Su edison...@citrix.com wrote: Currently, no, I can add an API for it. -Original Message- From: Jessica Wang [mailto:jessica.w...@citrix.com] Sent: Tuesday, June 18, 2013 4:24 PM To: Min Chen Cc: dev@cloudstack.apache.org Subject: RE: NFS Cache storage query Min, We may need to provide a way from UI to allow users to configure and display their NFS cache. Is there an API that list NFS cache? Jessica -Original Message- From: Min Chen Sent: Friday, June 14, 2013 9:26 AM To: dev@cloudstack.apache.org Cc: Jessica Wang Subject: Re: NFS Cache storage query Hi Sanjeev, In 4.2 release, we require that a NFS cache storage has to be added if you choose S3 as the storage provider since we haven't refactored hypervisor side code to handle s3 directly by bypassing NFS caching, which is the goal for 4.3 release. I see an issue with current UI, where user can only add cache storage when he/she adds a S3 storage. We may need to provide a way from UI to allow users to configure and display their NFS cache. You can file a JIRA ticket for this UI enhancement. Thanks -min On 6/14/13 6:35 AM, Chip Childers chip.child...@sungard.com wrote: On Fri, Jun 14, 2013 at 01:06:30PM +, Sanjeev Neelarapu wrote: Hi, I have a query on how to add NFS Cache storage. Before creating a zone if we create a secondary storage with s3 as the storage provider and don't select NFS Cache Storage then we treat it as S3 at region level. Later I create a zone and at add secondary storage creation wizard in UI if I select NFS as secondary storage provider will it be treated as NFS Cache Storage? If not is there a way to add NFS cache storage for that zone? Thanks, Sanjeev Based on the thread talking about this [1], I'm not sure that it will be implemented this way anymore. -chip [1] http://markmail.org/message/c73nagj45q6iktfh
Re: NFS Cache storage query
Nitin, If we provide any APIs for the staging/temporary area, they must be read-only. Allowing any external manipulation of its content could cause break in-flight transfers or cause data loss. I think we should consider the following APIs: List contents including name, reference count, and size Summary statistics for a staging area including consumed/available/total space and timestamp of the last garbage collection operation Force garbage collection/cleanup operation I think we should these are new API calls specific to the staging/temporary area mechanism rather than trying to overload existing API calls. Thanks, -John On Jun 19, 2013, at 5:48 AM, Nitin Mehta nitin.me...@citrix.com wrote: Agree with Min. How about CRUD operations on staging - all apis available ? Thanks, -Nitin On 19/06/13 6:44 AM, Min Chen min.c...@citrix.com wrote: Edison, we can just add a parameter in listImageStoreCmd to show cache store. Thanks -min On 6/18/13 6:06 PM, Edison Su edison...@citrix.com wrote: Currently, no, I can add an API for it. -Original Message- From: Jessica Wang [mailto:jessica.w...@citrix.com] Sent: Tuesday, June 18, 2013 4:24 PM To: Min Chen Cc: dev@cloudstack.apache.org Subject: RE: NFS Cache storage query Min, We may need to provide a way from UI to allow users to configure and display their NFS cache. Is there an API that list NFS cache? Jessica -Original Message- From: Min Chen Sent: Friday, June 14, 2013 9:26 AM To: dev@cloudstack.apache.org Cc: Jessica Wang Subject: Re: NFS Cache storage query Hi Sanjeev, In 4.2 release, we require that a NFS cache storage has to be added if you choose S3 as the storage provider since we haven't refactored hypervisor side code to handle s3 directly by bypassing NFS caching, which is the goal for 4.3 release. I see an issue with current UI, where user can only add cache storage when he/she adds a S3 storage. We may need to provide a way from UI to allow users to configure and display their NFS cache. You can file a JIRA ticket for this UI enhancement. Thanks -min On 6/14/13 6:35 AM, Chip Childers chip.child...@sungard.com wrote: On Fri, Jun 14, 2013 at 01:06:30PM +, Sanjeev Neelarapu wrote: Hi, I have a query on how to add NFS Cache storage. Before creating a zone if we create a secondary storage with s3 as the storage provider and don't select NFS Cache Storage then we treat it as S3 at region level. Later I create a zone and at add secondary storage creation wizard in UI if I select NFS as secondary storage provider will it be treated as NFS Cache Storage? If not is there a way to add NFS cache storage for that zone? Thanks, Sanjeev Based on the thread talking about this [1], I'm not sure that it will be implemented this way anymore. -chip [1] http://markmail.org/message/c73nagj45q6iktfh
Re: NFS Cache storage query
On Wed, Jun 19, 2013 at 01:07:29PM -0400, John Burwell wrote: Nitin, If we provide any APIs for the staging/temporary area, they must be read-only. Allowing any external manipulation of its content could cause break in-flight transfers or cause data loss. I think we should consider the following APIs: List contents including name, reference count, and size Summary statistics for a staging area including consumed/available/total space and timestamp of the last garbage collection operation Force garbage collection/cleanup operation I think we should these are new API calls specific to the staging/temporary area mechanism rather than trying to overload existing API calls. Thanks, -John What about creating / destroying them?
Re: NFS Cache storage query
Chip, Good catch. Yes, we need definitely need a create staging/temporary area API call. However, destroy is a bit more complicated due in-flight operations. Given the complexities and time pressures, I recommend supporting only create in 4.2, and addressing delete in 4.3. Does that make sense? Thanks, -John On Jun 19, 2013, at 1:11 PM, Chip Childers chip.child...@sungard.com wrote: On Wed, Jun 19, 2013 at 01:07:29PM -0400, John Burwell wrote: Nitin, If we provide any APIs for the staging/temporary area, they must be read-only. Allowing any external manipulation of its content could cause break in-flight transfers or cause data loss. I think we should consider the following APIs: List contents including name, reference count, and size Summary statistics for a staging area including consumed/available/total space and timestamp of the last garbage collection operation Force garbage collection/cleanup operation I think we should these are new API calls specific to the staging/temporary area mechanism rather than trying to overload existing API calls. Thanks, -John What about creating / destroying them?
Re: NFS Cache storage query
On Wed, Jun 19, 2013 at 01:23:47PM -0400, John Burwell wrote: Chip, Good catch. Yes, we need definitely need a create staging/temporary area API call. However, destroy is a bit more complicated due in-flight operations. Given the complexities and time pressures, I recommend supporting only create in 4.2, and addressing delete in 4.3. Does that make sense? If the existence of the staging datastore blocks the deleting of a zone, or any other entity, then that doesn't work for me. I'd rather give an operator the ability to decide how to best ensure that in-flight operations are halted (i.e.: block users from the environment or something else), than not give them a way to change their configuration. Thanks, -John On Jun 19, 2013, at 1:11 PM, Chip Childers chip.child...@sungard.com wrote: On Wed, Jun 19, 2013 at 01:07:29PM -0400, John Burwell wrote: Nitin, If we provide any APIs for the staging/temporary area, they must be read-only. Allowing any external manipulation of its content could cause break in-flight transfers or cause data loss. I think we should consider the following APIs: List contents including name, reference count, and size Summary statistics for a staging area including consumed/available/total space and timestamp of the last garbage collection operation Force garbage collection/cleanup operation I think we should these are new API calls specific to the staging/temporary area mechanism rather than trying to overload existing API calls. Thanks, -John What about creating / destroying them?
Re: NFS Cache storage query
Chip, Your concern had not occurred to me -- making me realize that either destroy or a zone attach/detach operation for the staging/temporary area mechanism in 4.2. Thinking through it, there are two types of operations occurring with the staging/temporary area. The first is data being pulled from the object store to support some activity (e.g. copying a template down to create a VM). From a data integrity perspective, it is safe to kill these operations since the data has already been persisted to the object store. The second is data being pushed to the object store which are much more problematic. Of particular concern would be snapshots that are in the staging/temporary area, but not yet transferred to the object store. Edison/Min: Does the current implementation provide a destroy or attach/detach behavior? If so, how are in-flight operations handled to ensure there is no data loss? Thanks, -John On Jun 19, 2013, at 1:26 PM, Chip Childers chip.child...@sungard.com wrote: On Wed, Jun 19, 2013 at 01:23:47PM -0400, John Burwell wrote: Chip, Good catch. Yes, we need definitely need a create staging/temporary area API call. However, destroy is a bit more complicated due in-flight operations. Given the complexities and time pressures, I recommend supporting only create in 4.2, and addressing delete in 4.3. Does that make sense? If the existence of the staging datastore blocks the deleting of a zone, or any other entity, then that doesn't work for me. I'd rather give an operator the ability to decide how to best ensure that in-flight operations are halted (i.e.: block users from the environment or something else), than not give them a way to change their configuration. Thanks, -John On Jun 19, 2013, at 1:11 PM, Chip Childers chip.child...@sungard.com wrote: On Wed, Jun 19, 2013 at 01:07:29PM -0400, John Burwell wrote: Nitin, If we provide any APIs for the staging/temporary area, they must be read-only. Allowing any external manipulation of its content could cause break in-flight transfers or cause data loss. I think we should consider the following APIs: List contents including name, reference count, and size Summary statistics for a staging area including consumed/available/total space and timestamp of the last garbage collection operation Force garbage collection/cleanup operation I think we should these are new API calls specific to the staging/temporary area mechanism rather than trying to overload existing API calls. Thanks, -John What about creating / destroying them?
RE: NFS Cache storage query
-Original Message- From: John Burwell [mailto:jburw...@basho.com] Sent: Wednesday, June 19, 2013 10:42 AM To: dev@cloudstack.apache.org Cc: Edison Su Subject: Re: NFS Cache storage query Chip, Your concern had not occurred to me -- making me realize that either destroy or a zone attach/detach operation for the staging/temporary area mechanism in 4.2. Thinking through it, there are two types of operations occurring with the staging/temporary area. The first is data being pulled from the object store to support some activity (e.g. copying a template down to create a VM). From a data integrity perspective, it is safe to kill these operations since the data has already been persisted to the object store. The second is data being pushed to the object store which are much more problematic. Of particular concern would be snapshots that are in the staging/temporary area, but not yet transferred to the object store. Edison/Min: Does the current implementation provide a destroy or attach/detach behavior? If so, how are in-flight operations handled to ensure there is no data loss? The current mater branch, there is no such operation for secondary storage yet, so does the object_store branch. Maybe we can discuss/implement a better life cycle management of both nfs secondary storage and staging area in collab next week. Thanks, -John On Jun 19, 2013, at 1:26 PM, Chip Childers chip.child...@sungard.com wrote: On Wed, Jun 19, 2013 at 01:23:47PM -0400, John Burwell wrote: Chip, Good catch. Yes, we need definitely need a create staging/temporary area API call. However, destroy is a bit more complicated due in-flight operations. Given the complexities and time pressures, I recommend supporting only create in 4.2, and addressing delete in 4.3. Does that make sense? If the existence of the staging datastore blocks the deleting of a zone, or any other entity, then that doesn't work for me. I'd rather give an operator the ability to decide how to best ensure that in-flight operations are halted (i.e.: block users from the environment or something else), than not give them a way to change their configuration. Thanks, -John On Jun 19, 2013, at 1:11 PM, Chip Childers chip.child...@sungard.com wrote: On Wed, Jun 19, 2013 at 01:07:29PM -0400, John Burwell wrote: Nitin, If we provide any APIs for the staging/temporary area, they must be read-only. Allowing any external manipulation of its content could cause break in-flight transfers or cause data loss. I think we should consider the following APIs: List contents including name, reference count, and size Summary statistics for a staging area including consumed/available/total space and timestamp of the last garbage collection operation Force garbage collection/cleanup operation I think we should these are new API calls specific to the staging/temporary area mechanism rather than trying to overload existing API calls. Thanks, -John What about creating / destroying them?
Re: NFS Cache storage query
Edison, For 4.2, are we going to state that the object store is just a backup of NFS (i.e. the same as 4.1)? Thanks, -John On Jun 19, 2013, at 1:58 PM, Edison Su edison...@citrix.com wrote: -Original Message- From: John Burwell [mailto:jburw...@basho.com] Sent: Wednesday, June 19, 2013 10:42 AM To: dev@cloudstack.apache.org Cc: Edison Su Subject: Re: NFS Cache storage query Chip, Your concern had not occurred to me -- making me realize that either destroy or a zone attach/detach operation for the staging/temporary area mechanism in 4.2. Thinking through it, there are two types of operations occurring with the staging/temporary area. The first is data being pulled from the object store to support some activity (e.g. copying a template down to create a VM). From a data integrity perspective, it is safe to kill these operations since the data has already been persisted to the object store. The second is data being pushed to the object store which are much more problematic. Of particular concern would be snapshots that are in the staging/temporary area, but not yet transferred to the object store. Edison/Min: Does the current implementation provide a destroy or attach/detach behavior? If so, how are in-flight operations handled to ensure there is no data loss? The current mater branch, there is no such operation for secondary storage yet, so does the object_store branch. Maybe we can discuss/implement a better life cycle management of both nfs secondary storage and staging area in collab next week. Thanks, -John On Jun 19, 2013, at 1:26 PM, Chip Childers chip.child...@sungard.com wrote: On Wed, Jun 19, 2013 at 01:23:47PM -0400, John Burwell wrote: Chip, Good catch. Yes, we need definitely need a create staging/temporary area API call. However, destroy is a bit more complicated due in-flight operations. Given the complexities and time pressures, I recommend supporting only create in 4.2, and addressing delete in 4.3. Does that make sense? If the existence of the staging datastore blocks the deleting of a zone, or any other entity, then that doesn't work for me. I'd rather give an operator the ability to decide how to best ensure that in-flight operations are halted (i.e.: block users from the environment or something else), than not give them a way to change their configuration. Thanks, -John On Jun 19, 2013, at 1:11 PM, Chip Childers chip.child...@sungard.com wrote: On Wed, Jun 19, 2013 at 01:07:29PM -0400, John Burwell wrote: Nitin, If we provide any APIs for the staging/temporary area, they must be read-only. Allowing any external manipulation of its content could cause break in-flight transfers or cause data loss. I think we should consider the following APIs: List contents including name, reference count, and size Summary statistics for a staging area including consumed/available/total space and timestamp of the last garbage collection operation Force garbage collection/cleanup operation I think we should these are new API calls specific to the staging/temporary area mechanism rather than trying to overload existing API calls. Thanks, -John What about creating / destroying them?
RE: NFS Cache storage query
Yes and No:) Yes, as all the hypervisors(KVM/Vmware/Xenserver) still need NFS in 4.2, even S3 is used as the place to store templates. No, we make NFS is optional, if you don't want to use NFS. E.g the HyperV implementation will not depend on NFS. In 4.3, we can start work on the hypervisor side refactor, to eliminate the dependence on NFS as much as possible, then we may can truly make the statement that, S3 will be the secondary storage. -Original Message- From: John Burwell [mailto:jburw...@basho.com] Sent: Wednesday, June 19, 2013 11:00 AM To: Edison Su Cc: dev@cloudstack.apache.org Subject: Re: NFS Cache storage query Edison, For 4.2, are we going to state that the object store is just a backup of NFS (i.e. the same as 4.1)? Thanks, -John On Jun 19, 2013, at 1:58 PM, Edison Su edison...@citrix.com wrote: -Original Message- From: John Burwell [mailto:jburw...@basho.com] Sent: Wednesday, June 19, 2013 10:42 AM To: dev@cloudstack.apache.org Cc: Edison Su Subject: Re: NFS Cache storage query Chip, Your concern had not occurred to me -- making me realize that either destroy or a zone attach/detach operation for the staging/temporary area mechanism in 4.2. Thinking through it, there are two types of operations occurring with the staging/temporary area. The first is data being pulled from the object store to support some activity (e.g. copying a template down to create a VM). From a data integrity perspective, it is safe to kill these operations since the data has already been persisted to the object store. The second is data being pushed to the object store which are much more problematic. Of particular concern would be snapshots that are in the staging/temporary area, but not yet transferred to the object store. Edison/Min: Does the current implementation provide a destroy or attach/detach behavior? If so, how are in-flight operations handled to ensure there is no data loss? The current mater branch, there is no such operation for secondary storage yet, so does the object_store branch. Maybe we can discuss/implement a better life cycle management of both nfs secondary storage and staging area in collab next week. Thanks, -John On Jun 19, 2013, at 1:26 PM, Chip Childers chip.child...@sungard.com wrote: On Wed, Jun 19, 2013 at 01:23:47PM -0400, John Burwell wrote: Chip, Good catch. Yes, we need definitely need a create staging/temporary area API call. However, destroy is a bit more complicated due in-flight operations. Given the complexities and time pressures, I recommend supporting only create in 4.2, and addressing delete in 4.3. Does that make sense? If the existence of the staging datastore blocks the deleting of a zone, or any other entity, then that doesn't work for me. I'd rather give an operator the ability to decide how to best ensure that in-flight operations are halted (i.e.: block users from the environment or something else), than not give them a way to change their configuration. Thanks, -John On Jun 19, 2013, at 1:11 PM, Chip Childers chip.child...@sungard.com wrote: On Wed, Jun 19, 2013 at 01:07:29PM -0400, John Burwell wrote: Nitin, If we provide any APIs for the staging/temporary area, they must be read-only. Allowing any external manipulation of its content could cause break in-flight transfers or cause data loss. I think we should consider the following APIs: List contents including name, reference count, and size Summary statistics for a staging area including consumed/available/total space and timestamp of the last garbage collection operation Force garbage collection/cleanup operation I think we should these are new API calls specific to the staging/temporary area mechanism rather than trying to overload existing API calls. Thanks, -John What about creating / destroying them?
RE: NFS Cache storage query
-Original Message- From: John Burwell [mailto:jburw...@basho.com] Sent: Wednesday, June 19, 2013 11:43 AM To: Edison Su Cc: dev@cloudstack.apache.org Subject: Re: NFS Cache storage query Edison, Based on the stance you have outlined, the usage of NFS in the object_store branch and 4.1 are not comparable. In 4.1, NFS is the store of record for data. Therefore, presence of the file in the NFS volume indicates that the data is permanently stored. However, in object_store, presence in NFS in the object_store branch means that the data may be awaiting permanent stage (dependent on the type of pending transfer operation). As such, I think we will need to provide insurance that in-flight operations are completed before a staging/temporary area is destroyed. Another option I can see is to change the way these staging/temporary areas are associated with zones. If we approached them as standalone entities that are attached or detached from a zone, we could define the detach operation as only disassociation from a zone without impacting in-flight operations. This solution would allow zones to be deleted without impacting in-flight operations. The interface is there: https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;a=blob;f=engine/api/src/org/apache/cloudstack/engine/subsystem/api/storage/DataStoreLifeCycle.java;h=1e893db6bb5b1dbae0142e8a26019ae34d44320a;hb=refs/heads/object_store Admin should be able to put the data store into maintenance mode, and then delete it, but the implementation is not there yet for both NFS secondary storage and staging area. For NFS, S3 secondary storage, we should at least implement maintenance/cancelMaintain in 4.2 For NFS staging area, we should at least implement maintenance/cancelMaintain in 4.2, the delete method in 4.3. How do you think? Thanks, -John On Jun 19, 2013, at 2:15 PM, Edison Su edison...@citrix.com wrote: Yes and No:) Yes, as all the hypervisors(KVM/Vmware/Xenserver) still need NFS in 4.2, even S3 is used as the place to store templates. No, we make NFS is optional, if you don't want to use NFS. E.g the HyperV implementation will not depend on NFS. In 4.3, we can start work on the hypervisor side refactor, to eliminate the dependence on NFS as much as possible, then we may can truly make the statement that, S3 will be the secondary storage. -Original Message- From: John Burwell [mailto:jburw...@basho.com] Sent: Wednesday, June 19, 2013 11:00 AM To: Edison Su Cc: dev@cloudstack.apache.org Subject: Re: NFS Cache storage query Edison, For 4.2, are we going to state that the object store is just a backup of NFS (i.e. the same as 4.1)? Thanks, -John On Jun 19, 2013, at 1:58 PM, Edison Su edison...@citrix.com wrote: -Original Message- From: John Burwell [mailto:jburw...@basho.com] Sent: Wednesday, June 19, 2013 10:42 AM To: dev@cloudstack.apache.org Cc: Edison Su Subject: Re: NFS Cache storage query Chip, Your concern had not occurred to me -- making me realize that either destroy or a zone attach/detach operation for the staging/temporary area mechanism in 4.2. Thinking through it, there are two types of operations occurring with the staging/temporary area. The first is data being pulled from the object store to support some activity (e.g. copying a template down to create a VM). From a data integrity perspective, it is safe to kill these operations since the data has already been persisted to the object store. The second is data being pushed to the object store which are much more problematic. Of particular concern would be snapshots that are in the staging/temporary area, but not yet transferred to the object store. Edison/Min: Does the current implementation provide a destroy or attach/detach behavior? If so, how are in-flight operations handled to ensure there is no data loss? The current mater branch, there is no such operation for secondary storage yet, so does the object_store branch. Maybe we can discuss/implement a better life cycle management of both nfs secondary storage and staging area in collab next week. Thanks, -John On Jun 19, 2013, at 1:26 PM, Chip Childers chip.child...@sungard.com wrote: On Wed, Jun 19, 2013 at 01:23:47PM -0400, John Burwell wrote: Chip, Good catch. Yes, we need definitely need a create staging/temporary area API call. However, destroy is a bit more complicated due in-flight operations. Given the complexities and time pressures, I recommend supporting only create in 4.2, and addressing delete in 4.3. Does that make sense? If the existence of the staging datastore blocks the deleting of a zone, or any other entity, then that doesn't work for me. I'd rather give an operator the ability to decide how to best ensure that in-flight operations are halted (i.e.: block users from
RE: NFS Cache storage query
Min, We may need to provide a way from UI to allow users to configure and display their NFS cache. Is there an API that list NFS cache? Jessica -Original Message- From: Min Chen Sent: Friday, June 14, 2013 9:26 AM To: dev@cloudstack.apache.org Cc: Jessica Wang Subject: Re: NFS Cache storage query Hi Sanjeev, In 4.2 release, we require that a NFS cache storage has to be added if you choose S3 as the storage provider since we haven't refactored hypervisor side code to handle s3 directly by bypassing NFS caching, which is the goal for 4.3 release. I see an issue with current UI, where user can only add cache storage when he/she adds a S3 storage. We may need to provide a way from UI to allow users to configure and display their NFS cache. You can file a JIRA ticket for this UI enhancement. Thanks -min On 6/14/13 6:35 AM, Chip Childers chip.child...@sungard.com wrote: On Fri, Jun 14, 2013 at 01:06:30PM +, Sanjeev Neelarapu wrote: Hi, I have a query on how to add NFS Cache storage. Before creating a zone if we create a secondary storage with s3 as the storage provider and don't select NFS Cache Storage then we treat it as S3 at region level. Later I create a zone and at add secondary storage creation wizard in UI if I select NFS as secondary storage provider will it be treated as NFS Cache Storage? If not is there a way to add NFS cache storage for that zone? Thanks, Sanjeev Based on the thread talking about this [1], I'm not sure that it will be implemented this way anymore. -chip [1] http://markmail.org/message/c73nagj45q6iktfh
RE: NFS Cache storage query
Currently, no, I can add an API for it. -Original Message- From: Jessica Wang [mailto:jessica.w...@citrix.com] Sent: Tuesday, June 18, 2013 4:24 PM To: Min Chen Cc: dev@cloudstack.apache.org Subject: RE: NFS Cache storage query Min, We may need to provide a way from UI to allow users to configure and display their NFS cache. Is there an API that list NFS cache? Jessica -Original Message- From: Min Chen Sent: Friday, June 14, 2013 9:26 AM To: dev@cloudstack.apache.org Cc: Jessica Wang Subject: Re: NFS Cache storage query Hi Sanjeev, In 4.2 release, we require that a NFS cache storage has to be added if you choose S3 as the storage provider since we haven't refactored hypervisor side code to handle s3 directly by bypassing NFS caching, which is the goal for 4.3 release. I see an issue with current UI, where user can only add cache storage when he/she adds a S3 storage. We may need to provide a way from UI to allow users to configure and display their NFS cache. You can file a JIRA ticket for this UI enhancement. Thanks -min On 6/14/13 6:35 AM, Chip Childers chip.child...@sungard.com wrote: On Fri, Jun 14, 2013 at 01:06:30PM +, Sanjeev Neelarapu wrote: Hi, I have a query on how to add NFS Cache storage. Before creating a zone if we create a secondary storage with s3 as the storage provider and don't select NFS Cache Storage then we treat it as S3 at region level. Later I create a zone and at add secondary storage creation wizard in UI if I select NFS as secondary storage provider will it be treated as NFS Cache Storage? If not is there a way to add NFS cache storage for that zone? Thanks, Sanjeev Based on the thread talking about this [1], I'm not sure that it will be implemented this way anymore. -chip [1] http://markmail.org/message/c73nagj45q6iktfh
Re: NFS Cache storage query
On Fri, Jun 14, 2013 at 01:06:30PM +, Sanjeev Neelarapu wrote: Hi, I have a query on how to add NFS Cache storage. Before creating a zone if we create a secondary storage with s3 as the storage provider and don't select NFS Cache Storage then we treat it as S3 at region level. Later I create a zone and at add secondary storage creation wizard in UI if I select NFS as secondary storage provider will it be treated as NFS Cache Storage? If not is there a way to add NFS cache storage for that zone? Thanks, Sanjeev Based on the thread talking about this [1], I'm not sure that it will be implemented this way anymore. -chip [1] http://markmail.org/message/c73nagj45q6iktfh
Re: NFS Cache storage query
Hi Sanjeev, In 4.2 release, we require that a NFS cache storage has to be added if you choose S3 as the storage provider since we haven't refactored hypervisor side code to handle s3 directly by bypassing NFS caching, which is the goal for 4.3 release. I see an issue with current UI, where user can only add cache storage when he/she adds a S3 storage. We may need to provide a way from UI to allow users to configure and display their NFS cache. You can file a JIRA ticket for this UI enhancement. Thanks -min On 6/14/13 6:35 AM, Chip Childers chip.child...@sungard.com wrote: On Fri, Jun 14, 2013 at 01:06:30PM +, Sanjeev Neelarapu wrote: Hi, I have a query on how to add NFS Cache storage. Before creating a zone if we create a secondary storage with s3 as the storage provider and don't select NFS Cache Storage then we treat it as S3 at region level. Later I create a zone and at add secondary storage creation wizard in UI if I select NFS as secondary storage provider will it be treated as NFS Cache Storage? If not is there a way to add NFS cache storage for that zone? Thanks, Sanjeev Based on the thread talking about this [1], I'm not sure that it will be implemented this way anymore. -chip [1] http://markmail.org/message/c73nagj45q6iktfh
