RE: NFS Cache storage query - an UI change where you can create a NFS Cache Storage independently
Sanjeev, I've made an UI change where you can create a NFS Cache Storage independently. After getting latest code from master branch, you'll see a new dropdown "Select view" in Infrastructure menu > Secondary Storage. Change "Select view" to "Cache Storage", then you'll see "Add NFS Cache Storage" button on right top corner. Jessica -Original Message- From: Sanjeev Neelarapu [mailto:sanjeev.neelar...@citrix.com] Sent: Friday, June 14, 2013 6:07 AM To: dev@cloudstack.apache.org Subject: NFS Cache storage query Hi, I have a query on how to add NFS Cache storage. Before creating a zone if we create a secondary storage with s3 as the storage provider and don't select NFS Cache Storage then we treat it as S3 at region level. Later I create a zone and at "add secondary storage" creation wizard in UI if I select NFS as secondary storage provider will it be treated as NFS Cache Storage? If not is there a way to add NFS cache storage for that zone? Thanks, Sanjeev
Re: NFS Cache storage query
Edison, As I understand Chip's concern, if we can't safely disassociate a staging area from a zone, we will break zone deletion. My understanding of your 4.2 proposal is that the system administrator/operation can place the staging area in maintenance mod How would this address disassociating from the zone to allow deletion? It feels like the shortest path would be to support detaching a staging area from a zone. It also seems like behavior we would want to support going forward. Also, what would it mean to put a secondary store in maintenance mode? My understanding is that we haven't worked out the functionality of secondary storage maintenance mode. Also, why don't we define a detach methods adjoining each of the attach methods? Thanks, -John On Jun 19, 2013, at 3:11 PM, Edison Su wrote: > > >> -Original Message- >> From: John Burwell [mailto:jburw...@basho.com] >> Sent: Wednesday, June 19, 2013 11:43 AM >> To: Edison Su >> Cc: dev@cloudstack.apache.org >> Subject: Re: NFS Cache storage query >> >> Edison, >> >> Based on the stance you have outlined, the usage of NFS in the object_store >> branch and 4.1 are not comparable. In 4.1, NFS is the store of record for >> data. >> Therefore, presence of the file in the NFS volume indicates that the data is >> permanently stored. However, in object_store, presence in NFS in the >> object_store branch means that the data may be awaiting permanent stage >> (dependent on the type of pending transfer operation). As such, I think we >> will need to provide insurance that in-flight operations are completed before >> a staging/temporary area is destroyed. Another option I can see is to change >> the way these staging/temporary areas are associated with zones. If we >> approached them as standalone entities that are attached or detached from >> a zone, we could define the detach operation as only disassociation from a >> zone without impacting in-flight operations. This solution would allow zones >> to be deleted without impacting in-flight operations. > > The interface is there: > https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;a=blob;f=engine/api/src/org/apache/cloudstack/engine/subsystem/api/storage/DataStoreLifeCycle.java;h=1e893db6bb5b1dbae0142e8a26019ae34d44320a;hb=refs/heads/object_store > Admin should be able to put the data store into maintenance mode, and then > delete it, but the implementation is not there yet for both NFS secondary > storage and staging area. > For NFS, S3 secondary storage, we should at least implement > maintenance/cancelMaintain in 4.2 > For NFS staging area, we should at least implement maintenance/cancelMaintain > in 4.2, the delete method in 4.3. > How do you think? > > >> >> Thanks, >> -John >> >> On Jun 19, 2013, at 2:15 PM, Edison Su wrote: >> >>> Yes and No:) >>> Yes, as all the hypervisors(KVM/Vmware/Xenserver) still need NFS in 4.2, >> even S3 is used as the place to store templates. >>> No, we make NFS is optional, if you don't want to use NFS. E.g the HyperV >> implementation will not depend on NFS. >>> >>> In 4.3, we can start work on the hypervisor side refactor, to eliminate the >> dependence on NFS as much as possible, then we may can truly make the >> statement that, S3 will be the "secondary storage". >>> >>>> -Original Message- >>>> From: John Burwell [mailto:jburw...@basho.com] >>>> Sent: Wednesday, June 19, 2013 11:00 AM >>>> To: Edison Su >>>> Cc: dev@cloudstack.apache.org >>>> Subject: Re: NFS Cache storage query >>>> >>>> Edison, >>>> >>>> For 4.2, are we going to state that the object store is just a backup of >>>> NFS >> (i.e. >>>> the same as 4.1)? >>>> >>>> Thanks, >>>> -John >>>> >>>> On Jun 19, 2013, at 1:58 PM, Edison Su wrote: >>>> >>>>> >>>>> >>>>>> -Original Message- >>>>>> From: John Burwell [mailto:jburw...@basho.com] >>>>>> Sent: Wednesday, June 19, 2013 10:42 AM >>>>>> To: dev@cloudstack.apache.org >>>>>> Cc: Edison Su >>>>>> Subject: Re: NFS Cache storage query >>>>>> >>>>>> Chip, >>>>>> >>>>>> Your concern had not occurred to me -- making me realize that >>>>>> either destroy or a zone attach/detach operation for the >>>>>
Re: NFS Cache storage query
On Wed, Jun 19, 2013 at 07:11:19PM +, Edison Su wrote: > > > > -Original Message- > > From: John Burwell [mailto:jburw...@basho.com] > > Sent: Wednesday, June 19, 2013 11:43 AM > > To: Edison Su > > Cc: dev@cloudstack.apache.org > > Subject: Re: NFS Cache storage query > > > > Edison, > > > > Based on the stance you have outlined, the usage of NFS in the object_store > > branch and 4.1 are not comparable. In 4.1, NFS is the store of record for > > data. > > Therefore, presence of the file in the NFS volume indicates that the data is > > permanently stored. However, in object_store, presence in NFS in the > > object_store branch means that the data may be awaiting permanent stage > > (dependent on the type of pending transfer operation). As such, I think we > > will need to provide insurance that in-flight operations are completed > > before > > a staging/temporary area is destroyed. Another option I can see is to > > change > > the way these staging/temporary areas are associated with zones. If we > > approached them as standalone entities that are attached or detached from > > a zone, we could define the detach operation as only disassociation from a > > zone without impacting in-flight operations. This solution would allow > > zones > > to be deleted without impacting in-flight operations. > > The interface is there: > https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;a=blob;f=engine/api/src/org/apache/cloudstack/engine/subsystem/api/storage/DataStoreLifeCycle.java;h=1e893db6bb5b1dbae0142e8a26019ae34d44320a;hb=refs/heads/object_store > Admin should be able to put the data store into maintenance mode, and then > delete it, but the implementation is not there yet for both NFS secondary > storage and staging area. > For NFS, S3 secondary storage, we should at least implement > maintenance/cancelMaintain in 4.2 > For NFS staging area, we should at least implement maintenance/cancelMaintain > in 4.2, the delete method in 4.3. > How do you think? Seems reasonable to me.
RE: NFS Cache storage query
> -Original Message- > From: John Burwell [mailto:jburw...@basho.com] > Sent: Wednesday, June 19, 2013 11:43 AM > To: Edison Su > Cc: dev@cloudstack.apache.org > Subject: Re: NFS Cache storage query > > Edison, > > Based on the stance you have outlined, the usage of NFS in the object_store > branch and 4.1 are not comparable. In 4.1, NFS is the store of record for > data. > Therefore, presence of the file in the NFS volume indicates that the data is > permanently stored. However, in object_store, presence in NFS in the > object_store branch means that the data may be awaiting permanent stage > (dependent on the type of pending transfer operation). As such, I think we > will need to provide insurance that in-flight operations are completed before > a staging/temporary area is destroyed. Another option I can see is to change > the way these staging/temporary areas are associated with zones. If we > approached them as standalone entities that are attached or detached from > a zone, we could define the detach operation as only disassociation from a > zone without impacting in-flight operations. This solution would allow zones > to be deleted without impacting in-flight operations. The interface is there: https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;a=blob;f=engine/api/src/org/apache/cloudstack/engine/subsystem/api/storage/DataStoreLifeCycle.java;h=1e893db6bb5b1dbae0142e8a26019ae34d44320a;hb=refs/heads/object_store Admin should be able to put the data store into maintenance mode, and then delete it, but the implementation is not there yet for both NFS secondary storage and staging area. For NFS, S3 secondary storage, we should at least implement maintenance/cancelMaintain in 4.2 For NFS staging area, we should at least implement maintenance/cancelMaintain in 4.2, the delete method in 4.3. How do you think? > > Thanks, > -John > > On Jun 19, 2013, at 2:15 PM, Edison Su wrote: > > > Yes and No:) > > Yes, as all the hypervisors(KVM/Vmware/Xenserver) still need NFS in 4.2, > even S3 is used as the place to store templates. > > No, we make NFS is optional, if you don't want to use NFS. E.g the HyperV > implementation will not depend on NFS. > > > > In 4.3, we can start work on the hypervisor side refactor, to eliminate the > dependence on NFS as much as possible, then we may can truly make the > statement that, S3 will be the "secondary storage". > > > >> -Original Message- > >> From: John Burwell [mailto:jburw...@basho.com] > >> Sent: Wednesday, June 19, 2013 11:00 AM > >> To: Edison Su > >> Cc: dev@cloudstack.apache.org > >> Subject: Re: NFS Cache storage query > >> > >> Edison, > >> > >> For 4.2, are we going to state that the object store is just a backup of > >> NFS > (i.e. > >> the same as 4.1)? > >> > >> Thanks, > >> -John > >> > >> On Jun 19, 2013, at 1:58 PM, Edison Su wrote: > >> > >>> > >>> > >>>> -Original Message- > >>>> From: John Burwell [mailto:jburw...@basho.com] > >>>> Sent: Wednesday, June 19, 2013 10:42 AM > >>>> To: dev@cloudstack.apache.org > >>>> Cc: Edison Su > >>>> Subject: Re: NFS Cache storage query > >>>> > >>>> Chip, > >>>> > >>>> Your concern had not occurred to me -- making me realize that > >>>> either destroy or a zone attach/detach operation for the > >>>> staging/temporary area mechanism in 4.2. Thinking through it, > >>>> there are two types of operations occurring with the > >>>> staging/temporary area. The first is data being pulled from the > >>>> object store to support some activity (e.g. copying a template down > >>>> to create a VM). From a data integrity perspective, it is safe to > >>>> kill these operations since the data has already > >> been persisted to the object store. > >>>> The second is data being pushed to the object store which are much > >>>> more problematic. Of particular concern would be snapshots that > >>>> are in the staging/temporary area, but not yet transferred to the object > store. > >>>> > >>>> Edison/Min: Does the current implementation provide a destroy or > >>>> attach/detach behavior? If so, how are in-flight operations > >>>> handled to ensure there is no data loss? > >>> > >>> The current mater branch, there is no such operation for secondary > >>
Re: NFS Cache storage query
Edison, Based on the stance you have outlined, the usage of NFS in the object_store branch and 4.1 are not comparable. In 4.1, NFS is the store of record for data. Therefore, presence of the file in the NFS volume indicates that the data is permanently stored. However, in object_store, presence in NFS in the object_store branch means that the data may be awaiting permanent stage (dependent on the type of pending transfer operation). As such, I think we will need to provide insurance that in-flight operations are completed before a staging/temporary area is destroyed. Another option I can see is to change the way these staging/temporary areas are associated with zones. If we approached them as standalone entities that are attached or detached from a zone, we could define the detach operation as only disassociation from a zone without impacting in-flight operations. This solution would allow zones to be deleted without impacting in-flight operations. Thanks, -John On Jun 19, 2013, at 2:15 PM, Edison Su wrote: > Yes and No:) > Yes, as all the hypervisors(KVM/Vmware/Xenserver) still need NFS in 4.2, even > S3 is used as the place to store templates. > No, we make NFS is optional, if you don't want to use NFS. E.g the HyperV > implementation will not depend on NFS. > > In 4.3, we can start work on the hypervisor side refactor, to eliminate the > dependence on NFS as much as possible, then we may can truly make the > statement that, S3 will be the "secondary storage". > >> -Original Message- >> From: John Burwell [mailto:jburw...@basho.com] >> Sent: Wednesday, June 19, 2013 11:00 AM >> To: Edison Su >> Cc: dev@cloudstack.apache.org >> Subject: Re: NFS Cache storage query >> >> Edison, >> >> For 4.2, are we going to state that the object store is just a backup of NFS >> (i.e. >> the same as 4.1)? >> >> Thanks, >> -John >> >> On Jun 19, 2013, at 1:58 PM, Edison Su wrote: >> >>> >>> >>>> -Original Message----- >>>> From: John Burwell [mailto:jburw...@basho.com] >>>> Sent: Wednesday, June 19, 2013 10:42 AM >>>> To: dev@cloudstack.apache.org >>>> Cc: Edison Su >>>> Subject: Re: NFS Cache storage query >>>> >>>> Chip, >>>> >>>> Your concern had not occurred to me -- making me realize that either >>>> destroy or a zone attach/detach operation for the staging/temporary >>>> area mechanism in 4.2. Thinking through it, there are two types of >>>> operations occurring with the staging/temporary area. The first is >>>> data being pulled from the object store to support some activity >>>> (e.g. copying a template down to create a VM). From a data integrity >>>> perspective, it is safe to kill these operations since the data has already >> been persisted to the object store. >>>> The second is data being pushed to the object store which are much >>>> more problematic. Of particular concern would be snapshots that are >>>> in the staging/temporary area, but not yet transferred to the object store. >>>> >>>> Edison/Min: Does the current implementation provide a destroy or >>>> attach/detach behavior? If so, how are in-flight operations handled >>>> to ensure there is no data loss? >>> >>> The current mater branch, there is no such operation for secondary storage >> yet, so does the object_store branch. >>> Maybe we can discuss/implement a better life cycle management of both >> nfs secondary storage and staging area in collab next week. >>> >>>> >>>> Thanks, >>>> -John >>>> >>>> On Jun 19, 2013, at 1:26 PM, Chip Childers >>>> >>>> wrote: >>>> >>>>> On Wed, Jun 19, 2013 at 01:23:47PM -0400, John Burwell wrote: >>>>>> Chip, >>>>>> >>>>>> Good catch. Yes, we need definitely need a create >>>>>> staging/temporary >>>> area API call. However, destroy is a bit more complicated due >>>> in-flight operations. Given the complexities and time pressures, I >>>> recommend supporting only create in 4.2, and addressing delete in >>>> 4.3. Does that make sense? >>>>>> >>>>> >>>>> If the existence of the staging datastore blocks the deleting of a >>>>> zone, or any other entity, then that doesn't work for me. >>>>> >>>>> I'd rather give an ope
RE: NFS Cache storage query
Yes and No:) Yes, as all the hypervisors(KVM/Vmware/Xenserver) still need NFS in 4.2, even S3 is used as the place to store templates. No, we make NFS is optional, if you don't want to use NFS. E.g the HyperV implementation will not depend on NFS. In 4.3, we can start work on the hypervisor side refactor, to eliminate the dependence on NFS as much as possible, then we may can truly make the statement that, S3 will be the "secondary storage". > -Original Message- > From: John Burwell [mailto:jburw...@basho.com] > Sent: Wednesday, June 19, 2013 11:00 AM > To: Edison Su > Cc: dev@cloudstack.apache.org > Subject: Re: NFS Cache storage query > > Edison, > > For 4.2, are we going to state that the object store is just a backup of NFS > (i.e. > the same as 4.1)? > > Thanks, > -John > > On Jun 19, 2013, at 1:58 PM, Edison Su wrote: > > > > > > >> -Original Message- > >> From: John Burwell [mailto:jburw...@basho.com] > >> Sent: Wednesday, June 19, 2013 10:42 AM > >> To: dev@cloudstack.apache.org > >> Cc: Edison Su > >> Subject: Re: NFS Cache storage query > >> > >> Chip, > >> > >> Your concern had not occurred to me -- making me realize that either > >> destroy or a zone attach/detach operation for the staging/temporary > >> area mechanism in 4.2. Thinking through it, there are two types of > >> operations occurring with the staging/temporary area. The first is > >> data being pulled from the object store to support some activity > >> (e.g. copying a template down to create a VM). From a data integrity > >> perspective, it is safe to kill these operations since the data has already > been persisted to the object store. > >> The second is data being pushed to the object store which are much > >> more problematic. Of particular concern would be snapshots that are > >> in the staging/temporary area, but not yet transferred to the object store. > >> > >> Edison/Min: Does the current implementation provide a destroy or > >> attach/detach behavior? If so, how are in-flight operations handled > >> to ensure there is no data loss? > > > > The current mater branch, there is no such operation for secondary storage > yet, so does the object_store branch. > > Maybe we can discuss/implement a better life cycle management of both > nfs secondary storage and staging area in collab next week. > > > >> > >> Thanks, > >> -John > >> > >> On Jun 19, 2013, at 1:26 PM, Chip Childers > >> > >> wrote: > >> > >>> On Wed, Jun 19, 2013 at 01:23:47PM -0400, John Burwell wrote: > >>>> Chip, > >>>> > >>>> Good catch. Yes, we need definitely need a create > >>>> staging/temporary > >> area API call. However, destroy is a bit more complicated due > >> in-flight operations. Given the complexities and time pressures, I > >> recommend supporting only create in 4.2, and addressing delete in > >> 4.3. Does that make sense? > >>>> > >>> > >>> If the existence of the staging datastore blocks the deleting of a > >>> zone, or any other entity, then that doesn't work for me. > >>> > >>> I'd rather give an operator the ability to decide how to best ensure > >>> that in-flight operations are halted (i.e.: block users from the > >>> environment or something else), than not give them a way to change > >>> their configuration. > >>> > >>>> Thanks, > >>>> -John > >>>> > >>>> On Jun 19, 2013, at 1:11 PM, Chip Childers > >>>> > >> wrote: > >>>> > >>>>> On Wed, Jun 19, 2013 at 01:07:29PM -0400, John Burwell wrote: > >>>>>> Nitin, > >>>>>> > >>>>>> If we provide any APIs for the staging/temporary area, they must > >>>>>> be > >> read-only. Allowing any external manipulation of its content could > >> cause break in-flight transfers or cause data loss. I think we > >> should consider the following APIs: > >>>>>> > >>>>>> List contents including name, reference count, and size Summary > >>>>>> statistics for a staging area including consumed/available/total > >>>>>> space and timestamp of the last garbage collection operation > >>>>>> Force garbage collection/cleanup operation > >>>>>> > >>>>>> I think we should these are new API calls specific to the > >> staging/temporary area mechanism rather than trying to overload > >> existing API calls. > >>>>>> > >>>>>> Thanks, > >>>>>> -John > >>>>> > >>>>> What about creating / destroying them? > >>>> > >>>> > >
Re: NFS Cache storage query
Edison, For 4.2, are we going to state that the object store is just a backup of NFS (i.e. the same as 4.1)? Thanks, -John On Jun 19, 2013, at 1:58 PM, Edison Su wrote: > > >> -Original Message- >> From: John Burwell [mailto:jburw...@basho.com] >> Sent: Wednesday, June 19, 2013 10:42 AM >> To: dev@cloudstack.apache.org >> Cc: Edison Su >> Subject: Re: NFS Cache storage query >> >> Chip, >> >> Your concern had not occurred to me -- making me realize that either destroy >> or a zone attach/detach operation for the staging/temporary area >> mechanism in 4.2. Thinking through it, there are two types of operations >> occurring with the staging/temporary area. The first is data being pulled >> from >> the object store to support some activity (e.g. copying a template down to >> create a VM). From a data integrity perspective, it is safe to kill these >> operations since the data has already been persisted to the object store. >> The second is data being pushed to the object store which are much more >> problematic. Of particular concern would be snapshots that are in the >> staging/temporary area, but not yet transferred to the object store. >> >> Edison/Min: Does the current implementation provide a destroy or >> attach/detach behavior? If so, how are in-flight operations handled to >> ensure there is no data loss? > > The current mater branch, there is no such operation for secondary storage > yet, so does the object_store branch. > Maybe we can discuss/implement a better life cycle management of both nfs > secondary storage and staging area in collab next week. > >> >> Thanks, >> -John >> >> On Jun 19, 2013, at 1:26 PM, Chip Childers >> wrote: >> >>> On Wed, Jun 19, 2013 at 01:23:47PM -0400, John Burwell wrote: >>>> Chip, >>>> >>>> Good catch. Yes, we need definitely need a create staging/temporary >> area API call. However, destroy is a bit more complicated due in-flight >> operations. Given the complexities and time pressures, I recommend >> supporting only create in 4.2, and addressing delete in 4.3. Does that make >> sense? >>>> >>> >>> If the existence of the staging datastore blocks the deleting of a >>> zone, or any other entity, then that doesn't work for me. >>> >>> I'd rather give an operator the ability to decide how to best ensure >>> that in-flight operations are halted (i.e.: block users from the >>> environment or something else), than not give them a way to change >>> their configuration. >>> >>>> Thanks, >>>> -John >>>> >>>> On Jun 19, 2013, at 1:11 PM, Chip Childers >> wrote: >>>> >>>>> On Wed, Jun 19, 2013 at 01:07:29PM -0400, John Burwell wrote: >>>>>> Nitin, >>>>>> >>>>>> If we provide any APIs for the staging/temporary area, they must be >> read-only. Allowing any external manipulation of its content could cause >> break in-flight transfers or cause data loss. I think we should consider the >> following APIs: >>>>>> >>>>>> List contents including name, reference count, and size Summary >>>>>> statistics for a staging area including consumed/available/total >>>>>> space and timestamp of the last garbage collection operation Force >>>>>> garbage collection/cleanup operation >>>>>> >>>>>> I think we should these are new API calls specific to the >> staging/temporary area mechanism rather than trying to overload existing >> API calls. >>>>>> >>>>>> Thanks, >>>>>> -John >>>>> >>>>> What about creating / destroying them? >>>> >>>> >
RE: NFS Cache storage query
> -Original Message- > From: John Burwell [mailto:jburw...@basho.com] > Sent: Wednesday, June 19, 2013 10:42 AM > To: dev@cloudstack.apache.org > Cc: Edison Su > Subject: Re: NFS Cache storage query > > Chip, > > Your concern had not occurred to me -- making me realize that either destroy > or a zone attach/detach operation for the staging/temporary area > mechanism in 4.2. Thinking through it, there are two types of operations > occurring with the staging/temporary area. The first is data being pulled > from > the object store to support some activity (e.g. copying a template down to > create a VM). From a data integrity perspective, it is safe to kill these > operations since the data has already been persisted to the object store. > The second is data being pushed to the object store which are much more > problematic. Of particular concern would be snapshots that are in the > staging/temporary area, but not yet transferred to the object store. > > Edison/Min: Does the current implementation provide a destroy or > attach/detach behavior? If so, how are in-flight operations handled to > ensure there is no data loss? The current mater branch, there is no such operation for secondary storage yet, so does the object_store branch. Maybe we can discuss/implement a better life cycle management of both nfs secondary storage and staging area in collab next week. > > Thanks, > -John > > On Jun 19, 2013, at 1:26 PM, Chip Childers > wrote: > > > On Wed, Jun 19, 2013 at 01:23:47PM -0400, John Burwell wrote: > >> Chip, > >> > >> Good catch. Yes, we need definitely need a create staging/temporary > area API call. However, destroy is a bit more complicated due in-flight > operations. Given the complexities and time pressures, I recommend > supporting only create in 4.2, and addressing delete in 4.3. Does that make > sense? > >> > > > > If the existence of the staging datastore blocks the deleting of a > > zone, or any other entity, then that doesn't work for me. > > > > I'd rather give an operator the ability to decide how to best ensure > > that in-flight operations are halted (i.e.: block users from the > > environment or something else), than not give them a way to change > > their configuration. > > > >> Thanks, > >> -John > >> > >> On Jun 19, 2013, at 1:11 PM, Chip Childers > wrote: > >> > >>> On Wed, Jun 19, 2013 at 01:07:29PM -0400, John Burwell wrote: > >>>> Nitin, > >>>> > >>>> If we provide any APIs for the staging/temporary area, they must be > read-only. Allowing any external manipulation of its content could cause > break in-flight transfers or cause data loss. I think we should consider the > following APIs: > >>>> > >>>> List contents including name, reference count, and size Summary > >>>> statistics for a staging area including consumed/available/total > >>>> space and timestamp of the last garbage collection operation Force > >>>> garbage collection/cleanup operation > >>>> > >>>> I think we should these are new API calls specific to the > staging/temporary area mechanism rather than trying to overload existing > API calls. > >>>> > >>>> Thanks, > >>>> -John > >>> > >>> What about creating / destroying them? > >> > >>
Re: NFS Cache storage query
Chip, Your concern had not occurred to me -- making me realize that either destroy or a zone attach/detach operation for the staging/temporary area mechanism in 4.2. Thinking through it, there are two types of operations occurring with the staging/temporary area. The first is data being pulled from the object store to support some activity (e.g. copying a template down to create a VM). From a data integrity perspective, it is safe to kill these operations since the data has already been persisted to the object store. The second is data being pushed to the object store which are much more problematic. Of particular concern would be snapshots that are in the staging/temporary area, but not yet transferred to the object store. Edison/Min: Does the current implementation provide a destroy or attach/detach behavior? If so, how are in-flight operations handled to ensure there is no data loss? Thanks, -John On Jun 19, 2013, at 1:26 PM, Chip Childers wrote: > On Wed, Jun 19, 2013 at 01:23:47PM -0400, John Burwell wrote: >> Chip, >> >> Good catch. Yes, we need definitely need a create staging/temporary area >> API call. However, destroy is a bit more complicated due in-flight >> operations. Given the complexities and time pressures, I recommend >> supporting only create in 4.2, and addressing delete in 4.3. Does that make >> sense? >> > > If the existence of the staging datastore blocks the deleting of a > zone, or any other entity, then that doesn't work for me. > > I'd rather give an operator the ability to decide how to best ensure > that in-flight operations are halted (i.e.: block users from the > environment or something else), than not give them a way to change their > configuration. > >> Thanks, >> -John >> >> On Jun 19, 2013, at 1:11 PM, Chip Childers wrote: >> >>> On Wed, Jun 19, 2013 at 01:07:29PM -0400, John Burwell wrote: Nitin, If we provide any APIs for the staging/temporary area, they must be read-only. Allowing any external manipulation of its content could cause break in-flight transfers or cause data loss. I think we should consider the following APIs: List contents including name, reference count, and size Summary statistics for a staging area including consumed/available/total space and timestamp of the last garbage collection operation Force garbage collection/cleanup operation I think we should these are new API calls specific to the staging/temporary area mechanism rather than trying to overload existing API calls. Thanks, -John >>> >>> What about creating / destroying them? >> >>
Re: NFS Cache storage query
On Wed, Jun 19, 2013 at 01:23:47PM -0400, John Burwell wrote: > Chip, > > Good catch. Yes, we need definitely need a create staging/temporary area API > call. However, destroy is a bit more complicated due in-flight operations. > Given the complexities and time pressures, I recommend supporting only create > in 4.2, and addressing delete in 4.3. Does that make sense? > If the existence of the staging datastore blocks the deleting of a zone, or any other entity, then that doesn't work for me. I'd rather give an operator the ability to decide how to best ensure that in-flight operations are halted (i.e.: block users from the environment or something else), than not give them a way to change their configuration. > Thanks, > -John > > On Jun 19, 2013, at 1:11 PM, Chip Childers wrote: > > > On Wed, Jun 19, 2013 at 01:07:29PM -0400, John Burwell wrote: > >> Nitin, > >> > >> If we provide any APIs for the staging/temporary area, they must be > >> read-only. Allowing any external manipulation of its content could cause > >> break in-flight transfers or cause data loss. I think we should consider > >> the following APIs: > >> > >> List contents including name, reference count, and size > >> Summary statistics for a staging area including consumed/available/total > >> space and timestamp of the last garbage collection operation > >> Force garbage collection/cleanup operation > >> > >> I think we should these are new API calls specific to the > >> staging/temporary area mechanism rather than trying to overload existing > >> API calls. > >> > >> Thanks, > >> -John > > > > What about creating / destroying them? > >
Re: NFS Cache storage query
Chip, Good catch. Yes, we need definitely need a create staging/temporary area API call. However, destroy is a bit more complicated due in-flight operations. Given the complexities and time pressures, I recommend supporting only create in 4.2, and addressing delete in 4.3. Does that make sense? Thanks, -John On Jun 19, 2013, at 1:11 PM, Chip Childers wrote: > On Wed, Jun 19, 2013 at 01:07:29PM -0400, John Burwell wrote: >> Nitin, >> >> If we provide any APIs for the staging/temporary area, they must be >> read-only. Allowing any external manipulation of its content could cause >> break in-flight transfers or cause data loss. I think we should consider >> the following APIs: >> >> List contents including name, reference count, and size >> Summary statistics for a staging area including consumed/available/total >> space and timestamp of the last garbage collection operation >> Force garbage collection/cleanup operation >> >> I think we should these are new API calls specific to the staging/temporary >> area mechanism rather than trying to overload existing API calls. >> >> Thanks, >> -John > > What about creating / destroying them?
Re: NFS Cache storage query
On Wed, Jun 19, 2013 at 01:07:29PM -0400, John Burwell wrote: > Nitin, > > If we provide any APIs for the staging/temporary area, they must be > read-only. Allowing any external manipulation of its content could cause > break in-flight transfers or cause data loss. I think we should consider the > following APIs: > > List contents including name, reference count, and size > Summary statistics for a staging area including consumed/available/total > space and timestamp of the last garbage collection operation > Force garbage collection/cleanup operation > > I think we should these are new API calls specific to the staging/temporary > area mechanism rather than trying to overload existing API calls. > > Thanks, > -John What about creating / destroying them?
Re: NFS Cache storage query
Nitin, If we provide any APIs for the staging/temporary area, they must be read-only. Allowing any external manipulation of its content could cause break in-flight transfers or cause data loss. I think we should consider the following APIs: List contents including name, reference count, and size Summary statistics for a staging area including consumed/available/total space and timestamp of the last garbage collection operation Force garbage collection/cleanup operation I think we should these are new API calls specific to the staging/temporary area mechanism rather than trying to overload existing API calls. Thanks, -John On Jun 19, 2013, at 5:48 AM, Nitin Mehta wrote: > Agree with Min. How about CRUD operations on staging - all apis available ? > > Thanks, > -Nitin > > On 19/06/13 6:44 AM, "Min Chen" wrote: > >> Edison, we can just add a parameter in listImageStoreCmd to show cache >> store. >> >> Thanks >> -min >> >> On 6/18/13 6:06 PM, "Edison Su" wrote: >> >>> Currently, no, I can add an API for it. >>> >>>> -Original Message- >>>> From: Jessica Wang [mailto:jessica.w...@citrix.com] >>>> Sent: Tuesday, June 18, 2013 4:24 PM >>>> To: Min Chen >>>> Cc: dev@cloudstack.apache.org >>>> Subject: RE: NFS Cache storage query >>>> >>>> Min, >>>> >>>>> We may need to provide a way from UI to allow users to configure and >>>> display their NFS cache. >>>> Is there an API that list NFS cache? >>>> >>>> Jessica >>>> >>>> -Original Message- >>>> From: Min Chen >>>> Sent: Friday, June 14, 2013 9:26 AM >>>> To: dev@cloudstack.apache.org >>>> Cc: Jessica Wang >>>> Subject: Re: NFS Cache storage query >>>> >>>> Hi Sanjeev, >>>> >>>>In 4.2 release, we require that a NFS cache storage has to be added if >>>> you choose S3 as the storage provider since we haven't refactored >>>> hypervisor side code to handle s3 directly by bypassing NFS caching, >>>> which is >>>> the goal for 4.3 release. I see an issue with current UI, where user >>>> can only >>>> add cache storage when he/she adds a S3 storage. We may need to provide >>>> a way from UI to allow users to configure and display their NFS cache. >>>> You >>>> can file a JIRA ticket for this UI enhancement. >>>> >>>>Thanks >>>>-min >>>> >>>> On 6/14/13 6:35 AM, "Chip Childers" wrote: >>>> >>>>> On Fri, Jun 14, 2013 at 01:06:30PM +, Sanjeev Neelarapu wrote: >>>>>> Hi, >>>>>> >>>>>> I have a query on how to add NFS Cache storage. >>>>>> Before creating a zone if we create a secondary storage with s3 as >>>>>> the storage provider and don't select NFS Cache Storage then we treat >>>>>> it as >>>>>> S3 at region level. >>>>>> Later I create a zone and at "add secondary storage" creation wizard >>>>>> in UI if I select NFS as secondary storage provider will it be >>>> treated >>>>>> as NFS Cache Storage? If not is there a way to add NFS cache storage >>>>>> for that zone? >>>>>> >>>>>> Thanks, >>>>>> Sanjeev >>>>>> >>>>> >>>>> Based on the thread talking about this [1], I'm not sure that it will >>>>> be implemented this way anymore. >>>>> >>>>> -chip >>>>> >>>>> [1] http://markmail.org/message/c73nagj45q6iktfh >>> >> >
RE: NFS Cache storage query
Min, Edison, Either way is fine with me. Jessica -Original Message- From: Min Chen [mailto:min.c...@citrix.com] Sent: Tuesday, June 18, 2013 6:07 PM To: Edison Su; dev@cloudstack.apache.org Subject: Re: NFS Cache storage query Edison, we can just add a parameter in listImageStoreCmd to show cache store. Thanks -min On 6/18/13 6:06 PM, "Edison Su" wrote: >Currently, no, I can add an API for it. > >> -Original Message- >> From: Jessica Wang [mailto:jessica.w...@citrix.com] >> Sent: Tuesday, June 18, 2013 4:24 PM >> To: Min Chen >> Cc: dev@cloudstack.apache.org >> Subject: RE: NFS Cache storage query >> >> Min, >> >> > We may need to provide a way from UI to allow users to configure and >> display their NFS cache. >> Is there an API that list NFS cache? >> >> Jessica >> >> -Original Message- >> From: Min Chen >> Sent: Friday, June 14, 2013 9:26 AM >> To: dev@cloudstack.apache.org >> Cc: Jessica Wang >> Subject: Re: NFS Cache storage query >> >> Hi Sanjeev, >> >> In 4.2 release, we require that a NFS cache storage has to be added if >> you choose S3 as the storage provider since we haven't refactored >> hypervisor side code to handle s3 directly by bypassing NFS caching, >>which is >> the goal for 4.3 release. I see an issue with current UI, where user >>can only >> add cache storage when he/she adds a S3 storage. We may need to provide >> a way from UI to allow users to configure and display their NFS cache. >>You >> can file a JIRA ticket for this UI enhancement. >> >> Thanks >> -min >> >> On 6/14/13 6:35 AM, "Chip Childers" wrote: >> >> >On Fri, Jun 14, 2013 at 01:06:30PM +, Sanjeev Neelarapu wrote: >> >> Hi, >> >> >> >> I have a query on how to add NFS Cache storage. >> >> Before creating a zone if we create a secondary storage with s3 as >> >>the storage provider and don't select NFS Cache Storage then we treat >> >>it as >> >>S3 at region level. >> >> Later I create a zone and at "add secondary storage" creation wizard >> >>in UI if I select NFS as secondary storage provider will it be treated >> >>as NFS Cache Storage? If not is there a way to add NFS cache storage >> >>for that zone? >> >> >> >> Thanks, >> >> Sanjeev >> >> >> > >> >Based on the thread talking about this [1], I'm not sure that it will >> >be implemented this way anymore. >> > >> >-chip >> > >> >[1] http://markmail.org/message/c73nagj45q6iktfh >
Re: NFS Cache storage query
Agree with Min. How about CRUD operations on staging - all apis available ? Thanks, -Nitin On 19/06/13 6:44 AM, "Min Chen" wrote: >Edison, we can just add a parameter in listImageStoreCmd to show cache >store. > >Thanks >-min > >On 6/18/13 6:06 PM, "Edison Su" wrote: > >>Currently, no, I can add an API for it. >> >>> -Original Message- >>> From: Jessica Wang [mailto:jessica.w...@citrix.com] >>> Sent: Tuesday, June 18, 2013 4:24 PM >>> To: Min Chen >>> Cc: dev@cloudstack.apache.org >>> Subject: RE: NFS Cache storage query >>> >>> Min, >>> >>> > We may need to provide a way from UI to allow users to configure and >>> display their NFS cache. >>> Is there an API that list NFS cache? >>> >>> Jessica >>> >>> -Original Message- >>> From: Min Chen >>> Sent: Friday, June 14, 2013 9:26 AM >>> To: dev@cloudstack.apache.org >>> Cc: Jessica Wang >>> Subject: Re: NFS Cache storage query >>> >>> Hi Sanjeev, >>> >>> In 4.2 release, we require that a NFS cache storage has to be added if >>> you choose S3 as the storage provider since we haven't refactored >>> hypervisor side code to handle s3 directly by bypassing NFS caching, >>>which is >>> the goal for 4.3 release. I see an issue with current UI, where user >>>can only >>> add cache storage when he/she adds a S3 storage. We may need to provide >>> a way from UI to allow users to configure and display their NFS cache. >>>You >>> can file a JIRA ticket for this UI enhancement. >>> >>> Thanks >>> -min >>> >>> On 6/14/13 6:35 AM, "Chip Childers" wrote: >>> >>> >On Fri, Jun 14, 2013 at 01:06:30PM +, Sanjeev Neelarapu wrote: >>> >> Hi, >>> >> >>> >> I have a query on how to add NFS Cache storage. >>> >> Before creating a zone if we create a secondary storage with s3 as >>> >>the storage provider and don't select NFS Cache Storage then we treat >>> >>it as >>> >>S3 at region level. >>> >> Later I create a zone and at "add secondary storage" creation wizard >>> >>in UI if I select NFS as secondary storage provider will it be >>>treated >>> >>as NFS Cache Storage? If not is there a way to add NFS cache storage >>> >>for that zone? >>> >> >>> >> Thanks, >>> >> Sanjeev >>> >> >>> > >>> >Based on the thread talking about this [1], I'm not sure that it will >>> >be implemented this way anymore. >>> > >>> >-chip >>> > >>> >[1] http://markmail.org/message/c73nagj45q6iktfh >> >
Re: NFS Cache storage query
Edison, Are we going to change the name of this mechanism to be more representative of its role as a staging/temporary area rather than a cache? Thanks, -John On Jun 18, 2013, at 9:06 PM, Edison Su wrote: > Currently, no, I can add an API for it. > >> -Original Message- >> From: Jessica Wang [mailto:jessica.w...@citrix.com] >> Sent: Tuesday, June 18, 2013 4:24 PM >> To: Min Chen >> Cc: dev@cloudstack.apache.org >> Subject: RE: NFS Cache storage query >> >> Min, >> >>> We may need to provide a way from UI to allow users to configure and >> display their NFS cache. >> Is there an API that list NFS cache? >> >> Jessica >> >> -Original Message- >> From: Min Chen >> Sent: Friday, June 14, 2013 9:26 AM >> To: dev@cloudstack.apache.org >> Cc: Jessica Wang >> Subject: Re: NFS Cache storage query >> >> Hi Sanjeev, >> >>In 4.2 release, we require that a NFS cache storage has to be added if >> you choose S3 as the storage provider since we haven't refactored >> hypervisor side code to handle s3 directly by bypassing NFS caching, which is >> the goal for 4.3 release. I see an issue with current UI, where user can only >> add cache storage when he/she adds a S3 storage. We may need to provide >> a way from UI to allow users to configure and display their NFS cache. You >> can file a JIRA ticket for this UI enhancement. >> >>Thanks >>-min >> >> On 6/14/13 6:35 AM, "Chip Childers" wrote: >> >>> On Fri, Jun 14, 2013 at 01:06:30PM +, Sanjeev Neelarapu wrote: >>>> Hi, >>>> >>>> I have a query on how to add NFS Cache storage. >>>> Before creating a zone if we create a secondary storage with s3 as >>>> the storage provider and don't select NFS Cache Storage then we treat >>>> it as >>>> S3 at region level. >>>> Later I create a zone and at "add secondary storage" creation wizard >>>> in UI if I select NFS as secondary storage provider will it be treated >>>> as NFS Cache Storage? If not is there a way to add NFS cache storage >>>> for that zone? >>>> >>>> Thanks, >>>> Sanjeev >>> >>> Based on the thread talking about this [1], I'm not sure that it will >>> be implemented this way anymore. >>> >>> -chip >>> >>> [1] http://markmail.org/message/c73nagj45q6iktfh >
Re: NFS Cache storage query
Edison, we can just add a parameter in listImageStoreCmd to show cache store. Thanks -min On 6/18/13 6:06 PM, "Edison Su" wrote: >Currently, no, I can add an API for it. > >> -Original Message- >> From: Jessica Wang [mailto:jessica.w...@citrix.com] >> Sent: Tuesday, June 18, 2013 4:24 PM >> To: Min Chen >> Cc: dev@cloudstack.apache.org >> Subject: RE: NFS Cache storage query >> >> Min, >> >> > We may need to provide a way from UI to allow users to configure and >> display their NFS cache. >> Is there an API that list NFS cache? >> >> Jessica >> >> -Original Message- >> From: Min Chen >> Sent: Friday, June 14, 2013 9:26 AM >> To: dev@cloudstack.apache.org >> Cc: Jessica Wang >> Subject: Re: NFS Cache storage query >> >> Hi Sanjeev, >> >> In 4.2 release, we require that a NFS cache storage has to be added if >> you choose S3 as the storage provider since we haven't refactored >> hypervisor side code to handle s3 directly by bypassing NFS caching, >>which is >> the goal for 4.3 release. I see an issue with current UI, where user >>can only >> add cache storage when he/she adds a S3 storage. We may need to provide >> a way from UI to allow users to configure and display their NFS cache. >>You >> can file a JIRA ticket for this UI enhancement. >> >> Thanks >> -min >> >> On 6/14/13 6:35 AM, "Chip Childers" wrote: >> >> >On Fri, Jun 14, 2013 at 01:06:30PM +, Sanjeev Neelarapu wrote: >> >> Hi, >> >> >> >> I have a query on how to add NFS Cache storage. >> >> Before creating a zone if we create a secondary storage with s3 as >> >>the storage provider and don't select NFS Cache Storage then we treat >> >>it as >> >>S3 at region level. >> >> Later I create a zone and at "add secondary storage" creation wizard >> >>in UI if I select NFS as secondary storage provider will it be treated >> >>as NFS Cache Storage? If not is there a way to add NFS cache storage >> >>for that zone? >> >> >> >> Thanks, >> >> Sanjeev >> >> >> > >> >Based on the thread talking about this [1], I'm not sure that it will >> >be implemented this way anymore. >> > >> >-chip >> > >> >[1] http://markmail.org/message/c73nagj45q6iktfh >
RE: NFS Cache storage query
Currently, no, I can add an API for it. > -Original Message- > From: Jessica Wang [mailto:jessica.w...@citrix.com] > Sent: Tuesday, June 18, 2013 4:24 PM > To: Min Chen > Cc: dev@cloudstack.apache.org > Subject: RE: NFS Cache storage query > > Min, > > > We may need to provide a way from UI to allow users to configure and > display their NFS cache. > Is there an API that list NFS cache? > > Jessica > > -Original Message- > From: Min Chen > Sent: Friday, June 14, 2013 9:26 AM > To: dev@cloudstack.apache.org > Cc: Jessica Wang > Subject: Re: NFS Cache storage query > > Hi Sanjeev, > > In 4.2 release, we require that a NFS cache storage has to be added if > you choose S3 as the storage provider since we haven't refactored > hypervisor side code to handle s3 directly by bypassing NFS caching, which is > the goal for 4.3 release. I see an issue with current UI, where user can only > add cache storage when he/she adds a S3 storage. We may need to provide > a way from UI to allow users to configure and display their NFS cache. You > can file a JIRA ticket for this UI enhancement. > > Thanks > -min > > On 6/14/13 6:35 AM, "Chip Childers" wrote: > > >On Fri, Jun 14, 2013 at 01:06:30PM +, Sanjeev Neelarapu wrote: > >> Hi, > >> > >> I have a query on how to add NFS Cache storage. > >> Before creating a zone if we create a secondary storage with s3 as > >>the storage provider and don't select NFS Cache Storage then we treat > >>it as > >>S3 at region level. > >> Later I create a zone and at "add secondary storage" creation wizard > >>in UI if I select NFS as secondary storage provider will it be treated > >>as NFS Cache Storage? If not is there a way to add NFS cache storage > >>for that zone? > >> > >> Thanks, > >> Sanjeev > >> > > > >Based on the thread talking about this [1], I'm not sure that it will > >be implemented this way anymore. > > > >-chip > > > >[1] http://markmail.org/message/c73nagj45q6iktfh
RE: NFS Cache storage query
Min, > We may need to provide a way from UI to allow users to configure and display > their NFS cache. Is there an API that list NFS cache? Jessica -Original Message- From: Min Chen Sent: Friday, June 14, 2013 9:26 AM To: dev@cloudstack.apache.org Cc: Jessica Wang Subject: Re: NFS Cache storage query Hi Sanjeev, In 4.2 release, we require that a NFS cache storage has to be added if you choose S3 as the storage provider since we haven't refactored hypervisor side code to handle s3 directly by bypassing NFS caching, which is the goal for 4.3 release. I see an issue with current UI, where user can only add cache storage when he/she adds a S3 storage. We may need to provide a way from UI to allow users to configure and display their NFS cache. You can file a JIRA ticket for this UI enhancement. Thanks -min On 6/14/13 6:35 AM, "Chip Childers" wrote: >On Fri, Jun 14, 2013 at 01:06:30PM +, Sanjeev Neelarapu wrote: >> Hi, >> >> I have a query on how to add NFS Cache storage. >> Before creating a zone if we create a secondary storage with s3 as the >>storage provider and don't select NFS Cache Storage then we treat it as >>S3 at region level. >> Later I create a zone and at "add secondary storage" creation wizard in >>UI if I select NFS as secondary storage provider will it be treated as >>NFS Cache Storage? If not is there a way to add NFS cache storage for >>that zone? >> >> Thanks, >> Sanjeev >> > >Based on the thread talking about this [1], I'm not sure that it will be >implemented this way anymore. > >-chip > >[1] http://markmail.org/message/c73nagj45q6iktfh
Re: NFS Cache storage query
Hi Sanjeev, In 4.2 release, we require that a NFS cache storage has to be added if you choose S3 as the storage provider since we haven't refactored hypervisor side code to handle s3 directly by bypassing NFS caching, which is the goal for 4.3 release. I see an issue with current UI, where user can only add cache storage when he/she adds a S3 storage. We may need to provide a way from UI to allow users to configure and display their NFS cache. You can file a JIRA ticket for this UI enhancement. Thanks -min On 6/14/13 6:35 AM, "Chip Childers" wrote: >On Fri, Jun 14, 2013 at 01:06:30PM +, Sanjeev Neelarapu wrote: >> Hi, >> >> I have a query on how to add NFS Cache storage. >> Before creating a zone if we create a secondary storage with s3 as the >>storage provider and don't select NFS Cache Storage then we treat it as >>S3 at region level. >> Later I create a zone and at "add secondary storage" creation wizard in >>UI if I select NFS as secondary storage provider will it be treated as >>NFS Cache Storage? If not is there a way to add NFS cache storage for >>that zone? >> >> Thanks, >> Sanjeev >> > >Based on the thread talking about this [1], I'm not sure that it will be >implemented this way anymore. > >-chip > >[1] http://markmail.org/message/c73nagj45q6iktfh
Re: NFS Cache storage query
On Fri, Jun 14, 2013 at 01:06:30PM +, Sanjeev Neelarapu wrote: > Hi, > > I have a query on how to add NFS Cache storage. > Before creating a zone if we create a secondary storage with s3 as the > storage provider and don't select NFS Cache Storage then we treat it as S3 at > region level. > Later I create a zone and at "add secondary storage" creation wizard in UI if > I select NFS as secondary storage provider will it be treated as NFS Cache > Storage? If not is there a way to add NFS cache storage for that zone? > > Thanks, > Sanjeev > Based on the thread talking about this [1], I'm not sure that it will be implemented this way anymore. -chip [1] http://markmail.org/message/c73nagj45q6iktfh
