[ceph-users] Delete a Pool - how hard should be?
I think this is a good question for everybody: How hard should be delete a Pool? We ask to tell the pool twice. We ask to add "--yes-i-really-really-mean-it" We ask to add ability to mons to delete the pool (and remove this ability ASAP after). ... and then somebody of course ask us to restore the pool. I think that all this stuff is not looking in the right direction. It's not the administrator that need to be warned from delete datas. It's the data owner that should be warned (which most of the time give it's approval by phone and gone). So, all this stuff just make the life of administrator harder, while not improving in any way the life of the Data Owner. Probably the best solution is to ...do not delete at all and instead apply a "deleting policy". Something like: ceph osd pool rm POOL_NAME -yes -> POOL_NAME is set to be deleted, removal is scheduled within 30 days. This allow us to do 2 things: * allow administrator to don't waste their time in CML with true strange command * allow data owner to have a grace period to verify if, after deletion, everything works as expected and that data that disapper wasn't usefull in some way. After 30 days data will be removed automatically. This is a safe policy for ADMIN and DATA OWNER. Of course ADMIN should be allowed to remove POOL scheduleded for deletion in order to save disk spaces if needed (but only if needed). What do you think? ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Re: [ceph-users] Delete a Pool - how hard should be?
On Thu, Mar 1, 2018 at 9:21 AM Max Cuttins wrote: > I think this is a good question for everybody: How hard should be delete a > Pool? > > We ask to tell the pool twice. > We ask to add "--yes-i-really-really-mean-it" > We ask to add ability to mons to delete the pool (and remove this ability > ASAP after). > > ... and then somebody of course ask us to restore the pool. > > I think that all this stuff is not looking in the right direction. > It's not the administrator that need to be warned from delete datas. > It's the data owner that should be warned (which most of the time give > it's approval by phone and gone). > > So, all this stuff just make the life of administrator harder, while not > improving in any way the life of the Data Owner. > Probably the best solution is to ...do not delete at all and instead apply > a "deleting policy". > Something like: > > ceph osd pool rm POOL_NAME -yes > -> POOL_NAME is set to be deleted, removal is scheduled within 30 days. > > > This allow us to do 2 things: > >- allow administrator to don't waste their time in CML with true >strange command >- allow data owner to have a grace period to verify if, after >deletion, everything works as expected and that data that disapper wasn't >usefull in some way. > > After 30 days data will be removed automatically. This is a safe policy > for ADMIN and DATA OWNER. > Of course ADMIN should be allowed to remove POOL scheduleded for deletion > in order to save disk spaces if needed (but only if needed). > > What do you think? > > You're not wrong, and indeed that's why I pushed back on the latest attempt to make deleting pools even more cumbersome. But having a "trash" concept is also pretty weird. If admins can override it to just immediately delete the data (if they need the space), how is that different from just being another hoop to jump through? If we want to give the data owners a chance to undo, how do we identify and notify *them* rather than the admin running the command? But if admins can't override the trash and delete immediately, what do we do for things like testing and proofs of concept where large-scale data creates and deletes are to be expected? -Greg ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Re: [ceph-users] Delete a Pool - how hard should be?
On Mon, Mar 5, 2018 at 2:17 PM, Gregory Farnum wrote: > On Thu, Mar 1, 2018 at 9:21 AM Max Cuttins wrote: >> >> I think this is a good question for everybody: How hard should be delete a >> Pool? >> >> We ask to tell the pool twice. >> We ask to add "--yes-i-really-really-mean-it" >> We ask to add ability to mons to delete the pool (and remove this ability >> ASAP after). >> >> ... and then somebody of course ask us to restore the pool. >> >> I think that all this stuff is not looking in the right direction. >> It's not the administrator that need to be warned from delete datas. >> It's the data owner that should be warned (which most of the time give >> it's approval by phone and gone). >> >> >> So, all this stuff just make the life of administrator harder, while not >> improving in any way the life of the Data Owner. >> Probably the best solution is to ...do not delete at all and instead apply >> a "deleting policy". >> Something like: >> >> ceph osd pool rm POOL_NAME -yes >> -> POOL_NAME is set to be deleted, removal is scheduled within 30 days. >> >> >> This allow us to do 2 things: >> >> allow administrator to don't waste their time in CML with true strange >> command >> allow data owner to have a grace period to verify if, after deletion, >> everything works as expected and that data that disapper wasn't usefull in >> some way. >> >> After 30 days data will be removed automatically. This is a safe policy >> for ADMIN and DATA OWNER. >> Of course ADMIN should be allowed to remove POOL scheduleded for deletion >> in order to save disk spaces if needed (but only if needed). >> >> What do you think? >> >> > > You're not wrong, and indeed that's why I pushed back on the latest attempt > to make deleting pools even more cumbersome. > > But having a "trash" concept is also pretty weird. If admins can override it > to just immediately delete the data (if they need the space), how is that > different from just being another hoop to jump through? If we want to give > the data owners a chance to undo, how do we identify and notify *them* > rather than the admin running the command? But if admins can't override the > trash and delete immediately, what do we do for things like testing and > proofs of concept where large-scale data creates and deletes are to be > expected? What about using the at command: ceph osd pool rm --yes-i-really-really-mean-it | at now + 30 days Regards, Alex > -Greg > > ___ > ceph-users mailing list > ceph-users@lists.ceph.com > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com > ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Re: [ceph-users] Delete a Pool - how hard should be?
What about using the at command: ceph osd pool rm --yes-i-really-really-mean-it | at now + 30 days Regards, Alex How do you know that this command is scheduled? How do you delete the scheduled command if is casted? This is weird. We need something within CEPH that make you see the "status" of the pool as "pending delete". ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Re: [ceph-users] Delete a Pool - how hard should be?
Il 05/03/2018 20:17, Gregory Farnum ha scritto: You're not wrong, and indeed that's why I pushed back on the latest attempt to make deleting pools even more cumbersome. But having a "trash" concept is also pretty weird. If admins can override it to just immediately delete the data (if they need the space), how is that different from just being another hoop to jump through? If we want to give the data owners a chance to undo, how do we identify and notify *them* rather than the admin running the command? But if admins can't override the trash and delete immediately, what do we do for things like testing and proofs of concept where large-scale data creates and deletes are to be expected? -Greg I'm talking about my experience: * Data Owner are a little bit in their LA LA LAND, and think that they can safely delete some of their data without losses. * Data Owner should think that their pool have been really deleted * Data Owner should not been akwnoledge about the existance of the "/trash/" * So Data Owner ask to restore from backup (but instead we'll use easily the trash). Said so, we also have to think that: * Administrator is always GOD, so he need to be in the possibility to override if needed whenever he needs. * However Administrator should just put in status delete without override this behaviour if there is not need to do so. * Override should be allowed only with many cumbersome telling you that YOU SHOULD NOT OVERRIDE - PLEASE AVOID OVERRIDE I don't like that the software can limit administrators to do his job... in the end Administrator'll always find its way to do what he want (it's the root). Of course I like the feature to push the Admin to follow the right behaviour. ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Re: [ceph-users] Delete a Pool - how hard should be?
On 06. mars 2018 10:26, Max Cuttins wrote: Il 05/03/2018 20:17, Gregory Farnum ha scritto: You're not wrong, and indeed that's why I pushed back on the latest attempt to make deleting pools even more cumbersome. But having a "trash" concept is also pretty weird. If admins can override it to just immediately delete the data (if they need the space), how is that different from just being another hoop to jump through? If we want to give the data owners a chance to undo, how do we identify and notify *them* rather than the admin running the command? But if admins can't override the trash and delete immediately, what do we do for things like testing and proofs of concept where large-scale data creates and deletes are to be expected? -Greg I'm talking about my experience: * Data Owner are a little bit in their LA LA LAND, and think that they can safely delete some of their data without losses. * Data Owner should think that their pool have been really deleted * Data Owner should not been akwnoledge about the existance of the "/trash/" * So Data Owner ask to restore from backup (but instead we'll use easily the trash). Said so, we also have to think that: * Administrator is always GOD, so he need to be in the possibility to override if needed whenever he needs. * However Administrator should just put in status delete without override this behaviour if there is not need to do so. * Override should be allowed only with many cumbersome telling you that YOU SHOULD NOT OVERRIDE - PLEASE AVOID OVERRIDE I don't like that the software can limit administrators to do his job... in the end Administrator'll always find its way to do what he want (it's the root). Of course I like the feature to push the Admin to follow the right behaviour. some sort of active/inactive toggle both on RBD images, pools, buckets and filesystems trees is nice to allow admins to perform scream tests. "data owner requests deletion - admin disables pool(kicks all clients) - data owner screams - admin reactivates" sounds much better then the last step beeing admin checking if the backups are good.,.. i try to do something similar by renaming pools to be deleted but that is not allways the same as inactive. kind regards Ronny Aasen ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Re: [ceph-users] Delete a Pool - how hard should be?
Il 06/03/2018 11:13, Ronny Aasen ha scritto: On 06. mars 2018 10:26, Max Cuttins wrote: Il 05/03/2018 20:17, Gregory Farnum ha scritto: You're not wrong, and indeed that's why I pushed back on the latest attempt to make deleting pools even more cumbersome. But having a "trash" concept is also pretty weird. If admins can override it to just immediately delete the data (if they need the space), how is that different from just being another hoop to jump through? If we want to give the data owners a chance to undo, how do we identify and notify *them* rather than the admin running the command? But if admins can't override the trash and delete immediately, what do we do for things like testing and proofs of concept where large-scale data creates and deletes are to be expected? -Greg I'm talking about my experience: * Data Owner are a little bit in their LA LA LAND, and think that they can safely delete some of their data without losses. * Data Owner should think that their pool have been really deleted * Data Owner should not been akwnoledge about the existance of the "/trash/" * So Data Owner ask to restore from backup (but instead we'll use easily the trash). Said so, we also have to think that: * Administrator is always GOD, so he need to be in the possibility to override if needed whenever he needs. * However Administrator should just put in status delete without override this behaviour if there is not need to do so. * Override should be allowed only with many cumbersome telling you that YOU SHOULD NOT OVERRIDE - PLEASE AVOID OVERRIDE I don't like that the software can limit administrators to do his job... in the end Administrator'll always find its way to do what he want (it's the root). Of course I like the feature to push the Admin to follow the right behaviour. some sort of active/inactive toggle both on RBD images, pools, buckets and filesystems trees is nice to allow admins to perform scream tests. "data owner requests deletion - admin disables pool(kicks all clients) - data owner screams - admin reactivates" sounds much better then the last step beeing admin checking if the backups are good.,.. i try to do something similar by renaming pools to be deleted but that is not allways the same as inactive. EXACTLY! :) I like the name "scream test"... it really look like that! :) ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Re: [ceph-users] Delete a Pool - how hard should be?
Il 06/03/2018 16:15, David Turner ha scritto: I've never deleted a bucket, pool, etc at the request of a user that they then wanted back because I force them to go through a process to have their data deleted. They have to prove to me, and I have to agree, that they don't need it before I'll delete it. Of course I cannot keep in touch with the customer of my reseller (which I don't know) .. or I've to say with the end customer [of the customer] [of the customer] [of the customer] of my resellers ...in order to obsessively ask to please PROVE ME that your data are not usefull anymore. And even if I could I neither want to call all the end customers making me wasting time to let me confirm _*I can go on *_and do my job. It just sounds like you need to either learn to be a storage admin, hire someone that is, or buy a solution that doesn't care if you are. Uh! That's bad. It is so sad when somebody cannot take a proposal as constructive criticism but need instead to mark other as incompetent. Everybody has different admin experience and different point-of-view and that's all folks. You don't have sub-sub-sub customer which you don't know? I do. You are the one that make everybody obey to "the process"? I can't. I need to solve the requests of my customers not yell when they are so dumb to delete important data. I just wrote to throw a proposal in order to improve the admin's life not of course to be offended. Thanks! ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Re: [ceph-users] Delete a Pool - how hard should be?
Il 06/03/2018 16:23, David Turner ha scritto: That said, I do like the idea of being able to disable buckets, rbds, pools, etc so that no client could access them. That is useful for much more than just data deletion and won't prevent people from deleting data prematurely. To me, if nobody can access data for 30 days and the customer didn't call me within those days, it's ok to delete definitly the data. Which is the way should be. Make easy to the admin delete data when he really wants. Make possible to the user to stay some days without it's data till these data is obsolete and useless. The autopurge of the trash of your mailbox works in the sameway and seems to me a reasonable way to handle precious data such personal emails. It could be added as a requisite step to deleting a pool, rbd, etc. The process would need to be refactored as adding another step isn't viable. This feature is much more complicated than it may seem on the surface. For pools, you could utilize cephx, except not everyone uses that... So maybe logic added to the osd map. Buckets would have to be completely in rgw. Rbds would probably have to be in the osd map as well. This is not a trivial change. Mine was just a "/nice-to-have/" proposal. There is no hurry in implement a secondary feature such this one. About the logic is it possible to use something like this: * snapshot the pool with a special poolname * remove the original pool * give the possibility to restore the snapshot with it's original name. I think this should suddenly stop all the connection to the original pool but leave all the data intact. Maybe. ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com