Re: [zfs-discuss] Re: [security-discuss] Thoughts on ZFS Secure Delete - without using Crypto
On Jan 2, 2007, at 6:48 AM, Darren Reed wrote: Darren J Moffat wrote: ... Of course. I didn't mention it because I thought it was obvious but this would NOT break the COW or the transactional integrity of ZFS. One of the possible ways that the to be bleached blocks are dealt with in the face of a crash is just like everything else - they would be in the ZFS Intent Log as things to do. Do NIST and other specifications that come into play here dictate what should be done in these and other situations? From http://csrc.nist.gov/publications/nistpubs/800-88/NISTSP800-88_rev1.pdf includes the statement that The security goal of the overwriting process is to replace written data with random data. How we achieve this is our problem. I expect that this is a subjective analysis if we meet the goals... Do they say how this feature must be provided or in which situations it is required to be covered in order to meet their criteria. They talk about Clearing vs Purging vs Destruction... Clearing is the lowest level and seems to be useful when repurposing storage within an organization. All of these are supposed to be NSA/CSS Approved. I do not see the exact approval requirements in this document. Or do they just document overwrite patterns, depending on the security of the information? From http://csrc.nist.gov/publications/nistpubs/800-88/NISTSP800-88_rev1.pdf it states Studies have shown that most of today’s media can be effectively cleared by one overwrite. Darren ___ security-discuss mailing list [EMAIL PROTECTED] ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Re: [security-discuss] Thoughts on ZFS Secure Delete - without using Crypto
David Bustos wrote: Quoth Darren J Moffat on Thu, Dec 21, 2006 at 03:31:59PM +: Pawel Jakub Dawidek wrote: I like the idea, I really do, but it will be s expensive because of ZFS' COW model. Not only file removal or truncation will call bleaching, but every single file system modification... Heh, well, if privacy of your data is important enough, you probably don't care too much about performance. I'm not sure it will be that slow, the bleaching will be done in a separate (new) transaction group in most (probably all) cases anyway so it shouldn't really impact your write performance unless you are very I/O bound and already running near the limit. However this is speculation until someone tries to implement this! Bleaching previously used blocks will corrupt files pointed to by older uberblocks. I think that means that you'd have to verify that the new uberblock is readable before you proceed, since part of ZFS's fault tolerance is falling back to the most recent good uberblock if the latest one is corrupt. I don't think this makes bleaching unworkable, but the interplay will require analysis. Of course. I didn't mention it because I thought it was obvious but this would NOT break the COW or the transactional integrity of ZFS. One of the possible ways that the to be bleached blocks are dealt with in the face of a crash is just like everything else - they would be in the ZFS Intent Log as things to do. -- Darren J Moffat ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Re: [security-discuss] Thoughts on ZFS Secure Delete - without using Crypto
Darren J Moffat wrote: ... Of course. I didn't mention it because I thought it was obvious but this would NOT break the COW or the transactional integrity of ZFS. One of the possible ways that the to be bleached blocks are dealt with in the face of a crash is just like everything else - they would be in the ZFS Intent Log as things to do. Do NIST and other specifications that come into play here dictate what should be done in these and other situations? Do they say how this feature must be provided or in which situations it is required to be covered in order to meet their criteria? Or do they just document overwrite patterns, depending on the security of the information? Darren ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Re: [security-discuss] Thoughts on ZFS Secure Delete - without using Crypto
Quoth Darren J Moffat on Thu, Dec 21, 2006 at 03:31:59PM +: Pawel Jakub Dawidek wrote: I like the idea, I really do, but it will be s expensive because of ZFS' COW model. Not only file removal or truncation will call bleaching, but every single file system modification... Heh, well, if privacy of your data is important enough, you probably don't care too much about performance. I'm not sure it will be that slow, the bleaching will be done in a separate (new) transaction group in most (probably all) cases anyway so it shouldn't really impact your write performance unless you are very I/O bound and already running near the limit. However this is speculation until someone tries to implement this! Bleaching previously used blocks will corrupt files pointed to by older uberblocks. I think that means that you'd have to verify that the new uberblock is readable before you proceed, since part of ZFS's fault tolerance is falling back to the most recent good uberblock if the latest one is corrupt. I don't think this makes bleaching unworkable, but the interplay will require analysis. David ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Re: [security-discuss] Thoughts on ZFS Secure Delete - without using Crypto
On Tue, 2006-12-26 at 13:59 -0500, Torrey McMahon wrote: clearly you'd need to store the unbleached list persistently in the pool. Which could then be easily referenced to find all the blocks that were recently deleted but not yet bleached? Is my paranoia running a bit too high? I think your paranoia is indeed running a bit high if the alternative is that some blocks escape bleaching forever when they were freed shortly before a crash. For a portable system, the risk of theft is highest when the laptop is unattended and idle -- and that's the point where the bleaching process would have time to catch up; most likely, the unbleached list would be small or empty.. - Bill ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Re: [security-discuss] Thoughts on ZFS Secure Delete - without using Crypto
On Wed, Dec 27, 2006 at 08:45:23AM -0500, Bill Sommerfeld wrote: I think your paranoia is indeed running a bit high if the alternative is that some blocks escape bleaching forever when they were freed shortly before a crash. Lazy bg bleaching of freed blocks is not enough if you're really paranoid about deleting things that might be cloned. (See sub-thread about bleach(2), which is off the table.) For a portable system, the risk of theft is highest when the laptop is unattended and idle -- and that's the point where the bleaching process would have time to catch up; most likely, the unbleached list would be small or empty.. For portable systems the risk is not the loss of unbleached freed blocks -- it's the loss of all those live blocks. Thus you'd need encryption. But encryption's still not enough if the system is stolen while the keys are in memory. Nico -- ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Re: [security-discuss] Thoughts on ZFS Secure Delete - without using Crypto
Darren J Moffat wrote: Pawel Jakub Dawidek wrote: I like the idea, I really do, but it will be s expensive because of ZFS' COW model. Not only file removal or truncation will call bleaching, but every single file system modification... Heh, well, if privacy of your data is important enough, you probably don't care too much about performance. I'm not sure it will be that slow, the bleaching will be done in a separate (new) transaction group in most (probably all) cases anyway so it shouldn't really impact your write performance unless you are very I/O bound and already running near the limit. However this is speculation until someone tries to implement this! What happens if fatal failure occurs after the txg which frees blocks have been written but before before txg doing bleaching will be started/completed? I for one would prefer encryption, which may turns out to be much faster than bleaching and also more secure. At least NIST, under I believe the guidance of the NSA, does not consider that encryption and key destruction alone is sufficient in all cases. Which is why I'm proposing this as complementary. True, dropping the keys leaves lots of encrypted material for determined cryptoanalytic to analyze, so it should be bleached in some good way. Victor ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Re: [security-discuss] Thoughts on ZFS Secure Delete - without using Crypto
On Tue, 2006-12-26 at 14:01 +0300, Victor Latushkin wrote: What happens if fatal failure occurs after the txg which frees blocks have been written but before before txg doing bleaching will be started/completed? clearly you'd need to store the unbleached list persistently in the pool. transactions which freed blocks (by punching holes in the allocation space map) would instead or additionally move them to the unbleached list; a separate bleaching task queue would pick blocks off the unbleached list, bleach them; only once bleaching was complete would they be removed from the unbleached list. In the face of a crash, some blocks might get bleached twice. - Bill ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Re: [security-discuss] Thoughts on ZFS Secure Delete - without using Crypto
Bill Sommerfeld wrote: On Tue, 2006-12-26 at 14:01 +0300, Victor Latushkin wrote: What happens if fatal failure occurs after the txg which frees blocks have been written but before before txg doing bleaching will be started/completed? clearly you'd need to store the unbleached list persistently in the pool. Which could then be easily referenced to find all the blocks that were recently deleted but not yet bleached? Is my paranoia running a bit too high? ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Re: [security-discuss] Thoughts on ZFS Secure Delete - without using Crypto
Pawel Jakub Dawidek wrote: I like the idea, I really do, but it will be s expensive because of ZFS' COW model. Not only file removal or truncation will call bleaching, but every single file system modification... Heh, well, if privacy of your data is important enough, you probably don't care too much about performance. I'm not sure it will be that slow, the bleaching will be done in a separate (new) transaction group in most (probably all) cases anyway so it shouldn't really impact your write performance unless you are very I/O bound and already running near the limit. However this is speculation until someone tries to implement this! I for one would prefer encryption, which may turns out to be much faster than bleaching and also more secure. At least NIST, under I believe the guidance of the NSA, does not consider that encryption and key destruction alone is sufficient in all cases. Which is why I'm proposing this as complementary. -- Darren J Moffat ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Re: [security-discuss] Thoughts on ZFS Secure Delete - without using Crypto
Frank Hofmann wrote: And this kind of deep bleaching would also break if you use snapshots - how do you reliably bleach if you need to keep the all of the old data around ? You only could do so once the last snapshot is gone. Kind of defeating the idea - automatic but delayed indefinitely till operator intervention (deleting the last snapshot). Right that doesn't break snapshots at all in fact it is working exactly like snapshots work today anyway. With user delegation (when is this integrating BTW?) the file system operator might be the end user anyway. -- Darren J Moffat ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Re: [security-discuss] Thoughts on ZFS Secure Delete - without using Crypto
On Thu, Dec 21, 2006 at 03:31:59PM +, Darren J Moffat wrote: Pawel Jakub Dawidek wrote: I like the idea, I really do, but it will be s expensive because of ZFS' COW model. Not only file removal or truncation will call bleaching, but every single file system modification... Heh, well, if privacy of your data is important enough, you probably don't care too much about performance. I'm not sure it will be that slow, the bleaching will be done in a separate (new) transaction group in most (probably all) cases anyway so it shouldn't really impact your write performance unless you are very I/O bound and already running near the limit. However this is speculation until someone tries to implement this! Yes, bleaching lazily would help with performance. You might even delay bleaching for very long periods of time if you want, as long as there's an interface by which to request that all outstanding free-but-not-yet- bleached blocks be bleached immediately and synchronously. I for one would prefer encryption, which may turns out to be much faster than bleaching and also more secure. At least NIST, under I believe the guidance of the NSA, does not consider that encryption and key destruction alone is sufficient in all cases. Which is why I'm proposing this as complementary. James makes a good argument that this scheme won't suffice for customers who need that level of assurance. I'm inclined to agree. For customers who don't need that level of assurance then encryption ought to suffice. Nico -- ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Re: [security-discuss] Thoughts on ZFS Secure Delete - without using Crypto
Nicolas Williams wrote: James makes a good argument that this scheme won't suffice for customers who need that level of assurance. I'm inclined to agree. For customers who don't need that level of assurance then encryption ought to suffice. Has anyone other than me actually read the current NIST guidelines on this ? [ the url was in my original email message ]. The current NIST guidelines, or at least my reading of it, says that even if you are using encryption and even if you are going to do physical destruction you still need to do something like this. So this is complementary to encrypting the data - not that we can't in ZFS encryption ALL ZFS metadata (we should be able to encrypt all the file system relevant meta data) at least thats my current belief based on my knowledge of ZFS. Maybe doing this in ZFS isn't necessary and what we have with format(1M) purge/analyze is the correct user interface. -- Darren J Moffat ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Re: [security-discuss] Thoughts on ZFS Secure Delete - without using Crypto
On Thu, Dec 21, 2006 at 03:47:07PM +, Darren J Moffat wrote: Nicolas Williams wrote: James makes a good argument that this scheme won't suffice for customers who need that level of assurance. I'm inclined to agree. For customers who don't need that level of assurance then encryption ought to suffice. Has anyone other than me actually read the current NIST guidelines on this ? [ the url was in my original email message ]. The current NIST guidelines, or at least my reading of it, says that even if you are using encryption and even if you are going to do physical destruction you still need to do something like this. I think it's a bit nuanced. Pages 15-16 obliquely mention encryption in the description of clearing: ... It must be resistant to keystore recovery attempts executed from standard input devices and from data scavenging tools. ... I'm not sure how to interpret that in the case of ZFS encryption. The actual keys used to encrypt file are not typed in by users, and data scavenging tools could only get at them if: a) they recovered user passwords from which master FS keys are derived, b) have access to the media. On page 4 (errata), it says that on 9-11-06 (version 10-06) text was deleted that had once declared encryption insufficient. So, altogether I would read this as allowing deletion of keys as a method of clearing. Since clearing is all we can hope to do in ZFS then I think this should be sufficient. Nico -- ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Re: [security-discuss] Thoughts on ZFS Secure Delete - without using Crypto
One other area where is is useful is when you are in a jurisdiction where a court order may require you to produce your encryption keys - yes such jurisdictions exist and I don't want to debate the human rights angle or social engineering aspects of this just state that it exists. For such environments you may not want to use encryption, because you could be forced to give up your key, or even if you are you want a background method of destroying the cipher text without doing full disk destruction. Think of court cases between companies rather than criminal activity. -- Darren J Moffat ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Re: [security-discuss] Thoughts on ZFS Secure Delete - without using Crypto
Darren J Moffat wrote: One other area where is is useful is when you are in a jurisdiction where a court order may require you to produce your encryption keys - yes such jurisdictions exist and I don't want to debate the human rights angle or social engineering aspects of this just state that it exists. I think in these cases you want plausable deniability where different encryption keys produce different view of the disk, none of which give away that there are any other correct views of the data. If it is possible to destroy a small piece of the ZFS meta data (key material) and that makes it thereafter impossible to encrypt data, sure, but otherwise, bleaching is probably going to take a bit too long once you hear the knock on the door... For such environments you may not want to use encryption, because you could be forced to give up your key, or even if you are you want a background method of destroying the cipher text without doing full disk destruction. Think of court cases between companies rather than criminal activity. For corporations there are different requirements, for examples laws that regulate data retention. Not only this but you also need to make sure that the data you want to make unavailable never got backed up or that those backups get wiped... Darren ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Re: [security-discuss] Thoughts on ZFS Secure Delete - without using Crypto
On Tue, Dec 19, 2006 at 02:04:37PM +, Darren J Moffat wrote: In case it wasn't clear I am NOT proposing a UI like this: $ zfs bleach ~/Documents/company-finance.odp Instead ~/Documents or ~ would be a ZFS file system with a policy set something like this: # zfs set erase=file:zero Or maybe more like this: # zfs create -o erase=file -o erasemethod=zero homepool/darrenm The goal is the same as the goal for things like compression in ZFS, no application change it is free for the applications. I like the idea, I really do, but it will be s expensive because of ZFS' COW model. Not only file removal or truncation will call bleaching, but every single file system modification... Heh, well, if privacy of your data is important enough, you probably don't care too much about performance. I for one would prefer encryption, which may turns out to be much faster than bleaching and also more secure. -- Pawel Jakub Dawidek http://www.wheel.pl [EMAIL PROTECTED] http://www.FreeBSD.org FreeBSD committer Am I Evil? Yes, I Am! pgpbmTXIeqnBE.pgp Description: PGP signature ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Re: [security-discuss] Thoughts on ZFS Secure Delete - without using Crypto
Pawel Jakub Dawidek writes: The goal is the same as the goal for things like compression in ZFS, no application change it is free for the applications. I like the idea, I really do, but it will be s expensive because of ZFS' COW model. Not only file removal or truncation will call bleaching, but every single file system modification... Heh, well, if privacy of your data is important enough, you probably don't care too much about performance. I for one would prefer encryption, which may turns out to be much faster than bleaching and also more secure. I think the idea here is that since ZFS encourages the use of lots of small file systems (rather than one or two very big ones), you'd have just one or two very small file systems with crucial data marked as needing bleach, while the others would get by with the usual complement of detergent and switch fabric softener. Having _every_ file modification result in dozens of I/Os would probably be bad, but perhaps not if it's not _every_ modification that's affected. -- James Carlson, KISS Network[EMAIL PROTECTED] Sun Microsystems / 1 Network Drive 71.232W Vox +1 781 442 2084 MS UBUR02-212 / Burlington MA 01803-2757 42.496N Fax +1 781 442 1677 ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Re: [security-discuss] Thoughts on ZFS Secure Delete - without using Crypto
On Wed, 20 Dec 2006, Pawel Jakub Dawidek wrote: On Tue, Dec 19, 2006 at 02:04:37PM +, Darren J Moffat wrote: In case it wasn't clear I am NOT proposing a UI like this: $ zfs bleach ~/Documents/company-finance.odp Instead ~/Documents or ~ would be a ZFS file system with a policy set something like this: # zfs set erase=file:zero Or maybe more like this: # zfs create -o erase=file -o erasemethod=zero homepool/darrenm The goal is the same as the goal for things like compression in ZFS, no application change it is free for the applications. I like the idea, I really do, but it will be s expensive because of ZFS' COW model. Not only file removal or truncation will call bleaching, but every single file system modification... Heh, well, if privacy of your data is important enough, you probably don't care too much about performance. I for one would prefer encryption, which may turns out to be much faster than bleaching and also more secure. And this kind of deep bleaching would also break if you use snapshots - how do you reliably bleach if you need to keep the all of the old data around ? You only could do so once the last snapshot is gone. Kind of defeating the idea - automatic but delayed indefinitely till operator intervention (deleting the last snapshot). FrankH. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Re: [security-discuss] Thoughts on ZFS Secure Delete - without using Crypto
In case it wasn't clear I am NOT proposing a UI like this: $ zfs bleach ~/Documents/company-finance.odp Instead ~/Documents or ~ would be a ZFS file system with a policy set something like this: # zfs set erase=file:zero Or maybe more like this: # zfs create -o erase=file -o erasemethod=zero homepool/darrenm The goal is the same as the goal for things like compression in ZFS, no application change it is free for the applications. All of the same reasons for doing crypto outside of a command like encrypt(1) apply here too - especially the temp file and rename problems. -- Darren J Moffat ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Re: [security-discuss] Thoughts on ZFS Secure Delete - without using Crypto
On Tue, Dec 19, 2006 at 02:04:37PM +, Darren J Moffat wrote: In case it wasn't clear I am NOT proposing a UI like this: $ zfs bleach ~/Documents/company-finance.odp Instead ~/Documents or ~ would be a ZFS file system with a policy set something like this: # zfs set erase=file:zero Or maybe more like this: # zfs create -o erase=file -o erasemethod=zero homepool/darrenm I get it. This should be lots easier than bleach(1). Snapshots/clones are mostly not an issue here. When a block is truly freed, then it is wiped. Clones are an issue here only if they have different settings for this property than the FS that spawned them (so you might want to disallow re-setting of this property). Nico -- ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Re: [security-discuss] Thoughts on ZFS Secure Delete - without using Crypto
Nicolas Williams wrote: On Tue, Dec 19, 2006 at 02:04:37PM +, Darren J Moffat wrote: In case it wasn't clear I am NOT proposing a UI like this: $ zfs bleach ~/Documents/company-finance.odp Instead ~/Documents or ~ would be a ZFS file system with a policy set something like this: # zfs set erase=file:zero Or maybe more like this: # zfs create -o erase=file -o erasemethod=zero homepool/darrenm I get it. This should be lots easier than bleach(1). Snapshots/clones are mostly not an issue here. When a block is truly freed, then it is wiped. Yep. Clones are an issue here only if they have different settings for this property than the FS that spawned them (so you might want to disallow re-setting of this property). I think you are saying it should have INHERITY set to YES and EDIT set to NO. We don't currently have any properties like that but crypto will need this as well - for a very similar reason with clones. -- Darren J Moffat ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Re: [security-discuss] Thoughts on ZFS Secure Delete - without using Crypto
On Tue, Dec 19, 2006 at 04:37:36PM +, Darren J Moffat wrote: I think you are saying it should have INHERITY set to YES and EDIT set to NO. We don't currently have any properties like that but crypto will need this as well - for a very similar reason with clones. What I mean is that if there's a block that's shared between two filesystems then what do you do if the two filesystems have different settings for this property? IMO you shouldn't allow this to happen. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
