Re: Secure deletion of blocks

2013-08-20 Thread Matt Fellows
Thanks for the heads up, but I think I've managed to implement it crudely by overwriting sequentially with 1s, 0s and random bytes and tested it successfully on an ext4 partition.  I tested it by dd-ing the entire partition to a file, confirming a particular string was not pre

Re: Secure deletion of blocks

2013-08-20 Thread Colin McCabe
Just to clarify, ext4 has the option to turn off journalling. ext3 does not. Not sure about reiser. Colin On Tue, Aug 20, 2013 at 12:42 PM, Colin McCabe wrote: > > If I've got the right idea about this at all? > > From the man page for wipe(1); > > "Journaling filesystems (such as Ext3 or Rei

Re: Secure deletion of blocks

2013-08-20 Thread Colin McCabe
> If I've got the right idea about this at all? >From the man page for wipe(1); "Journaling filesystems (such as Ext3 or ReiserFS) are now being used by default by most Linux distributions. No secure deletion program that does filesystem-level calls can sanitize files on such filesystems, because

Re: Secure deletion of blocks

2013-08-15 Thread Todd Lipcon
Hi Matt, I'd also recommend implementing this in a somewhat pluggable way -- eg a configuration for a Deleter class. The default Deleter can be the one we use today which just removes the file, and you could plug in a SecureDeleter. I'd also see some use cases for a Deleter implementation which do

Re: Secure deletion of blocks

2013-08-15 Thread Andrew Wang
Hi Matt, Here are some code pointers: - When doing a file deletion, the NameNode turns the file into a set of blocks that need to be deleted. - When datanodes heartbeat in to the NN (see BPServiceActor#offerService), the NN replies with blocks to be invalidated (see BlockCommand and DatanodeProto

Secure deletion of blocks

2013-08-15 Thread Matt Fellows
Hi, I'm looking into writing a patch for HDFS which will provide a new method within HDFS which can securely delete the contents of a block on all the nodes upon which it exists. By securely delete I mean, overwrite with 1's/0's/random data cyclically such that the data could not be recovered foren