Re: [zfs-discuss] Secure delete?

2010-04-16 Thread Miles Nordin
> "edm" == Eric D Mudama writes: edm> What you're suggesting is exactly what SSD vendors already do. no, it's not. You have to do it for them. edm> They present a 512B standard host interface sector size, and edm> perform their own translations and management inside the edm> de

Re: [zfs-discuss] Secure delete?

2010-04-16 Thread Ragnar Sundblad
On 16 apr 2010, at 17.05, Bob Friesenhahn wrote: > On Fri, 16 Apr 2010, Kyle McDonald wrote: >>> >> But doesn't the TRIM command help here. If as the OS goes along it makes >> sectors as unused, then the SSD will have a lighter wight lift to only >> need to read for example 1 out of 8 (assuming

Re: [zfs-discuss] Secure delete?

2010-04-16 Thread Eric D. Mudama
On Fri, Apr 16 at 14:42, Miles Nordin wrote: "edm" == Eric D Mudama writes: edm> How would you stripe or manage a dataset across a mix of edm> devices with different geometries? the ``geometry'' discussed is 1-dimensional: sector size. The way that you do it is to align all writes, and n

Re: [zfs-discuss] Secure delete?

2010-04-16 Thread Bob Friesenhahn
On Fri, 16 Apr 2010, Eric D. Mudama wrote: On Fri, Apr 16 at 10:05, Bob Friesenhahn wrote: It is much more efficient (from a housekeeping perspective) if filesystem sectors map directly to SSD pages, but we are not there yet. How would you stripe or manage a dataset across a mix of devices wi

Re: [zfs-discuss] Secure delete?

2010-04-16 Thread Miles Nordin
> "edm" == Eric D Mudama writes: edm> How would you stripe or manage a dataset across a mix of edm> devices with different geometries? the ``geometry'' discussed is 1-dimensional: sector size. The way that you do it is to align all writes, and never write anything smaller than the sec

Re: [zfs-discuss] Secure delete?

2010-04-16 Thread Eric D. Mudama
On Fri, Apr 16 at 10:05, Bob Friesenhahn wrote: It is much more efficient (from a housekeeping perspective) if filesystem sectors map directly to SSD pages, but we are not there yet. How would you stripe or manage a dataset across a mix of devices with different geometries? That would break m

Re: [zfs-discuss] Secure delete?

2010-04-16 Thread Bob Friesenhahn
On Fri, 16 Apr 2010, Kyle McDonald wrote: But doesn't the TRIM command help here. If as the OS goes along it makes sectors as unused, then the SSD will have a lighter wight lift to only need to read for example 1 out of 8 (assuming sectors of 512 bytes, and 4K FLASH Pages) before writing a new

Re: [zfs-discuss] Secure delete?

2010-04-16 Thread Kyle McDonald
On 4/16/2010 10:30 AM, Bob Friesenhahn wrote: > On Thu, 15 Apr 2010, Eric D. Mudama wrote: >> >> The purpose of TRIM is to tell the drive that some # of sectors are no >> longer important so that it doesn't have to work as hard in its >> internal garbage collection. > > The sector size does not typ

Re: [zfs-discuss] Secure delete?

2010-04-16 Thread Bob Friesenhahn
On Thu, 15 Apr 2010, Eric D. Mudama wrote: The purpose of TRIM is to tell the drive that some # of sectors are no longer important so that it doesn't have to work as hard in its internal garbage collection. The sector size does not typically match the FLASH page size so the SSD still has to d

Re: [zfs-discuss] Secure delete?

2010-04-15 Thread Eric D. Mudama
On Tue, Apr 13 at 9:52, Bob Friesenhahn wrote: On Mon, 12 Apr 2010, Eric D. Mudama wrote: The advantage of TRIM, even in high end SSDs, is that it allows you to effectively have additional "considerable extra space" available to the device for garbage collection and wear management when not al

Re: [zfs-discuss] Secure delete?

2010-04-13 Thread Bob Friesenhahn
On Tue, 13 Apr 2010, Joerg Schilling wrote: I believe you make a mistake with this assumption. I see that you make some mistakes with your own assumptions. :-) - The SSD cannot know which blocks are currently not in use. It does know that blocks in its spare pool are not in use. -

Re: [zfs-discuss] Secure delete?

2010-04-13 Thread Bob Friesenhahn
On Mon, 12 Apr 2010, Eric D. Mudama wrote: The advantage of TRIM, even in high end SSDs, is that it allows you to effectively have additional "considerable extra space" available to the device for garbage collection and wear management when not all sectors are in use on the device. For most use

Re: [zfs-discuss] Secure delete?

2010-04-13 Thread James Van Artsdalen
If you're concerned about someone reading the charge level of a Flash cell to infer the value of the cell before being erased, then overwrite with random data twice before issuing TRIM (remapping in an SSD probably makes this ineffective). Most people needing a secure erase feature need it to s

Re: [zfs-discuss] Secure delete?

2010-04-13 Thread Joerg Schilling
"David Magda" wrote: > Given that ZFS probably would not have to go back to "old" blocks until > it's reached the end of the disk, that should give the SSDs' firmware > plenty of time to do block-remapping and background erasing--something > that's done now anyway regardless of whether an SSD sup

Re: [zfs-discuss] Secure delete?

2010-04-13 Thread Joerg Schilling
Bob Friesenhahn wrote: > Yes of course. Properly built SSDs include considerable extra space > to support wear leveling, and this same space may be used to store > erased blocks. A block which is "overwritten" can simply be written > to a block allocated from the extra free pool, and the exi

Re: [zfs-discuss] Secure delete?

2010-04-13 Thread Joerg Schilling
Bob Friesenhahn wrote: > On Sun, 11 Apr 2010, James Van Artsdalen wrote: > > > OpenSolaris needs support for the TRIM command for SSDs. This > > command is issued to an SSD to indicate that a block is no longer in > > use and the SSD may erase it in preparation for future writes. > > There doe

Re: [zfs-discuss] Secure delete?

2010-04-12 Thread Eric D. Mudama
On Mon, Apr 12 at 10:50, Bob Friesenhahn wrote: On Mon, 12 Apr 2010, Tomas Ögren wrote: For flash to overwrite a block, it needs to clear it first.. so yes, clearing it out in the background (after erasing) instead of just before the timing critical write(), you can make stuff go faster. Yes

Re: [zfs-discuss] Secure delete?

2010-04-12 Thread Edward Ned Harvey
> From: zfs-discuss-boun...@opensolaris.org [mailto:zfs-discuss- > boun...@opensolaris.org] On Behalf Of Eric D. Mudama > > I believe the reason strings of bits "leak" on rotating drives you've > overwritten (other than grown defects) is because of minute off-track > occurances while writing (vibr

Re: [zfs-discuss] Secure delete?

2010-04-12 Thread Mattias Pantzare
On Mon, Apr 12, 2010 at 19:19, David Magda wrote: > On Mon, April 12, 2010 12:28, Tomas Ögren wrote: >> On 12 April, 2010 - David Magda sent me these 0,7K bytes: >> >>> On Mon, April 12, 2010 10:48, Tomas Ögren wrote: >>> >>> > For flash to overwrite a block, it needs to clear it first.. so yes, >

Re: [zfs-discuss] Secure delete?

2010-04-12 Thread Bob Friesenhahn
On Mon, 12 Apr 2010, James Van Artsdalen wrote: TRIM is not a Windows 7 command but rather a device command. I only called it the "Windows 7 TRIM command" since that is how almost all of the original reports in the media described it. It seems best to preserve this original name (as used by

Re: [zfs-discuss] Secure delete?

2010-04-12 Thread James Van Artsdalen
My point is not to advocate the TRIM command - those issues are already well-known - but rather suggest that the code that sends TRIM is also a good place to securely erase data on other media, such a hard disk. TRIM is not a Windows 7 command but rather a device command. FreeBSD's CAM layer a

Re: [zfs-discuss] Secure delete?

2010-04-12 Thread David Magda
On Mon, April 12, 2010 12:28, Tomas Ögren wrote: > On 12 April, 2010 - David Magda sent me these 0,7K bytes: > >> On Mon, April 12, 2010 10:48, Tomas Ögren wrote: >> >> > For flash to overwrite a block, it needs to clear it first.. so yes, >> > clearing it out in the background (after erasing) inst

Re: [zfs-discuss] Secure delete?

2010-04-12 Thread Mattias Pantzare
>> OpenSolaris needs support for the TRIM command for SSDs.  This command is >> issued to an SSD to indicate that a block is no longer in use and the SSD >> may erase it in preparation for future writes. > > There does not seem to be very much `need' since there are other ways that a > SSD can know

Re: [zfs-discuss] Secure delete?

2010-04-12 Thread Tomas Ögren
On 12 April, 2010 - David Magda sent me these 0,7K bytes: > On Mon, April 12, 2010 10:48, Tomas Ögren wrote: > > On 12 April, 2010 - Bob Friesenhahn sent me these 0,9K bytes: > > > >> Zfs is designed for high thoughput, and TRIM does not seem to improve > >> throughput. Perhaps it is most useful

Re: [zfs-discuss] Secure delete?

2010-04-12 Thread Bob Friesenhahn
On Mon, 12 Apr 2010, David Magda wrote: Except that ZFS does not overwrite blocks because it is copy-on-write. At some time in the (possibly distant) future the ZFS block might become free and then the Windows 7 TRIM command could be used to try to pre-erase it. This might help an intermitt

Re: [zfs-discuss] Secure delete?

2010-04-12 Thread Bob Friesenhahn
On Mon, 12 Apr 2010, Tomas Ögren wrote: For flash to overwrite a block, it needs to clear it first.. so yes, clearing it out in the background (after erasing) instead of just before the timing critical write(), you can make stuff go faster. Yes of course. Properly built SSDs include considera

Re: [zfs-discuss] Secure delete?

2010-04-12 Thread David Magda
On Mon, April 12, 2010 10:48, Tomas Ögren wrote: > On 12 April, 2010 - Bob Friesenhahn sent me these 0,9K bytes: > >> Zfs is designed for high thoughput, and TRIM does not seem to improve >> throughput. Perhaps it is most useful for low-grade devices like USB >> dongles and compact flash. > > For

Re: [zfs-discuss] Secure delete?

2010-04-12 Thread Tomas Ögren
On 12 April, 2010 - Bob Friesenhahn sent me these 0,9K bytes: > On Sun, 11 Apr 2010, James Van Artsdalen wrote: > >> OpenSolaris needs support for the TRIM command for SSDs. This command >> is issued to an SSD to indicate that a block is no longer in use and >> the SSD may erase it in preparati

Re: [zfs-discuss] Secure delete?

2010-04-12 Thread Bob Friesenhahn
On Sun, 11 Apr 2010, James Van Artsdalen wrote: OpenSolaris needs support for the TRIM command for SSDs. This command is issued to an SSD to indicate that a block is no longer in use and the SSD may erase it in preparation for future writes. There does not seem to be very much `need' since t

Re: [zfs-discuss] Secure delete?

2010-04-12 Thread Pablo Méndez Hernández
Hi James: On Mon, Apr 12, 2010 at 06:45, James Van Artsdalen wrote: > OpenSolaris needs support for the TRIM command for SSDs.  This command is > issued to an SSD to indicate that a block is no longer in use and the SSD may > erase it in preparation for future writes. That's what this RFE is a

Re: [zfs-discuss] Secure delete?

2010-04-12 Thread Eric D. Mudama
On Sun, Apr 11 at 22:45, James Van Artsdalen wrote: PS. It is faster for an SSD to write a block of 0xFF than 0 and it's possible some might make that optimization. That's why I suggest erase-to-ones rather than erase-to-zero. Do you have any data to back this up? While I understand the unde

Re: [zfs-discuss] Secure delete?

2010-04-11 Thread James Van Artsdalen
OpenSolaris needs support for the TRIM command for SSDs. This command is issued to an SSD to indicate that a block is no longer in use and the SSD may erase it in preparation for future writes. A SECURE_FREE dataset property might be added that says that when a block is released to free space

Re: [zfs-discuss] Secure delete?

2010-04-11 Thread Bill Sommerfeld
On 04/11/10 12:46, Volker A. Brandt wrote: The most paranoid will replace all the disks and then physically destroy the old ones. I thought the most paranoid will encrypt everything and then forget the key... :-) Actually, I hear that the most paranoid encrypt everything *and then* destroy th

Re: [zfs-discuss] Secure delete?

2010-04-11 Thread Volker A. Brandt
> The most paranoid will replace all the disks and then physically destroy > the old ones. I thought the most paranoid will encrypt everything and then forget the key... :-) Seriously, once encrypted zfs is integrated that's a viable method. Regards -- Volker -- --

Re: [zfs-discuss] Secure delete?

2010-04-11 Thread Bill Sommerfeld
On 04/11/10 10:19, Manoj Joseph wrote: Earlier writes to the file might have left older copies of the blocks lying around which could be recovered. Indeed; to be really sure you need to overwrite all the free space in the pool. If you limit yourself to worrying about data accessible via a re

Re: [zfs-discuss] Secure delete?

2010-04-11 Thread Manoj Joseph
joerg.schill...@fokus.fraunhofer.de wrote: > The secure deletion of the data would be something that hallens before > the file is actually unlinked (e.g. by rm). This secure deletion would > need open the file in a non COW mode. That may not be sufficient. Earlier writes to the file might have lef

Re: [zfs-discuss] Secure delete?

2010-04-11 Thread Michael Schuster, Oracle
On 10.04.10 21:06, Andrey Kuzmin wrote: No, until all snapshots referencing the file in question are removed. Simplest way to understand snapshots is to consider them as references. Any file-system object (say, file or block) is only removed when its reference count drops to zero. another thin

Re: [zfs-discuss] Secure delete?

2010-04-11 Thread Edward Ned Harvey
> From: zfs-discuss-boun...@opensolaris.org [mailto:zfs-discuss- > wrote: > > Hi all > > > > Is it possible to securely delete a file from a zfs dataset/zpool > once it's been snapshotted, meaning "delete (and perhaps overwrite) all > copies of this file"? > > No, until all snapshots referencing

Re: [zfs-discuss] Secure delete?

2010-04-10 Thread Joerg Schilling
Roy Sigurd Karlsbakk wrote: > I guess that's the way I thought it was. Perhaps it would be nice to add such > a feature? If something gets stuck in a truckload of snapshots, say a 40GB > file in the root fs, it'd be nice to just rm --killemall largefile Let us first asume the simple case where

Re: [zfs-discuss] Secure delete?

2010-04-10 Thread Roy Sigurd Karlsbakk
I guess that's the way I thought it was. Perhaps it would be nice to add such a feature? If something gets stuck in a truckload of snapshots, say a 40GB file in the root fs, it'd be nice to just rm --killemall largefile Best regards roy -- Roy Sigurd Karlsbakk (+47) 97542685 r...@karlsbakk.net

Re: [zfs-discuss] Secure delete?

2010-04-10 Thread Andrey Kuzmin
No, until all snapshots referencing the file in question are removed. Simplest way to understand snapshots is to consider them as references. Any file-system object (say, file or block) is only removed when its reference count drops to zero. Regards, Andrey On Sat, Apr 10, 2010 at 10:20 PM, R

[zfs-discuss] Secure delete?

2010-04-10 Thread Roy Sigurd Karlsbakk
Hi all Is it possible to securely delete a file from a zfs dataset/zpool once it's been snapshotted, meaning "delete (and perhaps overwrite) all copies of this file"? Best regards roy -- Roy Sigurd Karlsbakk (+47) 97542685 r...@karlsbakk.net http://blogg.karlsbakk.net/ -- I all pedagogikk er d