Re: [zfs-discuss] SSD vs "hybrid" drive - any advice?
On Thu, 28 Jul 2011, Nico Williams wrote: Seems to me that it'd be nicer to have an interface to raw flash (no wear leveling, direct access to erasure, read, write, read-modify-write [as an optimization]). Then the filesystem could do a much better job of using flash efficiently. But a raw interface wouldn't be a disk-compatible interface. Do you mean like as is already done by Linux UBI and ubifs? http://www.linux-mtd.infradead.org/doc/ubi.html http://www.linux-mtd.infradead.org/doc/ubifs.html This sort of thing is already done in embedded devices. Bob -- Bob Friesenhahn bfrie...@simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintainer,http://www.GraphicsMagick.org/ ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] SSD vs "hybrid" drive - any advice?
On Wed, Jul 27, 2011 at 9:22 PM, Daniel Carosone wrote: > Absent TRIM support, there's another way to do this, too. It's pretty > easy to dd /dev/zero to a file now and then. Just make sure zfs > doesn't prevent these being written to the SSD (compress and dedup are > off). I have a separate "fill" dataset for this purpose, to avoid > keeping these zeros in auto-snapshots too. Nice. Seems to me that it'd be nicer to have an interface to raw flash (no wear leveling, direct access to erasure, read, write, read-modify-write [as an optimization]). Then the filesystem could do a much better job of using flash efficiently. But a raw interface wouldn't be a disk-compatible interface. Nico -- ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] SSD vs "hybrid" drive - any advice?
> > "Processing" the request just means flagging the blocks, though, right? > > And the actual benefits only acrue if the garbage collection / block > > reshuffling background tasks get a chance to run? > > I think that's right. TRIM just gives hints to the garbage collector that > sectors are no longer in use. When the GC runs, it can find more flash > blocks more easily that aren't used or combine several mostly-empty > blocks and erase or otherwise free them for reuse later. Absent TRIM support, there's another way to do this, too. It's pretty easy to dd /dev/zero to a file now and then. Just make sure zfs doesn't prevent these being written to the SSD (compress and dedup are off). I have a separate "fill" dataset for this purpose, to avoid keeping these zeros in auto-snapshots too. At least the sandforce controllers recognise this, via their internal compression and dedup, and know that the blocks that have to be presented to the host as full of zeros can be reclaied internally and reassigned to the spare pool. As long as they have enough blocks to work with, it's fine. -- Dan. pgpUtDFSFlj9f.pgp Description: PGP signature ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] SSD vs "hybrid" drive - any advice?
On Mon, 25 Jul 2011, Edward Ned Harvey wrote: From: zfs-discuss-boun...@opensolaris.org [mailto:zfs-discuss- boun...@opensolaris.org] On Behalf Of Erik Trimble Honestly, I think TRIM isn't really useful for anyone. I'm going to have to disagree. As usual. :-) There are only two times when TRIM isn't useful: Hopefully you would agree that if there is a request to overwrite 128K of existing data that the SSD can then know that the FLASH pages which would be overwritten are no longer needed? This is step three of the three step process. In case #2, it is at least theoretically possible for devices to become smart enough to process the TRIM block erasures in parallel even while there are other operations taking place simultaneously. I don't know if device mfgrs implement things that way today. There is at least a common perception (misperception?) that devices cannot process TRIM requests while they are 100% busy processing other tasks. TRIM is really hard to do safely and efficiently. Consider that a drive may still have many pending TRIM requests while subsequent requests come in to write to those same areas. Improper re-ordering of TRIM and write requests can destroy data. This makes high-performance TRIM difficult to do in a low-cost product. Bob -- Bob Friesenhahn bfrie...@simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintainer,http://www.GraphicsMagick.org/ ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] SSD vs "hybrid" drive - any advice?
On 07/27/11 00:00, Peter Jeremy wrote: On 2011-Jul-26 17:24:05 +0800, "Fajar A. Nugraha" wrote: Shouldn't modern SSD controllers be smart enough already that they know: - if there's a request to overwrite a sector, then the old data on that sector is no longer needed ZFS never does update-in-place and UFS only does update-in-place for Note quite never, there are some very special cases where blocks are allocated ahead of time and could be written to "in place" more than once. In particular the special type of ZVOLs used for dump devices. -- Darren J Moffat ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] SSD vs "hybrid" drive - any advice?
>ZFS never does update-in-place and UFS only does update-in-place for >metadata and where the application forces update-in-place. ufs always updates in place (it will rewrite earlier allocated locks). The only time when it does is when the file is growing and it may move stuff around (when the end of the file is a fragment and it needs to grow) Casper ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] SSD vs "hybrid" drive - any advice?
On 2011-Jul-26 17:24:05 +0800, "Fajar A. Nugraha" wrote:
>Shouldn't modern SSD controllers be smart enough already that they know:
>- if there's a request to overwrite a sector, then the old data on
>that sector is no longer needed
ZFS never does update-in-place and UFS only does update-in-place for
metadata and where the application forces update-in-place. This means
there will generally (always for ZFS) be a delay between when a
filesystem "frees" (is no longer interested in the contents of) a
sector and when it overwrites that sector. Without TRIM support, a
SSD can only use "overwrite" to indicate that the contents of a sector
are not needed. Which, in turn, means there is a pool of sectors that
the FS knows are unused but the SSD doesn't - and is therefore forced
to preserve.
Since an overwrite almost never matches the erase page, this increases
wear on the SSD because it is forced to rewrite unwanted data in order
to free up pages for erasure to support external write requests. It
also reduces performance for several reasons:
- The SSD has to unnecessarily copy data - which takes time.
- The space recovered by each erasure is effectively reduced by the
amount of rewritten data so more time-consuming erasures are needed
for a given external write load.
- The pools of "unused but not erased" and "erased" ("available")
sectors are smaller, increasing the probability that an external
write will require a synchronous erase cycle to complete.
>- allocate a "clean" sector from pool of available sectors (part of
>wear-leveling mechanism)
As above, in the absence of TRIM, the pool will be smaller (and more
likely to be empty).
>- clear the old sector, and add it to the pool (possibly done in
>background operation)
Otherwise a sector could never be rewritten.
>It seems to be the case with sandforce-based SSDs. That would pretty
>much let the SSD work just fine even without TRIM (like when used
>under HW raid).
Better SSDs mitigate the problem by having more "hidden" space
(keeping the "available" pool larger to reduce the probability of a
synchronous erase being needed) and higher performance (masking the
impact of the additional internal writes and erasures).
If TRIM support was available then the performance would still
improve. This means you either get better system performance from
the same SSD, or you can get the same system performance from a
lower-performance (cheaper) SSD.
--
Peter Jeremy
pgpoOozgavEXj.pgp
Description: PGP signature
___
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] SSD vs "hybrid" drive - any advice?
On Tue, Jul 26, 2011 at 7:51 AM, David Dyer-Bennet wrote: > "Processing" the request just means flagging the blocks, though, right? > And the actual benefits only acrue if the garbage collection / block > reshuffling background tasks get a chance to run? > I think that's right. TRIM just gives hints to the garbage collector that sectors are no longer in use. When the GC runs, it can find more flash blocks more easily that aren't used or combine several mostly-empty blocks and erase or otherwise free them for reuse later. -B -- Brandon High : bh...@freaks.com ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] SSD vs "hybrid" drive - any advice?
On Tue, Jul 26, 2011 at 5:59 AM, Edward Ned Harvey < opensolarisisdeadlongliveopensola...@nedharvey.com> wrote: > like 4%, and for some reason (I don't know why) there's a benefit to > optimizing on 8k pages. Which means no. If you overwrite a sector of a > >From what I've heard it's due in large part to the FAT file system, since its used in a lot of embedded systems as well as on flash cards. The FAT cluster size is 32k, so any flash block that's a factor of 32k works well. Page sizes are usually 2k with a 128k erase block, 4k with a 256k erase block, or 4k with a 512k erase block. It's also due to ECC reasons, since a larger block size allows more efficient ECC over a larger block of data. This is similar to move to 4k sectors in magnetic drives. -B -- Brandon High : bh...@freaks.com ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] SSD vs "hybrid" drive - any advice?
On Mon, July 25, 2011 10:03, Orvar Korvar wrote: > "There is at least a common perception (misperception?) that devices > cannot process TRIM requests while they are 100% busy processing other > tasks." > > Just to confirm; SSD disks can do TRIM while processing other tasks? "Processing" the request just means flagging the blocks, though, right? And the actual benefits only acrue if the garbage collection / block reshuffling background tasks get a chance to run? -- David Dyer-Bennet, d...@dd-b.net; http://dd-b.net/ Snapshots: http://dd-b.net/dd-b/SnapshotAlbum/data/ Photos: http://dd-b.net/photography/gallery/ Dragaera: http://dragaera.info ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] SSD vs "hybrid" drive - any advice?
> From: zfs-discuss-boun...@opensolaris.org [mailto:zfs-discuss- > boun...@opensolaris.org] On Behalf Of Fajar A. Nugraha > > Shouldn't modern SSD controllers be smart enough already that they know: > - if there's a request to overwrite a sector, then the old data on > that sector is no longer needed In the present state of the world, somebody I know in the trade describes SSD's as "the pimple on the butt of the elephant" when it comes to flash manufacturing. In other words, mobile devices account for a huge majority (something like 90%) of flash produced in the world, and SSD's are something like 4%, and for some reason (I don't know why) there's a benefit to optimizing on 8k pages. Which means no. If you overwrite a sector of a SSD, that does not mean you can erase the page. Because you can only erase the whole page, and the disk can only interact with the OS using 4k blocks or smaller. So there's a minimum of 2 logical blocks per page in the SSD. When you trim a block, only half of the page gets marked as free. Eventually the controller needs to read half a block from page A, half a block from page B, write them both to blank page C, and then erase pages A and B. > - allocate a "clean" sector from pool of available sectors (part of > wear-leveling mechanism) > - clear the old sector, and add it to the pool (possibly done in > background operation) The complexity here is much larger... In all the storage pages in the SSD, some are marked used, some are marked unused, some are erased, and some are not erased. You can only write to a sector if it's both unused and erased. Each sector takes half a page. You can write to an individual sector, but you cannot erase an individual sector. At the OS interface, only sectors are logically addressed, but internally the controller must map those to physical halves of pages. So the controller maintains a completely arbitrary lookup table so any sector can map to any sector or page. When the OS requests to overwrite some sector, the controller will actually write to some formerly unused sector and remap and mark the old one as unused. Later in the background, if the other half of the page is also unused, the page will be erased. Does that clarify anything? ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] SSD vs "hybrid" drive - any advice?
>Shouldn't modern SSD controllers be smart enough already that they know: >- if there's a request to overwrite a sector, then the old data on >that sector is no longer needed >- allocate a "clean" sector from pool of available sectors (part of >wear-leveling mechanism) >- clear the old sector, and add it to the pool (possibly done in >background operation) > >It seems to be the case with sandforce-based SSDs. That would pretty >much let the SSD work just fine even without TRIM (like when used >under HW raid). That is possibly not sufficient. If ZFS writes bytes to every sector, even though the pool is not full, the controller cannot know where to reclaim the data. If it uses "spare" sectors then it can map them to the to the new data and add the overwritten sectors to the free pool. With TRIM, it gets more blocks to reuse and it gives more time to erase them, making the SSD faster. Casper ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] SSD vs "hybrid" drive - any advice?
On Tue, Jul 26, 2011 at 3:28 PM, wrote: > > >>Bullshit. I just got a OCZ Vertex 3, and the first fill was 450-500MB/s. >>Second and sequent fills are at half that speed. I'm quite confident >>that it's due to the flash erase cycle that's needed, and if stuff can >>be TRIM:ed (and thus flash erased as well), speed would be regained. >>Overwriting an previously used block requires a flash erase, and if that >>can be done in the background when the timing is not critical instead of >>just before you can actually write the block you want, performance will >>increase. > > I think TRIM is needed both for flash (for speed) and for > thin provisioning; ZFS will dirty all of the volume even though only a > small part of the volume is used at any particular time. That makes ZFS > more or less unusable with thin provisioning; support for TRIM would fix > that if the underlying volume management supports TRIM. > > Casper Shouldn't modern SSD controllers be smart enough already that they know: - if there's a request to overwrite a sector, then the old data on that sector is no longer needed - allocate a "clean" sector from pool of available sectors (part of wear-leveling mechanism) - clear the old sector, and add it to the pool (possibly done in background operation) It seems to be the case with sandforce-based SSDs. That would pretty much let the SSD work just fine even without TRIM (like when used under HW raid). -- Fajar ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] SSD vs "hybrid" drive - any advice?
>Bullshit. I just got a OCZ Vertex 3, and the first fill was 450-500MB/s. >Second and sequent fills are at half that speed. I'm quite confident >that it's due to the flash erase cycle that's needed, and if stuff can >be TRIM:ed (and thus flash erased as well), speed would be regained. >Overwriting an previously used block requires a flash erase, and if that >can be done in the background when the timing is not critical instead of >just before you can actually write the block you want, performance will >increase. I think TRIM is needed both for flash (for speed) and for thin provisioning; ZFS will dirty all of the volume even though only a small part of the volume is used at any particular time. That makes ZFS more or less unusable with thin provisioning; support for TRIM would fix that if the underlying volume management supports TRIM. Casper ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] SSD vs "hybrid" drive - any advice?
> From: zfs-discuss-boun...@opensolaris.org [mailto:zfs-discuss- > boun...@opensolaris.org] On Behalf Of Erik Trimble > > SSDs do Garbage Collection when the controller has spare cycles. I'm not > certain if there is a time factor (i.e. is it periodic, or just when > there's time in the controller's queue). So, theoretically, TRIM helps > GC when the drive is at low utilization, but not when the SSD is under > significant load. Under high load, the SSD doesn't have the luxury of > searching the NAND for "unused" blocks, aggregating them, writing them > to a new page, and then erasing the old location. It has to allocate > stuff NOW, so it goes right to the dreaded read-modify-erase-write > cycle. Even under high load, the SSD can "process" the TRIM request > (i.e. it will mark a block as unused), but that's not useful until a GC > is performed (unless you are so lucky as to mark an *entire* page as > unused), so, it doesn't really matter. The GC run is what "fixes" the > NAND allocation, not the TRIM command itself. This is really all a detail which is device specific. If the SSD controller is designed to do so, it can simultaneously erase one block on one bank, while performing RMEW or whatever operation on another bank. Again, I don't know if/when/which drives are built with that design, but I know it's not architecturally impossible. In which case, the TRIM would be useful, even for devices that are working under maximum load. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] SSD vs "hybrid" drive - any advice?
On 7/25/2011 8:03 AM, Orvar Korvar wrote: "There is at least a common perception (misperception?) that devices cannot process TRIM requests while they are 100% busy processing other tasks." Just to confirm; SSD disks can do TRIM while processing other tasks? I heard that Illumos is working on TRIM support for ZFS and will release something soon. Anyone knows more? SSDs do Garbage Collection when the controller has spare cycles. I'm not certain if there is a time factor (i.e. is it periodic, or just when there's time in the controller's queue). So, theoretically, TRIM helps GC when the drive is at low utilization, but not when the SSD is under significant load. Under high load, the SSD doesn't have the luxury of searching the NAND for "unused" blocks, aggregating them, writing them to a new page, and then erasing the old location. It has to allocate stuff NOW, so it goes right to the dreaded read-modify-erase-write cycle. Even under high load, the SSD can "process" the TRIM request (i.e. it will mark a block as unused), but that's not useful until a GC is performed (unless you are so lucky as to mark an *entire* page as unused), so, it doesn't really matter. The GC run is what "fixes" the NAND allocation, not the TRIM command itself. I can't speak for the ZFS developers as to TRIM support. I *believe* this would have to happen both at the device level and the filesystem level. But, I certainly could be wrong. (IllumOS currently supports TRIM in the SATA framework - not sure about the SAS framework) -- Erik Trimble Java Platform Group Infrastructure Mailstop: usca22-317 Phone: x67195 Santa Clara, CA Timezone: US/Pacific (UTC-0800) ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] SSD vs "hybrid" drive - any advice?
On 7/25/2011 4:49 AM, joerg.schill...@fokus.fraunhofer.de wrote: Erik Trimble wrote: On 7/25/2011 3:32 AM, Orvar Korvar wrote: How long have you been using a SSD? Do you see any performance decrease? I mean, ZFS does not support TRIM, so I wonder about long term effects... Frankly, for the kind of use that ZFS puts on a SSD, TRIM makes no impact whatsoever. TRIM is primarily useful for low-volume changes - that is, for a filesystem that generally has few deletes over time (i.e. rate of change is low). Using a SSD as a ZIL or L2ARC device puts a very high write load on the device (even as an L2ARC, there is a considerably higher write load than a "typical" filesystem use). SSDs in such a configuration can't really make use of TRIM, and depend on the internal SSD controller block re-allocation algorithms to improve block layout. Now, if you're using the SSD as primary media (i.e. in place of a Hard Drive), there is a possibility that TRIM could help. I honestly can't be sure that it would help, however, as ZFS's Copy-on-Write nature means that it tends to write entire pages of blocks, rather than just small blocks. Which is fine from the SSD's standpoint. Writing to an SSD is: clear + write + verify As the SSD cannot know that the rewritten blocks have been unused for a while, the SSD cannot handle the clear operation at a time when there is no interest in the block, the TRIM command is needed to give this knowledge to the SSD. Jörg Except in many cases with ZFS, that data is irrelevant by the time it can be used, or is much less useful than with other filesystems. Copy-on-Write tends to end up with whole SSD pages of blocks being rendered "unused", rather than individual blocks inside pages. So, the SSD often can avoid the read-erase-modify-write cycle, and just do erase-write instead. TRIM *might* help somewhat when you have a relatively quiet ZFS filesystem, but I'm not really convinced of how much of a benefit it would be. As I've mentioned in other posts, ZIL and L2ARC are too "hot" for TRIM to have any noticeable impact - the SSD is constantly being used, and has no time for GC. It's stuck in the read-erase-modify-write cycle even with TRIM. -- Erik Trimble Java Platform Group Infrastructure Mailstop: usca22-317 Phone: x67195 Santa Clara, CA Timezone: US/Pacific (UTC-0800) ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] SSD vs "hybrid" drive - any advice?
On 7/25/2011 4:28 AM, Tomas Ögren wrote: On 25 July, 2011 - Erik Trimble sent me these 2,0K bytes: On 7/25/2011 3:32 AM, Orvar Korvar wrote: How long have you been using a SSD? Do you see any performance decrease? I mean, ZFS does not support TRIM, so I wonder about long term effects... Frankly, for the kind of use that ZFS puts on a SSD, TRIM makes no impact whatsoever. TRIM is primarily useful for low-volume changes - that is, for a filesystem that generally has few deletes over time (i.e. rate of change is low). Using a SSD as a ZIL or L2ARC device puts a very high write load on the device (even as an L2ARC, there is a considerably higher write load than a "typical" filesystem use). SSDs in such a configuration can't really make use of TRIM, and depend on the internal SSD controller block re-allocation algorithms to improve block layout. Now, if you're using the SSD as primary media (i.e. in place of a Hard Drive), there is a possibility that TRIM could help. I honestly can't be sure that it would help, however, as ZFS's Copy-on-Write nature means that it tends to write entire pages of blocks, rather than just small blocks. Which is fine from the SSD's standpoint. You still need the flash erase cycle. On a related note: I've been using a OCZ Vertex 2 as my primary drive in a laptop, which runs Windows XP (no TRIM support). I haven't noticed any dropoff in performance in the year its be in service. I'm doing typical productivity laptop-ish things (no compiling, etc.), so it appears that the internal SSD controller is more than smart enough to compensate even without TRIM. Honestly, I think TRIM isn't really useful for anyone. It took too long to get pushed out to the OSes, and the SSD vendors seem to have just compensated by making a smarter controller able to do better reallocation. Which, to me, is the better ideal, in any case. Bullshit. I just got a OCZ Vertex 3, and the first fill was 450-500MB/s. Second and sequent fills are at half that speed. I'm quite confident that it's due to the flash erase cycle that's needed, and if stuff can be TRIM:ed (and thus flash erased as well), speed would be regained. Overwriting an previously used block requires a flash erase, and if that can be done in the background when the timing is not critical instead of just before you can actually write the block you want, performance will increase. /Tomas I should have been more clear: I consider the "native" speed of a SSD to be that which is obtained AFTER you've filled the entire drive once. That is, once you've blown through the extra reserve NAND, and are now into the full read/erase/write cycle. IMHO, that's really what the sustained performance of an SSD is, not the bogus numbers reported by venders. TRIM is really only useful for drives which have a low enough load factor to do background GC on unused blocks. For ZFS, that *might* be the case when the SSD is used as primary backing store, but certainly isn't the case when it's used as ZIL or L2ARC. Even with TRIM, performance after a complete fill of the SSD will drop noticeable, as the SSD has to do GC sometime. You might not notice it right away given your usage pattern, but, with OR without TRIM, a "used" SSD under load will perform the same. -- Erik Trimble Java Platform Group Infrastructure Mailstop: usca22-317 Phone: x67195 Santa Clara, CA Timezone: US/Pacific (UTC-0800) ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] SSD vs "hybrid" drive - any advice?
On 7/25/2011 6:43 AM, Edward Ned Harvey wrote: From: zfs-discuss-boun...@opensolaris.org [mailto:zfs-discuss- boun...@opensolaris.org] On Behalf Of Erik Trimble Honestly, I think TRIM isn't really useful for anyone. I'm going to have to disagree. There are only two times when TRIM isn't useful: 1) Your demand of the system is consistently so low that it never adds up to anything meaningful... Basically you always have free unused blocks so adding more unused blocks to the pile doesn't matter at all, or you never bother to delete anything... Or it's just a lightweight server processing requests where network latency greatly outweighs any disk latency, etc. AKA your demand is very low. or 2) Your demand of the system is consistently so high that even with TRIM, the device would never be able to find any idle time to perform an erase cycle on blocks marked for TRIM. In case #2, it is at least theoretically possible for devices to become smart enough to process the TRIM block erasures in parallel even while there are other operations taking place simultaneously. I don't know if device mfgrs implement things that way today. There is at least a common perception (misperception?) that devices cannot process TRIM requests while they are 100% busy processing other tasks. Or your disk is always 100% full. I guess that makes 3 cases, but the 3rd one is esoteric. What I'm saying is that #2 occurs all the time with ZFS, at least as a ZIL or L2ARC. TRIM is really only useful when the SSD has some "downtime" to work. As a ZIL or L2ARC, the SSD *has* no pauses, and can't do GC in the background usefully (which is what TRIM helps). Instead, what I've seen is that the increased "smarts" of the new generation SSD controllers do a better job of on-the-fly reallocation. -- Erik Trimble Java Platform Group Infrastructure Mailstop: usca22-317 Phone: x67195 Santa Clara, CA Timezone: US/Pacific (UTC-0800) ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] SSD vs "hybrid" drive - any advice?
"There is at least a common perception (misperception?) that devices cannot process TRIM requests while they are 100% busy processing other tasks." Just to confirm; SSD disks can do TRIM while processing other tasks? I heard that Illumos is working on TRIM support for ZFS and will release something soon. Anyone knows more? -- This message posted from opensolaris.org ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] SSD vs "hybrid" drive - any advice?
> From: zfs-discuss-boun...@opensolaris.org [mailto:zfs-discuss- > boun...@opensolaris.org] On Behalf Of Erik Trimble > > Honestly, I think TRIM isn't really useful for anyone. I'm going to have to disagree. There are only two times when TRIM isn't useful: 1) Your demand of the system is consistently so low that it never adds up to anything meaningful... Basically you always have free unused blocks so adding more unused blocks to the pile doesn't matter at all, or you never bother to delete anything... Or it's just a lightweight server processing requests where network latency greatly outweighs any disk latency, etc. AKA your demand is very low. or 2) Your demand of the system is consistently so high that even with TRIM, the device would never be able to find any idle time to perform an erase cycle on blocks marked for TRIM. In case #2, it is at least theoretically possible for devices to become smart enough to process the TRIM block erasures in parallel even while there are other operations taking place simultaneously. I don't know if device mfgrs implement things that way today. There is at least a common perception (misperception?) that devices cannot process TRIM requests while they are 100% busy processing other tasks. Or your disk is always 100% full. I guess that makes 3 cases, but the 3rd one is esoteric. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] SSD vs "hybrid" drive - any advice?
Erik Trimble wrote: > On 7/25/2011 3:32 AM, Orvar Korvar wrote: > > How long have you been using a SSD? Do you see any performance decrease? I > > mean, ZFS does not support TRIM, so I wonder about long term effects... > > Frankly, for the kind of use that ZFS puts on a SSD, TRIM makes no > impact whatsoever. > > TRIM is primarily useful for low-volume changes - that is, for a > filesystem that generally has few deletes over time (i.e. rate of change > is low). > > Using a SSD as a ZIL or L2ARC device puts a very high write load on the > device (even as an L2ARC, there is a considerably higher write load than > a "typical" filesystem use). SSDs in such a configuration can't really > make use of TRIM, and depend on the internal SSD controller block > re-allocation algorithms to improve block layout. > > Now, if you're using the SSD as primary media (i.e. in place of a Hard > Drive), there is a possibility that TRIM could help. I honestly can't > be sure that it would help, however, as ZFS's Copy-on-Write nature means > that it tends to write entire pages of blocks, rather than just small > blocks. Which is fine from the SSD's standpoint. Writing to an SSD is: clear + write + verify As the SSD cannot know that the rewritten blocks have been unused for a while, the SSD cannot handle the clear operation at a time when there is no interest in the block, the TRIM command is needed to give this knowledge to the SSD. Jörg -- EMail:jo...@schily.isdn.cs.tu-berlin.de (home) Jörg Schilling D-13353 Berlin j...@cs.tu-berlin.de(uni) joerg.schill...@fokus.fraunhofer.de (work) Blog: http://schily.blogspot.com/ URL: http://cdrecord.berlios.de/private/ ftp://ftp.berlios.de/pub/schily ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] SSD vs "hybrid" drive - any advice?
On 25 July, 2011 - Erik Trimble sent me these 2,0K bytes:
> On 7/25/2011 3:32 AM, Orvar Korvar wrote:
>> How long have you been using a SSD? Do you see any performance decrease? I
>> mean, ZFS does not support TRIM, so I wonder about long term effects...
>
> Frankly, for the kind of use that ZFS puts on a SSD, TRIM makes no
> impact whatsoever.
>
> TRIM is primarily useful for low-volume changes - that is, for a
> filesystem that generally has few deletes over time (i.e. rate of change
> is low).
>
> Using a SSD as a ZIL or L2ARC device puts a very high write load on the
> device (even as an L2ARC, there is a considerably higher write load than
> a "typical" filesystem use). SSDs in such a configuration can't really
> make use of TRIM, and depend on the internal SSD controller block
> re-allocation algorithms to improve block layout.
>
> Now, if you're using the SSD as primary media (i.e. in place of a Hard
> Drive), there is a possibility that TRIM could help. I honestly can't
> be sure that it would help, however, as ZFS's Copy-on-Write nature means
> that it tends to write entire pages of blocks, rather than just small
> blocks. Which is fine from the SSD's standpoint.
You still need the flash erase cycle.
> On a related note: I've been using a OCZ Vertex 2 as my primary drive
> in a laptop, which runs Windows XP (no TRIM support). I haven't noticed
> any dropoff in performance in the year its be in service. I'm doing
> typical productivity laptop-ish things (no compiling, etc.), so it
> appears that the internal SSD controller is more than smart enough to
> compensate even without TRIM.
>
>
> Honestly, I think TRIM isn't really useful for anyone. It took too long
> to get pushed out to the OSes, and the SSD vendors seem to have just
> compensated by making a smarter controller able to do better
> reallocation. Which, to me, is the better ideal, in any case.
Bullshit. I just got a OCZ Vertex 3, and the first fill was 450-500MB/s.
Second and sequent fills are at half that speed. I'm quite confident
that it's due to the flash erase cycle that's needed, and if stuff can
be TRIM:ed (and thus flash erased as well), speed would be regained.
Overwriting an previously used block requires a flash erase, and if that
can be done in the background when the timing is not critical instead of
just before you can actually write the block you want, performance will
increase.
/Tomas
--
Tomas Ögren, st...@acc.umu.se, http://www.acc.umu.se/~stric/
`- Sysadmin at {cs,acc}.umu.se
___
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] SSD vs "hybrid" drive - any advice?
On 7/25/2011 3:32 AM, Orvar Korvar wrote: How long have you been using a SSD? Do you see any performance decrease? I mean, ZFS does not support TRIM, so I wonder about long term effects... Frankly, for the kind of use that ZFS puts on a SSD, TRIM makes no impact whatsoever. TRIM is primarily useful for low-volume changes - that is, for a filesystem that generally has few deletes over time (i.e. rate of change is low). Using a SSD as a ZIL or L2ARC device puts a very high write load on the device (even as an L2ARC, there is a considerably higher write load than a "typical" filesystem use). SSDs in such a configuration can't really make use of TRIM, and depend on the internal SSD controller block re-allocation algorithms to improve block layout. Now, if you're using the SSD as primary media (i.e. in place of a Hard Drive), there is a possibility that TRIM could help. I honestly can't be sure that it would help, however, as ZFS's Copy-on-Write nature means that it tends to write entire pages of blocks, rather than just small blocks. Which is fine from the SSD's standpoint. On a related note: I've been using a OCZ Vertex 2 as my primary drive in a laptop, which runs Windows XP (no TRIM support). I haven't noticed any dropoff in performance in the year its be in service. I'm doing typical productivity laptop-ish things (no compiling, etc.), so it appears that the internal SSD controller is more than smart enough to compensate even without TRIM. Honestly, I think TRIM isn't really useful for anyone. It took too long to get pushed out to the OSes, and the SSD vendors seem to have just compensated by making a smarter controller able to do better reallocation. Which, to me, is the better ideal, in any case. -- Erik Trimble Java Platform Group Infrastructure Mailstop: usca22-317 Phone: x67195 Santa Clara, CA Timezone: US/Pacific (UTC-0800) ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] SSD vs "hybrid" drive - any advice?
How long have you been using a SSD? Do you see any performance decrease? I mean, ZFS does not support TRIM, so I wonder about long term effects... -- This message posted from opensolaris.org ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] SSD vs "hybrid" drive - any advice?
On Jul 21, 2011, at 4:08 PM, Gordon Ross wrote: > I'm looking to upgrade the disk in a high-end laptop (so called > "desktop replacement" type). I use it for development work, > runing OpenIndiana (native) with lots of ZFS data sets. > > These "hybrid" drives look kind of interesting, i.e. for about $100, > one can get: > Seagate Momentus XT ST95005620AS 500GB 7200 RPM 2.5" SATA 3.0Gb/s > with NCQ Solid State Hybrid Drive > http://www.newegg.com/Product/Product.aspx?Item=N82E16822148591 > And then for about $400 one can get an 250GB SSD, such as: > Crucial M4 CT256M4SSD2 2.5" 256GB SATA III MLC Internal Solid State > Drive (SSD) > http://www.newegg.com/Product/Product.aspx?Item=N82E16820148443 > > Anyone have experience with either one? (good or bad) > > Opinions whether the lower capacity and higher cost of > the SSD is justified in terms of performance for things > like software builds, etc? FWIW, I run a Toshiba SSD, 256G, all day, every day. Love it. No plans to go back to spinning rust on the laptop... ever :-) -- richard ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] SSD vs "hybrid" drive - any advice?
> From: zfs-discuss-boun...@opensolaris.org [mailto:zfs-discuss- > boun...@opensolaris.org] On Behalf Of Orvar Korvar > > So boot is much quicker. > > Everyday use, I dont notice anything. Every application boots quick, and I > dont think about application boot time anymore. OT but... At work, we wanted to evaluate SSD vs HDD as laptop boot drives. So we bought a batch of identical laptops, some with SSD and others with HDD. Built them with exactly the same OS image. I put a SSD side-by-side with a HDD laptop and did exactly the same things on each one. Result was: The SSD boots 2x faster. After bootup, applications launch 2x faster. After caching, of course, they're both identical. So it goes like this: Whenever somebody upgrades to a faster system, they say "I don't notice anything." But whenever somebody downgrades to a slower system, they complain about the end of the world and junk computers. Here's another OT anecdote: The company supported platform is windows. But my group is largely a bunch of mac users, so everyone gets a win7 VM inside of OSX. When somebody's new and looking at a mac for the first time, they don't really say anything about speed... You sit down with them and perform the exact same tasks inside of windows and inside of osx, and nobody says anything about speed. But then you give them the laptop, and after a week or two they start to notice... You perform some task such as launching firefox or anything else ... repeat in windows and osx... and windows is 2x faster. Inside the VM, that's quite surprising. ;-) This is something I always notice on their new laptops just because I've become hyper sensitive to it... But users don't notice it until they've become accustomed to it after a couple of weeks. Incidentally, these are all 8G laptops, with 3G assigned to windows and 5G assigned to osx. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] SSD vs "hybrid" drive - any advice?
I am using the OCZ Vertex 3, 240GB. When I boot Solaris 11 Express, on the splash screen, there is a small red line traveling from left to right. With this SSD, the red line is traveling two times over the screen before S11E has booted up. With hard disk, the red line traveled several times over the screen before boot. So boot is much quicker. Everyday use, I dont notice anything. Every application boots quick, and I dont think about application boot time anymore. One problem though, the OCZ Vertex 3 has bugs in it's firmware. This manifests as, randomly, an application will just hang. What happens is that an application window will be grayed out, and it will not accept any input or anything. So I just kill the application and restart it. Quite annoying, but I dont know what to do about it. -- This message posted from opensolaris.org ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] SSD vs "hybrid" drive - any advice?
On Thu, Jul 21, 2011 at 4:08 PM, Gordon Ross wrote: > And then for about $400 one can get an 250GB SSD, such as: > Crucial M4 CT256M4SSD2 2.5" 256GB SATA III MLC Internal Solid State > Drive (SSD) > http://www.newegg.com/Product/Product.aspx?Item=N82E16820148443 > > Anyone have experience with either one? (good or bad) The hybrid drive might accelerate some operations. No guarantees, though. It's about as fast as a WD Velociraptor in some operations, and the same as the regular Seagate 500gb in others. There is a decent review of it at Anandtech. The M4 is pretty decent, though the Vertex 3 and other Sandforce 2000-based drives beat it in benchmarks. Honestly though, you'll probably be very happy with any recent SSD, eg: C300, M4, Intel 320, Intel 510, Sandforce 1200-based (Vertex 2, Phoenix Pro, etc), Sandforce 2200-based (Vertex 3, Corsair Force GT, Patriot Wildfire, etc). -B -- Brandon High : bh...@freaks.com ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] SSD vs "hybrid" drive - any advice?
> From: zfs-discuss-boun...@opensolaris.org [mailto:zfs-discuss- > boun...@opensolaris.org] On Behalf Of Gordon Ross > > Anyone have experience with either one? (good or bad) > > Opinions whether the lower capacity and higher cost of > the SSD is justified in terms of performance for things > like software builds, etc? The hybrid has basically 4G NV cache. The rest is regular hard drive. It will work well if you care about accelerating your boot time and you reboot regularly. But if you don't reboot regularly, and you actually have enough ram in your system that you don't constantly fetch the same sectors over and over... Well the drive can only memorize what it's told to memorize. So it will only memorize things that your OS actually fetches over and over... Which isn't very likely to happen except by repeat reboots. Depends on your behavior patterns. They don't say anything about their algorithms, of course. So it's totally unclear if they use some of the SSD to buffer writes... The hybrid might only benefit reads under the right conditions, and it might not benefit writes at all... Don't really know. My opinion: I'm skeptical about the success of the hybrid drive accelerating performance. I'd like to be proven wrong. Do you have $0.98 change for a $1 bill? ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] SSD vs "hybrid" drive - any advice?
Gordon, If I were you, I would choose SSD. I have good experience with Intel 320 SSD. My company use it for L2ARC with good result. And quality is good Rocky -Original Message- From: zfs-discuss-boun...@opensolaris.org [mailto:zfs-discuss-boun...@opensolaris.org] On Behalf Of Gordon Ross Sent: Thursday, July 21, 2011 4:09 PM To: zfs-discuss@opensolaris.org Subject: [zfs-discuss] SSD vs "hybrid" drive - any advice? I'm looking to upgrade the disk in a high-end laptop (so called "desktop replacement" type). I use it for development work, runing OpenIndiana (native) with lots of ZFS data sets. These "hybrid" drives look kind of interesting, i.e. for about $100, one can get: Seagate Momentus XT ST95005620AS 500GB 7200 RPM 2.5" SATA 3.0Gb/s with NCQ Solid State Hybrid Drive http://www.newegg.com/Product/Product.aspx?Item=N82E16822148591 And then for about $400 one can get an 250GB SSD, such as: Crucial M4 CT256M4SSD2 2.5" 256GB SATA III MLC Internal Solid State Drive (SSD) http://www.newegg.com/Product/Product.aspx?Item=N82E16820148443 Anyone have experience with either one? (good or bad) Opinions whether the lower capacity and higher cost of the SSD is justified in terms of performance for things like software builds, etc? Thanks, Gordon ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
[zfs-discuss] SSD vs "hybrid" drive - any advice?
I'm looking to upgrade the disk in a high-end laptop (so called "desktop replacement" type). I use it for development work, runing OpenIndiana (native) with lots of ZFS data sets. These "hybrid" drives look kind of interesting, i.e. for about $100, one can get: Seagate Momentus XT ST95005620AS 500GB 7200 RPM 2.5" SATA 3.0Gb/s with NCQ Solid State Hybrid Drive http://www.newegg.com/Product/Product.aspx?Item=N82E16822148591 And then for about $400 one can get an 250GB SSD, such as: Crucial M4 CT256M4SSD2 2.5" 256GB SATA III MLC Internal Solid State Drive (SSD) http://www.newegg.com/Product/Product.aspx?Item=N82E16820148443 Anyone have experience with either one? (good or bad) Opinions whether the lower capacity and higher cost of the SSD is justified in terms of performance for things like software builds, etc? Thanks, Gordon ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
