On Fri, Apr 6, 2018 at 2:56 AM, Borja Marcos <bor...@sarenet.es> wrote:
> > > On 6 Apr 2018, at 10:41, Steven Hartland <kill...@multiplay.co.uk> wrote: > > That is very hw and use case dependent. > > The reason we originally sponsored the project to add TRIM to ZFS was that > in our case without TRIM the performance got so bad that we had to secure > erase disks every couple of weeks as they simply became so slow they where > unusable. > > Now admittedly that was a fair few years ago and hw has moved on since > then, but the point remains that it’s not totally true that just not > TRIMing is an option. > > > As far as I know, letting the device know that you have released a block > should be a Good Thing. > So I prefer to TRIM. I have also seen cases where not doing TRIMs resulted > in a terrible performance > although modern SSDs should have better algorithms to deal with it. > > But I have also seen bad cases of TRIM requests piling up and clogging the > queues for long periods > of time. Admittedly it was a purely synthetic situation (running bonnie++) > but, again, performing > operations like destroying a huge snapshot or dataset could trigger a > similar condition. It was > especially nasty when I tried NVMEs. I mentioned this here: > > https://lists.freebsd.org/pipermail/freebsd-fs/2016-May/023134.html > > > That’s why I think there is a mid point between the radical approach of > not doing TRIMs at all > and clogging the queues with too many of them. > nvd is especially troublesome. It doesn't do any trim shaping *AT*ALL* and we machine gun a huge number of TRIM requests when we delete large files. TRIM is normally not a fast path operation inside the drives, so sending lots of TRIMs down will often starve read/write resources for a variety of reasons (it often forces single threading limiting the parallelism in the drive, it often kicks off GC which causes lots of block erases which are slow and block reads/writes to other pages/blocks on the die (some or all mitigated by planes, but still), etc. > If the TRIM queue gets very large and the “mandatory” operations (read, > writes, etc) begin to > get large delays I think it’s safe to assume that *there is* a problem and > discarding at least part > of those TRIMs could help to solve it. > That's often a property of the devices, however. And how quickly we shove TRIMs down its throat. I've seen situations where we play out the TRIMs slowly enough that we can see huge latencies in TRIMs (seconds to minutes), but see little to no effect effect on read latency, even in the P99 number since that's often where effects in TRIMs show up and are easiest to measure. Warner > A “TRIM when you can” should still be better than a “Don’t TRIM at all”. > > Cheers, > > > > > > Borja. > > P.S: Attaching the graphs that were lost. > > > > > > _______________________________________________ freebsd-stable@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"
