On Thu, Jan 2, 2020 at 11:23 AM Mick <michaelkintz...@gmail.com> wrote:
>
> On Thursday, 2 January 2020 14:43:58 GMT Rich Freeman wrote:
>
>
> > Out of curiosity, what model drive is it?  Is it by chance an SMR /
> > archive drive?
>
> Good catch!  I hadn't thought of this - the Linux kernel will need to have
> DM_ZONED enabled I think, for the OS to manager the shingled writes
> sequentially, but I don't have this enabled here because AFAIK I have no such
> drives in my possession.

I haven't looked into the details of how it works.  Certainly the
kernel should be able to optimize writes to such disks and utilize
TRIM if supported by the drive.  But it isn't strictly needed as you
go on to say.

> > Due to the limitations on how those write data out I
> > could see them implementing an internal filesystem that journals
> > incoming data and then writes it back out after the fact.
>
> SMR drives which implement a 'device managed' write mechanism, will use their
> own firmware to control data storage.  The OS would not be aware of anything
> being different to a conventional drive.

Correct.

> > If so then
> > that might happen even after the kernel thinks it is unmounted.
> > However, such a drive firmware would probably use a journal that
> > ensures data is safe even if power is cut mid-operation.  The drive
> > isn't supposed to report that a write is completed until it is
> > durable.
>
> Which I take it to mean the drive would not be unmounted by the OS until it is
> safe to do so and for all intends and purposes it will also be safe to be
> powered down thereafter.

Yes - even if the drive is doing its own data shuffling after being
unmounted, it should still be safe to power off.

In any case, I was just speculating as to why it might be doing writes
when not mounted.  I don't know the internal details of how these
drives all work.

>  I would think this would be within seconds of
> successfully unmounting it.  Spinning for 30 minutes or more after it is
> unmounted sounds excessive to me, if it is only being spun by the firmware for
> flushing its journal buffers.

Indeed.  That really makes me wonder whether it is actually writing
anything.  It could just be that the drive has some vibration or
otherwise is giving the sensation that it is doing something when it
isn't.

Though, if the thing really does have a large journal inside to
improve performance it could actually buffer a pretty large number of
writes.  SMR drives can have a very large amount of write
amplification.  If each erase region on the disk contains 10k blocks
then every 1MB of data in the journal could potentially lead to 10GB
of disk rewrites, assuming that writes are randomly distributed.  It
also makes sense that when replaying the journal the drive is going to
prioritize erase regions with the most updates, which means that by
the time you're unmounting the drive you're going to expect the stuff
left in the journal to lead to the most write amplification.

I would imagine that a drive that works in this manner is going to use
a logfs for the journal (if the journal is even in SMR format), and
then it would just keep an erase region free at all times.  Anytime a
partial write happens in a region the data would get written to the
free region, then that region would be remapped in place of the old
region, and now the old region is unallocated for the next write.
This could be interrupted due to power loss at any time without any
real loss to data, since data is not overwritten in place.  Before the
journal record is closed out the old region is still valid, and after
the new region is valid.  An interrupted write is just repeated on
power up since the new region can just be overwritten again safely.

In any case, the bottom line is that a drive should be safe to unplug
if all filesystems on it are umounted and the umount commands all
return.  If the disk loses data after that point it is a manufacturing
design flaw.

-- 
Rich

Reply via email to