Re: Sending ATA commands?
[...], I wonder if you could attach the HPA area as an additional partition [...] >>> I don't see any reason why not. I'm not sure whether you're >>> proposing that the HPA not be accessible any other way or whether >>> this is just a default. >> or make it a ld device attached to the wd device. > That is a solution that gets around the entire problem of magic > partitions and the like. Yes. Like a lot of good ideas, it's really obvious once it's pointed out. I suspect it's more or less where I would have ended up if I'd followed through on the vague thoughts I had about making it a separate thing of its own. (I probably would have used a new pseudo-disk device rather than piggybacking on ld(4), if only because I care more about pre-ld(4) NetBSD than you do. Though admittedly naming might be a slight issue; there's a sense in which I'd rather see "hpa0 at wd0" instead of "ld0 at wd0\nld0: Host Protected Area"...and this thing, whether an ld instance or something else, is not a normal disk drive; for example, it usually will not be partitioned further.) /~\ The ASCII Mouse \ / Ribbon Campaign X Against HTMLmo...@rodents-montreal.org / \ Email! 7D C8 61 52 5D E7 2D 39 4E F1 31 3E E8 B3 27 4B
Re: Sending ATA commands?
On Aug 11, 9:47pm, Matt Thomas wrote: } Subject: Re: Sending ATA commands? } } On Aug 11, 2013, at 9:37 PM, Mouse wrote: } } >> [...], I wonder if you could attach the HPA area as an additional } >> partition on the default disklabel, or, if the disk is gpt } >> partitioned, fake up another partition in the gpt table. } > } > I don't see any reason why not. I'm not sure whether you're proposing } > that the HPA not be accessible any other way or whether this is just a } > default. } } or make it a ld device attached to the wd device. Hello. That is a solution that gets around the entire problem of magic partitions and the like. And, ld(4) exists in all versions of NetBSD we're interested in. -Brian
Re: Sending ATA commands?
On Aug 11, 2013, at 9:37 PM, Mouse wrote: >> [...], I wonder if you could attach the HPA area as an additional >> partition on the default disklabel, or, if the disk is gpt >> partitioned, fake up another partition in the gpt table. > > I don't see any reason why not. I'm not sure whether you're proposing > that the HPA not be accessible any other way or whether this is just a > default. or make it a ld device attached to the wd device.
Re: Sending ATA commands?
> [...], I wonder if you could attach the HPA area as an additional > partition on the default disklabel, or, if the disk is gpt > partitioned, fake up another partition in the gpt table. I don't see any reason why not. I'm not sure whether you're proposing that the HPA not be accessible any other way or whether this is just a default. > In any case, I think I would be most comfortable with an > implementation which set the HPA area as read-only and only became > read-write through some user command, perhaps a new command for > atactl(8). That strikes me as a reasonably sensible idea. I'm not sure whether I'd want it to be tacked onto the end of the drive or not (eg, for access through RAW_PART) - I don't like having even one partition with magic semantics and really don't want to add another, but I'm not sure how else to do this. > I realize that from a hardware perspective, you may not be able to > set read-only on the HPA once you enable access to it, but I imagine > you could set that restriction in the wd(4) driver. As I understand it, yes, this, would have to be a NetBSD-imposed limitation. I saw nothing in the doc that makes me think there's any way to enable read but not write access to the HPA. > Have you found any BIOS's that can use this HPA area to load saved > NetBSD rescue images as a way of restoring machines [whose] disks > have been corrupted? No, but I haven't looked, either. I have no idea what would need to be in an HPA in order for an HPA-aware BIOS to do anything useful with it; at a stab-in-the-dark guess, I'd guess that it would depend heavily on the particular BIOS in question - which at that point I'm no longer sure deserves the B part of the name. /~\ The ASCII Mouse \ / Ribbon Campaign X Against HTMLmo...@rodents-montreal.org / \ Email! 7D C8 61 52 5D E7 2D 39 4E F1 31 3E E8 B3 27 4B
Re: Sending ATA commands?
Hello. In thinking about your implementation, I wonder if you could attach the HPA area as an additional partition on the default disklabel, or, if the disk is gpt partitioned, fake up another partition in the gpt table. In any case, I think I would be most comfortable with an implementation which set the HPA area as read-only and only became read-write through some user command, perhaps a new command for atactl(8). I realize that from a hardware perspective, you may not be able to set read-only on the HPA once you enable access to it, but I imagine you could set that restriction in the wd(4) driver. That isn't full proof, as you could probably write a program to use the same interface as atactl(8) does to get around the restriction, but it would at least protect you from doing something catastrophic through dd(8) confusion or from a newfs typo. Have you found any BIOS's that can use this HPA area to load saved NetBSD rescue images as a way of restoring machines who's disks have been corrupted? -Brian On Aug 11, 7:19pm, Mouse wrote: } Subject: Re: Sending ATA commands? } [Brian - sorry for the off-list mail; I didn't notice you copied in } tech-kern until I got my copy of the list message!] } } > What does your support do? Does it let you write over the host } > protected area? Does it let you extract what's in there? } } Yes and yes. It simply removes the protection, letting the host see } the HPA as what it really is: more space appended to the space } advertised to HPA-unaware software. As my sample dmesg said, } } >> wd0: effective capacity raised from 18039071 to 18041184 } } and then the subsequent auto-sizing saw the higher limit: } } >> wd0: 8809 MB, 17898 cyl, 16 head, 63 sec, 512 bytes/sect x 18041184 sectors } } > Sorry for the silly questions, but I'm not sure what this support } > buys you. It sounds great, but I'm not sure what I'd get by making } > this work in 5.x or beyond? } } Well, I now have it in 5.2 myself. (Change 4.0.1 in that git repo name } to 5.2 to get that version. And, for anyone who's looking at the 4.0.1 } version, I made a 32-bit mistake which needed fixing; look also at } commits after the ones I named and read the commit messages to find all } the ones you need.) } } I'm not sure this is a good idea in its current form. It works, in } that it raises the soft limit to equal the hard limit - but that makes } it really easy to overwrite supposedly-protected disk by mistake. It's } true that overwriting important data is - and has always been - a } "don't do that, then" mistake, and Unix has run on a "don't prevent } stupid things because that also prevents clever things" philosophy, but } this does make it significantly easier to corrupt presumably-important } data by mistake. } } I'm wondering if it would be better to present the HPA as some kind of } secondary device, with the soft limit raised only long enough to access } it, then lowered back again once the HPA accesses are done. It would } make the HPA slower to access, but it would also make it safer. } } Another issue with what I've done is also related to leaving the limit } raised. According to the documentation I've been working from, on } "power-on or hardware reset", the soft limit is reset to the hard } limit. I know I've been changing the soft limit because it does revert } to what it used to be on power-cycle. But a reboot does not cause it } to revert; apparently whatever reset is done during a reboot on my test } machine isn't hard enough to provoke that change. Presumably it's not } the only such machine, too. } } There's one other interesting thing I noticed, and for this I need a } brief bit of background explanation. When I get a "new" disk, one of } the first things I od often (almost always, when it's second-hand and } thus likely has someone else's data on it) is a write-read-compare pass } over the whole drive. I have a program I wrote when playing with } RAIDframe, designed to detect issues leading to two conceptual sectors } ending up on the same real sector: it writes data to the whole disk } such that each sector's data uniquely identifies the sector. On } reading and comparing, I can then notice multiple conceptual sectors } ending up in the same real sector. On a real disk that's not likely, } but it _is_ an effective wipe (for the sort of threat model appropriate } to me, at least). } } This is relevant because, when I took my test disk and read the } 2113-sector HPA on it, I found my test pattern in many of the sectors. } The only explanation I've come up with is that I did a wipe when I } first got it, then put it in a machine whose BIOS used an HPA to stash } stuff, but didn't overwrite the whole HPA. (Possibly multiple such } machines, even, based on some strings I found in the HPA, with the } disk, probably, effectively slowly shrinking as I move it between } machines.) } } One thing I do not have is any way to change the hard limit
Re: Sending ATA commands?
[Brian - sorry for the off-list mail; I didn't notice you copied in tech-kern until I got my copy of the list message!] > What does your support do? Does it let you write over the host > protected area? Does it let you extract what's in there? Yes and yes. It simply removes the protection, letting the host see the HPA as what it really is: more space appended to the space advertised to HPA-unaware software. As my sample dmesg said, >> wd0: effective capacity raised from 18039071 to 18041184 and then the subsequent auto-sizing saw the higher limit: >> wd0: 8809 MB, 17898 cyl, 16 head, 63 sec, 512 bytes/sect x 18041184 sectors > Sorry for the silly questions, but I'm not sure what this support > buys you. It sounds great, but I'm not sure what I'd get by making > this work in 5.x or beyond? Well, I now have it in 5.2 myself. (Change 4.0.1 in that git repo name to 5.2 to get that version. And, for anyone who's looking at the 4.0.1 version, I made a 32-bit mistake which needed fixing; look also at commits after the ones I named and read the commit messages to find all the ones you need.) I'm not sure this is a good idea in its current form. It works, in that it raises the soft limit to equal the hard limit - but that makes it really easy to overwrite supposedly-protected disk by mistake. It's true that overwriting important data is - and has always been - a "don't do that, then" mistake, and Unix has run on a "don't prevent stupid things because that also prevents clever things" philosophy, but this does make it significantly easier to corrupt presumably-important data by mistake. I'm wondering if it would be better to present the HPA as some kind of secondary device, with the soft limit raised only long enough to access it, then lowered back again once the HPA accesses are done. It would make the HPA slower to access, but it would also make it safer. Another issue with what I've done is also related to leaving the limit raised. According to the documentation I've been working from, on "power-on or hardware reset", the soft limit is reset to the hard limit. I know I've been changing the soft limit because it does revert to what it used to be on power-cycle. But a reboot does not cause it to revert; apparently whatever reset is done during a reboot on my test machine isn't hard enough to provoke that change. Presumably it's not the only such machine, too. There's one other interesting thing I noticed, and for this I need a brief bit of background explanation. When I get a "new" disk, one of the first things I od often (almost always, when it's second-hand and thus likely has someone else's data on it) is a write-read-compare pass over the whole drive. I have a program I wrote when playing with RAIDframe, designed to detect issues leading to two conceptual sectors ending up on the same real sector: it writes data to the whole disk such that each sector's data uniquely identifies the sector. On reading and comparing, I can then notice multiple conceptual sectors ending up in the same real sector. On a real disk that's not likely, but it _is_ an effective wipe (for the sort of threat model appropriate to me, at least). This is relevant because, when I took my test disk and read the 2113-sector HPA on it, I found my test pattern in many of the sectors. The only explanation I've come up with is that I did a wipe when I first got it, then put it in a machine whose BIOS used an HPA to stash stuff, but didn't overwrite the whole HPA. (Possibly multiple such machines, even, based on some strings I found in the HPA, with the disk, probably, effectively slowly shrinking as I move it between machines.) One thing I do not have is any way to change the hard limit. Since I can change the soft limit, I have little doubt I could change the hard limit if I wanted, presumably in response to an ioctl or some such, but I haven't bothered. (It's not something to do casually. According to the doc, the device shall not accept an attempt to change the hard limit if a previous change to the hard limit has been made since the last power-on reset.) /~\ The ASCII Mouse \ / Ribbon Campaign X Against HTMLmo...@rodents-montreal.org / \ Email! 7D C8 61 52 5D E7 2D 39 4E F1 31 3E E8 B3 27 4B
Re: Sending ATA commands?
Hello. What does your support do? Does it let you write over the host protected area? Does it let you extract what's in there? Sorry for the silly questions, but I'm not sure what this support buys you. It sounds great, but I'm not sure what I'd get by making this work in 5.x or beyond? -thanks -Brian On Aug 11, 12:42am, Mouse wrote: } Subject: Re: Sending ATA commands? } In case anyone cares } } Back on } } > Date: Sat, 10 Aug 2013 01:13:12 -0400 (EDT) } } I wrote to the list asking about some peculiar issues trying to } implement HPA support for 4.0.1. Thanks to a very perceptive kre, I } got over that hump (I got careless and didn't quite follow the } interface specified; the results actually make sense in view of the way } I didn't). } } I now have minimal HPA support working. Here's a dmesg extract from } the latest boot of my test machine: } } wd0 at atabus3 drive 0: } wd0: drive supports 16-sector PIO transfers, LBA addressing } wd0: HPA enabled, protected area present } wd0: effective capacity raised from 18039071 to 18041184 } wd0: 8809 MB, 17898 cyl, 16 head, 63 sec, 512 bytes/sect x 18041184 sectors } wd0: 32-bit data port } wd0: drive supports PIO mode 4, DMA mode 2, Ultra-DMA mode 4 (Ultra/66) } } So, if anyone wants rudimentary HPA support for 4.0.1, it now exists. } I intend to put this into 5.2 as well; I don't know whether 6.* and/or } -current have it, but, if not, anyone who cares to is welcome to pick } this up and look at it as a possible basis for something there. } } It currently is available only two ways. One is on request from me, of } course; I can mail out a patchset. The other is a git repo. I have } 4.0.1's src in a git repo, with these changes being commits } 8ceee48..65301ff in that repo - I broke it up into six changesets, } adding support before using it. So, for those who have git, the disk } space (a little over a gig, with a fully-checked-out working tree), and } the patience to wait for git to fetch it, you can get it by cloning } git://git.rodents-montreal.org/Mouse/netbsd-fork/4.0.1/src. } } Here are the one-line summaries. Except for 53fe079, the commit } messages are only one line long; 53fe079's explains how the interface } in question is ugly. } } * 65301ff Add HPA support to the wd driver. } * 53fe079 Add AT_READREG48, which reads current-and-previous register values. Ugly interface. } * a915892 Add WDCTL_HOB, which makes reads of some registers read `previous' values. } * 19f8f04 Add AT_NORESEL, to suppress the wd_sdh-destroying write in __wdccommand_intr. } * e9d3f6b Add HPA-relevant WDC_ and WDCC_ definitions. } * 5d1dd61 Peculiar spurious whitespace (before end-of-statement ;) cleanup. } } Next up: DCO support. :-) } } /~\ The ASCII Mouse } \ / Ribbon Campaign } X Against HTML mo...@rodents-montreal.org } / \ Email! 7D C8 61 52 5D E7 2D 39 4E F1 31 3E E8 B3 27 4B >-- End of excerpt from Mouse
