On Thu, 17 Jan 2019, John Snow wrote:
Admittedly I'm not too sure of how the ATA support in Linux works to
know what the passthrough would actually look like, bit-wise. I know
there's some SCSI abstraction layer that can drive ATA devices, but I'm
not completely clear on the actual plumbing. How you want to attempt
this might depend on what the linux ATA drivers look like and at which
interface layer you could conceivably drive them. I don't know the
answer to this.
Hi John,
I looked into how to send ATA commands in Linux from userspace. It seems
to be a relatively obscure topic (it's not every day that one needs this,
after all).
All googling points me to SG_IO ioctl. Searching for info on that IOCTL
leads to some documentation.[1] But not only does that date to 2008, it
also talks only about SCSI, so I was wondering where/how does ATA come
into picture. Yet tools like hdparm and sedutil use SG_IO to send ATA
commands.
I dived into Linux kernel sources. There are other relevant IOCTLs, like
HDIO_DRIVE_CMD, but they seem quite constrained. SG_IO, at least when
issued on a disk that uses the "sd" driver, reaches kernel's libata and
libata is then able to unwrap the command and pass it on to an ATA disk.
(I believe this is the abstraction layer you mention.)
Specifically, ata_scsi_queuecmd is invoked. It then calls either
ata_scsi_translate or ata_scsi_simulate. The pass-through translation
(from a SCSI cmd to ATA taskfile) is implemented in ata_scsi_pass_thru.[2]
I think SG_IO IOCTL allows one to send a single ATA command and the call
blocks until the request is processed.
I don't know if there are other mechanisms how to pass ATA commands from
userspace.
If anybody reading this has more ideas/info on this topic, please let me
know.
[1] http://sg.danny.cz/sg/sg_io.html
[2]
https://github.com/torvalds/linux/blob/30bac164aca750892b93eef350439a0562a68647/drivers/ata/libata-scsi.c#L3138
I can give you maybe a brief overview of some of the obviously useful
choke points in QEMU, though...
the ATA support in QEMU comes in a few different levels:
(1) IDE/ATA/PATA disks use a register set and PIO to directly read and
write values to individual registers. You can see this interface in
hw/ide.core.c for ide_ioport_write, ide_status_read, and ide_cmd_write.
When the drive is in a PIO data loop, you can read or write data to a
buffer by repeatedly writing to a certain register address, implemented
with ide_data_[read|write][w|l].
ide_exec_cmd serves as the "start processing" signal in QEMU, and uses
the various registers manipulated in the above calls stored in `IDEState
*s` to know which command to emulate. The arguments to ide_exec_cmd
aren't sufficient instruction alone. ide_exec_cmd is triggered whenever
the guest updates the command register.
CDROM emulation actually does use SCSI packets. Generally the guest
sends the 0xA0 PACKET command to the drive and then the drive waits for
a SCSI CDB to arrive via PIO. When the packet has arrived in full,
ide_atapi_cmd() processes it. However, there are a few places in this
code where we dip into the ATA registers to formulate a reply, so the
logical split isn't perfect.
(2) PCI IDE utilizes additional BMDMA features outside of those core
registers and are driven separately. It does not fully wrap the register
interface present.
(3) SATA devices begin using FIS packets. They're a message format that
lets you send commands, update registers, read values, etc. They're the
basic interface unit at this level. Both NCQ and traditional ATA
commands are delivered using FIS Register Host-to-Device update packets.
(The command, as always, is activated when the ATA device itself
receives an update to its command register.)
QEMU doesn't have a clean separation for ATA and SATA emulation, so the
SATA device emulation actually happens at the interface layer in QEMU
instead, as a hack. See hw/ide/ahci.c and look for this blurb:
```
/* Check for NCQ command */
if (is_ncq(cmd_fis[2])) {
process_ncq_command(s, port, cmd_fis, slot);
return;
}
```
This feels like maybe high level and useful enough to be able to
intercept for passthrough purposes, but I'm not sure how to handle
things like DMA routing or PIO access, which is still... technically
allowed at this layer, and might be used by early bootup routines.
A problem is that QEMU does not really disentangle the concept of a
"SATA Device" and the "AHCI controller", so a lot of the FIS responses
in QEMU go straight into the controller's buffer, and we'd have to split
all of that out.
(4) the AHCI controller manages sending and receiving the FIS packets.
You fill a buffer with the FIS packet to send and manipulate AHCI PCI
registers to send it off. FIS responses are buffered from the SATA
drives informing the controller of the new register values.
the AHCI command buffers include space for guests to pre-write their
SCSI CDBs, and the controller handles sending both the outer ATA command
and the inner ATAPI packet to the device.
Again, in QEMU, we cheat a little and layers (3) and (4) are pretty well
smooshed together. In general, the AHCI layer sends FIS packets back and
forth from the SATA layer, which decomposes the FIS packets into
constituent register updates, which are sent into layer (1) for
processing. None of these layers are really truly strictly separated,
unfortunately.
A naive question... which of these "details" (as in... was the command
dispatched via PIO, or in a FIS packet) are needed for ATA pass-through?
The only mechanism I'm currently aware of, in Linux, lets us pass a single
command, synchronously, and it doesn't let us tune much more. This sounds
like a horrible thing for performance, but this might in itself be OK for
the fiddling/debugging scenarios I'm after. But even ignoring performance,
is this sufficient? I.e. to extract - through whatever means - a sequence
of ATA commands (no matter how they were delivered from the guest), then
sequentially execute them, and then deliver them to the guest in the
appripriate way (depending on how they were received)?
Hope this helps even 1% instead of just being a useless info dump.
Thank you for this "executive summary". It helps me find the relevant code
in QEMU and the terms to google for, as I don't know much about this.
Best regards,
David
