Signed-off-by: Hongren (Zenithal) Zheng <i...@zenithal.me> --- docs/system/device-emulation.rst | 1 + docs/system/devices/canokey.rst | 168 +++++++++++++++++++++++++++++++ 2 files changed, 169 insertions(+) create mode 100644 docs/system/devices/canokey.rst
Note on the qemu-xhci issue: For FIDO2 packets, they follow the pattern below Interrupt IN (size 64) Interrupt OUT (size 128 with payload) Interrupt OUT ACK (size 64) Interrupt IN ACK (size 128 with payload) This can be captured by `fido2-token -I /dev/hidraw0` or `systemd-cryptenroll --fido2-device=auto`. I doubt this is a feature of libfido2 that it sends IN before OUT. I've been coping with this pattern for a long miserable time, you can see the url below for a example https://github.com/canokeys/canokey-usbip/blob/e9db44f1aa7f46c7dd7266a682d778921edbeb49/Src/usbip.c#L33-L36 In qemu-xhci, it assumes a pattern like this Interrupt IN (size 64) -> usb_handle_packet Interrupt IN ACK (size 128 with payload (not possible)) <- usb_handle_packet returns Interrupt OUT (size 128 with payload) -> the next usb_handle_packet Interrupt OUT ACK (size 64) <- the next usb_handle_packet returns However, for device, it can not ack the IN token if the payload in OUT token has not been passed. Also, we can not have a trace like this because we are single threaded (not async) and the next usb_handle_packet must be called after the return of the first usb-handle_packet (emulating async interrupt ep with sync thread is painful) Interrupt IN (size 64) -> usb_handle_packet Interrupt OUT (size 128 with payload) -> the next usb_handle_packet (not possible) Interrupt OUT ACK (size 64) <- the next usb_handle_packet returns Interrupt IN ACK (size 128 with payload) <- the first usb_handle_packet returns The code works for uhci/ehci in the following way Interrupt IN (size 64) -> usb_handle_packet Interrupt IN NAK (size 64) <- usb_handle_packet returns ... there are many IN NAK here ... uhci/ehci reschedule OUT before IN now Interrupt OUT (size 128 with payload) -> the next usb_handle_packet Interrupt OUT ACK (size 64) <- the next usb_handle_packet returns Interrupt IN (size 64) -> last usb_handle_packet Interrupt IN ACK (size 128 with payload) <- last usb_handle_packet returns I think qemu-xhci should retry/schedule the failed IN token after receiving NAK instead of failing immediately, because interrupt endpoint is async. However, I'm not so familiar with qemu-xhci and I tried to debug it but in vain. The debug trace only showed that after the first IN NAK it retried and then use `xhci_ep_nuke_xfers` to cancel the IN packet. After handling the OUT packet, the IN packet is not retried and the whole QEMU stuck there. Interrupt IN (size 64) -> usb_handle_packet Interrupt IN NAK (size 64) <- usb_handle_packet returns ... nuke_xfers Interrupt IN (size 64) -> usb_handle_packet Interrupt IN NAK (size 64) <- usb_handle_packet returns Interrupt OUT (size 128 with payload) -> the next usb_handle_packet Interrupt OUT ACK (size 64) <- the next usb_handle_packet returns ... xhci_try_complete_packet ... the whole QEMU stuck here as there is no more IN token scheduled You can see that in canokey_handle_data of hw/usb/canokey.c I NAKed for early INTR IN, this is what u2f_key_handle_data also does so I think the bug is not in canokey.c Also, canokey.c works with uhci/ehci controllers diff --git a/docs/system/device-emulation.rst b/docs/system/device-emulation.rst index 3b729b920d..0506006056 100644 --- a/docs/system/device-emulation.rst +++ b/docs/system/device-emulation.rst @@ -92,3 +92,4 @@ Emulated Devices devices/vhost-user.rst devices/virtio-pmem.rst devices/vhost-user-rng.rst + devices/canokey.rst diff --git a/docs/system/devices/canokey.rst b/docs/system/devices/canokey.rst new file mode 100644 index 0000000000..169f99b8eb --- /dev/null +++ b/docs/system/devices/canokey.rst @@ -0,0 +1,168 @@ +.. _canokey: + +CanoKey QEMU +------------ + +CanoKey [1]_ is an open-source secure key with supports of + +* U2F / FIDO2 with Ed25519 and HMAC-secret +* OpenPGP Card V3.4 with RSA4096, Ed25519 and more [2]_ +* PIV (NIST SP 800-73-4) +* HOTP / TOTP +* NDEF + +All these platform-independent features are in canokey-core [3]_. + +For different platforms, CanoKey has different implementations, +including both hardware implementions and virtual cards: + +* CanoKey STM32 [4]_ +* CanoKey Pigeon [5]_ +* (virt-card) CanoKey USB/IP +* (virt-card) CanoKey FunctionFS + +In QEMU, yet another CanoKey virt-card is implemented. +CanoKey QEMU exposes itself as a USB device to the guest OS. + +With the same software configuration as a hardware key, +the guest OS can use all the functionalities of a secure key as if +there was actually an hardware key plugged in. + +CanoKey QEMU provides much convenience for debuging: + +* libcanokey-qemu supports debuging output thus developers can + inspect what happens inside a secure key +* CanoKey QEMU supports trace event thus event +* QEMU USB stack supports pcap thus USB packet between the guest + and key can be captured and analysed + +Then for developers: + +* For developers on software with secure key support (e.g. FIDO2, OpenPGP), + they can see what happens inside the secure key +* For secure key developers, USB packets between guest OS and CanoKey + can be easily captured and analysed + +Also since this is a virtual card, it can be easily used in CI for testing +on code coping with secure key. + +Building +======== + +libcanokey-qemu is required to use CanoKey QEMU. + +.. code-block:: shell + + git clone https://github.com/canokeys/canokey-qemu + mkdir canokey-qemu/build + pushd canokey-qemu/build + +If you want to install libcanokey-qemu in a different place, +add ``-DCMAKE_INSTALL_PREFIX=/path/to/your/place`` to cmake below. + +.. code-block:: shell + + cmake .. + make + make install # may need sudo + popd + +Then configuring and building: + +.. code-block:: shell + + # depending on your env, lib/pkgconfig can be lib64/pkgconfig + export PKG_CONFIG_PATH=/path/to/your/place/lib/pkgconfig:$PKG_CONFIG_PATH + ./configure --enable-canokey && make + +Using CanoKey QEMU +================== + +CanoKey QEMU stores all its data on a file of the host specified by the argument +when invoking qemu. + +.. parsed-literal:: + + |qemu_system| -usb -device canokey,file=$HOME/.canokey-file + +Note: you should keep this file carefully as it may contain your private key! + +The first time when the file is used, it is created and initialized by CanoKey, +afterwards CanoKey QEMU would just read this file. + +After the guest OS boots, you can check that there is a USB device. + +For example, If the guest OS is an Linux machine. You may invoke lsusb +and find CanoKey QEMU there: + +.. code-block:: shell + + $ lsusb + Bus 001 Device 002: ID 20a0:42d4 Clay Logic CanoKey QEMU + +You may setup the key as guided in [6]_. The console for the key is at [7]_. + +Debuging +======== + +CanoKey QEMU consists of two parts, ``libcanokey-qemu.so`` and ``canokey.c``, +the latter of which resides in QEMU. The former provides core functionality +of a secure key while the latter provides platform-dependent functions: +USB packet handling. + +If you want to trace what happens inside the secure key, when compiling +libcanokey-qemu, you should add ``-DQEMU_DEBUG_OUTPUT=ON`` in cmake command +line: + +.. code-block:: shell + + cmake .. -DQEMU_DEBUG_OUTPUT=ON + +If you want to trace events happened in canokey.c, use + +.. parsed-literal:: + + |qemu_system| --trace "canokey_*" \\ + -usb -device canokey,file=$HOME/.canokey-file + +If you want to capture USB packets between the guest and the host, you can: + +.. parsed-literal:: + + |qemu_system| -usb -device canokey,file=$HOME/.canokey-file,pcap=key.pcap + +Limitations +=========== + +Currently libcanokey-qemu.so has dozens of global variables as it was originally +designed for embedded systems. Thus one qemu instance can not have +multiple CanoKey QEMU running, namely you can not + +.. parsed-literal:: + + |qemu_system| -usb -device canokey,file=$HOME/.canokey-file \\ + -device canokey,file=$HOME/.canokey-file2 + +Also, there is no lock on canokey-file, thus two CanoKey QEMU instance +can not read one canokey-file at the same time. + +Another limitation is that this device is not compatible with ``qemu-xhci``, +in that this device would hang when there are FIDO2 packets (traffic on +interrupt endpoints). If you do not use FIDO2 then it works as intended, +but for full functionality you should use old uhci/ehci bus and attach canokey +to it, for example + +.. parsed-literal:: + + |qemu_system| -device piix3-usb-uhci,id=uhci -device canokey,bus=uhci.0 + +References +========== + +.. [1] `<https://canokeys.org>`_ +.. [2] `<https://docs.canokeys.org/userguide/openpgp/#supported-algorithm>`_ +.. [3] `<https://github.com/canokeys/canokey-core>`_ +.. [4] `<https://github.com/canokeys/canokey-stm32>`_ +.. [5] `<https://github.com/canokeys/canokey-pigeon>`_ +.. [6] `<https://docs.canokeys.org/>`_ +.. [7] `<https://console.canokeys.org/>`_ -- 2.35.1
