Hi--
On 12/1/20 2:34 PM, mgr...@linux.intel.com wrote:
> From: mark gross <mgr...@linux.intel.com>
>
>
> Reviewed-by: Mark Gross <mgr...@linux.intel.com>
> Signed-off-by: Mark Gross <mgr...@linux.intel.com>
My reading of submitting-patches.rst seems to indicate that
the Reviewer and Submitter are probably not the same person.
Are you sure that you reviewed it?
> ---
> Documentation/index.rst | 3 +-
> Documentation/vpu/index.rst | 16 ++
> Documentation/vpu/vpu-stack-overview.rst | 267 +++++++++++++++++++++++
> 3 files changed, 285 insertions(+), 1 deletion(-)
> create mode 100644 Documentation/vpu/index.rst
> create mode 100644 Documentation/vpu/vpu-stack-overview.rst
>
> diff --git a/Documentation/index.rst b/Documentation/index.rst
> index 57719744774c..0a2cc0204e8f 100644
> --- a/Documentation/index.rst
> +++ b/Documentation/index.rst
> @@ -1,4 +1,4 @@
> -.. SPDX-License-Identifier: GPL-2.0
> +.. SPDX-License-Identifier: GPL-2.0-only
That looks both inappropriate for this patch and incorrect AFAICT.
>
>
> .. The Linux Kernel documentation master file, created by
> @@ -137,6 +137,7 @@ needed).
> misc-devices/index
> scheduler/index
> mhi/index
> + vpu/index
>
> Architecture-agnostic documentation
> -----------------------------------
> diff --git a/Documentation/vpu/index.rst b/Documentation/vpu/index.rst
> new file mode 100644
> index 000000000000..7e290e048910
> --- /dev/null
> +++ b/Documentation/vpu/index.rst
> @@ -0,0 +1,16 @@
> +.. SPDX-License-Identifier: GPL-2.0-only
license-rules.rst says:
For 'GNU General Public License (GPL) version 2 only' use:
SPDX-License-Identifier: GPL-2.0
> +
> +============================================
> +Vision Processor Unit Documentation
> +============================================
> +
> +This documentation contains information for the Intel VPU stack.
> +
> +.. class:: toc-title
> +
> + Table of contents
> +
> +.. toctree::
> + :maxdepth: 2
> +
> + vpu-stack-overview
> diff --git a/Documentation/vpu/vpu-stack-overview.rst
> b/Documentation/vpu/vpu-stack-overview.rst
> new file mode 100644
> index 000000000000..53c06a7d9a52
> --- /dev/null
> +++ b/Documentation/vpu/vpu-stack-overview.rst
> @@ -0,0 +1,267 @@
> +.. SPDX-License-Identifier: GPL-2.0-only
Nope.
> +
> +======================
> +Intel VPU architecture
> +======================
> +
> +Overview
> +========
> +
> +The Intel Movidius acquisition has developed a Vision Processing Unit (VPU)
> +roadmap of products starting with Keem Bay (KMB). The HW configurations the
s/HW/hardware/
> +VPU can support include:
> +
> +1. Standalone smart camera that does local CV processing in camera
Tell us what CV is before using it.
> +2. Standalone appliance or SBC device connected to a network and tethered
Tell us what SBC is before using it. (yeah, I know)
> + cameras doing local CV processing
> +3. Embedded in a USB dongle or M.2 as an CV accelerator.
> +4. Multiple VPU enabled SOC's on a PCIE card as a CV accelerator in a larger
> IA
PCIe (?)
> + box or server.
> +
> +Keem Bay is the first instance of this family of products. This document
> +provides an architectural overview of the SW stack supporting the VPU enabled
s/SW/software/
> +products.
> +
> +Keem Bay (KMB) is a Computer Vision AI processing SoC based on ARM A53 CPU
> that
> +provides Edge neural network acceleration (inference) and includes a Vision
> +Processing Unit (VPU) hardware. The ARM CPU SubSystem (CPUSS) interfaces
> +locally to the VPU and enables integration/interfacing with a remote host
> over
> +PCIe or USB or Ethernet interfaces. The interface between the CPUSS and the
> VPU
> +is implemented with HW FIFOs (Control) and coherent memory mapping (Data)
> such
> +that zero copy processing can happen within the VPU.
> +
> +The KMB can be used in all 4 of the above classes of designs.
> +
> +We refer to the 'local host' as being the ARM part of the SoC, while the
> +'remote host' as the IA system hosting the KMB device(s). The KMB SoC boots
> +from an eMMC via uBoot and ARM Linux compatible device tree interface with an
> +expectation to fully boot within hundreds of milliseconds. There is also
> +support for downloading the kernel and root file system image from a remote
> +host.
> +
> +The eMMC can be updated with standard mender update process.
Mender
> +See https://github.com/mendersoftware/mender
> +
> +The VPU is started and controlled from the A53 local host. Its firmware
> image
> +is loaded using the drive FW helper KAPI's.
s/FW/firmware/
> +
> +The VPU IP FW payload consists of a SPARC ISA RTEMS bootloader and/or
> +application binary.
> +
> +The interface allowing (remote or local) host clients to access VPU IP
^^drop one space
> +capabilities is realized through an abstracted programming model, which
> +provides Remote Proxy APIs for a host CPU application to dynamically create
> and
> +execute CV and NN workloads on the VPU. All frameworks exposed through
Tell us what NN is.
> +programming model’s APIs are contained in the pre-compiled standard firmware
> +image.
> +
> +There is a significant SW stack built up to support KMB and the use cases.
> The
> +rest of this documentation provides an overview of the components of the
> stack.
> +
> +Keem Bay IPC
> +============
> +
> +Directly interfaces with the KMB HW FIFOs to provide zero copy processing
> from
> +the VPU. It implements the lowest level protocol for interacting with the
> VPU.
> +
> +The Keem Bay IPC mechanism is based on shared memory and hardware FIFOs,
FIFOs.
> +specifically there are:
Specifically
> +
> +* Two 128-entry HW FIFOs, one for the CPU and one for the VPU.
> +* Two shared memory regions, used as memory pool for allocating IPC buffers
end with a period since the previous line did that.
> +
> +An IPC channel is a software abstraction allowing communication multiplexing,
> +so that multiple applications / users can concurrently communicated to the
> VPU.
communicate
> +IPC channels area conceptually similar to socket ports.
> +
> +There is a total of 1024 channels, each one identified by a channel ID,
> ranging
are
> +from 0 to 1023.
> +
> +Channels are divided in two categories:
> +
> +* High-Speed (HS) channels, having IDs in the 0-9 range.
> +* General-Purpose (GP) channels, having IDs in the 10-1023 range.
> +
> +HS channels have higher priority over GP channels and can be used by
> +applications requiring higher throughput or lower latency.
> +
> +Since all the channels share the same HW resources (i.e., the HW FIFOs and
> the
> +IPC memory pools), the Keem Bay IPC driver uses software queues to give a
> +higher priority to HS channels.
> +
> +The driver supports a build-time configurable number of communication
> channels
> +defined in a so called Channel Mapping Table.
so-called
> +
> +An IPC channel is full duplex: a pending operation from a certain channel
> does
> +not block other operations on the same channel, regardless of their operation
> +mode (blocking or non-blocking).
> +
> +Operation mode is individually selectable for each channel, per operation
> +direction (read or write). All operations for that direction comply to
> +selection.
> +
> +
> +Keem Bay-VPU-IPC
> +================
> +
> +This is the MMIO driver of the VPU IP block inside the SOC. It is a control
> +driver mapping IPC channel communication to Xlink virtual channels.
> +
> +This driver provides the following functionality to other drivers in the
> +communication stack:
> +
> +* VPU IP execution control (firmware load, start, reset)
> +* VPU IP event notifications (device connected, device disconnected, WDT
> event)
> +* VPU IP device status query (OFF, BUSY, READY, ERROR, RECOVERY)
> +* Communication via the IPC protocol (wrapping the Keem Bay IPC driver and
> + exposing it to higher level Xlink layer)
> +
> +In addition to the above, the driver exposes SoC information (like stepping,
> +device ID, etc.) to user-space via sysfs.
> +
> +This driver depends on the 'Keem Bay IPC' driver, which enables the Keem Bay
> +IPC communication protocol.
> +
> +The driver uses the Firmware API to load the VPU firmware from user-space.
> +
> +Xlink-IPC
> +=========
> +This component is implementing the IPC specific Xlink protocol. It maps
> channel
component implements IPC-specific
> +IDs to HW FIFO entries, using the Keem Bay VPU IPC driver.
> +
> +Some of the main functions this driver provides:
> +
> +* establishing a connection with an IPC device
> +* obtaining a list with the available devices
> +* obtaining the status for a device
> +* booting a device
> +* resetting a device
> +* opening and closing channels
> +* issuing read and write operations
> +
> +Xlink-core
> +==========
> +
> +This component implements an abstracted set of control and communication APIs
> +based on channel identification. It is intended to support VPU technology
> both
> +at SoC level as well as at IP level, over multiple interfaces.
> +
> +It provides symmetrical services, where the producer and the consumer have
> +the same privileges.
> +
> +Xlink driver has the ability to abstract several types of communication
> +channels underneath, allowing the usage of different interfaces with the same
> +function calls.
> +
> +Xlink services are available to both kernel and user space clients and
> include:
> +
> +* interface abstract control and communication API
> +* multi device support
> +* concurrent communication across 4096 communication channels (from 0 to
> + 0xFFF), with customizable properties
> +* full duplex channels with multiprocess and multithread support
> +* channel IDs can be mapped to desired physical interface (PCIE, USB, ETH,
> IPC)
PCIe
> + via a Channel Mapping Table
> +* asynchronous fast pass through mode: remote host data packets are directly
passthrough
> + dispatched using interrupt systems running on local host to IPC calls for
> low
> + overhead
> +* channel handshaking mechanism for peer to peer communication, without the
> + need of static channel preallocation
> +* channel resource management
> +* asynchronous data and device notifications to subscribers
> +
> +Xlink transports: PCIe, USB, ETH, IPCXLink-PCIe
IPC,
> +
> +XLink-PCIE
PCIe
> +==========
> +This is an endpoint driver that is mapping Xlink channel IDs to PCIE
> channels.
that maps PCIe
> +
> +This component ensures (remote)host-to-(local)host communication, and VPU IP
> +communication via an asynchronous pass through mode, where PCIE data loads
> are
passthrough PCIe
> +directly dispatched to Xlink-IPC.
> +
> +The component builds and advertises Device IDs that can are used by local
> host
that are used
> +application in case of multi device scenarios.
> +
> +XLink-USB
> +==========
> +This is an endpoint driver that is mapping Xlink channel IDs to bidirectional
that maps
> +USB endpoints and supports CDC USB class protocol. More than one Xlink
> channels
channel
> +can be mapped to a single USB endpoint.
> +
> +This component ensures host-to-host communication, and, as well, asynchronous
> +pass through communication, where USB transfer packets are directly
> dispatched
passthrough
> +to Xlink-IPC.
> +
> +The component builds and advertises Device IDs that can are used by local
> host
that are used
> +application in case of multi device scenarios.
> +
> +XLink-ETH
> +=========
> +
> +This is an endpoint driver that is mapping Xlink channel IDs to Ethernet
that maps
> +sockets.
> +
> +This component ensures host-to-host communication, and, as well, asynchronous
> +pass through communication, where Ethernet data loads are directly
> dispatched to
passthrough
> +Xlink-IPC.
> +
> +The component builds and advertises Device IDs that can are used by local
> host
that are used
> +application in case of multi device scenarios.
> +
> +Assorted drivers that depend on this stack:
> +
> +Xlink-SMB
> +=========
> +The Intel Edge.AI Computer Vision platforms have to be monitored using
> platform
> +devices like sensors, fan controller, IO expander etc. Some of these devices
> +are memory mapped and some are i2c based. Either of these devices are not
I2C-based. None of these devices is
> +directly accessible to the host.
> +
> +The host here refers to the server to which the vision accelerators are
> +connected over PCIe Interface. The Host needs to do a consolidated action
> based
> +on the parameters of platform devices. In general, most of the standard
> devices
> +(includes sensors, fan controller, IO expander etc) are I2C/SMBus based and
> are
SMBus-based
> +used to provide the status of the accelerator. Standard drivers for these
> +devices are available based on i2c/smbus APIs.
I2C/SMBus
> +
> +Instead of changing the sensor drivers to adapt to PCIe interface, a generic
> +i2c adapter "xlink-smbus" which underneath uses xlink as physical medium is
I2C Xlink
> +used. With xlink-smbus, the drivers for the platform devices doesn't need to
don't
> +undergo any interface change.
> +
> +TSEN
> +====
> +
> +Thermal sensor driver for exporting thermal events to the local Arm64 host as
> +well as to the remote X86 host if in the PCIe add in CV accelerator
add-in
> +configuration.
> +
> +The driver receiving the junction temperature from different heating points
receives
> +inside the SOC. The driver will receive the temperature on SMBUS connection
> and
SMBus
> +forward over xlink-smb when in a remote host configuration.
> +
> +In Keem Bay, the four thermal junction temperature points are, Media
> Subsystem
^no comma
> +(mss), NN subsystem (nce), Compute subsystem (cse) and SOC(Maximum of mss,
> nce
and SOC (maximum of mss,
nce
> +and cse)
cse).
> +
> +HDDL
> +====
> +
> +- Exports details of temperature sensor, current sensor and fan controller
> + present in Intel Edge.AI Computer Vision platforms to IA host.
> +- Enable Time sync of Intel Edge.AI Computer Vision platform with IA host.
> +- Handles device connect and disconnect events.
> +- Receives slave address from the IA host for memory mapped thermal sensors
> + present in SoC (Documentation/hwmon/intel_tsens_sensors.rst).
> +- Registers i2c slave device for slaves present in Intel Edge.AI Computer
I2C
> + Vision platform
> +
> +
> +VPUMGR (VPU Manager)
> +====================
> +
> +Bridges firmware on VPU side and applications on CPU user-space, it assists
> +firmware on VPU side serving multiple user space application processes on CPU
> +side concurrently while also performing necessary data buffer management on
> +behalf of VPU IP.
>
--
~Randy
