JannePeltonen replied on github web page:
doc/users-guide/users-guide-ipsec.adoc
line 218
@@ -0,0 +1,443 @@
+== IPsec services
+
+In addition to general cryptographic services, ODP offers offload support for
+the IPsec protocol. IPsec is a general term referencing a suite of protocols
+and packet formats and as such a full discussion of IPsec is beyond the scope
+of this document. See https://tools.ietf.org/html/rfc4301[RFC 4301] and
+related RFCs for more detail. This section assumes the reader is already
+familiar with IPsec and focuses on explaining the ODP APIs that support it.
+
+ODP provides APIs for the following IPsec services:
+
+* General IPsec configuration
+* Security Association (SA) configuration and lifecycle management
+* Synchronous and Asynchronous IPsec lookaside processing
+* Inline processing for full IPsec RX and/or TX offload
+* Pipelining for RX traffic
+* Fragmentation support for TX traffic
+* IPsec event management
+
+=== IPsec Capabilities and Configuration
+As with other features, ODP provides APIs that permit applications to query
+platform-specific IPsec capabilities. The `odp_ipsec_capability()` API queries
+the general IPsec features available while the `odp_ipsec_cipher_capability()`
+and `odp_ipsec_auth_capability()` APIs provide detail on the range of
+cipher and authentication algorithms supported by IPsec on this platform.
+
+General IPsec capabilities that are reported include:
+
+* The IPsec operation modes supported by this implementation. Different
+operation modes may be _not supported_, _supported_, or _preferred_. A
+preferred form means that this mode takes advantage of hardware
+acceleration features to achieve best performance.
+* Whether IPsec AH processing is supported. All ODP platforms must provide
+support for IPsec ESP processing, however since AH is relatively rare, it
+may not be supported, or supported only via software emulation (_e.g.,_ be
+non-preferred).
+* Whether IPsec headers should be retained on decrypt for inbound inline
+operations.
+* Whether classification pipelining is supported (to be discussed below).
+
+In addition, capabilities also inform the application of the maximum number
+of destination queues and classification CoS targets supported. These
+will be discussed further later.
+
+==== IPsec Operation Modes
+IPsec operates in one of three modes: Synchronous, Asynchronous, and Inline.
+
+==== Lookaside Processing
+Synchronous and Asynchronous are types of _lookaside_ processing. In lookaside
+mode, the application receives (or creates) an IPsec packet and then uses ODP
+to perform one of two functions:
+
+* To decrypt an IPsec packet into a "normal" packet
+* To take a "normal" packet and encrypt it into an IPsec packet.
+
+This process may be performed _synchronously_ with the APIs `odp_ipsec_in()`
+(to decrypt) and `odp_ipsec_out()` (to encrypt). Upon return from these calls
+the requested packet transformation is complete, or an error return code
+indicates that it could not be performed (_e.g.,_ packet decryption failed).
+
+Synchronous processing may be preferred if the application has a large number
+of worker threads so that blocking any individual worker while IPsec processing
+is performed represents a reasonable design. The alternative is to use
+_asynchronous_ forms of these APIs:
+
+* `odp_ipsec_in_enq()` for decrypt
+* `odp_ipsec_out_enq()` for encrypt
+
+These simply pass packets to IPsec for processing. When this processing is
+complete, the resulting packets are sent to the completion queue associated
+with the SA used by the operation, serving as IPsec completion events as
+shown here:
+
+image::ipsec-lookaside.svg[align="center"]
+
+If the operation fails because SA lookup failed for inbound processing, then
+these result packets are sent to the default queue specified as part of the
+`odp_ipsec_inbound_config_t` used in the `odp_ipsec_config()` call.
+
+Following an asynchronous IPsec call, the worker thread moves on to process
+other events until the IPsec completion shows up. At that point the worker
+thread sees whether the operation was successful or not and continues
+processing for that packet. These events may be direct-polled with
+`odp_queue_deq()` if the completion queue was created as a plain queue, or
+processed via the ODP scheduler if the completion queue was created as a
+scheduled queue.
+
+==== Inline Processing
+While lookaside processing offers flexibility, it still requires extra
+processing steps not required by modern hardware. To avoid this overhead
+ODP also offers _inline_ processing support for IPsec. In this mode the
+processing of IPsec packets on the RX and TX paths is fully offloaded as
+shown here:
+
+image::ipsec-inline.svg[align="center"]
+
+It is worth noting that, depending on the implementation and application
+needs, inline processing may be enabled only for one direction (inbound or
+outbound) or for both directions.
+
+On the receive side, once configured for inline processing, arriving IPsec
+packets that are recognized at the PktIO interface are decrypted automatically
+before the application ever sees them. On the transmit side, the application
+calls `odp_ipsec_out_inline()` and the packet is encrypted and queued for
+transmission as a single operation without further application involvement.
+Note that if an inbound IPsec packet is not recognized (_e.g.,_ it belongs to
+an unknown SA) then it will be presented to the application as-is without
+further processing. The application may then use a lookaside call to process
+the packet if it is able to supply a matching SA by other means.
+
+On the receive side, after an IPsec packet is decrypted, it may be
+_pipelined_ to the ODP classifier or added to a poll queue, as the
+application wishes. The advantage of classification pipelining is that inbound
+IPsec traffic is automatically decrypted and classified into appropriate
+flow-based queues for ease of processing.
+
+On the transmit side, since IPsec encryption and tunneling may exceed an
+output MTU, ODP also offers support for MTU configuration and automatic IPsec
+TX fragmentation.
+
+Both classification pipelining and TX fragmentation support are support
+features that are indicated by `odp_ipsec_capability()`.
+
+Note that at present inline IPsec output support sends resulting packets
+directly to an output PktIO. If it's desired to send them to the ODP
+Traffic Manager for shaping prior to transmission, use the lookaside APIs
+to perform the IPsec encrypt and then call `odp_tm_enq()` on the resulting
+packet.
+
+=== IPsec Configuration
+Prior to making use of IPsec services, the `odp_ipsec_config()` API is used to
+configure IPsec processing options. This API takes a pointer to an
+`odp_ipsec_config_t` struct as its argument. SAs in ODP are represented by the
+abstract type `odp_ipsec_sa_t`.
+
+The `odp_ipsec_config_t` struct specifies the inbound and outbound
+processing modes that the application plans to use, the maximum number of
+Security Associations it will use, and sets inbound and outbound
+processing options.
+
+==== IPsec Inbound Configuration
+Inbound configuration options for IPsec specify the default `odp_queue_t` to
+be used for processing global events like SA lookup failures, how Security
+Parameter Index (SPI) lookup is to be performed, and whether the application
+requires ODP to retain outer headers for decrypted IPsec packets.
+
+Parsing options specify how "deep" decrypted packets are to be parsed
+after IPsec processing by specifying the packet layers of interest to the
+application (None, L2, L3, L4, or All). And which checksums should be verified
+on decrypted packets.
+
+==== IPsec Outbound Configuration
+Outbound configuration options for IPsec specify checksum insertion processing
+that should be performed prior to encryption.
+
+=== IPsec Events
+IPsec introduces one new event type and one new event subtype. These are:
+
+* IPsec packet events. These are events of type `ODP_EVENT_PACKET` that have
+subtype `ODP_EVENT_PACKET_IPSEC`. These are packets that carry additional
+IPsec-related metadata in the form of an `odp_ipsec_packet_result_t` struct
+that can be retrieved from the packet via the `odp_ipsec_result()` API.
+
+* IPsec status notifications. These are events of type `ODP_EVENT_IPSEC_STATUS`
+that indicate status events not associated with any particular IPsec
+packet. Such events carry status in the form of an `odp_ipsec_status_t`
+struct that is retrieved from the event via the `odp_ipsec_status()` API.
+
+IPsec-related events are thus part of normal and exception processing when
+working with IPsec.
+
+=== Security Associations (SAs)
+The fundamental "building block" for IPsec processing is the _Security
+Association (SA)_. Similar to a crypto session, the SA encapsulates the keying
+material and context needed to perform IPsec protocol processing for inbound
+or outbound packets on a given flow, as well as additional processing options
+that control how IPsec is to be used for packets processed under this
+SA. Security Associations are unidirectional (RX or TX) so a flow that
+requires both inbound (decrypt) and outbound (encrypt) IPsec functions will
+have two SAs associated with it.
+
+After ODP initialization, IPsec support is dormant until it is configured
+by a call to `odp_ipsec_config()` as described earlier. Once configured,
+SAs may be created by calling `odp_ipsec_sa_create()`.
+
+==== SA Creation and Configuration
+The `odp_ipsec_sa_create()` API takes an `odp_ipsec_sa_param_t` argument that
+describes the SA to be created. Use the `odp_ipsec_sa_param_init()` API to
+initialize this to its default state and then override selected fields within
+the param struct as needed.
+
+Items specified in the `odp_ipsec_sa_param_t` struct include:
+
+* The direction of the SA (inbound or outbound).
+
+* The IPsec protocol being used (ESP or AH).
+
+* The IPsec protocol mode (Transport or Tunnel).
+
+* The parameters needed for the crypto and authentication algorithms to be
+used by this SA.
+
+* Miscellaneous SA options that control behavior such as use of Extended
+Sequence Numbers (ESNs), the use of UDP encapsulation, various copy
+options for header fields, and whether the TTL (Hop Limit) field should be
+decremented when operating in tunnel mode.
+
+* Parameters controlling the SA lifetime.
+
+* The Security Parameter Index (SPI) that packets will use to indicate that
+they belong to this SA.
+
+* The pipeline mode used by this SA.
+
+* The destination `odp_queue_t` to be used for status events associated with
+this SA.
+
+* The user context pointer (and length) associated with this SA for
Comment:
Maybe it is not useful to mention length here since it is only a prefetching
hint.
> JannePeltonen wrote
> Maybe it should be made more clear somewhere that the synchronous processing
> calls are allowed only in the synchronous operating mode and asynchronous
> calls in asynchronous and inline operation mode (and inline calls only in the
> inline mode).
>> JannePeltonen wrote
>> This dummy packet stuff is not yet (if ever?) part of the API spec. I think
>> this patch should be in sync with what is the current state of api-next,
>> which still uses the status event for indicating SA disable completion.
>>> JannePeltonen wrote
>>> "This call" could be replaced with odp_ipsec_sa_disable() to make the text
>>> more clear.
>>>> JannePeltonen wrote
>>>> I think this may give the impression that calling odp_ipsec_sa_disable()
>>>> is optional, when in fact it is mandatory before odp_ipsec_sa_destroy().
>>>>> JannePeltonen wrote
>>>>> According to the API spec soft limit alerts do not need to be edge
>>>>> triggered. The spec says: "It's implementation defined how many times
>>>>> soft lifetime expiration is reported: only once, first N or all packets
>>>>> following the limit crossing."
>>>>>> JannePeltonen wrote
>>>>>> Is it clear enough here that the mtu error bit is set only when MTU
>>>>>> checking was requested and not in case fragmentation offload was
>>>>>> requested?
>>>>>>> JannePeltonen wrote
>>>>>>> Same comment as for odp_ipsec_{in,out}(), the description of the return
>>>>>>> code is not correct.
>>>>>>>> JannePeltonen wrote
>>>>>>>> The return code does not tell whether the application of the SA was
>>>>>>>> successful but how many input packets were consumed by the operation.
>>>>>>>> The per-packet operation status is retrieved via the
>>>>>>>> odp_ipsec_result() api for the outputted packets, which will be
>>>>>>>> available only when the function succeeds.
>>>>>>>>> JannePeltonen wrote
>>>>>>>>> The operating mode is a global setting, not per-SA setting as the
>>>>>>>>> text implies. And the relevant type is odp_ipsec_op_mode_t, not
>>>>>>>>> odp_ipsec_mode_t which is for selecting between tunnel and transport
>>>>>>>>> mode.
>>>>>>>>>> JannePeltonen wrote
>>>>>>>>>> "How this is done" sounds a bit awkward as it is not clear what
>>>>>>>>>> "this" refers to (the way the IPsec services are used). Maybe the
>>>>>>>>>> sentence should be reworded.
>>>>>>>>>>> JannePeltonen wrote
>>>>>>>>>>> Would "minimum size" be better than "size" here?
>>>>>>>>>>>> JannePeltonen wrote
>>>>>>>>>>>> The queue is not only for status events but for ipsec packet
>>>>>>>>>>>> events too. Maybe just remove the word "status"?
>>>>>>>>>>>>> JannePeltonen wrote
>>>>>>>>>>>>> This should be: "...can be retained..." instead of "...should be
>>>>>>>>>>>>> retained..." as this is a capability and application chooses
>>>>>>>>>>>>> whether it wants to retain the headers if the ODP is capable.
>>>>>>>>>>>>>> Bill Fischofer(Bill-Fischofer-Linaro) wrote:
>>>>>>>>>>>>>> I'd like to see us all make it a priority to close on #197 next
>>>>>>>>>>>>>> week. I'll post a doc update once we're all agreed on the
>>>>>>>>>>>>>> precise semantics and operation flow. So I'm fine with holding
>>>>>>>>>>>>>> off merging this PR until then.
>>>>>>>>>>>>>>> Dmitry Eremin-Solenikov(lumag) wrote:
>>>>>>>>>>>>>>> Any update here? @NikhilA-Linaro, @bala-manoharan can you
>>>>>>>>>>>>>>> please comment wrt your implementations?
>>>>>>>>>>>>>>>> Dmitry Eremin-Solenikov(lumag) wrote:
>>>>>>>>>>>>>>>> Well, we do not have #197 merged, so it is too early to depend
>>>>>>>>>>>>>>>> on that.
>>>>>>>>>>>>>>>> Also, frankly speaking, this `sa_disabled` warning inside
>>>>>>>>>>>>>>>> `odp_ipsec_result_t` is a backup plan. It is expected that
>>>>>>>>>>>>>>>> most of the implementations will report this as a proper
>>>>>>>>>>>>>>>> `ODP_IPSEC_STATUS` event, carrying this warning bit inside.
>>>>>>>>>>>>>>>>> Bill Fischofer(Bill-Fischofer-Linaro) wrote:
>>>>>>>>>>>>>>>>> OK, changed in v6.
>>>>>>>>>>>>>>>>>> Dmitry Eremin-Solenikov(lumag) wrote:
>>>>>>>>>>>>>>>>>> s/default //
>>>>>>>>>>>>>>>>>>> Bill Fischofer(Bill-Fischofer-Linaro) wrote:
>>>>>>>>>>>>>>>>>>> OK, corrected in v5.
>>>>>>>>>>>>>>>>>>>> Dmitry Eremin-Solenikov(lumag) wrote:
>>>>>>>>>>>>>>>>>>>> In lookaside mode soft limit expiration is reported as
>>>>>>>>>>>>>>>>>>>> `warn` part of `ipsec_op_status` packet metadata.
>>>>>>>>>>>>>>>>>>>>> Bill Fischofer(Bill-Fischofer-Linaro) wrote:
>>>>>>>>>>>>>>>>>>>>> OK, in v4 I've added a new terminal state `SA_Expired` to
>>>>>>>>>>>>>>>>>>>>> the FSM and have updated the doc to say "expired" rather
>>>>>>>>>>>>>>>>>>>>> than "disabled". From the expired state the only valid
>>>>>>>>>>>>>>>>>>>>> operation is `odp_ipsec_sa_destroy()`.
>>>>>>>>>>>>>>>>>>>>>> Dmitry Eremin-Solenikov(lumag) wrote:
>>>>>>>>>>>>>>>>>>>>>> Yep. It is just not a 'disabled' state, because we have
>>>>>>>>>>>>>>>>>>>>>> separate definition for 'disabled SA'.
>>>>>>>>>>>>>>>>>>>>>>> Dmitry Eremin-Solenikov(lumag) wrote:
>>>>>>>>>>>>>>>>>>>>>>> IIRC, one can call it only once in NXP case.
>>>>>>>>>>>>>>>>>>>>>>> @NikhilA-Linaro, could you please comment?
>>>>>>>>>>>>>>>>>>>>>>>> Bill Fischofer(Bill-Fischofer-Linaro) wrote:
>>>>>>>>>>>>>>>>>>>>>>>> So how is this indicated in lookaside mode? The whole
>>>>>>>>>>>>>>>>>>>>>>>> point of ODP providing the limit support is so the
>>>>>>>>>>>>>>>>>>>>>>>> application doesn't have to track byte/packet counts
>>>>>>>>>>>>>>>>>>>>>>>> itself, so it's expected that soft limit overruns will
>>>>>>>>>>>>>>>>>>>>>>>> happen as part of lookaside processing as well.
>>>>>>>>>>>>>>>>>>>>>>>>> Bill Fischofer(Bill-Fischofer-Linaro) wrote:
>>>>>>>>>>>>>>>>>>>>>>>>> If you let yourself run out of gas, the car can stop
>>>>>>>>>>>>>>>>>>>>>>>>> at inconvenient times, which is why one pays
>>>>>>>>>>>>>>>>>>>>>>>>> attention to that low fuel warning light. A hard
>>>>>>>>>>>>>>>>>>>>>>>>> limit is a hard limit. That's what makes it hard. Any
>>>>>>>>>>>>>>>>>>>>>>>>> other definition seems confusingly fuzzy and
>>>>>>>>>>>>>>>>>>>>>>>>> unnecessary.
>>>>>>>>>>>>>>>>>>>>>>>>>> Bill Fischofer(Bill-Fischofer-Linaro) wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>> I thought the restriction is that you can call this
>>>>>>>>>>>>>>>>>>>>>>>>>> repeatedly as long as an SA hasn't yet been created.
>>>>>>>>>>>>>>>>>>>>>>>>>> I can change this (and the state diagram) if that's
>>>>>>>>>>>>>>>>>>>>>>>>>> not the case.
>>>>>>>>>>>>>>>>>>>>>>>>>>> Bill Fischofer(Bill-Fischofer-Linaro) wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>> It's part of the `enum`. In this case L2 would
>>>>>>>>>>>>>>>>>>>>>>>>>>> effectively be None.
>>>>>>>>>>>>>>>>>>>>>>>>>>>> Dmitry Eremin-Solenikov(lumag) wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>> Only in OUT INLINE mode, if I remember the outcome
>>>>>>>>>>>>>>>>>>>>>>>>>>>> of discussions correctly.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Dmitry Eremin-Solenikov(lumag) wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Ahem. It does not enter disabled state per se:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> - It is an undefined behaviour (iow, an
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> application error) to submit packets to disabled
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> SA
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> - It is a perfectly valid to submit packets to SA
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> after hard limit overrun (e.g. because other
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> packets might be already queued at this moment).
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Dmitry Eremin-Solenikov(lumag) wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It is worth mentioning that depending on the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> implementation and application needs, inline
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> processing might be enabled either in both
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> directions or in just one direction.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Dmitry Eremin-Solenikov(lumag) wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> TBD: mention that it MUST be called at most
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> once per IPsec application.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Dmitry Eremin-Solenikov(lumag) wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> L2 does not make sense here, does it?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Dmitry Eremin-Solenikov(lumag) wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ... parsed after IPsec processing ...
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Dmitry Eremin-Solenikov(lumag) wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> TBD: describe that some IPsec packets still
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> might be reported via plain PktIO interface
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (e.g. because of SA lookup failure). They
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> can be resubmitted to IPsec in lookaside
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mode.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Dmitry Eremin-Solenikov(lumag) wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If SA was not determined (because SA lookup
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> failed for inbound packet), event will be
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> sent to the default queue.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Dmitry Eremin-Solenikov(lumag) wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ... resulting packet is sent back serving
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> as IPsec completion event ...
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Dmitry Eremin-Solenikov(lumag) wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ... in inbound inline operations.
https://github.com/Linaro/odp/pull/185#discussion_r148319871
updated_at 2017-11-01 17:01:09
