JannePeltonen replied on github web page: doc/users-guide/users-guide-ipsec.adoc line 218 @@ -0,0 +1,443 @@ +== IPsec services + +In addition to general cryptographic services, ODP offers offload support for +the IPsec protocol. IPsec is a general term referencing a suite of protocols +and packet formats and as such a full discussion of IPsec is beyond the scope +of this document. See https://tools.ietf.org/html/rfc4301[RFC 4301] and +related RFCs for more detail. This section assumes the reader is already +familiar with IPsec and focuses on explaining the ODP APIs that support it. + +ODP provides APIs for the following IPsec services: + +* General IPsec configuration +* Security Association (SA) configuration and lifecycle management +* Synchronous and Asynchronous IPsec lookaside processing +* Inline processing for full IPsec RX and/or TX offload +* Pipelining for RX traffic +* Fragmentation support for TX traffic +* IPsec event management + +=== IPsec Capabilities and Configuration +As with other features, ODP provides APIs that permit applications to query +platform-specific IPsec capabilities. The `odp_ipsec_capability()` API queries +the general IPsec features available while the `odp_ipsec_cipher_capability()` +and `odp_ipsec_auth_capability()` APIs provide detail on the range of +cipher and authentication algorithms supported by IPsec on this platform. + +General IPsec capabilities that are reported include: + +* The IPsec operation modes supported by this implementation. Different +operation modes may be _not supported_, _supported_, or _preferred_. A +preferred form means that this mode takes advantage of hardware +acceleration features to achieve best performance. +* Whether IPsec AH processing is supported. All ODP platforms must provide +support for IPsec ESP processing, however since AH is relatively rare, it +may not be supported, or supported only via software emulation (_e.g.,_ be +non-preferred). +* Whether IPsec headers should be retained on decrypt for inbound inline +operations. +* Whether classification pipelining is supported (to be discussed below). + +In addition, capabilities also inform the application of the maximum number +of destination queues and classification CoS targets supported. These +will be discussed further later. + +==== IPsec Operation Modes +IPsec operates in one of three modes: Synchronous, Asynchronous, and Inline. + +==== Lookaside Processing +Synchronous and Asynchronous are types of _lookaside_ processing. In lookaside +mode, the application receives (or creates) an IPsec packet and then uses ODP +to perform one of two functions: + +* To decrypt an IPsec packet into a "normal" packet +* To take a "normal" packet and encrypt it into an IPsec packet. + +This process may be performed _synchronously_ with the APIs `odp_ipsec_in()` +(to decrypt) and `odp_ipsec_out()` (to encrypt). Upon return from these calls +the requested packet transformation is complete, or an error return code +indicates that it could not be performed (_e.g.,_ packet decryption failed). + +Synchronous processing may be preferred if the application has a large number +of worker threads so that blocking any individual worker while IPsec processing +is performed represents a reasonable design. The alternative is to use +_asynchronous_ forms of these APIs: + +* `odp_ipsec_in_enq()` for decrypt +* `odp_ipsec_out_enq()` for encrypt + +These simply pass packets to IPsec for processing. When this processing is +complete, the resulting packets are sent to the completion queue associated +with the SA used by the operation, serving as IPsec completion events as +shown here: + +image::ipsec-lookaside.svg[align="center"] + +If the operation fails because SA lookup failed for inbound processing, then +these result packets are sent to the default queue specified as part of the +`odp_ipsec_inbound_config_t` used in the `odp_ipsec_config()` call. + +Following an asynchronous IPsec call, the worker thread moves on to process +other events until the IPsec completion shows up. At that point the worker +thread sees whether the operation was successful or not and continues +processing for that packet. These events may be direct-polled with +`odp_queue_deq()` if the completion queue was created as a plain queue, or +processed via the ODP scheduler if the completion queue was created as a +scheduled queue. + +==== Inline Processing +While lookaside processing offers flexibility, it still requires extra +processing steps not required by modern hardware. To avoid this overhead +ODP also offers _inline_ processing support for IPsec. In this mode the +processing of IPsec packets on the RX and TX paths is fully offloaded as +shown here: + +image::ipsec-inline.svg[align="center"] + +It is worth noting that, depending on the implementation and application +needs, inline processing may be enabled only for one direction (inbound or +outbound) or for both directions. + +On the receive side, once configured for inline processing, arriving IPsec +packets that are recognized at the PktIO interface are decrypted automatically +before the application ever sees them. On the transmit side, the application +calls `odp_ipsec_out_inline()` and the packet is encrypted and queued for +transmission as a single operation without further application involvement. +Note that if an inbound IPsec packet is not recognized (_e.g.,_ it belongs to +an unknown SA) then it will be presented to the application as-is without +further processing. The application may then use a lookaside call to process +the packet if it is able to supply a matching SA by other means. + +On the receive side, after an IPsec packet is decrypted, it may be +_pipelined_ to the ODP classifier or added to a poll queue, as the +application wishes. The advantage of classification pipelining is that inbound +IPsec traffic is automatically decrypted and classified into appropriate +flow-based queues for ease of processing. + +On the transmit side, since IPsec encryption and tunneling may exceed an +output MTU, ODP also offers support for MTU configuration and automatic IPsec +TX fragmentation. + +Both classification pipelining and TX fragmentation support are support +features that are indicated by `odp_ipsec_capability()`. + +Note that at present inline IPsec output support sends resulting packets +directly to an output PktIO. If it's desired to send them to the ODP +Traffic Manager for shaping prior to transmission, use the lookaside APIs +to perform the IPsec encrypt and then call `odp_tm_enq()` on the resulting +packet. + +=== IPsec Configuration +Prior to making use of IPsec services, the `odp_ipsec_config()` API is used to +configure IPsec processing options. This API takes a pointer to an +`odp_ipsec_config_t` struct as its argument. SAs in ODP are represented by the +abstract type `odp_ipsec_sa_t`. + +The `odp_ipsec_config_t` struct specifies the inbound and outbound +processing modes that the application plans to use, the maximum number of +Security Associations it will use, and sets inbound and outbound +processing options. + +==== IPsec Inbound Configuration +Inbound configuration options for IPsec specify the default `odp_queue_t` to +be used for processing global events like SA lookup failures, how Security +Parameter Index (SPI) lookup is to be performed, and whether the application +requires ODP to retain outer headers for decrypted IPsec packets. + +Parsing options specify how "deep" decrypted packets are to be parsed +after IPsec processing by specifying the packet layers of interest to the +application (None, L2, L3, L4, or All). And which checksums should be verified +on decrypted packets. + +==== IPsec Outbound Configuration +Outbound configuration options for IPsec specify checksum insertion processing +that should be performed prior to encryption. + +=== IPsec Events +IPsec introduces one new event type and one new event subtype. These are: + +* IPsec packet events. These are events of type `ODP_EVENT_PACKET` that have +subtype `ODP_EVENT_PACKET_IPSEC`. These are packets that carry additional +IPsec-related metadata in the form of an `odp_ipsec_packet_result_t` struct +that can be retrieved from the packet via the `odp_ipsec_result()` API. + +* IPsec status notifications. These are events of type `ODP_EVENT_IPSEC_STATUS` +that indicate status events not associated with any particular IPsec +packet. Such events carry status in the form of an `odp_ipsec_status_t` +struct that is retrieved from the event via the `odp_ipsec_status()` API. + +IPsec-related events are thus part of normal and exception processing when +working with IPsec. + +=== Security Associations (SAs) +The fundamental "building block" for IPsec processing is the _Security +Association (SA)_. Similar to a crypto session, the SA encapsulates the keying +material and context needed to perform IPsec protocol processing for inbound +or outbound packets on a given flow, as well as additional processing options +that control how IPsec is to be used for packets processed under this +SA. Security Associations are unidirectional (RX or TX) so a flow that +requires both inbound (decrypt) and outbound (encrypt) IPsec functions will +have two SAs associated with it. + +After ODP initialization, IPsec support is dormant until it is configured +by a call to `odp_ipsec_config()` as described earlier. Once configured, +SAs may be created by calling `odp_ipsec_sa_create()`. + +==== SA Creation and Configuration +The `odp_ipsec_sa_create()` API takes an `odp_ipsec_sa_param_t` argument that +describes the SA to be created. Use the `odp_ipsec_sa_param_init()` API to +initialize this to its default state and then override selected fields within +the param struct as needed. + +Items specified in the `odp_ipsec_sa_param_t` struct include: + +* The direction of the SA (inbound or outbound). + +* The IPsec protocol being used (ESP or AH). + +* The IPsec protocol mode (Transport or Tunnel). + +* The parameters needed for the crypto and authentication algorithms to be +used by this SA. + +* Miscellaneous SA options that control behavior such as use of Extended +Sequence Numbers (ESNs), the use of UDP encapsulation, various copy +options for header fields, and whether the TTL (Hop Limit) field should be +decremented when operating in tunnel mode. + +* Parameters controlling the SA lifetime. + +* The Security Parameter Index (SPI) that packets will use to indicate that +they belong to this SA. + +* The pipeline mode used by this SA. + +* The destination `odp_queue_t` to be used for status events associated with +this SA. + +* The user context pointer (and length) associated with this SA for
Comment: Maybe it is not useful to mention length here since it is only a prefetching hint. > JannePeltonen wrote > Maybe it should be made more clear somewhere that the synchronous processing > calls are allowed only in the synchronous operating mode and asynchronous > calls in asynchronous and inline operation mode (and inline calls only in the > inline mode). >> JannePeltonen wrote >> This dummy packet stuff is not yet (if ever?) part of the API spec. I think >> this patch should be in sync with what is the current state of api-next, >> which still uses the status event for indicating SA disable completion. >>> JannePeltonen wrote >>> "This call" could be replaced with odp_ipsec_sa_disable() to make the text >>> more clear. >>>> JannePeltonen wrote >>>> I think this may give the impression that calling odp_ipsec_sa_disable() >>>> is optional, when in fact it is mandatory before odp_ipsec_sa_destroy(). >>>>> JannePeltonen wrote >>>>> According to the API spec soft limit alerts do not need to be edge >>>>> triggered. The spec says: "It's implementation defined how many times >>>>> soft lifetime expiration is reported: only once, first N or all packets >>>>> following the limit crossing." >>>>>> JannePeltonen wrote >>>>>> Is it clear enough here that the mtu error bit is set only when MTU >>>>>> checking was requested and not in case fragmentation offload was >>>>>> requested? >>>>>>> JannePeltonen wrote >>>>>>> Same comment as for odp_ipsec_{in,out}(), the description of the return >>>>>>> code is not correct. >>>>>>>> JannePeltonen wrote >>>>>>>> The return code does not tell whether the application of the SA was >>>>>>>> successful but how many input packets were consumed by the operation. >>>>>>>> The per-packet operation status is retrieved via the >>>>>>>> odp_ipsec_result() api for the outputted packets, which will be >>>>>>>> available only when the function succeeds. >>>>>>>>> JannePeltonen wrote >>>>>>>>> The operating mode is a global setting, not per-SA setting as the >>>>>>>>> text implies. And the relevant type is odp_ipsec_op_mode_t, not >>>>>>>>> odp_ipsec_mode_t which is for selecting between tunnel and transport >>>>>>>>> mode. >>>>>>>>>> JannePeltonen wrote >>>>>>>>>> "How this is done" sounds a bit awkward as it is not clear what >>>>>>>>>> "this" refers to (the way the IPsec services are used). Maybe the >>>>>>>>>> sentence should be reworded. >>>>>>>>>>> JannePeltonen wrote >>>>>>>>>>> Would "minimum size" be better than "size" here? >>>>>>>>>>>> JannePeltonen wrote >>>>>>>>>>>> The queue is not only for status events but for ipsec packet >>>>>>>>>>>> events too. Maybe just remove the word "status"? >>>>>>>>>>>>> JannePeltonen wrote >>>>>>>>>>>>> This should be: "...can be retained..." instead of "...should be >>>>>>>>>>>>> retained..." as this is a capability and application chooses >>>>>>>>>>>>> whether it wants to retain the headers if the ODP is capable. >>>>>>>>>>>>>> Bill Fischofer(Bill-Fischofer-Linaro) wrote: >>>>>>>>>>>>>> I'd like to see us all make it a priority to close on #197 next >>>>>>>>>>>>>> week. I'll post a doc update once we're all agreed on the >>>>>>>>>>>>>> precise semantics and operation flow. So I'm fine with holding >>>>>>>>>>>>>> off merging this PR until then. >>>>>>>>>>>>>>> Dmitry Eremin-Solenikov(lumag) wrote: >>>>>>>>>>>>>>> Any update here? @NikhilA-Linaro, @bala-manoharan can you >>>>>>>>>>>>>>> please comment wrt your implementations? >>>>>>>>>>>>>>>> Dmitry Eremin-Solenikov(lumag) wrote: >>>>>>>>>>>>>>>> Well, we do not have #197 merged, so it is too early to depend >>>>>>>>>>>>>>>> on that. >>>>>>>>>>>>>>>> Also, frankly speaking, this `sa_disabled` warning inside >>>>>>>>>>>>>>>> `odp_ipsec_result_t` is a backup plan. It is expected that >>>>>>>>>>>>>>>> most of the implementations will report this as a proper >>>>>>>>>>>>>>>> `ODP_IPSEC_STATUS` event, carrying this warning bit inside. >>>>>>>>>>>>>>>>> Bill Fischofer(Bill-Fischofer-Linaro) wrote: >>>>>>>>>>>>>>>>> OK, changed in v6. >>>>>>>>>>>>>>>>>> Dmitry Eremin-Solenikov(lumag) wrote: >>>>>>>>>>>>>>>>>> s/default // >>>>>>>>>>>>>>>>>>> Bill Fischofer(Bill-Fischofer-Linaro) wrote: >>>>>>>>>>>>>>>>>>> OK, corrected in v5. >>>>>>>>>>>>>>>>>>>> Dmitry Eremin-Solenikov(lumag) wrote: >>>>>>>>>>>>>>>>>>>> In lookaside mode soft limit expiration is reported as >>>>>>>>>>>>>>>>>>>> `warn` part of `ipsec_op_status` packet metadata. >>>>>>>>>>>>>>>>>>>>> Bill Fischofer(Bill-Fischofer-Linaro) wrote: >>>>>>>>>>>>>>>>>>>>> OK, in v4 I've added a new terminal state `SA_Expired` to >>>>>>>>>>>>>>>>>>>>> the FSM and have updated the doc to say "expired" rather >>>>>>>>>>>>>>>>>>>>> than "disabled". From the expired state the only valid >>>>>>>>>>>>>>>>>>>>> operation is `odp_ipsec_sa_destroy()`. >>>>>>>>>>>>>>>>>>>>>> Dmitry Eremin-Solenikov(lumag) wrote: >>>>>>>>>>>>>>>>>>>>>> Yep. It is just not a 'disabled' state, because we have >>>>>>>>>>>>>>>>>>>>>> separate definition for 'disabled SA'. >>>>>>>>>>>>>>>>>>>>>>> Dmitry Eremin-Solenikov(lumag) wrote: >>>>>>>>>>>>>>>>>>>>>>> IIRC, one can call it only once in NXP case. >>>>>>>>>>>>>>>>>>>>>>> @NikhilA-Linaro, could you please comment? >>>>>>>>>>>>>>>>>>>>>>>> Bill Fischofer(Bill-Fischofer-Linaro) wrote: >>>>>>>>>>>>>>>>>>>>>>>> So how is this indicated in lookaside mode? The whole >>>>>>>>>>>>>>>>>>>>>>>> point of ODP providing the limit support is so the >>>>>>>>>>>>>>>>>>>>>>>> application doesn't have to track byte/packet counts >>>>>>>>>>>>>>>>>>>>>>>> itself, so it's expected that soft limit overruns will >>>>>>>>>>>>>>>>>>>>>>>> happen as part of lookaside processing as well. >>>>>>>>>>>>>>>>>>>>>>>>> Bill Fischofer(Bill-Fischofer-Linaro) wrote: >>>>>>>>>>>>>>>>>>>>>>>>> If you let yourself run out of gas, the car can stop >>>>>>>>>>>>>>>>>>>>>>>>> at inconvenient times, which is why one pays >>>>>>>>>>>>>>>>>>>>>>>>> attention to that low fuel warning light. A hard >>>>>>>>>>>>>>>>>>>>>>>>> limit is a hard limit. That's what makes it hard. Any >>>>>>>>>>>>>>>>>>>>>>>>> other definition seems confusingly fuzzy and >>>>>>>>>>>>>>>>>>>>>>>>> unnecessary. >>>>>>>>>>>>>>>>>>>>>>>>>> Bill Fischofer(Bill-Fischofer-Linaro) wrote: >>>>>>>>>>>>>>>>>>>>>>>>>> I thought the restriction is that you can call this >>>>>>>>>>>>>>>>>>>>>>>>>> repeatedly as long as an SA hasn't yet been created. >>>>>>>>>>>>>>>>>>>>>>>>>> I can change this (and the state diagram) if that's >>>>>>>>>>>>>>>>>>>>>>>>>> not the case. >>>>>>>>>>>>>>>>>>>>>>>>>>> Bill Fischofer(Bill-Fischofer-Linaro) wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>> It's part of the `enum`. In this case L2 would >>>>>>>>>>>>>>>>>>>>>>>>>>> effectively be None. >>>>>>>>>>>>>>>>>>>>>>>>>>>> Dmitry Eremin-Solenikov(lumag) wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>> Only in OUT INLINE mode, if I remember the outcome >>>>>>>>>>>>>>>>>>>>>>>>>>>> of discussions correctly. >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Dmitry Eremin-Solenikov(lumag) wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Ahem. It does not enter disabled state per se: >>>>>>>>>>>>>>>>>>>>>>>>>>>>> - It is an undefined behaviour (iow, an >>>>>>>>>>>>>>>>>>>>>>>>>>>>> application error) to submit packets to disabled >>>>>>>>>>>>>>>>>>>>>>>>>>>>> SA >>>>>>>>>>>>>>>>>>>>>>>>>>>>> - It is a perfectly valid to submit packets to SA >>>>>>>>>>>>>>>>>>>>>>>>>>>>> after hard limit overrun (e.g. because other >>>>>>>>>>>>>>>>>>>>>>>>>>>>> packets might be already queued at this moment). >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Dmitry Eremin-Solenikov(lumag) wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It is worth mentioning that depending on the >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> implementation and application needs, inline >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> processing might be enabled either in both >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> directions or in just one direction. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Dmitry Eremin-Solenikov(lumag) wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> TBD: mention that it MUST be called at most >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> once per IPsec application. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Dmitry Eremin-Solenikov(lumag) wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> L2 does not make sense here, does it? >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Dmitry Eremin-Solenikov(lumag) wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ... parsed after IPsec processing ... >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Dmitry Eremin-Solenikov(lumag) wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> TBD: describe that some IPsec packets still >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> might be reported via plain PktIO interface >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (e.g. because of SA lookup failure). They >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> can be resubmitted to IPsec in lookaside >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mode. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Dmitry Eremin-Solenikov(lumag) wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If SA was not determined (because SA lookup >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> failed for inbound packet), event will be >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> sent to the default queue. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Dmitry Eremin-Solenikov(lumag) wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ... resulting packet is sent back serving >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> as IPsec completion event ... >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Dmitry Eremin-Solenikov(lumag) wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ... in inbound inline operations. https://github.com/Linaro/odp/pull/185#discussion_r148319871 updated_at 2017-11-01 17:01:09