JannePeltonen replied on github web page: doc/users-guide/users-guide-ipsec.adoc line 266 @@ -0,0 +1,443 @@ +== IPsec services + +In addition to general cryptographic services, ODP offers offload support for +the IPsec protocol. IPsec is a general term referencing a suite of protocols +and packet formats and as such a full discussion of IPsec is beyond the scope +of this document. See https://tools.ietf.org/html/rfc4301[RFC 4301] and +related RFCs for more detail. This section assumes the reader is already +familiar with IPsec and focuses on explaining the ODP APIs that support it. + +ODP provides APIs for the following IPsec services: + +* General IPsec configuration +* Security Association (SA) configuration and lifecycle management +* Synchronous and Asynchronous IPsec lookaside processing +* Inline processing for full IPsec RX and/or TX offload +* Pipelining for RX traffic +* Fragmentation support for TX traffic +* IPsec event management + +=== IPsec Capabilities and Configuration +As with other features, ODP provides APIs that permit applications to query +platform-specific IPsec capabilities. The `odp_ipsec_capability()` API queries +the general IPsec features available while the `odp_ipsec_cipher_capability()` +and `odp_ipsec_auth_capability()` APIs provide detail on the range of +cipher and authentication algorithms supported by IPsec on this platform. + +General IPsec capabilities that are reported include: + +* The IPsec operation modes supported by this implementation. Different +operation modes may be _not supported_, _supported_, or _preferred_. A +preferred form means that this mode takes advantage of hardware +acceleration features to achieve best performance. +* Whether IPsec AH processing is supported. All ODP platforms must provide +support for IPsec ESP processing, however since AH is relatively rare, it +may not be supported, or supported only via software emulation (_e.g.,_ be +non-preferred). +* Whether IPsec headers should be retained on decrypt for inbound inline +operations. +* Whether classification pipelining is supported (to be discussed below). + +In addition, capabilities also inform the application of the maximum number +of destination queues and classification CoS targets supported. These +will be discussed further later. + +==== IPsec Operation Modes +IPsec operates in one of three modes: Synchronous, Asynchronous, and Inline. + +==== Lookaside Processing +Synchronous and Asynchronous are types of _lookaside_ processing. In lookaside +mode, the application receives (or creates) an IPsec packet and then uses ODP +to perform one of two functions: + +* To decrypt an IPsec packet into a "normal" packet +* To take a "normal" packet and encrypt it into an IPsec packet. + +This process may be performed _synchronously_ with the APIs `odp_ipsec_in()` +(to decrypt) and `odp_ipsec_out()` (to encrypt). Upon return from these calls +the requested packet transformation is complete, or an error return code +indicates that it could not be performed (_e.g.,_ packet decryption failed). + +Synchronous processing may be preferred if the application has a large number +of worker threads so that blocking any individual worker while IPsec processing +is performed represents a reasonable design. The alternative is to use +_asynchronous_ forms of these APIs: + +* `odp_ipsec_in_enq()` for decrypt +* `odp_ipsec_out_enq()` for encrypt + +These simply pass packets to IPsec for processing. When this processing is +complete, the resulting packets are sent to the completion queue associated +with the SA used by the operation, serving as IPsec completion events as +shown here: + +image::ipsec-lookaside.svg[align="center"] + +If the operation fails because SA lookup failed for inbound processing, then +these result packets are sent to the default queue specified as part of the +`odp_ipsec_inbound_config_t` used in the `odp_ipsec_config()` call. + +Following an asynchronous IPsec call, the worker thread moves on to process +other events until the IPsec completion shows up. At that point the worker +thread sees whether the operation was successful or not and continues +processing for that packet. These events may be direct-polled with +`odp_queue_deq()` if the completion queue was created as a plain queue, or +processed via the ODP scheduler if the completion queue was created as a +scheduled queue. + +==== Inline Processing +While lookaside processing offers flexibility, it still requires extra +processing steps not required by modern hardware. To avoid this overhead +ODP also offers _inline_ processing support for IPsec. In this mode the +processing of IPsec packets on the RX and TX paths is fully offloaded as +shown here: + +image::ipsec-inline.svg[align="center"] + +It is worth noting that, depending on the implementation and application +needs, inline processing may be enabled only for one direction (inbound or +outbound) or for both directions. + +On the receive side, once configured for inline processing, arriving IPsec +packets that are recognized at the PktIO interface are decrypted automatically +before the application ever sees them. On the transmit side, the application +calls `odp_ipsec_out_inline()` and the packet is encrypted and queued for +transmission as a single operation without further application involvement. +Note that if an inbound IPsec packet is not recognized (_e.g.,_ it belongs to +an unknown SA) then it will be presented to the application as-is without +further processing. The application may then use a lookaside call to process +the packet if it is able to supply a matching SA by other means. + +On the receive side, after an IPsec packet is decrypted, it may be +_pipelined_ to the ODP classifier or added to a poll queue, as the +application wishes. The advantage of classification pipelining is that inbound +IPsec traffic is automatically decrypted and classified into appropriate +flow-based queues for ease of processing. + +On the transmit side, since IPsec encryption and tunneling may exceed an +output MTU, ODP also offers support for MTU configuration and automatic IPsec +TX fragmentation. + +Both classification pipelining and TX fragmentation support are support +features that are indicated by `odp_ipsec_capability()`. + +Note that at present inline IPsec output support sends resulting packets +directly to an output PktIO. If it's desired to send them to the ODP +Traffic Manager for shaping prior to transmission, use the lookaside APIs +to perform the IPsec encrypt and then call `odp_tm_enq()` on the resulting +packet. + +=== IPsec Configuration +Prior to making use of IPsec services, the `odp_ipsec_config()` API is used to +configure IPsec processing options. This API takes a pointer to an +`odp_ipsec_config_t` struct as its argument. SAs in ODP are represented by the +abstract type `odp_ipsec_sa_t`. + +The `odp_ipsec_config_t` struct specifies the inbound and outbound +processing modes that the application plans to use, the maximum number of +Security Associations it will use, and sets inbound and outbound +processing options. + +==== IPsec Inbound Configuration +Inbound configuration options for IPsec specify the default `odp_queue_t` to +be used for processing global events like SA lookup failures, how Security +Parameter Index (SPI) lookup is to be performed, and whether the application +requires ODP to retain outer headers for decrypted IPsec packets. + +Parsing options specify how "deep" decrypted packets are to be parsed +after IPsec processing by specifying the packet layers of interest to the +application (None, L2, L3, L4, or All). And which checksums should be verified +on decrypted packets. + +==== IPsec Outbound Configuration +Outbound configuration options for IPsec specify checksum insertion processing +that should be performed prior to encryption. + +=== IPsec Events +IPsec introduces one new event type and one new event subtype. These are: + +* IPsec packet events. These are events of type `ODP_EVENT_PACKET` that have +subtype `ODP_EVENT_PACKET_IPSEC`. These are packets that carry additional +IPsec-related metadata in the form of an `odp_ipsec_packet_result_t` struct +that can be retrieved from the packet via the `odp_ipsec_result()` API. + +* IPsec status notifications. These are events of type `ODP_EVENT_IPSEC_STATUS` +that indicate status events not associated with any particular IPsec +packet. Such events carry status in the form of an `odp_ipsec_status_t` +struct that is retrieved from the event via the `odp_ipsec_status()` API. + +IPsec-related events are thus part of normal and exception processing when +working with IPsec. + +=== Security Associations (SAs) +The fundamental "building block" for IPsec processing is the _Security +Association (SA)_. Similar to a crypto session, the SA encapsulates the keying +material and context needed to perform IPsec protocol processing for inbound +or outbound packets on a given flow, as well as additional processing options +that control how IPsec is to be used for packets processed under this +SA. Security Associations are unidirectional (RX or TX) so a flow that +requires both inbound (decrypt) and outbound (encrypt) IPsec functions will +have two SAs associated with it. + +After ODP initialization, IPsec support is dormant until it is configured +by a call to `odp_ipsec_config()` as described earlier. Once configured, +SAs may be created by calling `odp_ipsec_sa_create()`. + +==== SA Creation and Configuration +The `odp_ipsec_sa_create()` API takes an `odp_ipsec_sa_param_t` argument that +describes the SA to be created. Use the `odp_ipsec_sa_param_init()` API to +initialize this to its default state and then override selected fields within +the param struct as needed. + +Items specified in the `odp_ipsec_sa_param_t` struct include: + +* The direction of the SA (inbound or outbound). + +* The IPsec protocol being used (ESP or AH). + +* The IPsec protocol mode (Transport or Tunnel). + +* The parameters needed for the crypto and authentication algorithms to be +used by this SA. + +* Miscellaneous SA options that control behavior such as use of Extended +Sequence Numbers (ESNs), the use of UDP encapsulation, various copy +options for header fields, and whether the TTL (Hop Limit) field should be +decremented when operating in tunnel mode. + +* Parameters controlling the SA lifetime. + +* The Security Parameter Index (SPI) that packets will use to indicate that +they belong to this SA. + +* The pipeline mode used by this SA. + +* The destination `odp_queue_t` to be used for status events associated with +this SA. + +* The user context pointer (and length) associated with this SA for +application use. + +In addition, there are specific direction-specific parameters that vary +based on whether the SA is for inbound or outbound use. For inbound SAs: + +* Controls for how this SA is to be looked up. + +* The size of the anti-replay window to be used. + +* The default CoS to use when classification pipelining packets matching this +SA. + +For outbound SAs: + +* Tunnel parameters to use when doing outbound processing in tunnel mode. + +* The fragmentation mode to be used. + +* The MTU to be used to control the maximum length IP packets that outbound +IPsec operations may produce. This can be changed dynamically by the +`odp_ipsec_sa_mtu_update()` API. + +As can be seen, SAs have a large degree of configurability. + +==== SA Lifecycle Management +In discussing the lifecycle of an SA, it is useful to refer to the following +state diagram: + +image::ipsec_fsm.svg[align="center"] + +After creation, IPsec services are active for this security association. How +this is done depends on the IPsec operating mode being used + +===== IPsec Lookaside Processing +If the SA is operating in lookaside mode (the `odp_ipsec_mode_t` is +`ODP_IPSEC_OP_MODE_SYNC` or `ODP_IPSEC_OP_MODE_ASYNC`), then inbound or +outbound lookaside operations may be performed. + +The synchronous forms of these APIs are: + +* `odp_ipsec_in()` +* `odp_ipsec_out()` + +Upon return from these calls, the return code tells the application whether +the application was successful and if not the reason is contained in the +`odp_ipsec_result_t` struct that is attached to the failing packet. This +struct is retrieved via the `odp_ipsec_result()` API. +
Comment: The return code does not tell whether the application of the SA was successful but how many input packets were consumed by the operation. The per-packet operation status is retrieved via the odp_ipsec_result() api for the outputted packets, which will be available only when the function succeeds. > JannePeltonen wrote > The operating mode is a global setting, not per-SA setting as the text > implies. And the relevant type is odp_ipsec_op_mode_t, not odp_ipsec_mode_t > which is for selecting between tunnel and transport mode. >> JannePeltonen wrote >> "How this is done" sounds a bit awkward as it is not clear what "this" >> refers to (the way the IPsec services are used). Maybe the sentence should >> be reworded. >>> JannePeltonen wrote >>> Would "minimum size" be better than "size" here? >>>> JannePeltonen wrote >>>> The queue is not only for status events but for ipsec packet events too. >>>> Maybe just remove the word "status"? >>>>> JannePeltonen wrote >>>>> This should be: "...can be retained..." instead of "...should be >>>>> retained..." as this is a capability and application chooses whether it >>>>> wants to retain the headers if the ODP is capable. >>>>>> Bill Fischofer(Bill-Fischofer-Linaro) wrote: >>>>>> I'd like to see us all make it a priority to close on #197 next week. >>>>>> I'll post a doc update once we're all agreed on the precise semantics >>>>>> and operation flow. So I'm fine with holding off merging this PR until >>>>>> then. >>>>>>> Dmitry Eremin-Solenikov(lumag) wrote: >>>>>>> Any update here? @NikhilA-Linaro, @bala-manoharan can you please >>>>>>> comment wrt your implementations? >>>>>>>> Dmitry Eremin-Solenikov(lumag) wrote: >>>>>>>> Well, we do not have #197 merged, so it is too early to depend on that. >>>>>>>> Also, frankly speaking, this `sa_disabled` warning inside >>>>>>>> `odp_ipsec_result_t` is a backup plan. It is expected that most of the >>>>>>>> implementations will report this as a proper `ODP_IPSEC_STATUS` event, >>>>>>>> carrying this warning bit inside. >>>>>>>>> Bill Fischofer(Bill-Fischofer-Linaro) wrote: >>>>>>>>> OK, changed in v6. >>>>>>>>>> Dmitry Eremin-Solenikov(lumag) wrote: >>>>>>>>>> s/default // >>>>>>>>>>> Bill Fischofer(Bill-Fischofer-Linaro) wrote: >>>>>>>>>>> OK, corrected in v5. >>>>>>>>>>>> Dmitry Eremin-Solenikov(lumag) wrote: >>>>>>>>>>>> In lookaside mode soft limit expiration is reported as `warn` part >>>>>>>>>>>> of `ipsec_op_status` packet metadata. >>>>>>>>>>>>> Bill Fischofer(Bill-Fischofer-Linaro) wrote: >>>>>>>>>>>>> OK, in v4 I've added a new terminal state `SA_Expired` to the FSM >>>>>>>>>>>>> and have updated the doc to say "expired" rather than "disabled". >>>>>>>>>>>>> From the expired state the only valid operation is >>>>>>>>>>>>> `odp_ipsec_sa_destroy()`. >>>>>>>>>>>>>> Dmitry Eremin-Solenikov(lumag) wrote: >>>>>>>>>>>>>> Yep. It is just not a 'disabled' state, because we have separate >>>>>>>>>>>>>> definition for 'disabled SA'. >>>>>>>>>>>>>>> Dmitry Eremin-Solenikov(lumag) wrote: >>>>>>>>>>>>>>> IIRC, one can call it only once in NXP case. @NikhilA-Linaro, >>>>>>>>>>>>>>> could you please comment? >>>>>>>>>>>>>>>> Bill Fischofer(Bill-Fischofer-Linaro) wrote: >>>>>>>>>>>>>>>> So how is this indicated in lookaside mode? The whole point of >>>>>>>>>>>>>>>> ODP providing the limit support is so the application doesn't >>>>>>>>>>>>>>>> have to track byte/packet counts itself, so it's expected that >>>>>>>>>>>>>>>> soft limit overruns will happen as part of lookaside >>>>>>>>>>>>>>>> processing as well. >>>>>>>>>>>>>>>>> Bill Fischofer(Bill-Fischofer-Linaro) wrote: >>>>>>>>>>>>>>>>> If you let yourself run out of gas, the car can stop at >>>>>>>>>>>>>>>>> inconvenient times, which is why one pays attention to that >>>>>>>>>>>>>>>>> low fuel warning light. A hard limit is a hard limit. That's >>>>>>>>>>>>>>>>> what makes it hard. Any other definition seems confusingly >>>>>>>>>>>>>>>>> fuzzy and unnecessary. >>>>>>>>>>>>>>>>>> Bill Fischofer(Bill-Fischofer-Linaro) wrote: >>>>>>>>>>>>>>>>>> I thought the restriction is that you can call this >>>>>>>>>>>>>>>>>> repeatedly as long as an SA hasn't yet been created. I can >>>>>>>>>>>>>>>>>> change this (and the state diagram) if that's not the case. >>>>>>>>>>>>>>>>>>> Bill Fischofer(Bill-Fischofer-Linaro) wrote: >>>>>>>>>>>>>>>>>>> It's part of the `enum`. In this case L2 would effectively >>>>>>>>>>>>>>>>>>> be None. >>>>>>>>>>>>>>>>>>>> Dmitry Eremin-Solenikov(lumag) wrote: >>>>>>>>>>>>>>>>>>>> Only in OUT INLINE mode, if I remember the outcome of >>>>>>>>>>>>>>>>>>>> discussions correctly. >>>>>>>>>>>>>>>>>>>>> Dmitry Eremin-Solenikov(lumag) wrote: >>>>>>>>>>>>>>>>>>>>> Ahem. It does not enter disabled state per se: >>>>>>>>>>>>>>>>>>>>> - It is an undefined behaviour (iow, an application >>>>>>>>>>>>>>>>>>>>> error) to submit packets to disabled SA >>>>>>>>>>>>>>>>>>>>> - It is a perfectly valid to submit packets to SA after >>>>>>>>>>>>>>>>>>>>> hard limit overrun (e.g. because other packets might be >>>>>>>>>>>>>>>>>>>>> already queued at this moment). >>>>>>>>>>>>>>>>>>>>>> Dmitry Eremin-Solenikov(lumag) wrote: >>>>>>>>>>>>>>>>>>>>>> It is worth mentioning that depending on the >>>>>>>>>>>>>>>>>>>>>> implementation and application needs, inline processing >>>>>>>>>>>>>>>>>>>>>> might be enabled either in both directions or in just >>>>>>>>>>>>>>>>>>>>>> one direction. >>>>>>>>>>>>>>>>>>>>>>> Dmitry Eremin-Solenikov(lumag) wrote: >>>>>>>>>>>>>>>>>>>>>>> TBD: mention that it MUST be called at most once per >>>>>>>>>>>>>>>>>>>>>>> IPsec application. >>>>>>>>>>>>>>>>>>>>>>>> Dmitry Eremin-Solenikov(lumag) wrote: >>>>>>>>>>>>>>>>>>>>>>>> L2 does not make sense here, does it? >>>>>>>>>>>>>>>>>>>>>>>>> Dmitry Eremin-Solenikov(lumag) wrote: >>>>>>>>>>>>>>>>>>>>>>>>> ... parsed after IPsec processing ... >>>>>>>>>>>>>>>>>>>>>>>>>> Dmitry Eremin-Solenikov(lumag) wrote: >>>>>>>>>>>>>>>>>>>>>>>>>> TBD: describe that some IPsec packets still might be >>>>>>>>>>>>>>>>>>>>>>>>>> reported via plain PktIO interface (e.g. because of >>>>>>>>>>>>>>>>>>>>>>>>>> SA lookup failure). They can be resubmitted to IPsec >>>>>>>>>>>>>>>>>>>>>>>>>> in lookaside mode. >>>>>>>>>>>>>>>>>>>>>>>>>>> Dmitry Eremin-Solenikov(lumag) wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>> If SA was not determined (because SA lookup failed >>>>>>>>>>>>>>>>>>>>>>>>>>> for inbound packet), event will be sent to the >>>>>>>>>>>>>>>>>>>>>>>>>>> default queue. >>>>>>>>>>>>>>>>>>>>>>>>>>>> Dmitry Eremin-Solenikov(lumag) wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>> ... resulting packet is sent back serving as IPsec >>>>>>>>>>>>>>>>>>>>>>>>>>>> completion event ... >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Dmitry Eremin-Solenikov(lumag) wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>>> ... in inbound inline operations. https://github.com/Linaro/odp/pull/185#discussion_r148314907 updated_at 2017-11-01 16:42:42