Hi Tom, I agree that using IOAM in IPv6 both e2e and hbh is a powerful and useful combo!
My point, sorry if I was not clear, is that an “SFC Hop” does not correspond to a transport encapsulation hop, and that IOAM can be in-situ’ed to the encapsulation that realizes the (service, overlay, otherwise higher) topology (which can be IPv6 natively or something else as well) Thanks, Thumb typed by Carlos Pignataro. Excuze typofraphicak errows > On Apr 21, 2018, at 11:05, Tom Herbert <t...@herbertland.com> wrote: > > On Fri, Apr 20, 2018 at 12:03 PM, Carlos Pignataro (cpignata) > <cpign...@cisco.com> wrote: >> Tom, >> >> On Apr 17, 2018, at 10:22 AM, Tom Herbert <t...@herbertland.com> wrote: >> >> On Tue, Apr 17, 2018 at 12:51 AM, Frank Brockners (fbrockne) >> <fbroc...@cisco.com> wrote: >> >> >> Hi Tianran, >> >> Tom's note already includes the hint: You'll add IOAM data to the >> protocol/layer that you're interested in monitoring. Again using Geneve over >> IPv6 as an example: >> * If you're interested in the overlay, i.e. Geneve (e.g. timestamping the >> packet when it enters and exists the tunnel) - you'd add IOAM data to Geneve >> * If you're interested in the underlay, i.e. IPv6 (e.g. you'd like to >> understand which path packets take in the v6 network) - you'd add IOAM data >> to IPv6 >> * If you're interested in both, then you'd add IOAM data to Geneve and IPv6 >> >> Frank, >> >> In that case why not just use a hop-by-hop option for measuring the >> underlay and a destination option for measuring the overlay? The >> advantage is that this works _any_ IP encapsulation method or any IP >> protocol for that matter. >> >> >> Because you want to instrument the layer that you want to measure. >> Because there’s cases with more unnatural layering where there’s a desire to >> correlate and compare measurements across layers (in a way in which, for >> example, the Service layer is tested in a service chaining scenario, not the >> IPv6 hop-by-hop. >> Because different topologies expose different Hops and IPv6 HBH goes by the >> IPv6 node topology. >> Because not everything is IPv6, and because you have cases of IPv6 over >> something as well. >> Those are quick ones that come to mind. >> > Carlos, > > Please see my other email that details some use cases that shows > destination options are functionally equivalent to ippm in > encapsulation, and also my comments that the IPv6 has superior > capabilities to cover in-situ ippm requirements (in particular that IP > options are the _only_ protocol conformant means for intermediate > nodes to change IP payloads needed for IOAM tracing). > > I don't have a general issue with supporting ippm in encapsulation, > but I do think this should be viewed as legacy support. Note there is > no concept of segment routing in IPv4, they are blazing forward only > on IPv6 so it is reasonable to take this view. Personally, I don't > think this is a disadvantage to SR. IPv6 does have more capabilities > than IPv4 and we're now seeing protocols that will take advantage of > those. Features like this are good motivation for moving to IPv6, > which in the long run is good for the Internet! > > Tom > >> Frank, >> I don't believe adding ippm to every >> encapsulation protocol is straightforward: e.g. >> draft-brockners-ippm-ioam-geneve describe but notes the limited size >> of header, draft-weis-ippm-ioam-gre states that a new EtherType would >> be needed just for this purpose. This also entails additional >> encapsulation-specific HW support also, whereas support destination >> and hbh options could be more generic. >> >> >> Engineering is about trade-offs. If you want to measure Geneve, there are >> limitations. But instead of trying to prove why it does not work, I’ll point >> to working demos of where it does — many of which on different HW/SW and >> encaps, shown at various IETF events. >> >> Thanks, >> >> — Carlos Pignataro >> >> Tom >> >> Draft draft-ietf-ippm-ioam-data-02 already mentions layering (see section >> 3): >> "Layering: If several encapsulation protocols (e.g., in case of tunneling) >> are stacked on top of each other, IOAM data-records could be present at >> every layer. The behavior follows the ships-in-the-night model." >> >> Given the discussion here, we'll add some additional text in the next >> revision to make things crisper (e.g. adding an example might help). >> >> Frank >> >> -----Original Message----- >> From: Tianran Zhou <zhoutian...@huawei.com> >> Sent: Dienstag, 17. April 2018 03:18 >> To: Tom Herbert <t...@herbertland.com> >> Cc: Shwetha Bhandari (shwethab) <shwet...@cisco.com>; Frank Brockners >> (fbrockne) <fbroc...@cisco.com>; Mickey Spiegel >> <mspie...@barefootnetworks.com>; NVO3 <nvo3@ietf.org>; Service Function >> Chaining IETF list <s...@ietf.org>; IETF IPPM WG <i...@ietf.org> >> Subject: RE: [ippm] [Int-area] encapsulation of IOAM data in various >> protocols - follow up from WG discussion in London >> >> I think it's better that Frank or Shwetha can explain the multi-layer use >> case in detail. >> >> Tianran >> >> -----Original Message----- >> From: Tom Herbert [mailto:t...@herbertland.com] >> Sent: Monday, April 16, 2018 10:40 PM >> To: Tianran Zhou <zhoutian...@huawei.com> >> Cc: Shwetha Bhandari (shwethab) <shwet...@cisco.com>; Frank Brockners >> (fbrockne) <fbroc...@cisco.com>; Mickey Spiegel >> <mspie...@barefootnetworks.com>; NVO3 <nvo3@ietf.org>; int-area >> <int-a...@ietf.org>; Service Function Chaining IETF list >> <s...@ietf.org>; IETF IPPM WG <i...@ietf.org> >> Subject: Re: [ippm] [Int-area] encapsulation of IOAM data in various >> protocols - follow up from WG discussion in London >> >> On Mon, Apr 16, 2018 at 6:31 AM, Tianran Zhou <zhoutian...@huawei.com> >> wrote: >> >> Hi Shwetha, >> >> You are talking about the outer encapsution. It is straight forward >> for the underlay to record by the header. But what about the >> overlay, i.e., inner encapsulation(e.g. geneve)? Without special >> configuration, intermediate node will not read the inner header, >> hence not be able to process IOAM.e >> >> >> Hi Tianran, >> >> I believe that is also not protocol conformant. Intermediate nodes >> should not be processing transport layer data as this can lead to >> misinterpretation and possibly silent data corruption. >> >> For instance, Geneve is a UDP encapsulation protocol with assigned port >> 6081. >> In order for an intermediate device to process the Geneve >> encapsulation header it would need to look for packets with >> destination port of 6081 since that is the only possible >> discriminator. However, transport port numbers do not have global >> meaning and hosts may use port numbers for other purposes (RFC7605 >> describes this). So a packet to port 6081 might be something other >> than Geneve and may be misinterpreted. If a misinterpreted packet is changed >> (like ippm data is written) then that would be systematic silent data >> corruption. >> >> As far as I know, hop-by-hop options is the only protocol confirming >> mechanism that allows an intermediate note to change data of packet in >> flight. >> Encpasulation is the only conforming mechanism that allows an >> intermediate node to add data (like extension headers) to a packet in >> flight. >> >> Tom >> >> Maybe we are not synced by this overlay/underlay use case. :-) >> >> Tianran >> >> >> >> ________________________________ >> Sent from WeLink >> >> 发件人: Shwetha Bhandari (shwethab) >> 收件人: Tianran Zhou<zhoutian...@huawei.com>;Frank Brockners >> (fbrockne)<fbroc...@cisco.com>;Mickey >> Spiegel<mspie...@barefootnetworks.com>;Tom >> Herbert<t...@herbertland.com> >> 抄送: NVO3<nvo3@ietf.org>;int-area<int-a...@ietf.org>;Service Function >> Chaining IETF list<s...@ietf.org>;IETF IPPM WG<i...@ietf.org> >> 主题: Re: [ippm] [Int-area] encapsulation of IOAM data in various >> protocols - follow up from WG discussion in London >> 时间: 2018-04-16 18:17:01 >> >> Hi Tianran, >> >> If I recall right, it is not written in the ioam data draft. >> >> >> Data draft is defining the data to be carried in IOAM in an >> encapsulation agnostic way, it does not specify how the >> encapsulation protocol is configured. >> >> >> >> Yes, node by node configuration is an easy way. >> >> >> While it is, it does not have to be a node by node configuration. It >> can be part of the encapsulation definition. >> >> For e.g. If the encapsulation is IPv6 and if we define the data to >> be carried as HbH options, then based on the Option Type with >> highest order 2 bits set to 00 then the v6 nodes that implement IOAM >> will process the option and others will skip over. >> >> >> >> >> >> Thanks, >> >> Shwetha >> >> >> >> From: ippm <ippm-boun...@ietf.org> on behalf of Tianran Zhou >> <zhoutian...@huawei.com> >> Date: Monday, April 16, 2018 at 2:36 PM >> To: "Frank Brockners (fbrockne)" <fbroc...@cisco.com>, Mickey >> Spiegel <mspie...@barefootnetworks.com>, Tom Herbert >> <t...@herbertland.com> >> Cc: NVO3 <nvo3@ietf.org>, "int-a...@ietf.org" <int-a...@ietf.org>, >> Service Function Chaining IETF list <s...@ietf.org>, IETF IPPM WG >> <i...@ietf.org> >> Subject: Re: [ippm] [Int-area] encapsulation of IOAM data in various >> protocols - follow up from WG discussion in London >> >> >> >> Hi Frank, >> >> >> >> If I recall right, it is not written in the ioam data draft. >> >> Yes, node by node configuration is an easy way. In the >> draft-zhou-ippm-ioam-yang, we have the “protocol-type” to indicate >> the layering. >> >> +--rw ioam >> >> +--rw ioam-profiles >> >> +--rw enabled? boolean >> >> +--rw ioam-profile* [profile-name] >> >> +--rw profile-name string >> >> +--rw filter >> >> | +--rw filter-type? ioam-filter-type >> >> | +--rw acl-name? -> /acl:acls/acl/name >> >> +--rw protocol-type? ioam-protocol-type >> >> +--rw incremental-tracing-profile {incremental-trace}? >> >> | ... >> >> +--rw preallocated-tracing-profile {preallocated-trace}? >> >> | ... >> >> +--rw pot-profile {proof-of-transit}? >> >> | ... >> >> +--rw e2e-profile {edge-to-edge}? >> >> ... >> >> >> >> >> >> Tianran >> >> From: Frank Brockners (fbrockne) [mailto:fbroc...@cisco.com] >> Sent: Monday, April 16, 2018 4:51 PM >> To: Tianran Zhou <zhoutian...@huawei.com>; Mickey Spiegel >> <mspie...@barefootnetworks.com>; Tom Herbert <t...@herbertland.com> >> Cc: NVO3 <nvo3@ietf.org>; int-a...@ietf.org; Service Function >> Chaining IETF list <s...@ietf.org>; IETF IPPM WG <i...@ietf.org> >> Subject: RE: [ippm] [Int-area] encapsulation of IOAM data in various >> protocols - follow up from WG discussion in London >> >> >> >> Hi Tianran, >> >> >> >> IOAM is a domain specific feature (see also >> draft-ietf-ippm-ioam-data-02 sections 3 and 4), which allows an >> operator to control by means of configuration where and for which >> traffic IOAM data fields are added/updated/removed from the customer >> traffic. Using your example of Geneve over IPv6 – with IOAM data in >> both the Geneve and the IPv6 protocol, one would expect that the >> operator configures the endpoints of the Geneve tunnel to operate on >> the IOAM data in Geneve, and the IPv6 routers that the Geneve tunnel >> >> traverses to operate on the IOAM data in IPv6. >> >> >> >> >> Frank >> >> >> >> From: Tianran Zhou <zhoutian...@huawei.com> >> Sent: Montag, 16. April 2018 10:37 >> To: Frank Brockners (fbrockne) <fbroc...@cisco.com>; Mickey Spiegel >> <mspie...@barefootnetworks.com>; Tom Herbert <t...@herbertland.com> >> Cc: NVO3 <nvo3@ietf.org>; int-a...@ietf.org; Service Function >> Chaining IETF list <s...@ietf.org>; IETF IPPM WG <i...@ietf.org> >> Subject: RE: [ippm] [Int-area] encapsulation of IOAM data in various >> protocols - follow up from WG discussion in London >> >> >> >> Hi Frank, >> >> >> >> How does a forwarder know when and where to insert the data? >> >> In the case of Geneve over IPv6, do you mean the device need to scan >> all the protocol stack? Or just the outer encapsulation? >> >> >> >> Tianran >> >> >> >> From: ippm [mailto:ippm-boun...@ietf.org] On Behalf Of Frank >> Brockners >> (fbrockne) >> Sent: Monday, April 16, 2018 3:08 PM >> To: Mickey Spiegel <mspie...@barefootnetworks.com>; Tom Herbert >> <t...@herbertland.com> >> Cc: NVO3 <nvo3@ietf.org>; int-a...@ietf.org; Service Function >> Chaining IETF list <s...@ietf.org>; IETF IPPM WG <i...@ietf.org> >> Subject: Re: [ippm] [Int-area] encapsulation of IOAM data in various >> protocols - follow up from WG discussion in London >> >> >> >> >> >> Tom, >> >> >> >> a quick addition to what Mickey mentioned below: What you seem to >> have in mind is what draft-ietf-ippm-ioam-data-02 refers to as “layering” >> (see section 3.), i.e. if you’re running for example Geneve over >> IPv6, then IOAM data could be encapsulated in both protocols, Geneve >> and >> IPv6 – giving you visibility into the “underlay” (IPv6) and the “overlay” >> >> (Geneve). >> >> >> >> >> Frank >> >> >> >> From: ippm <ippm-boun...@ietf.org> On Behalf Of Mickey Spiegel >> Sent: Freitag, 13. April 2018 20:22 >> To: Tom Herbert <t...@herbertland.com> >> Cc: NVO3 <nvo3@ietf.org>; int-a...@ietf.org; Service Function >> Chaining IETF list <s...@ietf.org>; IETF IPPM WG <i...@ietf.org> >> Subject: Re: [ippm] [Int-area] encapsulation of IOAM data in various >> protocols - follow up from WG discussion in London >> >> >> >> Tom, >> >> >> >> On Thu, Apr 12, 2018 at 10:17 PM, Tom Herbert <t...@herbertland.com> wrote: >> >> Mickey, >> >> Looking at these ippm drafts more closely, I have a much more >> fundamental concern. >> >> In draft-brockners-ippm-ioam-geneve-00 for instance, there is the >> text in the introduction: >> >> "In-situ OAM (IOAM) records OAM information within the packet while >> the packet traverses a particular network domain. The term "in-situ" >> refers to the fact that the IOAM data fields are added to the data >> packets rather than is being sent within packets specifically >> dedicated to OAM. This document defines how IOAM data fields are >> transported as part of the Geneve [I-D.ietf-nvo3-geneve] >> encapsulation." >> >> I assume this means that as packets with Geneve encapsulation >> traverse the network they are interpreted by intermediate nodes as >> being Geneve. Since Geneve is a UDP encapsulation, then the >> destination UDP port number would be used to identify packets as >> being Geneve. So an intermediate device might be looking for UDP >> packets destined to port >> 6081 (the assigned UDP port for Geneve). If my understanding is >> correct, then this is a problem. >> >> UDP port numbers do not have global meaning. An intermediate device >> may very well see UDP packets destined to port 6081 that are not >> actually Geneve. This scenario is discussed in RFC7605: >> >> "...intermediate device interprets traffic based on the port number. >> It is important to recognize that any interpretation of port numbers >> -- except at the endpoints -- may be incorrect, because port numbers >> are meaningful only at the endpoints." >> >> If the UDP data is modified, as the draft would imply, then >> misinterpretation may also mean silent data corruption of packets. A >> protocol that would allow this seems pretty incorrect! Note that >> this would be true also for any UDP encapsulation that the network >> tries to interpret. >> >> >> >> The intention is to allow for multiple nodes that a packet traverses >> >> to be able to insert IOAM node information in the same trace option, >> >> but leave some flexibility regarding which nodes actually do the >> >> IOAM processing and the node information. This may vary >> >> depending on the transport. >> >> >> >> In case of a tunneled encapsulation such as Geneve or VXLAN, >> >> there may still be multiple hops. For example a network may use >> >> Geneve or VXLAN, but only do L2 processing at ToRs, with L3 >> >> processing done at aggregation or core switches. In this case >> >> many packets would do 2 Geneve or VXLAN hops, so the packet >> >> would contain IOAM node information from two nodes. >> >> >> >> Another example is service function chaining using Geneve or >> >> VXLAN rather than NSH. >> >> >> >> >> I am also wondering if hop-by-hop options been considered for this >> application? Their interpretation in the network is unabiguous and >> they also have the advantage that the work with any IP protocol or >> encapsulation. >> >> >> >> IPv6 hop-by-hop options has been considered. See >> >> draft-brockners-inband-oam-transport-05. This has not yet been >> >> broken out into a separate draft. >> >> >> >> Mickey >> >> >> >> >> Thanks, >> Tom >> >> >> On Thu, Apr 12, 2018 at 3:31 PM, Mickey Spiegel >> <mspie...@barefootnetworks.com> wrote: >> >> Tom, >> >> On Thu, Apr 12, 2018 at 2:46 PM, Tom Herbert <t...@herbertland.com> wrote: >> >> >> On Thu, Apr 12, 2018 at 9:54 AM, Greg Mirsky >> <gregimir...@gmail.com> >> wrote: >> >> Hi Frank, >> thank you for sharing your points. Please find my notes in-line >> and tagged >> GIM>>. I believe that this is very much relevant to work of >> GIM>>other >> working >> groups that directly work on the overlay encapsulations in the >> center of the discussion and hence I've added them to the list. >> Hope we'll have more opinions to reach the conclusion that is >> acceptable to all. >> >> Regards, >> Greg >> >> On Wed, Apr 11, 2018 at 12:02 PM, Frank Brockners (fbrockne) >> <fbroc...@cisco.com> wrote: >> >> >> Back at the IPPM meeting in London, we discussed several drafts >> dealing with the encapsulation of IOAM data in various >> protocols (draft-brockners-ippm-ioam-vxlan-gpe-00, >> draft-brockners-ippm-ioam-geneve-00, >> draft-weis-ippm-ioam-gre-00). One discussion topic that we >> decided to take to the list was the question on whether >> draft-ooamdt-rtgwg-ooam-header could be leveraged.. After >> carefully considering draft-ooamdt-rtgwg-ooam-header, I came to >> the conclusion that the “OOAM header” does not meet the needs >> of >> IOAM: >> >> * Efficiency: IOAM adds data to live user traffic. As such, an >> encapsulation needs to be as efficient as possible. The “OOAM header” >> is 8 >> bytes long. The approach for IOAM data encapsulation in the >> above mentioned drafts only requires 4 bytes. Using the OOAM >> header approach would add an unnecessary overhead of 4 bytes – >> which is significant. >> >> Greg, >> >> I'm missing something here. I looked at the drafts you referenced >> and each of them looks like the overhead for OAM is greater that >> four bytes. In each there is some overhead equivalent to >> type/length, for instance in Geneve four bytes are needed for >> option class, type, and length. Unless the the OAM data is zero >> length, I don't see how this adds up to only four bytes of overhead. >> >> >> >> The four versus eight bytes just refers to the fields in the four >> bytes of IOAM info, that is common to all IOAM options. Beyond >> that, there are IOAM option specific fields. For example if doing >> one of the IOAM trace options, there are four bytes of trace option >> header, including the IOAM-trace-type, NodeLen, Flags, and >> RemainingLen fields. These are followed by the node data list >> containing the per hop IOAM information. >> >> In looking at the OOAM header content, it has nothing to do with >> any of the IOAM information after the first four bytes. It contains >> another variant of the information in the first four bytes of IOAM >> info, spread out over eight bytes. >> >> >> Tom >> >> >> GIM>> The difference in four octets is because OOAM Header: >> >> provides more flexibility, e.g. Flags field and Reserved fields; >> >> >> >> The flags field only has one defined flag at the moment, for a >> timestamp block. For IOAM trace we need per hop timestamps, which >> the timestamp block cannot address, i.e. the timestamp block is >> redundant for >> >> IOAM. >> >> >> >> supports larger OAM packets than iOAM header; >> >> >> >> For IOAM purposes, 1020 octets is more than enough. >> >> >> is future proof by supporting versioning (Version field). >> >> >> >> IMO, taking the first two bits of the IOAM-Type to define a Version >> field would be a good thing. This does not require adding four more >> bytes of overhead. 64 IOAM-Types is more than enough. >> >> >> >> * Maturity: IOAM has several implementations, which were also >> shown at recent IETF hackathons – and we’re expecting >> additional implementations to be publicized soon. Interoperable >> implementations need timely specifications. Despite the >> question being asked, the recent thread on OOAM in the NVO3 >> list hasn’t revealed any implementation of the OOAM header. >> In >> addition, the thread revealed that several fundamental >> questions about the OOAM header are still open, such as whether >> or how active OAM mechanisms within protocols such as Geneve >> would apply to the OOAM header. This ultimately means that we >> won’t get to a timely specification. >> >> >> GIM>> May I ask which encapsulations supported by the >> GIM>> implementations >> you >> refer to. Until very recently all iOAM proposals were to use >> meta-data TLV in, e.g. Geneve and NSH. And if these or some of >> these implementations already updated to the newly proposed iOAM >> shim, I don't see problem in making them use OOAM Header. Would >> you agree? >> >> >> * Scope: It isn’t entirely clear to which protocols the OOAM >> header would ultimately apply to. The way the OOAM header is >> defined, OOAM uses a 8-bit field for “Next Prot”, the next >> protocol. Some protocols that IOAM data needs to be >> encapsulated into use 16-bits for their next protocol code points. See e.g. >> the GRE encapsulation – as specified in >> draft-weis-ippm-ioam-gre-00. >> >> >> GIM>> The first paragraph of the Introduction section states: >> New protocols that support overlay networks like VxLAN-GPE >> [I-D.ietf-nvo3-vxlan-gpe], GUE [I-D.ietf-nvo3-gue], Geneve >> [I-D.ietf-nvo3-geneve], BIER >> [I-D.ietf-bier-mpls-encapsulation], >> >> and >> >> NSH [I-D.ietf-sfc-nsh] support multi-protocol payload, e.g. >> Ethernet, IPv4/IPv6, and recognize Operations, Administration, and >> Maintenance (OAM) as one of distinct types. That ensures that >> Overlay OAM (OOAM)packets are sharing fate with Overlay data packet >> traversing the underlay. >> I'm updating the OOAM Header draft and along with cleaning nits >> will update reference to GUE. I think that the list and the >> statemnt are quite clear in identifying the scope of networks >> that may benefit from using not only common OOAM Header but >> common OOAM mechanisms, e.g. Echo Request/Reply. >> >> With the above in mind, I’d suggest that the WG moves forward >> with specific definitions for encapsulating IOAM data into >> protocols – per the above mentioned drafts. >> >> >> >> Regards, Frank >> >> >> _______________________________________________ >> ippm mailing list >> i...@ietf.org >> https://www.ietf.org/mailman/listinfo/ippm >> >> >> >> _______________________________________________ >> Int-area mailing list >> int-a...@ietf.org >> https://www.ietf.org/mailman/listinfo/int-area >> >> >> _______________________________________________ >> ippm mailing list >> i...@ietf.org >> https://www.ietf.org/mailman/listinfo/ippm >> >> >> >> >> >> >> _______________________________________________ >> sfc mailing list >> s...@ietf.org >> https://www.ietf.org/mailman/listinfo/sfc >> >> _______________________________________________ nvo3 mailing list nvo3@ietf.org https://www.ietf.org/mailman/listinfo/nvo3