| RFC 9259 | SRv6 OAM | May 2022 |
| Ali, et al. | Standards Track | [Page] |
This document describes how the existing IPv6 mechanisms for ping and traceroute can be used in an SRv6 network. The document also specifies the OAM flag (O-flag) in the Segment Routing Header (SRH) for performing controllable and predictable flow sampling from segment endpoints. In addition, the document describes how a centralized monitoring system performs a path continuity check between any nodes within an SRv6 domain.¶
This is an Internet Standards Track document.¶
This document is a product of the Internet Engineering Task Force (IETF). It represents the consensus of the IETF community. It has received public review and has been approved for publication by the Internet Engineering Steering Group (IESG). Further information on Internet Standards is available in Section 2 of RFC 7841.¶
Information about the current status of this document, any errata, and how to provide feedback on it may be obtained at https://www.rfc-editor.org/info/rfc9259.¶
Copyright (c) 2022 IETF Trust and the persons identified as the document authors. All rights reserved.¶
This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (https://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Revised BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Revised BSD License.¶
As Segment Routing with IPv6 data plane (SRv6) [RFC8402] simply adds a new type of Routing Extension Header, existing IPv6 OAM mechanisms can be used in an SRv6 network. This document describes how the existing IPv6 mechanisms for ping and traceroute can be used in an SRv6 network. This includes illustrations of pinging an SRv6 Segment Identifier (SID) to verify that the SID is reachable and is locally programmed at the target node. This also includes illustrations for tracerouting to an SRv6 SID for hop-by-hop fault localization as well as path tracing to a SID.¶
This document also introduces enhancements for the OAM mechanism for SRv6 networks for performing controllable and predictable flow sampling from segment endpoints using, e.g., the IP Flow Information Export (IPFIX) protocol [RFC7011]. Specifically, the document specifies the OAM flag (O-flag) in the SRH as a marking bit in the user packets to trigger telemetry data collection and export at the segment endpoints.¶
This document also outlines how the centralized OAM technique in [RFC8403] can be extended for SRv6 to perform a path continuity check between any nodes within an SRv6 domain. Specifically, the document illustrates how a centralized monitoring system can monitor arbitrary SRv6 paths by creating loopback probes that originate and terminate at the centralized monitoring system.¶
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in BCP 14 [RFC2119] [RFC8174] when, and only when, they appear in all capitals, as shown here.¶
The following abbreviations are used in this document:¶
The terminology and simple topology in this section are used for illustration throughout the document.¶
+--------------------------| N100 |---------------------------------+
| |
| ====== link1====== link3------ link5====== link9------ ====== |
||N1||------||N2||------| N3 |------||N4||------| N5 |---||N7||
|| ||------|| ||------| |------|| ||------| |---|| ||
====== link2====== link4------ link6======link10------ ======
| | | |
---+-- | ------ | --+---
|CE1 | +-------| N6 |---------+ |CE2 |
------ link7 | | link8 ------
------
In the reference topology:¶
(SA,DA) (S3, S2, S1; SL)(payload) represents an IPv6 packet with:¶
SRH with SID list <S1, S2, S3> with SegmentsLeft = SL¶
Note the difference between the < > and () symbols: <S1, S2, S3> represents a SID list where S1 is the first SID and S3 is the last SID to traverse. (S3, S2, S1; SL) represents the same SID list but encoded in the SRH format where the rightmost SID in the SRH is the first SID and the leftmost SID in the SRH is the last SID. When referring to an SR policy in a high-level use case, it is simpler to use the <S1, S2, S3> notation. When referring to an illustration of the detailed packet behavior, the (S3, S2, S1; SL) notation is more convenient.¶
This section defines OAM enhancements for SRv6 networks.¶
[RFC8754] describes the Segment Routing Header (SRH) and how SR-capable nodes use it. The SRH contains an 8-bit Flags field.¶
This document defines the following bit in the SRH Flags field to carry the O-flag:¶
0 1 2 3 4 5 6 7
+-+-+-+-+-+-+-+-+
| |O| |
+-+-+-+-+-+-+-+-+
¶
Where:¶
The O-flag in the SRH is used as a marking bit in user packets to trigger telemetry data collection and export at the segment endpoints.¶
An SR domain ingress edge node encapsulates packets traversing the SR domain as defined in [RFC8754]. The SR domain ingress edge node MAY use the O-flag in the SRH for marking the packet to trigger the telemetry data collection and export at the segment endpoints. Based on local configuration, the SR domain ingress edge node may implement a classification and sampling mechanism to mark a packet with the O-flag in the SRH. Specification of the classification and sampling method is outside the scope of this document.¶
This document does not specify the data elements that need to be exported and the associated configurations. Similarly, this document does not define any formats for exporting the data elements. Nonetheless, without the loss of generality, this document assumes that the IP Flow Information Export (IPFIX) protocol [RFC7011] is used for exporting the traffic flow information from the network devices to a controller for monitoring and analytics. Similarly, without the loss of generality, this document assumes that requested information elements are configured by the management plane through data set templates (e.g., as in IPFIX [RFC7012]).¶
Implementation of the O-flag is OPTIONAL. If a node does not support the O-flag, then it simply ignores it upon reception. If a node supports the O-flag, it can optionally advertise its potential via control plane protocol(s).¶
When N receives a packet destined to S and S is a local SID, line S01 of the pseudocode associated with the SID S (as defined in Section 4.3.1.1 of [RFC8754]) is appended to as follows for O-flag processing.¶
S01.1. IF the O-flag is set and local configuration permits
O-flag processing {
a. Make a copy of the packet.
b. Send the copied packet, along with a timestamp,
to the OAM process for telemetry data collection
and export. ;; Ref1
}
Ref1: To provide an accurate timestamp, an implementation should
copy and record the timestamp as soon as possible during packet
processing. Timestamp and any other metadata are not carried in
the packet forwarded to the next hop.
¶
Please note that the O-flag processing happens before execution of regular processing of the local SID S. Specifically, line S01.1 of the pseudocode specified in this document is inserted between lines S01 and S02 of the pseudocode defined in Section 4.3.1.1 of [RFC8754].¶
Based on the requested information elements configured by the management plane through data set templates [RFC7012], the OAM process exports the requested information elements. The information elements include parts of the packet header and/or parts of the packet payload for flow identification. The OAM process uses information elements defined in IPFIX [RFC7011] and Packet Sampling (PSAMP) [RFC5476] for exporting the requested sections of the mirrored packets.¶
If the penultimate segment of a segment list is a PSP SID, telemetry data from the ultimate segment cannot be requested. This is because, when the penultimate segment is a PSP SID, the SRH is removed at the penultimate segment, and the O-flag is not processed at the ultimate segment.¶
The processing node MUST rate-limit the number of packets punted to the OAM process to a configurable rate. This is to avoid hitting any performance impact on the OAM and telemetry collection processes. Failure to implement the rate limit can lead to a denial-of-service attack, as detailed in Section 3.¶
The OAM process MUST NOT process the copy of the packet or respond to any upper-layer header (like ICMP, UDP, etc.) payload to prevent multiple evaluations of the datagram.¶
The OAM process is expected to be located on the routing node processing the packet. Although the specification of the OAM process or the external controller operations are beyond the scope of this document, the OAM process SHOULD NOT be topologically distant from the routing node, as this is likely to create significant security and congestion issues. How to correlate the data collected from different nodes at an external controller is also outside the scope of this document. Appendix A illustrates use of the O-flag for implementing a hybrid OAM mechanism, where the "hybrid" classification is based on [RFC7799].¶
IPv6 OAM operations can be performed for any SRv6 SID whose behavior allows Upper-Layer Header processing for an applicable OAM payload (e.g., ICMP, UDP).¶
Ping to an SRv6 SID is used to verify that the SID is reachable and is locally programmed at the target node. Traceroute to a SID is used for hop-by-hop fault localization as well as path tracing to a SID. Appendix A illustrates the ICMPv6-based ping and UDP-based traceroute mechanisms for ping and traceroute to an SRv6 SID. Although this document only illustrates ICMPv6-based ping and UDP-based traceroute to an SRv6 SID, the procedures are equally applicable to other IPv6 OAM probing to an SRv6 SID (e.g., Bidirectional Forwarding Detection (BFD) [RFC5880], Seamless BFD (S-BFD) [RFC7880], and Simple Two-way Active Measurement Protocol (STAMP) probe message processing [STAMP-SR]). Specifically, as long as local configuration allows the Upper-layer Header processing of the applicable OAM payload for SRv6 SIDs, the existing IPv6 OAM techniques can be used to target a probe to a (remote) SID.¶
IPv6 OAM operations can be performed with the target SID in the IPv6 destination address without an SRH or with an SRH where the target SID is the last segment. In general, OAM operations to a target SID may not exercise all of its processing depending on its behavior definition. For example, ping to an End.X SID [RFC8986] only validates the SID is locally programmed at the target node and does not validate switching to the correct outgoing interface. To exercise the behavior of a target SID, the OAM operation should construct the probe in a manner similar to a data packet that exercises the SID behavior, i.e. to include that SID as a transit SID in either an SRH or IPv6 DA of an outer IPv6 header or as appropriate based on the definition of the SID behavior.¶
[RFC8754] defines the notion of an SR domain and use of the SRH within the SR domain. The use of OAM procedures described in this document is restricted to an SR domain. For example, similar to SID manipulation, O-flag manipulation is not considered a threat within the SR domain. Procedures for securing an SR domain are defined in Sections 5.1 and 7 of [RFC8754].¶
As noted in Section 7.1 of [RFC8754], compromised nodes within the SR domain may mount attacks. The O-flag may be set by an attacking node attempting a denial-of-service attack on the OAM process at the segment endpoint node. An implementation correctly implementing the rate limiting described in Section 2.1.1 is not susceptible to that denial-of-service attack. Additionally, SRH flags are protected by the Hashed Message Authentication Code (HMAC) TLV, as described in Section 2.1.2.1 of [RFC8754]. Once an HMAC is generated for a segment list with the O-flag set, it can be used for an arbitrary amount of traffic using that segment list with the O-flag set.¶
The security properties of the channel used to send exported packets marked by the O-flag will depend on the specific OAM processes used. An on-path attacker able to observe this OAM channel could conduct traffic analysis, or potentially eavesdropping (depending on the OAM configuration), of this telemetry for the entire SR domain from such a vantage point.¶
This document does not impose any additional security challenges to be considered beyond the security threats described in [RFC4884], [RFC4443], [RFC0792], [RFC8754], and [RFC8986].¶
The per-packet marking capabilities of the O-flag provide a granular mechanism to collect telemetry. When this collection is deployed by an operator with the knowledge and consent of the users, it will enable a variety of diagnostics and monitoring to support the OAM and security operations use cases needed for resilient network operations. However, this collection mechanism will also provide an explicit protocol mechanism to operators for surveillance and pervasive monitoring use cases done contrary to the user's consent.¶
IANA has registered the following in the "Segment Routing Header Flags" subregistry in the "Internet Protocol Version 6 (IPv6) Parameters" registry:¶
| Bit | Description | Reference |
|---|---|---|
| 2 | O-flag | RFC 9259 |
This appendix shows how some of the existing IPv6 OAM mechanisms can be used in an SRv6 network. It also illustrates an OAM mechanism for performing controllable and predictable flow sampling from segment endpoints. How the centralized OAM technique in [RFC8403] can be extended for SRv6 is also described in this appendix.¶
The existing mechanism to perform the reachability checks, along the shortest path, continues to work without any modification. Any IPv6 node (SRv6-capable or non-SRv6-capable) can initiate, transit, and egress a ping packet.¶
The following subsections outline some additional use cases of ICMPv6 ping in SRv6 networks.¶
If an SRv6-capable ingress node wants to ping an IPv6 address via an arbitrary segment list <S1, S2, S3>, it needs to initiate an ICMPv6 ping with an SR header containing the SID list <S1, S2, S3>. This is illustrated using the topology in Figure 1. The user issues a ping from node N1 to a loopback of node N5 via segment list <2001:db8:K:2:X31::, 2001:db8:K:4:X52::>. The SID behavior used in the example is End.X, as described in [RFC8986], but the procedure is equally applicable to any other (transit) SID type.¶
Figure 2 contains sample output for a ping request initiated at node N1 to a loopback address of node N5 via segment list <2001:db8:K:2:X31::, 2001:db8:K:4:X52::>.¶
> ping 2001:db8:L:5:: via segment list 2001:db8:K:2:X31::,
2001:db8:K:4:X52::
Sending 5, 100-byte ICMPv6 Echos to B5::, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 0.625
/0.749/0.931 ms
All transit nodes process the echo request message like any other data packet carrying an SR header and hence do not require any change. Similarly, the egress node does not require any change to process the ICMPv6 echo request. For example, in the example in Figure 2:¶
The ping mechanism described above applies equally to perform SID reachability check and to validate the SID is locally programmed at the target node. This is explained in the following example. The example uses ping to an End SID, as described in [RFC8986], but the procedure is equally applicable to ping any other SID behaviors.¶
Consider the example where the user wants to ping a remote SID 2001:db8:K:4::, via 2001:db8:K:2:X31::, from node N1. The ICMPv6 echo request is processed at the individual nodes along the path as follows:¶
The existing traceroute mechanisms, along the shortest path, continue to work without any modification. Any IPv6 node (SRv6-capable or a non-SRv6-capable) can initiate, transit, and egress a traceroute probe.¶
The following subsections outline some additional use cases of traceroute in SRv6 networks.¶
If an SRv6-capable ingress node wants to traceroute to an IPv6 address via an arbitrary segment list <S1, S2, S3>, it needs to initiate a traceroute probe with an SR header containing the SID list <S1, S2, S3>. The user issues a traceroute from node N1 to a loopback of node N5 via segment list <2001:db8:K:2:X31::, 2001:db8:K:4:X52::>. The SID behavior used in the example is End.X, as described in [RFC8986], but the procedure is equally applicable to any other (transit) SID type. Figure 3 contains sample output for the traceroute request.¶
> traceroute 2001:db8:L:5:: via segment list 2001:db8:K:2:X31::,
2001:db8:K:4:X52::
Tracing the route to 2001:db8:L:5::
1 2001:db8:2:1:21:: 0.512 msec 0.425 msec 0.374 msec
DA: 2001:db8:K:2:X31::,
SRH:(2001:db8:L:5::, 2001:db8:K:4:X52::, 2001:db8:K:2:X31::, SL=2)
2 2001:db8:3:2:31:: 0.721 msec 0.810 msec 0.795 msec
DA: 2001:db8:K:4:X52::,
SRH:(2001:db8:L:5::, 2001:db8:K:4:X52::, 2001:db8:K:2:X31::, SL=1)
3 2001:db8:4:3::41:: 0.921 msec 0.816 msec 0.759 msec
DA: 2001:db8:K:4:X52::,
SRH:(2001:db8:L:5::, 2001:db8:K:4:X52::, 2001:db8:K:2:X31::, SL=1)
4 2001:db8:5:4::52:: 0.879 msec 0.916 msec 1.024 msec
DA: 2001:db8:L:5::
In the sample traceroute output, the information displayed at each hop is obtained using the contents of the "Time Exceeded" or "Destination Unreachable" ICMPv6 responses. These ICMPv6 responses are IP routed.¶
In the sample traceroute output, the information for link3 is returned by N3, which is a non-SRv6-capable node. Nonetheless, the ingress node is able to display SR header contents as the packet travels through the non-SRv6-capable node. This is because the "Time Exceeded" ICMPv6 message can contain as much of the invoking packet as possible without the ICMPv6 packet exceeding the minimum IPv6 MTU [RFC4443]. The SR header is included in these ICMPv6 messages initiated by the non-SRv6-capable transit nodes that are not running SRv6 software. Specifically, a node generating an ICMPv6 message containing a copy of the invoking packet does not need to understand the extension header(s) in the invoking packet.¶
The segment list information returned for the first hop is returned by N2, which is an SRv6-capable node. Just like for the second hop, the ingress node is able to display SR header contents for the first hop.¶
There is no difference in processing of the traceroute probe at an SRv6-capable and a non-SRv6-capable node. Similarly, both SRv6-capable and non-SRv6-capable nodes may use the address of the interface on which probe was received as the source address in the ICMPv6 response. ICMPv6 extensions defined in [RFC5837] can be used to display information about the IP interface through which the datagram would have been forwarded had it been forwardable, the IP next hop to which the datagram would have been forwarded, the IP interface upon which the datagram arrived, and the sub-IP component of an IP interface upon which the datagram arrived.¶
The IP address of the interface on which the traceroute probe was received is useful. This information can also be used to verify if SIDs 2001:db8:K:2:X31:: and 2001:db8:K:4:X52:: are executed correctly by N2 and N4, respectively. Specifically, the information displayed for the second hop contains the incoming interface address 2001:db8:2:3:31:: at N3. This matches the expected interface bound to End.X behavior 2001:db8:K:2:X31:: (link3). Similarly, the information displayed for the fourth hop contains the incoming interface address 2001:db8:4:5::52:: at N5. This matches the expected interface bound to the End.X behavior 2001:db8:K:4:X52:: (link10).¶
The mechanism to traceroute an IPv6 address via a segment list described in the previous section applies equally to traceroute a remote SID behavior, as explained in the following example. The example uses traceroute to an End SID, as described in [RFC8986], but the procedure is equally applicable to tracerouting any other SID behaviors.¶
Please note that traceroute to a SID is exemplified using UDP probes. However, the procedure is equally applicable to other implementations of traceroute mechanism. The UDP encoded message to traceroute a SID would use the UDP ports assigned by IANA for "traceroute use".¶
Consider the example where the user wants to traceroute a remote SID 2001:db8:K:4::, via 2001:db8:K:2:X31::, from node N1. The traceroute probe is processed at the individual nodes along the path as follows:¶
Figure 4 displays a sample traceroute output for this example.¶
> traceroute 2001:db8:K:4:X52:: via segment list 2001:db8:K:2:X31::
Tracing the route to SID 2001:db8:K:4:X52::
1 2001:db8:2:1:21:: 0.512 msec 0.425 msec 0.374 msec
DA: 2001:db8:K:2:X31::,
SRH:(2001:db8:K:4:X52::, 2001:db8:K:2:X31::; SL=1)
2 2001:db8:3:2:21:: 0.721 msec 0.810 msec 0.795 msec
DA: 2001:db8:K:4:X52::,
SRH:(2001:db8:K:4:X52::, 2001:db8:K:2:X31::; SL=0)
3 2001:db8:4:3:41:: 0.921 msec 0.816 msec 0.759 msec
DA: 2001:db8:K:4:X52::,
SRH:(2001:db8:K:4:X52::, 2001:db8:K:2:X31::; SL=0)
This section illustrates a hybrid OAM mechanism using the O-flag. Without loss of the generality, the illustration assumes N100 is a centralized controller.¶
This illustration is different from the "in situ OAM" defined in [RFC9197]. This is because in situ OAM records operational and telemetry information in the packet as the packet traverses a path between two points in the network [RFC9197]. The illustration in this subsection does not require the recording of OAM data in the packet.¶
The illustration does not assume any formats for exporting the data elements or the data elements that need to be exported. The illustration assumes system clocks among all nodes in the SR domain are synchronized.¶
Consider the example where the user wants to monitor sampled IPv4 VPN 999 traffic going from CE1 to CE2 via a low-latency SR policy P installed at node N1. To exercise a low-latency path, the SR Policy P forces the packet via segments 2001:db8:K:2:X31:: and 2001:db8:K:4:X52::. The VPN SID at N7 associated with VPN 999 is 2001:db8:K:7:DT999::. 2001:db8:K:7:DT999:: is a USP SID. N1, N4, and N7 are capable of processing the O-flag, but N2 is not capable of processing the O-flag. N100 is the centralized controller capable of processing and correlating the copy of the packets sent from nodes N1, N4, and N7. N100 is aware of O-flag processing capabilities. Controller N100, with help from nodes N1, N4, and N7, implements a hybrid OAM mechanism using the O-flag as follows:¶
In the recent past, network operators demonstrated interest in performing network OAM functions in a centralized manner. [RFC8403] describes such a centralized OAM mechanism. Specifically, [RFC8403] describes a procedure that can be used to perform path continuity checks between any nodes within an SR domain from a centralized monitoring system. However, while [RFC8403] focuses on SR networks with MPLS data plane, this document describes how the concept can be used to perform path monitoring in an SRv6 network from a centralized controller.¶
In the reference topology in Figure 1, N100 uses an IGP protocol like OSPF or IS-IS to get a view of the topology within the IGP domain. N100 can also use BGP-LS to get the complete view of an inter-domain topology. The controller leverages the visibility of the topology to monitor the paths between the various endpoints.¶
The controller N100 advertises an End SID [RFC8986] 2001:db8:K:100:1::. To monitor any arbitrary SRv6 paths, the controller can create a loopback probe that originates and terminates on node N100. To distinguish between a failure in the monitored path and loss of connectivity between the controller and the network, node N100 runs a suitable mechanism to monitor its connectivity to the monitored network.¶
The following example illustrates loopback probes in which controller N100 needs to verify a segment list <2001:db8:K:2:X31::, 2001:db8:K:4:X52::>:¶
The OAM payload type or the information carried in the OAM probe is a local implementation decision at the controller and is outside the scope of this document.¶
The authors would like to thank Joel M. Halpern, Greg Mirsky, Bob Hinden, Loa Andersson, Gaurav Naik, Ketan Talaulikar, and Haoyu Song for their review comments.¶
The following people contributed to this document:¶