]> Cisco Systems, Inc.

Canada rgandhi@cisco.com

Cisco Systems, Inc.

cfilsfil@cisco.com

Colt

Bart.Janssens@colt.net

Individual

mach.chen@outlook.com

Nokia

footer.foote@nokia.com

SPRING Working Group Segment Routing (SR) can be used to steer packets through a network employing source routing. SR can be applied to both MPLS (SR-MPLS) and IPv6 (SRv6) data planes. This document describes the procedures for performance measurement in SR-MPLS networks using the Simple Two-Way Active Measurement Protocol (STAMP), as specified in RFC 8762, along with its optional extensions specified in RFC 8972 and further augmented in RFC 9503. The procedures described in this document are used for SR-MPLS paths (including Segment Lists of SR-MPLS Policies, SR-MPLS IGP best paths, and SR-MPLS IGP Flexible Algorithm (Flex-Algo) paths), as well as Layer-3 and Layer-2 services carried over the SR-MPLS paths.

Introduction Segment Routing (SR) can be used to steer packets through a network employing source routing. SR can be applied to both MPLS (SR-MPLS) and IPv6 (SRv6) data planes. SR can take advantage of Equal-Cost Multipath (ECMP) between source and transit nodes, between transit nodes, and between transit and destination nodes. SR Policies, as defined in , are used to steer traffic through specific user-defined paths using a list of segments. A comprehensive SR performance measurement toolset is an essential requirement for measuring network performance and providing Service Level Agreements (SLAs). The Simple Two-Way Active Measurement Protocol (STAMP), as specified in , provides the capability to measure various performance metrics in IP networks without the use of a control channel to pre-signal session parameters. specifies optional extensions in the form of Type-Length-Value (TLV) objects for STAMP, and further augments that framework to define STAMP extensions for SR networks. This document describes the procedures for performance measurement in SR-MPLS networks, using STAMP as specified in , along with its optional extensions specified in and augmented in . The procedures described in this document are used for SR-MPLS paths (including Segment Lists of SR-MPLS Policies , SR-MPLS IGP best paths, and SR-MPLS IGP Flexible Algorithm (Flex-Algo) paths ), as well as Layer-3 (L3) and Layer-2 (L2) services carried over the SR-MPLS paths. STAMP requires protocol support on the Session-Reflector to process the received test packets. As a result, the received test packets need to be punted from the fast path in the data plane for control-plane processing, and return test packets need to be generated. This limits the frequency of STAMP test packets and the ability to provide shorter measurement intervals. This document defines new mechanisms to enhance the procedures for performance measurement using STAMP, improve scalability by supporting a larger number of STAMP sessions, and shorten the measurement interval for SR-MPLS paths by defining three new measurement modes: one-way, loopback, and loopback with Timestamp and Forward (TSF). The new measurement modes, loopback and loopback with TSF, take advantage of source routing. The procedure for performance measurement of MPLS LSPs using the measurement modes defined in this document is outside the scope of this document.

Conventions Used in This Document

Requirements Language The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in BCP 14 when, and only when, they appear in all capitals, as shown here.

Abbreviations Abbreviations

Abbreviation	Expansion	Reference
BoS	Bottom of Stack
ECMP	Equal-Cost Multipath
HMAC	Hashed Message Authentication Code
L2VPN	Layer-2 Virtual Private Network
L3VPN	Layer-3 Virtual Private Network
LSE	Label Stack Entry
MBZ	Must Be Zero
MNA	MPLS Network Action
MPLS	Multiprotocol Label Switching
NTP	Network Time Protocol
PHP	Penultimate Hop Popping
PSID	Path Segment Identifier
PTP	Precision Time Protocol
S bit	Bottom of Stack bit
SHA	Secure Hash Algorithms
SID	Segment Identifier
SR	Segment Routing
SR-MPLS	Segment Routing with MPLS data plane
SSID	STAMP Session Identifier
STAMP	Simple Two-Way Active Measurement Protocol
TC	Traffic Class
TLV	Type-Length-Value
TSF	Timestamp and Forward	This document
TTL	Time to Live
VPN	Virtual Private Network

Overview For performance measurement in SR-MPLS networks, the STAMP Session-Sender and Session-Reflector use the STAMP test packets specified in , along with optional extensions specified in . The STAMP test packets are encapsulated using an IP/UDP header, as specified in . In this document, the STAMP test packets using the IP/UDP header are used for SR-MPLS networks, where the STAMP test packets are further encapsulated with an MPLS header. STAMP test packets are transmitted in one of the following performance measurement modes in SR-MPLS networks where processing on Session-Reflector varies:

1. Two-Way measurement: Session-Reflector generates and transmits Session-Reflector test packets (see ).
2. One-Way measurement: Session-Reflector does not generate and transmit Session-Reflector test packets (see ).
3. Loopback measurement: Session-Reflector does not perform STAMP processing (see ).
4. Loopback measurement with TSF: Session-Reflector writes the receive timestamp in fast path but does not perform STAMP processing (see ).

Note that the two-way measurement mode is described as part of the STAMP process in and is further described for SR-MPLS networks in this document. The other measurement modes are new, specific to SR-MPLS networks, and are not defined in . STAMP test packets are transmitted on the same path as the data traffic flow under measurement to measure the delay and packet loss experienced by the data traffic flow, using the same MPLS encapsulation.

• STAMP test packets are transmitted on various transport data paths in the network to measure the delay and packet loss experienced by the traffic forwarded on those paths.
• STAMP test packets are transmitted over L3 and L2 services in the network to measure the delay and packet loss experienced by the traffic carried by those services.

Typically, STAMP Session-Reflector test packets are transmitted along an IP path between the Session-Reflector and Session-Sender. Matching the forward-direction path and return path for STAMP test packets, even for directly connected nodes, is not guaranteed. In SR-MPLS networks, the same path (i.e., the same set of links and nodes) between the Session-Sender and Session-Reflector may be desired for the STAMP test packets in both directions, for example, in an ECMP environment. This is achieved as follows:

• In two-way measurement mode: The optional STAMP extensions for SR-MPL networks, as specified in , are used. The STAMP Session-Reflector uses the return path parameters for the Session-Reflector test packet from the STAMP extensions in the received Session-Sender test packet, as specified in .
• In loopback and loopback with TSF measurement modes: Both the forward direction path and the return path are added in the MPLS encapsulation of the Session-Sender test packets using source routing.

The performance measurement procedures defined in this document are used to measure both delay and packet loss in SR-MPLS networks based on the transmission and reception of STAMP test packets. The optional STAMP extensions, as defined in , are used for direct measurement in SR-MPLS networks.

Measurement Modes In to , the nodes S1 and R1 may be connected via an SR-MPLS path . The SR-MPLS path may be a Segment List of an SR-MPLS Policy on node S1 (referred to as the "head-end") with node R1 as the destination (referred to as the "endpoint"), an SR-MPLS IGP best path, or an SR-MPLS IGP Flex-Algo path . Additionally, an L3 or L2 VPN service may be carried over the SR-MPLS path between nodes S1 and R1.

Two-Way Measurement Mode As shown in , in the reference topology for two-way measurement mode, the STAMP Session-Sender S1 initiates a Session-Sender test packet, and the STAMP Session-Reflector R1 generates and transmits a Session-Reflector test packet. The Session-Reflector test packets are transmitted to the Session-Sender S1 on the same path (i.e., the same set of links and nodes) or on a different path in the reverse direction from the path taken towards the Session-Reflector R1.

Reference Topology for Two-Way Measurement Mode | | | S1 |=====================| R1 | | |<- - - - - - - - - - | | +-------+ Reply Test Packet +-------+ \ / T4 T3 STAMP Session-Sender STAMP Session-Reflector ]]>

T1 is a transmit timestamp, and T4 is a receive timestamp added by node S1. T2 is a receive timestamp, and T3 is a transmit timestamp added by node R1. All four timestamps are used by the Session-Sender to measure the two-way delay metric as ((T4 - T1) - (T3 - T2)). Timestamps T1 and T2 are used by the Session-Sender to measure the one-way delay metric as (T2 - T1), also referred to as the near-end (forward direction) delay metric. Note that the delay value (T4 - T3), measured by the Session-Sender, is referred to as the far-end (backward direction) one-way delay metric. The "two-way delay" is the sum of the one-way delays in each direction and reflects the delay of the bidirectional path, irrespective of processing delays within the Session-Reflector. The computation of the one-way delay metric requires the clocks on the Session-Sender and Session-Reflector to be synchronized using either PTPv2 or NTPv4.

One-Way Measurement Mode As shown in , in the reference topology for one-way measurement mode, the STAMP Session-Sender S1 initiates a Session-Sender test packet. The STAMP Session-Reflector does not transmit Session-Reflector test packets upon receiving the Session-Sender test packets.

Reference Topology for One-Way Measurement Mode | | | S1 |=====================| R1 | | | | | +-------+ +-------+ STAMP Session-Sender STAMP Session-Reflector ]]>

T1 is a transmit timestamp added by node S1, and T2 is a receive timestamp added by node R1. Timestamps T1 and T2 are used by the Session-Reflector to measure the one-way delay metric as (T2 - T1). The computation of the one-way delay metric requires the clocks on the Session-Sender and Session-Reflector to be synchronized using either PTPv2 or NTPv4.

Loopback Measurement Mode As shown in , in the reference topology for loopback measurement mode, the STAMP Session-Sender S1 initiates a Session-Sender test packet to measure the loopback delay using source routing. At the STAMP Session-Reflector, the received STAMP test packets remain in the fast path in the data plane and are simply forwarded. In other words, the Session-Reflector does not perform STAMP functions or generate Session-Reflector test packets.

Reference Topology for Loopback Measurement Mode

The Session-Sender retrieves timestamp T1 from the received Session-Sender test packet and collects the receive timestamp T4 locally. The loopback delay is measured as (T4 - T1). This delay includes STAMP test packet processing on the Session-Reflector in data plane. The processing delay includes only the time required to forward the test packet from the incoming interface to the outgoing interface in the data plane. The Session-Reflector does not timestamp the test packets and therefore does not require timestamping capability. The loopback delay is the round-trip delay as specified in .

Loopback Measurement Mode with TSF As shown in , in the reference topology for "loopback measurement mode with TSF", the STAMP Session-Sender S1 initiates a Session-Sender test packet in loopback measurement mode using source routing. The TSF mechanism is used to optimize the operation of punting the test packet from the fast path in the data plane for control-plane processing and generating the return test packet on the STAMP Session-Reflector, as writing timestamp is implemented in the fast path in the data plane. This helps achieve a higher number of STAMP sessions and faster measurement intervals.

Reference Topology for Loopback Measurement Mode with TSF

The Session-Sender adds the transmit timestamp (T1) to the payload of the Session-Sender test packet. The Session-Reflector writes the receive timestamp (T2) in the received STAMP test packet in the fast path in the data plane, without punting the test packet from the fast path in the data plane for control-plane STAMP processing. The Session-Sender retrieves timestamps T1 and T2 from the received Session-Sender test packet and collects receive timestamp T4 locally. Timestamps T1 and T2 are used by the Session-Sender to measure the one-way delay metric as (T2 - T1). Timestamps T1 and T4 are used by the Session-Sender to measure the loopback delay metric as (T4 - T1).

STAMP Reference Model The STAMP Reference Model, along with some typical measurement parameters, as defined in for a STAMP session, is shown in .

STAMP Reference Model

The procedure defined in uses the two-way measurement mode. The base STAMP test packet payloads specified in are transported using an IP/UDP header and a destination UDP port , selected as specified in . The same destination port can be used for STAMP sessions for links, SR-MPLS paths, and L3 and L2 services carried over the SR-MPLS paths. The source UDP port is selected by the Session-Sender. The same or different source UDP ports may be used for different STAMP sessions. The Session-Sender and Session-Reflector IP addresses for a STAMP session are provisioned on both endpoints of the session. Session-Reflector mode can be either Stateful or Stateless, as specified in . Stateless Session-Reflector mode is applicable only in two-way measurement mode. The SSID in the STAMP test packets must be set to a non-zero value in both directions. The SSID in a STAMP test packet, along with the local configuration for the performance measurement mode, is used to identify STAMP sessions. When authentication mode is enabled for STAMP sessions, the matching Authentication Type (e.g., HMAC-SHA-256) and Keychain must be configured on both the Session-Sender and Session-Reflector . Examples of timestamp formats include 64-bit truncated Precision Time Protocol (PTPv2) and 64-bit Network Time Protocol (NTPv4) . By default, the Session-Reflector replies using the same timestamp format as received in the Session-Sender test packet, as indicated by the "Z" flag in the Error Estimate field, as specified in . This behavior depends on the Session-Reflector's capability. Examples of delay metrics are one-way delay, two-way delay, near-end delay (forward direction), and far-end delay (backward direction), as specified in . Examples of packet loss metric types are round-trip packet loss, near-end packet loss (forward direction), and far-end packet loss (backward direction), as specified in . The IPv4 TTL, MPLS TTL, and IPv6 Hop Limit fields follow the specification in . The Flow Label field in the IPv6 header of the Session-Sender test packets is set to the value used by the data packets for the IPv6 traffic flow being measured by the Session-Sender. The Session-Reflector sets the Flow Label in its test packet to the value received in the Session-Sender test packet, subject to local policy. A Software-Defined Networking (SDN) controller can be used for the configuration and management of STAMP sessions, as specified in . The controller can also receive streaming telemetry of operational data. The YANG data model for STAMP, defined in , can be used to configure Session-Senders and Session-Reflectors and to stream telemetry of operational data.

STAMP for One-Way Measurement Mode In one-way measurement mode, the Stateful mode of the Session-Reflector is used. The SSID field in the received Session-Sender test packets at the Session-Reflector, along with the local configuration, is used to identify the STAMP sessions that use one-way measurement mode on the Stateful Session-Reflector. A different destination UDP port can be selected for one-way measurement mode than the one used by the Session-Reflector for two-way measurement mode. When the same Session-Reflector UDP port is selected for one-way measurement mode, the Session-Sender requests, in the test packets, that the Session-Reflector not transmit Session-Reflector test packets. To achieve this, it uses the "No Reply Requested" flag in the Control Code Sub-TLV within the Return Path TLV defined in .

STAMP for Loopback Measurement Mode The Session-Reflector does not perform the STAMP process. Instead, its loopback function simply processes the IPv6/SRH header, ignoring the UDP header, to forward the test packet back to the Session-Sender without any STAMP modification. The Session-Sender sets the destination UDP port to the UDP port it uses to receive return Session-Reflector test packets (other than UDP port 862, which is used by the Session-Reflector). The same UDP port is used as both the destination and source UDP port in the Session-Sender test packets. At the Session-Sender, the "Session-Sender Sequence Number", the "Session-Sender Timestamp", the "Session-Sender Error Estimate", and the "Session-Sender TTL" fields are all set to zero in the transmitted Session-Sender test packets and are ignored in the received test packets.

STAMP for Loopback Measurement Mode with TSF The loopback measurement mode with TSF for the authenticated mode Session-Reflector test packet defined in Figure 4 of requires recomputation of the HMAC key after writing the timestamp in the packet on the Session-Reflector.

Measurement Mode Comparison for STAMP Measurement Mode Comparison for STAMP

Mode	Reflector	Timestamp	Clock Sync	Loss Metrics Applicable	Direct	Reference
Two-Way	Stateful or Stateless	T1/T2/T3/T4	N1, N2	One-Way, Round-trip	Yes
One-Way	Stateful	T1/T2	N1	One-Way	Yes	This document
Loopback	N/A	T1/T4	N3	Round-trip	No	This document
Loopback with TSF	N/A	T1/T2/T4	N1, N3	Round-trip	No	This document

N1: One-way delay metric computation requires clock synchronization. N2: Two-way delay metric computation does not require clock synchronization. N3: Loopback delay metric computation does not require clock synchronization.

Encapsulations for Two-Way Measurement Mode

Session-Sender Test Packet The content of a Session-Sender test packet is shown in . The Session-Sender test packet payload, as defined in , is transmitted with an IP header and a UDP header .

Content of Session-Sender Test Packet

Session-Sender Test Packet for SR-MPLS Data Plane

Session-Sender Test Packet for SR-MPLS Paths An SR-MPLS Policy Candidate-Path contains one or more Segment Lists (i.e., a stack of MPLS labels) . For delay measurement of an SR-MPLS Policy, the Session-Sender test packets are transmitted for every Segment List of the Candidate-Path of the SR-MPLS Policy, by creating a separate STAMP session for each Segment List. Each SR-MPLS Segment List contains a list of 32-bit Label Stack Entries (LSEs), where each LSE includes a 20-bit label value, an 8-bit Time to Live (TTL) field, a 3-bit Traffic Class (TC) field, and a 1-bit Bottom of Stack (BoS) field . The content of a Session-Sender test packet for an SR-MPLS path, using the SR-MPLS encapsulation of the data traffic transmitted over the path, is shown in .

Content of Session-Sender Test Packet for SR-MPLS Path

An IP header is added that contains the head-end node address of the SR-MPLS Policy as the Source Address. There are two cases for the SR-MPLS Policy endpoints, as described below.

1. The endpoint address of the SR-MPLS Policy is used as the Destination Address in the IP header when it is specified and is not the null endpoint. In the case of Penultimate Hop Popping (PHP), the MPLS header is removed by the penultimate node. In this case, the specified Destination Address in the IP header ensures that the test packets reach the Session-Reflector at the SR-MPLS Policy endpoint.
2. For an SR-MPLS Policy with Color-Only Destination Steering, where the endpoint is an unspecified address (the null endpoint is 0.0.0.0 for IPv4, as defined in ), a loopback address from the range 127/8 for IPv4 is used as the Destination Address in the IPv4 header. For IPv6 traffic, the IPv6 address of the Session-Sender is used as the Source Address, and an IPv6 address from the Dummy IPv6 Prefix 100:0:0:1::/64 block is used as the Destination Address in the IPv6 header. In this case, the Session-Sender ensures that the Session-Sender test packets using the Segment List reach the Session-Reflector at the SR-MPLS Policy endpoint (for example, by adding the Prefix SID label of the SR-MPLS Policy endpoint to the Segment List). In addition, the Session-Sender test packets may carry the "Destination Node IPv4 or IPv6 Address" STAMP TLV as defined in to identify the intended Session-Reflector address.

Each IGP Flex-Algo path in SR-MPLS networks has Prefix SID labels advertised by the nodes. For delay measurement of SR-MPLS IGP Flex-Algo paths, the Session-Sender test packets carry the Flex-Algo Prefix SID labels of the Session-Sender and Session-Reflector in the MPLS header for that IGP Flex-Algo path under measurement. Similarly, each IGP best path in SR-MPLS networks has Prefix SID labels advertised by the nodes. For delay measurement of SR-MPLS IGP best paths, the Session-Sender test packets carry the IGP Prefix SID labels of the Session-Sender and Session-Reflector in the MPLS header for that IGP best path under measurement.

Session-Sender Test Packet for Layer-3 Services over SR-MPLS Path For delay measurement of the L3 service carried over an SR-MPLS path, the SR-MPLS label stack of the data packets transmitted over the L3 service, including the L3 Virtual Private Network (L3VPN) label (advertised by the Session-Reflector), is used to encapsulate the Session-Sender test packets, as shown in .

Content of Session-Sender Test Packet for L3 Service over SR-MPLS Path

An IP header, as shown in , is added to the Session-Sender test packets after the MPLS header. The Destination Address in the IP header is reachable via the IP table lookup associated with the L3VPN label added for the L3 service on the Session-Reflector. The Source Address in the IP header of the Session-Sender test packets is reachable via the IP table lookup associated with the L3 service in the reverse direction.

Session-Sender Test Packet for Layer-2 Services over SR-MPLS Path For delay measurement of the L2 service carried over an SR-MPLS path, the SR-MPLS label stack of the data packets transmitted over the L2 service, including the L2 Virtual Private Network (L2VPN) label (advertised by the Session-Reflector), is used to encapsulate the Session-Sender test packets, as shown in .

Content of Session-Sender Test Packet for L2 Service over SR-MPLS Path

The L2VPN label is added with a TTL value of 1 to punt the Session-Sender STAMP test packet from the fast path in the data plane for control-plane processing on the Session-Reflector when using the Type 3 exception specified in . An IP header, as shown in , is added to the Session-Sender test packets after the MPLS header. This header contains the Session-Sender Address as the Source Address and the Session-Reflector Address as the Destination Address.

Session-Reflector Test Packet In two-way measurement mode, the Session-Reflector test packets are transmitted on the same SR-MPLS path (i.e., the same set of links and nodes) in the reverse direction to the Session-Sender to perform accurate two-way delay measurement. The Session-Reflector decapsulates the MPLS header, if present, from the received Session-Sender test packets. The Session-Reflector test packet is generated using the information from the received IP/UDP header of the Session-Sender test packet, as shown in .

Content of Session-Reflector Test Packet

The payload contains the Session-Reflector test packet defined in . For SR-MPLS paths, the Session-Sender uses the Segment List sub-TLV in the Return Path TLV defined in to request that the Session-Reflector transmit the Session-Reflector test packet on a specific SR-MPLS return path. Examples of specific SR-MPLS return paths include:

• The reverse SR-MPLS path associated with the forward direction SR-MPLS path.
• The Binding SID label of the reverse SR-MPLS Policy.
• The Prefix SID of the Session-Sender.

For SR-MPLS IGP Flex-Algo paths, the Session-Sender uses the Segment List sub-TLV in the Return Path TLV defined in to request that the Session-Reflector transmit the Session-Reflector test packet on the same SR-MPLS IGP Flex-Algo path in the reverse direction.

Encapsulations for One-Way Measurement Mode In one-way measurement mode, for links, SR-MPLS paths, and L3 and L2 services carried over the SR-MPLS paths, the Session-Sender test packets with encapsulations as defined in for STAMP sessions, are transmitted. As no Session-Reflector test packets are transmitted, the encapsulation for them defined in does not apply.

Encapsulations for Loopback Measurement Mode In loopback measuement mode for SR-MPLS paths, and L3 and L2 services carried over the SR-MPLS paths, the Session-Sender test packets with encapsulations as defined for STAMP sessions, are transmitted. An IP header is added for the return path in the Session-Sender test packets, setting the Destination Address equal to the Session-Sender address, as shown in , to return the test packets to the Session-Sender.

Content of Session-Sender Return Test Packet in Loopback Measurement Mode

Loopback Measurement Mode for SR-MPLS Paths In loopback measurement mode for SR-MPLS paths, the Session-Sender test packet carries either the Segment List of the forward direction path only or both the forward direction and return paths in the MPLS header, as specified in , as shown in .

Content of Session-Sender Test Packet in Loopback Measurement Mode for SR-MPLS Path

In the case of an SR-MPLS Policy using PHP, the Session-Sender ensures that the STAMP test packets reach the SR-MPLS Policy endpoint, for example, by adding the Prefix SID label of the SR-MPLS Policy endpoint to the Segment List of the forward direction path. The IP header for the return path is added to the Session-Sender test packets, and the Destination Address is set to the Session-Sender address in the IP header.

SR-MPLS Return Path The Session-Sender test packets, in the SR-MPLS label stack, carry the return path in addition to the forward direction path, as shown in Example 1 of . Examples of specific SR-MPLS return paths include:

• The SR-MPLS label stack of the Segment List of the associated reverse Candidate-Path.
• The Binding SID label of the reverse SR-MPLS Policy.
• The SR-MPLS Prefix SID label of the Session-Sender.

For SR-MPLS IGP Flex-Algo paths, the Session-Sender test packets carry the SR-MPLS Prefix SID label of the Session-Sender on the same SR-MPLS IGP Flex-Algo path in the reverse direction. The Binding SID label of the reverse SR-MPLS Policy can be configured on the Session-Sender using, for example, an SDN controller.

IP Return Path The Session-Sender test packets, in the MPLS header, carry only the SR-MPLS label stack of the forward direction path, as shown in Example 2 of . The Session-Reflector decapsulates the MPLS header and forwards the test packet using the IP header back to the Session-Sender.

Loopback Measurement Mode for Layer-3 Services over SR-MPLS Path In loopback measurement mode for the L3 service carried over an SR-MPLS path, the SR-MPLS label stack of the data packets transmitted over the L3 service is used to encapsulate the Session-Sender test packets, as shown in .

Content of Session-Sender Test Packet in Loopback Measurement Mode for L3 Service over SR-MPLS Path

The IP header for the return path of the Session-Sender test packets is added, setting the Destination Address to the Session-Sender address. The Destination Address added in the IP header for the return path must be reachable via the IP table lookup associated with the L3VPN label added to the test packets.

SR-MPLS Return Path The SR-MPLS label stack for the forward direction L3 service, excluding the L3VPN label advertised by the Session-Reflector, is added to the Session-Sender test packets. In addition, the SR-MPLS label stack for the reverse direction L3 service, including its L3VPN label advertised by the Session-Sender, is added to the Session-Sender test packets.

IP Return Path The SR-MPLS label stack, including the L3VPN label (advertised by the Session-Reflector) for the forward direction L3 service, is added to the Session-Sender test packets. The Session-Reflector decapsulates the MPLS header and forwards the Session-Sender test packet back to the Session-Sender using the IP header, after adding SR-MPLS encapsulation for the reverse direction L3 service.

Loopback Measurement Mode for Layer-2 Services over SR-MPLS Path In loopback measurement mode for the L2 service carried over an SR-MPLS path, the SR-MPLS label stack of the data packets transmitted over the L2 service is used to encapsulate the Session-Sender test packets, as shown in .

Content of Session-Sender Test Packet in Loopback Measurement Mode for L2 Service over SR-MPLS Path

The IP header for the return path is added to the Session-Sender test packets, and the Destination Address is set to the Session-Sender address.

SR-MPLS Return Path The SR-MPLS label stack for the forward direction L2 service, excluding the L2VPN label advertised by the Session-Reflector, is added to the Session-Sender test packets. In addition, the SR-MPLS label stack for the reverse direction L2 service, including its L2VPN label advertised by the Session-Sender, is added to the Session-Sender test packets with a TTL value of 1 to punt STAMP test packets from the fast path in the data plane for control-plane processing on the Session-Sender when using the Type 3 exception specified in .

IP Return Path The STAMP test packets that do not use the SR-MPLS return path are not supported.

Encapsulations for Loopback Measurement Mode with TSF The encapsulations for loopback measurement mode with TSF is defined for SR-MPLS paths and does not support L3 and L2 services carried over the SR-MPLS paths.

STAMP TSF Network Actions The MPLS Network Action (MNA) Sub-Stack is specified in . This document defines two MPLS Network Action opcodes for TSF:

• STAMP TSF with PTPv2 (opcode TBA1): A 64-bit PTPv2 timestamp written at a begin offset of 16 bytes from the start of the STAMP test packet payload.
• STAMP TSF with NTPv4 (opcode TBA2): A 64-bit NTPv4 timestamp written at a begin offset of 16 bytes from the start of the STAMP test packet payload.
• Format: The LSE Format B or the LSE Format C can carry either TSF opcode.
• Scope: The Ingress-to-Egress (I2E), Hop-by-Hop, and Select (IHS) field . Must be set to "Select" when the return path is SR-MPLS (see ) because the node that writes the timestamp pops the top label but does not remove the MPLS header. Must be set to "I2E" when the return path is IP/UDP (see ) because the node that writes the timestamp removes the MPLS header and forwards the packet using the IP header.
• Ancillary Data: Set to 0.
• Interactions: The TSF opcodes do not interact with other network action opcodes.
• The U bit, Network Action Sub-Stack Length (NASL), and Network Action Length (NAL) are set as defined in .

The timestamp is written in the "Receive Timestamp" field , located at a begin offset of 16 bytes from the start of the STAMP test packet payload, as shown in the Session-Reflector test packet in Figure 2 of . In the Session-Sender test packets for SR-MPLS paths in loopback measurement mode with TSF, the MNA Sub-Stack with applicable TSF opcode is added to the MPLS header, as shown in .

Content of Session-Sender Test Packet in Loopback Measurement Mode with TSF Network Action in Format-B LSE for SR-MPLS Paths

The SR-MPLS label stack of the return path can be added after the MNA Sub-Stack to receive the return test packet on a specific path, as described in the loopback measurement mode for SR-MPLS paths in this document. When a Session-Reflector receives a STAMP test packet with an MNA Sub-Stack containing opcode TBA1 or TBA2, it writes the timestamp in the STAMP test packet payload, pops the MNA Sub-Stack (after completing any other network actions), and forwards the test packet as defined in the loopback measurement mode for SR-MPLS paths.

TSF Network Action Node Capability The Session-Sender needs to know if the Session-Reflector is capable of processing applicable opcode TBA1 or TBA2, to avoid dropping the test packets. This capability can be locally configured on the Session-Sender or signaled. Signaling extensions for this capability exchange are outside the scope of this document.

Packet Loss Measurement in SR-MPLS Networks The procedure described for two-way measurement mode supports inferred measurements of round-trip, near-end (forward direction), and far-end (backward direction) packet loss. However, this provides only an approximate view of data packet loss. The loopback measurement mode and loopback measurement mode with TSF, defined in this document, allow only round-trip packet loss measurement. Note that the packet loss measurement does not require the clocks on the Session-Sender and Session-Reflector to be synchronized using either PTPv2 or NTPv4.

Direct Measurement in SR-MPLS Networks The STAMP "Direct Measurement" TLV (Type 5), defined in , is used for data packet loss measurement. To collect direct-measurement counters for data packet flows, STAMP test packets containing this TLV are transmitted using the two-way measurement-mode procedure. The procedure collects Session-Sender transmit counters and Session-Reflector receive and transmit counters. The receive data traffic can be measured as follows:

• The Path Segment Identifier (PSID) of an SR-MPLS Policy (for the Segment List or for the Candidate-Path) may be carried in the data packets to measure received data traffic (for the receive packet counter) on the associated SR-MPLS path when the egress node supports PSID processing.
• In the case of L3 and L2 services in SR-MPLS networks, the associated SR-MPLS service labels are used to measure received data traffic (for the receive packet counters) on the Session-Reflector.

In loopback measurement mode and loopback measurement mode with TSF, direct measurement is not applicable.

ECMP Measurement in SR-MPLS Networks The Segment List of an SR-MPLS path can have ECMP paths between the source and transit nodes, between transit nodes, and between transit and destination nodes, due to, for example:

• Use of a node SID .
• Use of an Anycast SID , which can result in ECMP paths via transit nodes that are part of that anycast group.

The STAMP test packets are transmitted to traverse different ECMP paths to measure the delay of each ECMP path of a Segment List, and can use the following mechanisms:

• Different entropy label values are used in the Session-Sender and Session-Reflector test packets to take advantage of the hashing function in the forwarding plane and influence the ECMP path taken by the packets.
• Different Destination Address values from the IPv4 range 127/8 are used in the Session-Sender and Session-Reflector test packets to traverse different IPv4 ECMP paths, as described in . In this case, the Session-Sender test packets may carry the "Destination Node IPv4 or IPv6 Address" STAMP TLV, as defined in , to identify the intended Session-Reflector IP address.

The considerations for loss measurement for different ECMP paths of an SR-MPLS path are outside the scope of this document.

Implementation Status Editorial note: Please remove this section prior to publication.

Cisco Implementation The following Cisco routing platforms running IOS-XR operating system have participated in interoperability testing for one-way, two-way, and loopback measurement modes for SR-MPLS: * Cisco 8000 (based on Cisco Silicon One ASIC) * Cisco ASR9904 with Lightspeed linecard and Tomahawk linecard * Cisco NCS5500 (based on Broadcom Jericho1 ASIC) * Cisco NCS5700 (based on Broadcom Jericho2 ASIC)

Operational and Manageability Considerations The operational considerations specified in also apply to the procedures specified in this document. Further, the operation and management considerations for performance measurement based on STAMP specified in also apply to the procedures specified in this document. The manageability considerations described in apply to this specification. When a destination UDP port number other than the default port 862 is used, the same network-impact study and agreement requirements specified in apply. The operational considerations specified in are also applicable to the procedures described in this document. Various statistics for one-way (near-end, far-end), two-way, and loopback delay metrics (such as average delay, minimum delay, maximum delay, and delay variance) as well as for one-way (near-end, far-end) or round-trip packet loss metrics (such as percentage loss and consecutive packets lost) and the STAMP session state changes can be computed using the performance measurement procedures described in this document. Operator alerts are generated for anomaly detection when delay or loss metrics cross user-configured thresholds or when the STAMP session state changes. When STAMP sessions are created for the Segment Lists of the SR-MPLS Policies, the scalability regarding the number of STAMP sessions needs to be carefully considered. The operational considerations specified in apply when selecting a routable or non-routable IP address as a destination address.

STAMP Session State Notification The threshold-based notification for delay and packet loss metrics is generated only when the metrics change significantly. For unambiguous monitoring, the controller needs to distinguish whether the STAMP session is active but delay and packet loss metrics did not cross the thresholds, or whether the STAMP session has failed and is not transmitting or receiving test packets. The STAMP session state monitoring allows the node to determine whether the performance measurement test is active, idle, or failed. The failed state of the STAMP session also indicates the connectivity failure of the SR-MPLS path or of the L3/L2 service carried over the SR-MPLS path, where the STAMP session was active. In all measurement modes, the STAMP session state is notified as idle when the Session-Sender is not transmitting test packets. In two-way measurement mode, loopback measurement mode, and loopback measurement mode with TSF, STAMP session state is notified on Session-Sender as follows:

• The STAMP session state is initially notified as active on the Session-Sender when the Session-Sender is transmitting test packets and at least one Session-Reflector test packet has been received.
• The STAMP session state is notified as failed when N consecutive Session-Reflector test packets are not received at the Session-Sender after the STAMP session state is notified as active, where N (the consecutive packet loss count) is a locally provisioned value.

Similarly, in one-way measurement mode, STAMP session state is notified on Session-Reflector as follows:

• The STAMP session state is initially notified as active on the Session-Reflector once one or more Session-Sender test packets are received.
• The STAMP session state is notified as failed when N consecutive Session-Sender test packets are not received at the Session-Reflector after the STAMP session state is notified as active, where N (the consecutive packet loss count) is a locally provisioned value.

Operational Considerations for TSF The TSF network action processing depends on the applicable TSF opcode and the corresponding timestamp format capability. Operators should verify Session-Reflector support for the applicable TSF opcode before enabling STAMP sessions with the TSF network action. Implementations should maintain per-network-action counters for the following TSF Network Action events:

• Packets with TSF Network Action received.
• Packets with TSF Network Action invocations.
• Packets with TSF Network Action dropped because the action was unknown.
• Packets with TSF Network Action forwarded when the action was unknown.
• Packets with TSF Network Action dropped because of malformed packet.
• Packets with TSF Network Action timestamp write failures.

Successful and failed TSF network action invocations should be distinguishable. Notifications for sustained failures, malformed packets, or excessive packets with the TSF network action should be rate-limited.

Security Considerations The security considerations specified in , , and also apply to the procedures described in this document. The measures specified in to mitigate attacks also apply. The security considerations specified in and are also applicable to the procedures described in this document. The use of HMAC-SHA-256 in authenticated mode protects the data integrity of the STAMP test packets. The message integrity protection using HMAC, as specified in , can be used with the procedures described in this document. The source UDP port number should be selected using a randomized allocation method as specified in to provide protection against off-path attacks, as recommended in . Furthermore, SSIDs should not be assigned predictably. To avoid predictability, implementations can use a cryptographically secure pseudorandom number generator . The procedures defined in this document are intended for deployment in a single network administrative domain. As such, the Session-Sender address, Session-Reflector address, and the forward direction and return paths are provisioned by the operator for the STAMP session. It is assumed that the operator has verified the integrity of the forward direction and return paths of the STAMP test packets. Implementations can mitigate attacks by performing basic validation checks on Session-Reflector test packets received at the Session-Sender, such as verifying that timestamp T2 is later than timestamp T1 (when the Session-Sender and Session-Reflector clocks are synchronized) in the STAMP Reference Topology shown in . The minimal state associated with this protocol also limits the extent of measurement disruption that can be caused by a corrupt or invalid test packet to a single test cycle. STAMP test packets received through a transport path or a service context must be processed only in that context. This document does not provide a mechanism for cross-service OAM interactions.

Security Considerations for TSF The TSF network action uses the IANA-assigned opcodes TBA1 and TBA2 with timestamp formats and begin offsets. Their processing therefore needs to be restricted to trusted nodes and trusted STAMP sessions. An attacker that can inject packets with the TSF network action could cause unauthorized data-plane timestamping or influence measured paths. Network operators need to filter MPLS packets carrying the TSF Network Action at administrative-domain boundaries and are expected to restrict the action to Session-Reflector nodes that support the applicable TSF opcode. The Session-Reflector writes the timestamp specified by opcode TBA1 or TBA2 in the STAMP test packet payload. Implementations need to validate the MNA Sub-Stack, opcode, timestamp format, and available payload length before writing the timestamp. Bounds checking is required to prevent malformed packets from causing memory corruption, packet corruption, or denial-of-service conditions. Any malformed packet with the TSF network action needs to be dropped.

IANA Considerations IANA is requested to assign code points from the Network Action Opcodes registry created in as specified in . Network Action Opcodes

Opcode	Description	In-Stack Only, Post-Stack Only, In-Stack and Post-Stack	Reference
TBA1	STAMP TSF with PTPv2	In-Stack Only	This document
TBA2	STAMP TSF with NTPv4	In-Stack Only	This document

References Normative References &RFC768; &RFC2119; &RFC4026; &RFC6335; &RFC8174; &RFC8762; &RFC8972; &RFC9503; &RFC9994; &I-D.ietf-mpls-stamp-pw; Informative References &RFC3031; &RFC3032; &RFC2681; &RFC5462; &RFC5905; &RFC6056; &RFC6234; &RFC6790; &RFC8029; &RFC8085; &RFC8402; &RFC8403; &RFC9256; &RFC9350; &RFC9545; &RFC9780; &I-D.ietf-ippm-stamp-yang; IEEE National Institute of Standards and Technology IANA

Acknowledgments The authors would like to thank Ianik Semco and Thierry Couture for their discussions on the use cases for Performance Measurement in Segment Routing. The authors would also like to thank Greg Mirsky, Gyan Mishra, Xie Jingrong, Zafar Ali, Boris Hassanov, Ruediger Geib, Liyan Gong, Zhenqiang Li, Maria Matejka, William Hawkins, Mike Koldychev, and Bruno Decraene for reviewing this document and providing useful comments and suggestions. Additionally, Patrick Khordoc, Haowei Shi, Amila Tharaperiya Gamage, Pengyan Zhang, Ruby Lin, Senni Tan, and Radu Valceanu have helped improve the mechanisms described in this document. The authors would also like to thank Haoyu Song for the Shepherd's review and Alvaro Retana for the WG chair's review, which helped improve this document.

Contributors The following people have substantially contributed to this document: