Network Working Group X. Xu Internet-Draft China Mobile Intended status: Standards Track 22 November 2023 Expires: 25 May 2024 Flooding Reduction in CLOS Networks draft-xu-lsr-flooding-reduction-in-clos-01 Abstract In a CLOS topology, an OSPF (or ISIS) router may receive identical copies of an LSA (or LSP) from multiple OSPF (or ISIS) neighbors. Moreover, two OSPF (or ISIS) neighbors may exchange the same LSA (or LSP) simultaneously. This results in unnecessary flooding of link- state information, which wastes the precious resources of OSPF (or ISIS) routers. Therefore, this document proposes extensions to OSPF (or ISIS) to reduce this flooding within CLOS networks. The reduction of OSPF (or ISIS) flooding is highly beneficial for improving the scalability of CLOS networks. Requirements Language The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC 2119 [RFC2119]. Status of This Memo This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF). Note that other groups may also distribute working documents as Internet-Drafts. The list of current Internet- Drafts is at https://datatracker.ietf.org/drafts/current/. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." This Internet-Draft will expire on 25 May 2024. Copyright Notice Copyright (c) 2023 IETF Trust and the persons identified as the document authors. All rights reserved. Xu Expires 25 May 2024 [Page 1] Internet-Draft Flooding Reduction in CLOS Networks November 2023 This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (https://trustee.ietf.org/ license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Revised BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Revised BSD License. Table of Contents 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 2 2. Solution Description . . . . . . . . . . . . . . . . . . . . 3 3. Terminology . . . . . . . . . . . . . . . . . . . . . . . . . 5 4. Modifications to OSPF and ISIS Behavior . . . . . . . . . . . 5 5. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 5 6. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 5 7. Security Considerations . . . . . . . . . . . . . . . . . . . 5 8. References . . . . . . . . . . . . . . . . . . . . . . . . . 5 8.1. Normative References . . . . . . . . . . . . . . . . . . 5 8.2. Informative References . . . . . . . . . . . . . . . . . 5 Author's Address . . . . . . . . . . . . . . . . . . . . . . . . 6 1. Introduction In a CLOS topology, OSPF or ISIS routers might receive multiple identical copies of an LSA or LSP from different neighbors, two neighbors may even exchange the same LSA or LSP simultaneously. The unnecessary flooding of link-state information waste valuable resources of OSPF or ISIS routers. To tackle this issue, this document proposes some extensions to OSPF or ISIS so as to reduce the unnecessary flooding within CLOS networks and therefore improve the scalability of CLOS networks. Due to the flooding issue as mentioned above, some data center network operators have opted for BGP [RFC7938] as the underlay routing protocol for their CLOS networks. However, with the advent of high-performance Ethernet networks for AI and HPC, it has become necessary to have visibility of the whole network topology, including link capacity and load information, to enable end-to-end path load-balancing, also known as global load- balancing or adaptive routing. This implies that link-state routing protocols like OSPF or ISIS may need to be reviewed as the routing protocol for large-scale AI and HPC Ethernet networks, provided the scaling issue associated with link-state routing protocols is well addressed. Xu Expires 25 May 2024 [Page 2] Internet-Draft Flooding Reduction in CLOS Networks November 2023 To address the scaling issue, this document proposes a pragmatic approach. The idea is to configure partial routers as non-reflectors for a given OSPF area or a given ISIS level. These routers cannot reflect the LSAs or LSPs they receive from neighbors in that OSPF area or ISIS level to their other neighbors in the same OSPF area or ISIS level. 2. Solution Description +----+ +----+ +----+ +----+ | S1 | | S2 | | S3 | | S4 | (Spine) +----+ +----+ +----+ +----+ +----+ +----+ +----+ +----+ +----+ +----+ +----+ +----+ | L1 | | L2 | | L3 | | L4 | | L5 | | L6 | | L7 | | L8 | (Leaf) +----+ +----+ +----+ +----+ +----+ +----+ +----+ +----+ Figure 1 (Note that the diagram above does not include the connections between nodes. However, it can be assumed that leaf nodes are connected to every spine node in their CLOS topology.) In a three-stage CLOS network as shown in Figure 1, also known as a leaf-spine network, all nodes should be in OSPF area zero or ISIS Level-2. All leaf nodes SHOULD be configured as non-reflectors. To further reduce the flooding of link-state, all spine nodes, except for two, COULD also be configured as non-reflectors. Xu Expires 25 May 2024 [Page 3] Internet-Draft Flooding Reduction in CLOS Networks November 2023 ========================================= # +----+ +----+ +----+ +----+ # # | L1 | | L2 | | L3 | | L4 | (Leaf) # # +----+ +----+ +----+ +----+ # # PoD-1 # # +----+ +----+ +----+ +----+ # # | S1 | | S2 | | S3 | | S4 | (Spine) # # +----+ +----+ +----+ +----+ # ========================================= =============================== =============================== # +----+ +----+ +----+ +----+ # # +----+ +----+ +----+ +----+ # # |SS1 | |SS2 | |SS3 | |SS4 | # # |SS1 | |SS2 | |SS3 | |SS4 | # # +----+ +----+ +----+ +----+ # # +----+ +----+ +----+ +----+ # # (Super-Spine@Plane-1) # # (Super-Spine@Plane-4) # #============================== ... =============================== ========================================= # +----+ +----+ +----+ +----+ # # | S1 | | S2 | | S3 | | S4 | (Spine) # # +----+ +----+ +----+ +----+ # # PoD-8 # # +----+ +----+ +----+ +----+ # # | L1 | | L2 | | L3 | | L4 | (Leaf) # # +----+ +----+ +----+ +----+ # ========================================= Figure 2 (Note that the diagram above does not include the connections between nodes. However, it can be assumed that the leaf nodes in a given PoD are connected to every spine node in that PoD. Similarly, each spine node (e.g., S1) is connected to all super-spine nodes in the corresponding PoD-interconnect plane (e.g., Plane-1).) For a five-stage CLOS network as illustrated in Figure 2, each Pod consisting of leaf and spine nodes is configured as an OSPF non-zero area or an ISIS Level-1 area. The Pod-interconnect plane consisting of spine and super-spine nodes is configured as an OSPF area zero or an ISIS Level-2 area. Therefore, spine nodes play the role of OSPF area border routers or ISIS Level-1-2 routers. All leaf nodes SHOULD be configured as non-reflectors, and all spine nodes SHOULD be configured as non-reflectors for OSPF area zero or ISIS Level-2. To reduce the link-state flooding further, all spine nodes in each Pod, except for at least two of them, COULD be configured as non- reflectors for the associated non-zero OSPF area or ISIS Level-1 area. Similarly, all super-spine nodes in each Pod-interconnect plane, except for two of them, COULD be configured as non-reflectors. Xu Expires 25 May 2024 [Page 4] Internet-Draft Flooding Reduction in CLOS Networks November 2023 3. Terminology This memo makes use of the terms defined in [RFC2328] and [RFC1195]. 4. Modifications to OSPF and ISIS Behavior Routers that are configured as non-reflectors for a given OSPF area or ISIS level SHOULD NOT reflect the LSAs or LSPs they receive from neighbors in that OSPF area or ISIS level to their other neighbors in the same OSPF area or ISIS level. 5. Acknowledgements TBD. 6. IANA Considerations TBD. 7. Security Considerations TBD. 8. References 8.1. Normative References [RFC1195] Callon, R., "Use of OSI IS-IS for routing in TCP/IP and dual environments", RFC 1195, DOI 10.17487/RFC1195, December 1990, . [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, DOI 10.17487/RFC2119, March 1997, . [RFC2328] Moy, J., "OSPF Version 2", STD 54, RFC 2328, DOI 10.17487/RFC2328, April 1998, . [RFC5340] Coltun, R., Ferguson, D., Moy, J., and A. Lindem, "OSPF for IPv6", RFC 5340, DOI 10.17487/RFC5340, July 2008, . 8.2. Informative References Xu Expires 25 May 2024 [Page 5] Internet-Draft Flooding Reduction in CLOS Networks November 2023 [RFC7938] Lapukhov, P., Premji, A., and J. Mitchell, Ed., "Use of BGP for Routing in Large-Scale Data Centers", RFC 7938, DOI 10.17487/RFC7938, August 2016, . Author's Address Xiaohu Xu China Mobile Email: xuxiaohu_ietf@hotmail.com Xu Expires 25 May 2024 [Page 6]