VRF HLD version 1.1 (#408)

* Updating VRF HLD Version 1.0 * VRF HLD v1.1 * Addressing comments from Nephos: Removed test plan since its tracked as separate document * Addressing comments from Nephos: Removed test plan since its tracked as separate document Addressing comments * Addressing comments from Nephos: Removed test plan since its tracked as separate document Addressing comments * Addressing few review comments from VRF HLD review * Rename VRF HLD file to not have version numbre in file name
sonic-net · Jul 17, 2019 · 56a4b71 · 56a4b71
1 parent eb59050
commit 56a4b71
Show file tree

Hide file tree

Showing 2 changed files with 116 additions and 46 deletions.
diff --git a/doc/sonic-vrf-hld-v1.0.md → doc/sonic-vrf-hld.md b/doc/sonic-vrf-hld-v1.0.md → doc/sonic-vrf-hld.md
@@ -7,6 +7,10 @@ Table of Contents
   - [Document History](#document-history)
   - [Abbreviations](#abbreviations)
   - [VRF feature Requirement](#vrf-feature-requirement)
+  - [Functionality](#functionality)
+    - [Target Deployment Use Case](#target-deployment-use-case)
+    - [Functionality Description](#functionality-description)
+    - [VRF Route Leak Support](#vrf-route-leak-support)
   - [Dependencies](#dependencies)
   - [SONiC system diagram for VRF](#sonic-system-diagram-for-vrf)
   - [The schema changes](#the-schema-changes)
@@ -34,7 +38,6 @@ Table of Contents
     - [Add VRF and bind/unbind interfaces to this VRF](#add-vrf-and-bindunbind-interfaces-to-this-vrf)
     - [Delete vrf](#delete-vrf)
   - [Impact to other service after import VRF feature](#impact-to-other-service-after-import-vrf-feature)
-  - [Progress](#progress)
   - [Test plan](#test-plan)
   - [Appendix - An alternative proposal](#appendix---an-alternative-proposal)
     - [vrf as key](#vrf-as-key)
@@ -54,6 +57,7 @@ Table of Contents
 | v.05    | 04/17/2019 | Xin Liu, Prince Sunny (MSFT)  | Update the status                                |
 | v.06    | 05/09/2019 | Shine Chen, Jeffrey Zeng, Tyler Li | Add Some description and format adjustment |
 | v1.0    | 05/26/2019 | Shine Chen, Jeffrey Zeng, Tyler Li, Ryan Guo | After review, move proposal-2 in v0.6 to Appendix
+| v1.1    | 06/04/2019 | Preetham Singh, Nikhil Kelapure, Utpal Kant Pintoo | Update on VRF Leak feature support |
 
 ## Abbreviations
 
@@ -81,10 +85,57 @@ The fallback feature which defined by RFC4364 is very useful for specified VRF u
 
 7. VRF Scalability: Currently VRF number can be supported up to 1000 after fixing a bug in FRR.
 
-In this release, supporting requirement 5) and 6) are not supported. See next section for details.
+In this release, requirement 5) is not supported. See next section for details.
 
 Note: linux kernel use VRF master device to support VRF and it supports admin up/down on VRF master device. But we don't plan to support VRF level up/down state on SONIC.
 
+## Functionality
+### Target Deployment Use Cases
+Virtual Routing and Forwarding is used to achieve network virtualization and traffic separation over on a shared network infrastructure in Enterprise and DC delpoyments.
+
+![Deployment use case](https://github.com/preetham-singh/SONiC-1/blob/master/images/vrf_hld/Multi-VRF_Deployment.png "Figure 1: Multi VRF Deployment use case")
+__Figure 1: Multi VRF Deployment use case__
+
+In above customer deployment:
+Customer A and Customer B in Site 1 or Site 2 are customer facing devices referred as Customer Edge routers.
+Router-1 and Router-2 are routers which provide interconnectivity between customers across sites referred as Provider Edge Routers.
+
+Above figure depicts typical customer deployment where multiple Customer facing devices are connected to Provider edge routers.
+With such deployment Provider Edge routers associate each input interface with one VRF instance.
+
+In Figure 1, All cutomer facing devices belonging to Customer-A irrespective of the site, will belong to VRF Green and those in Customer-B will belong to VRF Red.
+With this deployment, isolation of traffic is achieved across customers maintaining connectivity within same customer sites.
+
+Note that multiple input interfaces can be associated to a VRF instance. This input interface can be Physical interface, Port-channel or L3 Vlan interface.
+
+### Functional Description
+
+Multi-VRF is the ability to maintain multiple "Virtual Routing and Forwarding"(VRF) tables on the same Provider Edge router.
+Multi-VRF aware routing protocol such as BGP is used to exchange routing information among peer Provider Edge routers.
+The Multi-VRF capable Provider Edge router maps an input customer interface to a unique VRF instance. Provider Edge router maintains unique VRF table for each VRF instance on that Provider Edge router.
+
+Multi-VRF routers communicate with one another by exchanging route information in the VRF table with the neighboring Provider Edge router.
+This exchange of information among the Provider Edge routers is done using routing protocol like BGP.
+Customers connect to Provider Edge routers in the network using Customer Edge routers as shown in Figure 1. 
+
+Due to this overlapping address spaces can be maintained among the different VRF instances.
+
+FRR's BGP implementation is capable of running multiple autonomous systems (AS) at once. Each configured AS is associated with a VRF instance. The  SONiC VRF implementation will enable this capability of FRR BGP in SONiC.
+
+#### VRF route leak support:
+
+VRF route leak is a case where route and nexthops are in different VRF.
+
+VRF route leak can be achieved via both Static or Dynamic approach.
+
+In Static VRF route leak, FRR CLI can be used where user can specify nexthop IP along with nexthop VRF, where the nexthop IP is reachable through a nexthop VRF.
+
+In Dynamic VRF route leak, Route maps can be used to import routes from other VRFs.
+Prefix lists within route maps are used to match route prefixes in source VRF and various action can be applied on these matching prefixes.
+If route map action is permit, these routes will be installed into destination VRF.
+
+Leaked routes will be automatically deleted when corresponding routes are deleted in source VRF.
+
 ## Dependencies
 
 VRF feature needs the following software package/upgrade
@@ -99,9 +150,9 @@ the master net device.
 Application can get creation or deletion event of VRF master device via RTNETLINK,
 as well as information about slave net device joining a VRF.
 
-Linux kernel supports VRF forwarding using PBR scheme. It will fall to main
-routing table to check do IP lookup. VRF also can have its own default network
-instruction in case VRF lookup fails.
+Linux kernel supports VRF forwarding using PBR scheme. All route lookups will be performed on
+main routing table associated with the VRF. VRF also can have its own default network
+instruction in case route lookup fails.
 
 2. FRRouting is needed to support BGP VRF aware routing.
 
@@ -130,20 +181,8 @@ ip [-6] rule add pref 32765 table local && ip [-6] rule del pref 0
 
 SAI right now does not seem to have VRF concept, it does have VR.
 
-We propose to implement VR as "virtual router" and VRF as "virtual router
-forwarding"
-
-VR is defined as a logical routing system. VRF is defined as forwarding
-domain within a VR.
-
-As this stage, we assume one VR per system. Only implement VRFs within this VR.
-
-Accordingly, we need to add vrf_id to sai_Route_entry and add vrf attribute
-to sai_routeInterface object.
+Hence in this implementation release we use VR object as VRF object.
 
-An alternative method is using VR as VRF, this requires to add two attribution
-to VR object to support Requirement 5) (fallback lookup). SAI community has
-decided to take VR as VRF. So in this implementation release we use VR object as VRF object.
 Here are the new flags we propose to add in the SAI interface:
 
 ```jason
@@ -238,7 +277,7 @@ Add vrf-binding information in config_db.json file.
 
 With this approach, there is no redundant vrf info configured with an interface where multiple IP addresses are configured.
 
-Logically IP address configuration must be processed after interface binding to vrf is processed. In intfmgrd/intfOrch process intf-bind-vrf event must be handled before IP address event. So interface-name entry in config_db.json is necessary even user doesn't use VRF feature. e.g. `"Ethernet2":{}` in the above example configuration. For version upgrade compatibility we need to add a script, this script will convert old config_db.json to new config_db.json at bootup, then the new config_db.json would contain the interface-name entry for interfaces associated in the global VRF table.
+Logically IP address configuration must be processed after interface binding to vrf is processed. In intfmgrd/intfOrch process intf-bind-vrf event must be handled before IP address event. So interface-name entry in config_db.json is necessary even though user doesn't use VRF feature. e.g. `"Ethernet2":{}` in the above example configuration. For version upgrade compatibility we need to add a script, this script will convert old config_db.json to new config_db.json at bootup automatically, then the new config_db.json would contain the interface-name entry for interfaces associated in the global VRF table.
 
 ### Change redirect syntax in acl_rule_table of configdb
 
@@ -451,20 +490,21 @@ update kernel using iproute2 CLIs and write VRF information to app-VRF-table.
 
 ### intfsmgrd changes
 
-Ip address event and vrf binding event need to be handled seperately. These two events has sequency dependency.
+IP address event and VRF binding event need to be handled seperately. These two events has sequence dependency.
 
 - Listening to interface binding to specific VRF configuration in config_db.
-  - bind to vrf event:
-    - bind kernel device to master vrf
-    - add interface entry with vrf attribute to app-intf-table.
-    - set intf-bind-vrf flag on statedb
-  - unbind from vrf event:
+  - bind to VRF event:
+    - bind kernel device to master VRF
+    - add interface entry with VRF attribute to app-intf-table.
+    - set intf-bind-vrf flag on STATE_DB
+  - unbind from VRF event:
     - wait until all ip addresses associated with the interface is removed. Ip address infomation can be retrieved from kernel.
-    - bind kernel device to global vrf
-    - del interface entry with vrf attribute from app-intf-table
-    - unset vrf-binding flag on statedb
+    - bind kernel device to global VRF
+    - del interface entry with VRF attribute from app-intf-table
+    - unset vrf-binding flag on STATE_DB
 - Listening to interface ip address configuration in config_db.
-  - add ip address event: wait until intf-bind-vrf flag is set, set ip address on kernel device and add {interface_name:ip address} entry to app-intf-prefix-table
+  - add ip address event: 
+    - wait until intf-bind-vrf flag is set, set ip address on kernel device and add {interface_name:ip address} entry to app-intf-prefix-table
   - del ip address event:
     - unset ip address on kernel device
     - del {interface_name:ip address} entry from app-intf-prefix-table.
@@ -545,22 +585,19 @@ $ config vrf add <vrf_name>
 $ config vrf del <vrf_name>
 
 //bind an interface to a VRF
-$ config interface <interface_name> vrf bind <vrf_name>
+$ config interface vrf bind <interface_name> <vrf_name>
 
 //unbind an interface from a VRF
-$ config interface <interface_name> vrf unbind
+$ config interface vrf unbind <interface_name> <vrf_name>
 
 // show attributes for a given vrf
 $ show vrf [<vrf_name>]
 
-// show the list of router interfaces
-$ show router-interface [vrf <vrf_name>]
-
 //add IP address to an interface.  The command already exists in SONiC, but will be enhanced
-$ config interface <interface_name> ip add <ip_addr/mask>
+$ config interface ip add <interface_name> <ip_addr/mask>
 
 //remove an IP address from an interface. The command already exists in SONiC, but will be enhanced.
-$ config interface <interface_name> ip del <ip_addr/mask>
+$ config interface ip del <interface_name> <ip_addr/mask>
 
 //add a prefix to a VRF
 $ config route add [vrf <vrf_name>] prefix <route_prefix/mask> nexthop <[vrf <vrf_name>] <ip> | dev <dev_name>>
@@ -571,6 +608,27 @@ $ config route del [vrf <vrf_name>] prefix <route_prefix/mask> nexthop <[vrf <vr
 //show prefixes in a given VRF. The existing command is enhanced to take VRF as the key
 $ show ip route [vrf < all | vrf_name>]
 
+//show ip interface command updated to show VRF name to which interface is bound to
+$ show ip interface 
+
+//show ipv6 interface command updated to show VRF name to which interface is bound to
+$ show ipv6 interface 
+
+```
+
+## Other Linux utilities
+
+Standard linux ping, ping6 and traceroute commands can be used on VRF by explicitly specifying kernel VRF device.
+Since Kernel device name is same as user configured VRF name, VRF name itself can be used as interface in below commands.
+
+```bash
+Ping/ping6:
+ping [-I <vrf_name>] destination
+ping6 [-I <vrf_name>] destination
+
+traceroute:
+traceroute [-i <vrf_name>] destination
+traceroute6 [-i <vrf_name>] destination
 ```
 
 ## User scenarios
@@ -585,7 +643,7 @@ Lets use Ethernet0 as an example in this document.
 
 ```bash
 
-$ config interface Ethernet0 ip add 1.1.1.1/24
+$ config interface ip add Ethernet0 1.1.1.1/24
 
 ```
 
@@ -600,7 +658,7 @@ To remove IP address from an interface:
 
 ```bash
 
-$ config interface Ethernet0 ip remove 1.1.1.1/24
+$ config interface ip remove Ethernet0 1.1.1.1/24
 
 ```
 
@@ -619,7 +677,7 @@ In this case, user wants to configure a VRF "Vrf-blue", with interfaces attached
 ```bash
 
 $ config vrf add Vrf-blue
-$ config interface Ethernet0 vrf bind Vrf-blue
+$ config interface bind Ethernet0 Vrf-blue
 
 ```
 
@@ -631,7 +689,7 @@ The Bind command will do the following:
 
 ```bash
 
-$ config interface Ethernet0 ip add 1.1.1.1/24
+$ config interface ip add Ethernet0 1.1.1.1/24
 
 ```
 
@@ -643,7 +701,7 @@ To unbind an interface from VRF:
 
 ```bash
 
-$ config interface Ethernet0 vrf unbind
+$ config interface unbind Ethernet0 Vrf-blue 
 
 ```
 
@@ -662,7 +720,7 @@ This set of commands will perform the work:
 
 $ show vrf Vrf-blue
 This will to get interface list belonging to Vrf-blue from app_db
-$ config interface Ethernet0 ip remove 1.1.1.1/24
+$ config interface ip remove Ethernet0 1.1.1.1/24
 This will remove all IP addresses from the interfaces belonging to the VRF.
 $ config interface Ethernet0 vrf unbind
 This will unbind all interfaces from this VRF
@@ -687,6 +745,22 @@ This command will do the following:
 - unbind interfaces(s) from Vrf-blue
 - del Vrf-blue
 
+### Route Leak Configuration
+
+User can configure static leak routes using below CLIs:
+
+```bash
+
+$ config route add vrf Vrf-red prefix 10.1.1.1/32 nexthop vrf Vrf-green 100.1.1.2
+This installs route 10.1.1.1/32 in Vrf-red and with nexthop pointing to 100.1.1.2 in Vrf-green, provided below conditions are met:
+  - Vrf-green is configured in kernel
+  - 100.1.1.2 is reachable in Vrf-green
+
+$ config route del vrf Vrf-red prefix 10.1.1.1/32 nexthop vrf Vrf-green 100.1.1.2
+This deletes route 10.1.1.1/32 in Vrf-red.
+
+```
+
 ## Impact to other service after import VRF feature
 
 For apps that don't care VRF they don't need to modify after sonic import VRF.
@@ -719,10 +793,6 @@ be modified or restarted for VRF binding event.
 For layer 3 apps such as snmpd or ntpd they are using vrf-global socket too.
 So they are vrf-transparent too.
 
-## Progress
-
-In the diagram, fpmsyncd, vrfmgrd, intfsmgrd, intfsorch are checked into the master branch. There may need changes to support VRF. Other components are working in progress, will be committed as planned.
-
 ## Test plan
 
 A separate test plan document will be uploaded and reviewed by the community

diff --git a/images/vrf_hld/Multi-VRF_Deployment.png b/images/vrf_hld/Multi-VRF_Deployment.png