Using OSPF sham-links on a basic MPLS VPN

原文：http://blog.initialdraft.com/archives/117/

OSPF is used very often in MPLS/VPN environments. It’s common to see several sites communicate through a MPLS backbone with the routing protocol used between CEs-PEs being OSPF.

The common behavior is that after prefixes were announced from the CE to the PE by OSPF, they are later redistributed into MP-BGP and the vice versa on the remote site. The prefixes are redistributed back into OSPF as LSA Type-3 (inter-area) or LSA Type-5 (external), depending if the Domain-ID is configured and not on the area configured on each site.

If both sites have the same Domain-ID, prefixes are redistributed from MP-BGP into OSPF as LSA Type-3. They are redistributed as LSA Type-3 and not as LSA Type-1 because PEs don’t have a direct neighbor session between them (remember that the MPLS Backbone introduces a discontinuity in the OSPF domain). The goal of the OSPF sham-link feature is to build a direct OSPF neighbor session between PEs, and then redistribute prefixes from MP-BGP into OSPF as LSA Type-1 instead of LSA Type-3.

In our case, we’ll assume all sites area on the same OSPF area, and that all PEs involved in the VPN use the same process number for OSPF.

Sham-Link Topology

MP-BGP uses two extended communities when redistributing from OSPF. These communities help other PEs to identify how they should announce in OSPF those received prefixes. Those communities are “OSPF Route Type” and “OSPF Domain-ID”.

When all PEs use the same Domain-ID (we will see in another post how to use this parameter in OSPF and what implications it has), prefixes will be redistributed from MP-BGP into OSPF as LSA Type-3.
Check in the next output how the prefix 1.1.1.1/32 from Site-1 is seen at PE-2 in Site-2 (the Domain-ID pushed by PE-1 to the prefix is the same Domain-ID configured on PE-2)

PE2#sh ip bgp vpn all
....
Network          Next Hop            Metric     LocPrf     Weight     Path
Route Distinguisher: 1:1 (default for vrf VPN_A)
*>i1.1.1.1/32       100.100.100.1       11        100          0     ?
*> 2.2.2.2/32       10.10.22.2          11                 32768     ?
*>i10.10.11.0/24    100.100.100.1        0        100          0     ?
*> 10.10.22.0/24    0.0.0.0              0                 32768     ?

Let’s look at the BGP communities for prefix 1.1.1.1/32:

PE2#sh bgp vpn uni all 1.1.1.1/32
BGP routing table entry for 1:1:1.1.1.1/32, version 34
Paths: (1 available, best #1, table VPN_A)
Flag: 0x820
  Not advertised to any peer
  Local
    100.100.100.1 (metric 11) from 100.100.100.1 (100.100.100.1)
      Origin incomplete, metric 11, localpref 100, valid, internal, best
      Extended Community: RT:1:1 OSPF DOMAIN ID:0x0005:0x010101010200
        OSPF RT:0.0.0.1:2:0 OSPF ROUTER ID:10.10.11.2:0
      mpls labels in/out nolabel/17

This makes PE-2 redistribute the prefix as an LSA Type-3, as we can confirm by looking at R2 routing table, here R2 is using the link to PE2 to reach the prefix 1.1.1.1/32 and this is seen as a Inter-Area (IA) route :

R2#sh ip route
...
1.0.0.0/32 is subnetted, 1 subnets
O IA    1.1.1.1 [110/21] via 10.10.22.1, 00:01:10, FastEthernet0/0
2.0.0.0/32 is subnetted, 1 subnets
C       2.2.2.2 is directly connected, Loopback0
10.0.0.0/24 is subnetted, 2 subnets
O IA    10.10.11.0 [110/11] via 10.10.22.1, 00:01:10, FastEthernet0/0
C       10.10.22.0 is directly connected, FastEthernet0/0

If we take a look at the prefix 1.1.1.1/32 on the actual OSPF database we can see that PE2 is the ABR that injected the prefix, and that is on the DB as Inter-Area LSA.

R2#sh ip ospf datab

            OSPF Router with ID (2.2.2.2) (Process ID 100)
        Router Link States (Area 1)
Link ID         ADV Router      Age         Seq#       Checksum Link count
1.1.1.1         1.1.1.1         366         0x80000003 0x00F926 5
2.2.2.2         2.2.2.2         89          0x80000006 0x002F32 3
10.10.11.2      10.10.11.2      156         0x80000005 0x00B4CE 2
10.10.22.1      10.10.22.1      160         0x80000005 0x0089CC 2

        Summary Net Link States (Area 1)
Link ID         ADV Router      Age         Seq#       Checksum
1.1.1.1         10.10.22.1      85          0x80000001 0x00F68B
10.10.11.0      10.10.22.1      85          0x80000001 0x004C25

If  backdoor links are not built between sites, traffic will always cross the MPLS backbone. But if you happen to have a backdoor link between two or more routers on the sites and OSPF is enabled on those links, we’ll find out that traffic doesn’t cross the MPLS backbone, but the backdoor link instead. In this case, the same prefixes announced through the backbone MPLS as LSA Type-3 (inter-area) are also announced through the backdoor links as LSA Type-1 (intra-area). And LSA Type-1 are preferred over LSA Type-3 in OSPF, as we can see here:

R2 prefers to use the backdoor-link, and the prefix is seen as Intra-Area:

R2#sh ip route
<snip>

1.0.0.0/32 is subnetted, 1 subnets
O       1.1.1.1 [110/2] via 12.12.12.1, 00:00:17, FastEthernet1/0
2.0.0.0/32 is subnetted, 1 subnets
C       2.2.2.2 is directly connected, Loopback0
10.0.0.0/24 is subnetted, 2 subnets
O       10.10.11.0 [110/11] via 12.12.12.1, 00:00:17, FastEthernet1/0
C       10.10.22.0 is directly connected, FastEthernet0/0
12.0.0.0/24 is subnetted, 1 subnets
C       12.12.12.0 is directly connected, FastEthernet1/0

R2 uses the link with R1 to reach the prefix 1.1.1.1/32

R2#traceroute 1.1.1.1

<snip>


Tracing the route to 1.1.1.1 1 12.12.12.1 8 msec * 8 msec

If these backdoor links are supposed to be for backup purposes, then we got an issue.

In order to change this behavior, we need that the PEs redistribute MP-BGP into OSPF as LSA Type-1 instead of LSA3. We can use an ospf sham-link between PEs for that. The steps to configure a sham-link are:

1.- Create a new Loopback on each PE
2.- Insert the new Loopback in the VRF
3.- Announce those loopbacks in MP-BGP (not in OSPF)
4.- Build an ospf sham-link between each pair of PEs, using the new loopbacks as origin and destination of the link.
5.- Increment the OSPF cost in the backdoor links
Configuration

This are the steps to be used to build the sham-link in PE2. Something similar needs to be done in PE1, with the IP 111.111.111.111 on the Loopback100:

PE2# conf term
Enter configuration commands, one per line.  End with CNTL/Z.
PE2(config)# int lo100
PE2(config-if)# ip vrf forw VPN_A
PE2(config-if)# ip add 222.222.222.222 255.255.255.255
PE2(config-if)#
PE2(config-if)# router bgp 65000
PE2(config-router)# address ipv4 vrf VPN_A
PE2(config-router-af)# net 222.222.222.222 mask 255.255.255.255
PE2(config-router-af)#
PE2(config-router-af)# router ospf 100 vrf VPN_A
PE2(config-router)# area 1 sham-link 222.222.222.222 111.111.111.111 cost 1

Once the sham-link is built, the prefixes between sites using the MPLS backbone will be seen as LSA Type-1:

We can verify that the sham-link is established:

PE2#sh ip ospf 100 sham-link

Sham Link OSPF_SL0 to address 111.111.111.111 is up
Area 1 source address 222.222.222.222
  Run as demand circuit
  DoNotAge LSA allowed. Cost of using 1 State POINT_TO_POINT,
  Timer intervals configured, Hello 10, Dead 40, Wait 40,
    Hello due in 00:00:07
    Adjacency State FULL (Hello suppressed)
    Index 2/2, retransmission queue length 0, number of retransmission 0
    First 0x0(0)/0x0(0) Next 0x0(0)/0x0(0)
    Last retransmission scan length is 0, maximum is 0
    Last retransmission scan time is 0 msec, maximum is 0 msec

An adjacency is built through the sham-link

PE2#sh ip ospf 100 neig

Neighbor ID     Pri   State           Dead Time   Address         Interface
10.10.11.2        0   FULL/  -           -        111.111.111.111  OSPF_SL0
2.2.2.2           0   FULL/  -        00:00:31    10.10.22.2      FastEthernet0/1

We can see the new loopback prefixes on the BGP: PE1′s and PE2′s Loopbacks

PE2#sh ip bgp vpn all

BGP table version is 53, local router ID is 100.100.100.2
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal,
              r RIB-failure, S Stale
Origin codes: i - IGP, e - EGP, ? - incomplete
   Network          Next Hop            Metric LocPrf Weight Path
Route Distinguisher: 1:1 (default for vrf VPN_A)
r>i1.1.1.1/32       100.100.100.1           11    100      0   ?
*> 2.2.2.2/32       10.10.22.2              11         32768   ?
r>i10.10.11.0/24    100.100.100.1            0    100      0   ?
*> 10.10.22.0/24    0.0.0.0                  0         32768   ?
* i12.12.12.0/24    100.100.100.1           11    100      0   ?
*>                  10.10.22.2              11         32768   ?
*>i111.111.111.111/32
                    100.100.100.1            0    100      0   i
*> 222.222.222.222/32
                    0.0.0.0                  0         32768   i

And the details of how the prefix is seen on the BGP table:

PE2#sh bgp vpn uni all 1.1.1.1/32

BGP routing table entry for 1:1:1.1.1.1/32, version 52
Paths: (1 available, best #1, table VPN_A, RIB-failure(17))
Flag: 0x820
  Not advertised to any peer
  Local
    100.100.100.1 (metric 11) from 100.100.100.1 (100.100.100.1)
      Origin incomplete, metric 11, localpref 100, valid, internal, best
      Extended Community: RT:1:1 OSPF DOMAIN ID:0x0005:0x010101010200
        OSPF RT:0.0.0.1:2:0 OSPF ROUTER ID:10.10.11.2:0
      mpls labels in/out nolabel/17

After changing the OSPF cost of the backdoor link, R2 prefers to use the MPLS backbone

R2# conf term
R2(config)# interface fa1/0
R2(config-if)# ip ospf cost 100
R2(config-if)# end
R2#sh ip route
...
222.222.222.0/32 is subnetted, 1 subnets
O E2    222.222.222.222 [110/1] via 10.10.22.1, 00:01:36, FastEthernet0/0
1.0.0.0/32 is subnetted, 1 subnets
O       1.1.1.1 [110/2] via 10.10.22.1, 00:01:36, FastEthernet0/0
2.0.0.0/32 is subnetted, 1 subnets
C       2.2.2.2 is directly connected, Loopback0
111.0.0.0/32 is subnetted, 1 subnets
O E2    111.111.111.111 [110/1] via 10.10.22.1, 00:01:36, FastEthernet0/0
10.0.0.0/24 is subnetted, 2 subnets
O       10.10.11.0 [110/11] via 10.10.22.1, 00:01:36, FastEthernet0/0
C       10.10.22.0 is directly connected, FastEthernet0/0
12.0.0.0/24 is subnetted, 1 subnets
C       12.12.12.0 is directly connected, FastEthernet1/0

The prefix 1.1.1.1/32 is seen as Intra-Area, and PE1′s and PE2′s loopbacks as External

R2#sh ip ospf data

            OSPF Router with ID (2.2.2.2) (Process ID 100)
        Router Link States (Area 1)
Link ID         ADV Router      Age         Seq#       Checksum Link count
1.1.1.1         1.1.1.1         187         0x80000005 0x00F528 5
2.2.2.2         2.2.2.2         186         0x80000008 0x0015DB 5
10.10.11.2      10.10.11.2      116         0x80000006 0x001E22 3
10.10.22.1      10.10.22.1      115         0x80000006 0x00C656 3

        Type-5 AS External Link States
Link ID         ADV Router      Age         Seq#       Checksum Tag
111.111.111.111 10.10.11.2      124         0x80000001 0x009983 3489725928
111.111.111.111 10.10.22.1      115         0x80000001 0x0052C0 3489725928
222.222.222.222 10.10.11.2      127         0x80000001 0x0093CB 3489725928
222.222.222.222 10.10.22.1      129         0x80000001 0x004C09 3489725928

R2 uses the MPLS Backbone to reach the prefix 1.1.1.1/32

R2#traceroute 1.1.1.1
Type escape sequence to abort.
Tracing the route to 1.1.1.1
1 10.10.22.1 4 msec 4 msec 4 msec
2 10.10.11.2 [MPLS: Label 17 Exp 0] 8 msec 8 msec 8 msec
3 10.10.11.1 8 msec *  24 msec

Verification

If the sham-link doesn’t come up, we could use these basic commands for troubleshooting:

    sh ip bgp vpn all: the loopback to build the sham-link must be announced in BGP on each PE.
    sh ip ospf sham-link: once the sham-link has been built between two PEs, the link must be shown as up.
    sh ip ospf neig: this allows us to verify that an adjacency is established through the sham-link.
    sh ip route: eventually, the loopbacks used to build the sham-link must be seen as External routes at the CEs, and internal prefixes from the remote site of the sham-link must be seen as intra-area routes.
    sh ip ospf database: at CEs, internal prefixes from the remote site of the sham-link must be seen as LSA Type-1.