With route validation now operational, this lab explores how to share validation state between iBGP peers, and apply policy to prefixes depending on their validation state.
The diagram below is a reminder of the lab topology:
Typical network design would see network operators configuring all EBGP speaking routers to have a session talking to the two redundant validators deployed on the network. Valids would be propagated, NotFound would be propagated as well (perhaps lower priority), and Invalids would be dropped as they arrive from EBGP peers.
Some operators may also wish to propagate the validation state of all prefixes received to all IBGP neighbours as well; and this would include Valid, Invalid, and NotFound. This is usually to support functionality such as Remote Trigger Blackhole Filtering, as we studied earlier. The Cisco IOS default is to automatically drop Invalid prefixes, and to prefer Valid over NotFound regardless of local-preference setting (an inserted step in the Best Path Selection Algorithm).
An extended BGP Community has been defined to allow this propagation of validation state. This means that all routers participating in the IBGP mesh can act on the validation state as determined by the routers which have a RTR connection to the validation cache.
Cisco IOS, by default, does not conform to the route origin validation standards. The RPKI state attached to the prefix should be just that; information for the network operator to implement policy on. However, Cisco IOS does three things automatically:
drops prefixes which have RPKI state Invalid
prefers paths with Valid state over paths with NotFound state regardless of local preference setting
marks all prefixes announced by IBGP as Valid (even if they are NotFound).
We need to fix the first problem. The other two have no manual override in the IOS version used for this lab and remain issues which still have to be resolved.
We have set up the Border, Peering, and Access Routers to talk with the validator cache. We now need to tell these routers to not drop Invalids. We also need to set up this configuration on the Core Router, as we will be propagating validation state to it from the EBGP speaking routers.
To do this, enter the following on Border, Core, Peering, and Access routers:
router bgp X0
address-family ipv4
bgp bestpath prefix-validate allow-invalid
!
address-family ipv6
bgp bestpath prefix-validate allow-invalid
!These four routers will now pass on prefixes marked as Invalid, allowing the operator to implement the policy they want (including dropping the prefixes).
We are now going to share the validation state from the Border, Peering and Access routers across the iBGP mesh. We aren’t going to set this up on the Trigger router, as we still need RTBH to work for destinations with potentially different origins from the ones we originate (more later).
To distribute validation state, Cisco IOS has the following two commands:
neighbor <xxxx> send-community both
neighbor <xxxx> announce rpki stateExplaining these two:
send-community both means that the iBGP session will send not only the standard BGP communities but also any defined extended BGP communities. The default send-community command only sends the standard BGP communities (remember that IOS default is to send no communities at all).
announce rpki state means that the router will set the extended BGP community values for Valid (0x4300:0:0), NotFound (0x4300:0:1) and Invalid (0x4300:0:2) as per RFC8097.
The two commands need to be applied on all routers, not just the routers with a validator configuration set up on it. The IBGP neighbour receiving the prefixes needs the announce rpki state command as well (even though it is only accepting the extended community indicating the RPKI state).
Now add the above two commands to all the iBGP peer-groups on the Border, Core, Peering and Access Routers. Do not add them to the Trigger Router as that is used only for RTBH - we don’t need to know validation state there. Here is an example for the Core Router:
router bgp X0
address-family ipv4
neighbor ibgpv4-full send-community both
neighbor ibgpv4-full announce rpki state
neighbor ibgpv4-partial send-community both
neighbot ibgpv4-partial announce rpki state
!
address-family ipv6
neighbor ibgpv6-full send-community both
neighbor ibgpv6-full announce rpki state
neighbor ibgpv6-partial send-community both
neighbor ibgpv6-partial announce rpki state
!
The Core Router has no configuration connecting it to a validator. If you now display the BGP table, what do you observe?
You should see validation states for the prefixes in the BGP table.
Our final test is to remove the RPKI validator configuration on the Access and Peering Routers. This is just to test the effect of propagating the validation information by the extended BGP community from the Border Router.
On Peering and Access Router, delete the validator lines:
router bgp X0
no bgp rpki server tcp 100.68.X.30 port 3323 refresh 3600
no bgp rpki server tcp 100.64.0.1 port 3323 refresh 3600
!Once you have done this, check the BGP tables on all 4 routers in the autonomous system.
What do you see in the BGP table now?
What valid routes to you see?
Do you see any invalid routes now?
What are the best paths?
Here is an example on the Core router of Group 1 after the validator configuration was removed from the Peering router:
C1#sh ip bgp
BGP table version is 45, local router ID is 100.68.1.2
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal,
r RIB-failure, S Stale, m multipath, b backup-path, f RT-Filter,
x best-external, a additional-path, c RIB-compressed,
Origin codes: i - IGP, e - EGP, ? - incomplete
RPKI validation codes: V valid, I invalid, N Not found
Network Next Hop Metric LocPrf Weight Path
V*>i 61.45.248.0/24 100.68.1.4 0 100 0 135533 i
V*>i 61.45.249.0/24 100.68.1.1 0 50 0 121 20 135534 i
N* i 100.68.1.3 0 200 0 20 135534 i
V*>i 61.45.250.0/24 100.68.1.1 0 50 0 121 30 135535 i
N* i 100.68.1.3 0 150 0 30 135535 i
V*>i 61.45.251.0/24 100.68.1.1 0 50 0 121 40 135536 i
N* i 100.68.1.3 0 150 0 40 135536 i
V*>i 61.45.252.0/24 100.68.1.1 0 50 0 121 122 50 135537 i
N* i 100.68.1.3 0 150 0 50 135537 i
V*>i 61.45.253.0/24 100.68.1.1 0 50 0 121 122 60 135538 i
N* i 100.68.1.3 0 150 0 60 135538 i
V*>i 61.45.254.0/24 100.68.1.1 0 50 0 121 122 70 135539 i
N* i 100.68.1.3 0 150 0 70 135539 i
V*>i 61.45.255.0/24 100.68.1.1 0 50 0 121 122 80 135540 i
N* i 100.68.1.3 0 150 0 80 135540 iWhat has happened1?
Important: Discuss in your group, and with the instructors when this portion of the lab exercise is completed. What are the implications?
Once you have completed the last step above, restore the validator configuration on the Peering and Access Routers. We now have a fully working configuration for our network infrastructure in our group.
On the Access Router, restore the validator lines:
router bgp X0
bgp rpki server tcp 100.68.X.30 port 3323 refresh 3600
bgp rpki server tcp 100.64.0.1 port 3323 refresh 3600
!and on the Peering Router, restore the validator line:
router bgp X0
bgp rpki server tcp 100.68.X.30 port 3323 refresh 3600
!
This lab has demonstrated how to propagate validation information between iBGP speaking routers in an AS, and showed the results of doing this.
Important Note: most operators will test validation state on their EBGP speaking routers only and will not have any need or reason to distribute state elsewhere in their IBGP mesh unless they are supporting inter-AS RTBH. Consult the RPKI presentation for more information.
Cisco IOS has added an automatic option in the BGP Path Selection process, before checking local-preference, to prefer valid paths over not found. There is no way of turning this off in the release of IOS used for this workshop. This means that if an EBGP speaking router loses its connectivity to the validator, the routes learned by that router will be distributed in IBGP with state not found - which means that the best path selection on the other routers will prefer valid over not found. In this case, traffic between two ASes will go over a transit link rather than the peering link - with big traffic and economic impact on the networks involved.↩︎