## Introduction

With route validation now operational, this lab explores how to share validation state between iBGP peers, and apply policy to prefixes depending on their validation state.

## Lab Topology

The diagram below is a reminder of the lab topology:

## Background

Typical network design would see network operators configuring all EBGP speaking routers to have a session talking to the two redundant validators deployed on the network. `Valid`s would be propagated, `NotFound` would be propagated as well (perhaps lower priority), and `Invalid`s would be dropped as they arrive from EBGP peers.

Some operators may also wish to propagate the validation state of all prefixes received to all IBGP neighbours as well; and this would include `Valid`, `Invalid`, and `NotFound`. This is usually to support functionality such as Remote Trigger Blackhole Filtering, as we studied earlier. The Cisco IOS default is to automatically drop `Invalid` prefixes, and to prefer `Valid` over `NotFound` regardless of local-preference setting (an inserted step in the Best Path Selection Algorithm).

An extended BGP Community has been defined to allow this propagation of validation state. This means that all routers participating in the IBGP mesh can act on the validation state as determined by the routers which have a RTR connection to the validation cache.

## Propagating Validation State

### Disabling automatic dropping of invalids

Cisco IOS, by default, does not conform to the route origin validation standards. The RPKI state attached to the prefix should be just that; information for the network operator to implement policy on. However, Cisco IOS does three things automatically:

- drops prefixes which have RPKI state `Invalid`

- prefers paths with `Valid` state over paths with `NotFound` state regardless of local preference setting

- marks all prefixes announced by IBGP as `Valid` (even if they are `NotFound`).

We need to fix the first problem. The other two have no manual override in the IOS version used for this lab and remain issues which still have to be resolved.

We have set up the Border, Peering, and Access Routers to talk with the validator cache. We now need to tell these routers to not drop `Invalid`s. We also need to set up this configuration on the Core Router, as we will be propagating validation state to it from the EBGP speaking routers.

To do this, enter the following on Border, Core, Peering, and Access routers:

    router bgp X0
     address-family ipv4
      bgp bestpath prefix-validate allow-invalid
    !
     address-family ipv6
      bgp bestpath prefix-validate allow-invalid
    !

These four routers will now pass on prefixes marked as `Invalid`, allowing the operator to implement the policy they want (including dropping the prefixes).

### Extended Community to Propagate Validation State

We are now going to share the validation state from the Border, Peering and Access routers across the iBGP mesh. We aren’t going to set this up on the Trigger router, as we still need RTBH to work for destinations with potentially different origins from the ones we originate (more later).

To distribute validation state, Cisco IOS has the following two commands:

    neighbor <xxxx> send-community both
    neighbor <xxxx> announce rpki state

Explaining these two:

- `send-community both` means that the iBGP session will send not only the standard BGP communities but also any defined extended BGP communities. The default `send-community` command only sends the standard BGP communities (remember that IOS default is to send no communities at all).

- `announce rpki state` means that the router will set the extended BGP community values for `Valid` (0x4300:0:0), `NotFound` (0x4300:0:1) and `Invalid` (0x4300:0:2) as per RFC8097.

The two commands need to be applied on all routers, not just the routers with a validator configuration set up on it. The IBGP neighbour receiving the prefixes needs the `announce rpki state` command as well (even though it is only accepting the extended community indicating the RPKI state).

### Configuring Border, Core, Peering and Access Routers

Now add the above two commands to all the iBGP peer-groups on the Border, Core, Peering and Access Routers. Do not add them to the Trigger Router as that is used only for RTBH - we don’t need to know validation state there. Here is an example for the Core Router:

    router bgp X0
     address-family ipv4
      neighbor ibgpv4-full send-community both
      neighbor ibgpv4-full announce rpki state
      neighbor ibgpv4-partial send-community both
      neighbot ibgpv4-partial announce rpki state
    !
     address-family ipv6
      neighbor ibgpv6-full send-community both
      neighbor ibgpv6-full announce rpki state
      neighbor ibgpv6-partial send-community both
      neighbor ibgpv6-partial announce rpki state
    !

### Testing Propagation of Validation information

The Core Router has no configuration connecting it to a validator. If you now display the BGP table, what do you observe?

You should see validation states for the prefixes in the BGP table.

## Further Testing

Our final test is to remove the RPKI validator configuration on the Access and Peering Routers. This is just to test the effect of propagating the validation information by the extended BGP community from the Border Router.

On Peering and Access Router, delete the validator lines:

    router bgp X0
     no bgp rpki server tcp 100.68.X.30 port 3323 refresh 3600
     no bgp rpki server tcp 100.64.0.1 port 3323 refresh 3600
    !

Once you have done this, check the BGP tables on all 4 routers in the autonomous system.

What do you see in the BGP table now?

What valid routes to you see?

Do you see any invalid routes now?

What are the best paths?

Here is an example on the Core router of Group 1 after the validator configuration was removed from the Peering router:

    C1#sh ip bgp
    BGP table version is 45, local router ID is 100.68.1.2
    Status codes: s suppressed, d damped, h history, * valid, > best, i - internal,
                  r RIB-failure, S Stale, m multipath, b backup-path, f RT-Filter,
                  x best-external, a additional-path, c RIB-compressed,
    Origin codes: i - IGP, e - EGP, ? - incomplete
    RPKI validation codes: V valid, I invalid, N Not found

         Network          Next Hop          Metric LocPrf Weight Path
    V*>i 61.45.248.0/24   100.68.1.4             0    100      0 135533 i
    V*>i 61.45.249.0/24   100.68.1.1             0     50      0 121 20 135534 i
    N* i                  100.68.1.3             0    200      0 20 135534 i
    V*>i 61.45.250.0/24   100.68.1.1             0     50      0 121 30 135535 i
    N* i                  100.68.1.3             0    150      0 30 135535 i
    V*>i 61.45.251.0/24   100.68.1.1             0     50      0 121 40 135536 i
    N* i                  100.68.1.3             0    150      0 40 135536 i
    V*>i 61.45.252.0/24   100.68.1.1             0     50      0 121 122 50 135537 i
    N* i                  100.68.1.3             0    150      0 50 135537 i
    V*>i 61.45.253.0/24   100.68.1.1             0     50      0 121 122 60 135538 i
    N* i                  100.68.1.3             0    150      0 60 135538 i
    V*>i 61.45.254.0/24   100.68.1.1             0     50      0 121 122 70 135539 i
    N* i                  100.68.1.3             0    150      0 70 135539 i
    V*>i 61.45.255.0/24   100.68.1.1             0     50      0 121 122 80 135540 i
    N* i                  100.68.1.3             0    150      0 80 135540 i

What has happened1?

Important: Discuss in your group, and with the instructors when this portion of the lab exercise is completed. What are the implications?

## Finishing Steps

Once you have completed the last step above, restore the validator configuration on the Peering and Access Routers. We now have a fully working configuration for our network infrastructure in our group.

On the Access Router, restore the validator lines:

    router bgp X0
     bgp rpki server tcp 100.68.X.30 port 3323 refresh 3600
     bgp rpki server tcp 100.64.0.1 port 3323 refresh 3600
    !

and on the Peering Router, restore the validator line:

    router bgp X0
     bgp rpki server tcp 100.68.X.30 port 3323 refresh 3600
    !

## Summary

This lab has demonstrated how to propagate validation information between iBGP speaking routers in an AS, and showed the results of doing this.

Important Note: most operators will test validation state on their EBGP speaking routers only and will not have any need or reason to distribute state elsewhere in their IBGP mesh unless they are supporting inter-AS RTBH. Consult the RPKI presentation for more information.

1. Cisco IOS has added an automatic option in the BGP Path Selection process, before checking local-preference, to prefer `valid` paths over `not found`. There is no way of turning this off in the release of IOS used for this workshop. This means that if an EBGP speaking router loses its connectivity to the validator, the routes learned by that router will be distributed in IBGP with state `not found` - which means that the best path selection on the other routers will prefer `valid` over `not found`. In this case, traffic between two ASes will go over a transit link rather than the peering link - with big traffic and economic impact on the networks involved.↩︎