Communities Lab

Introduction

The purpose of this part of the lab is introduce BGP Communities into our lab. We are doing this for ease of management of the various prefixes we are carrying in our iBGP. There is some careful preparatory work needed and all participants are advised to follow the lab notes carefully.

Lab Topology

The lab topology is unchanged from the topology used in the previous section.

BGP Communities

We will be updating our BGP configurations to use BGP Communities. Following the presentation, we specifically want to tag prefixes we learn from our private peers, our IXP peers, and our own prefixes we introduce into our iBGP. This way we can manage what we announce to all external networks simply by applying community filters, removing the need to maintain prefix-lists, and therefore simplifying the management of the network configuration.

Community Policy

The communities we are going to adopt are as follows:

Prefix Type	Community	Description
Our aggregate	AS:1000	Our assigned address block(s)
Subnets of our aggregate	AS:1001	Any subnets we carve out of our address block(s)
Customer independent addresses	AS:1005	Any address space customers are assigned independently of what they receive from us
Private Peer addresses	AS:1100	Any addresses our private peers send to us
IXP Bilateral peer addresses	AS:1200	Any addresses our IXP Bilateral peers send to us
IXP RS Peer addresses	AS:1300	Any addresses our IXP RS peers send to us

Each group will now set up these community definitions on their Border, Core, Peering and Access routers (not the Customer router, which is not in the group’s AS).

Here is a configuration example using Cisco IOS named community lists:

ip community-list standard aggregate      permit X0:1000
ip community-list standard subnets        permit X0:1001
ip community-list standard customer-pi    permit X0:1005
ip community-list standard private-peer   permit X0:1100
ip community-list standard bilateral-peer permit X0:1200
ip community-list standard RS-peer        permit X0:1300

We also need to tell Cisco IOS to use the industry format for BGP communities (as described in RFC1998) rather than the specification standard (as described in RFC1997). The latter represents communities as a 32-bit integer; the former represents communities as two 16-bit integers separated by a colon which is deemed more human readable by network operators.

The following configuration needs to be done on all the routers which have been configured with BGP (Border, Core, Peering, Access). The new-format is actually referring to the community format used in RFC1998, rather than in the specification in RFC1997 - it’s not a new format as such, but an “easier for humans” format.

ip bgp-community new-format

Setting Community on our aggregate

We introduced our aggregate for our AS on the Core router using the BGP network statement. We are now going to augment this network statement on the Core Router so that the community is set on the prefix we introduce. Here is an example:

route-map set-aggregate-community permit 5
 description Set community on Aggregate
 set community X0:1000
!
router bgp X0
 address-family ipv4
  network 100.68.X.0 mask 255.255.255.0 route-map set-aggregate-community
!
 address-family ipv6
  network 2001:DB8:X::/48 route-map set-aggregate-community
!

Once this is done, check the BGP table. Run the:

show ip bgp community

and

show bgp ipv6 unicast community

commands. What do you see now?

All being well, you should see your IPv4 and IPv6 aggregates displayed in the output. This command shows you all prefixes which have a BGP community set on them. Then show the actual BGP table entry - you will now see an extra line in the output, indicating which community has been set. It should be in the format AS:1000.

Setting Community on our Customer and Server Routes

We introduced our customer route on the Access router using the BGP network statement.

First we will augment the network statement on the Access Router so that the community is set on the prefix we introduce. Here is an example:

route-map set-subnet-community permit 5
 description Set community on Customer subnet
 set community X0:1001
!
router bgp X0
 address-family ipv4
  network 100.68.X.64 mask 255.255.255.192 route-map set-subnet-community
!
 address-family ipv6
  network 2001:DB8:X:4000::/52 route-map set-subnet-community
!

We also need to tag the server subnet connected to the core router with our Subnet community. Use the same route-map above on the configuration on the Core Router:

route-map set-subnet-community permit 5
 description Set community on Server subnet
 set community X0:1001
!
router bgp X0
 address-family ipv4
  network 100.68.X.28 mask 255.255.255.252 route-map set-subnet-community
!
 address-family ipv6
  network 2001:DB8:X:21::/64 route-map set-subnet-community
!

Once this is done, check the BGP table on the Core and Access Routers. Again run the:

show ip bgp community

and

show bgp ipv6 unicast community

commands. What do you see now?

All being well, you should see your IPv4 and IPv6 customer route displayed in the output, as well as the aggregate generated earlier on the Core Router.

Setting Community on Private Peer Routes

We now set the community on the prefix we learn from our private peer on the Peering router. We look for the prefix they are sending us, and tag it with the appropriate community. Here is a sample configuration - note that we are adding to the existing configuration (which includes the private-peer-in route-map and the inbound prefix-list for the private peer):

route-map private-peer-in permit 5
 description Tag prefixes from our Private Peer
 set community X0:1100
!
router bgp X0
 address-family ipv4
  neighbor <ptp-v4> route-map private-peer-in in
!
 address-family ipv6
  neighbor <ptp-v6> route-map private-peer-in in
!

Once the route-map and the BGP neighbour configuration has been updated, remember to do the inbound route-refresh. Cisco IOS does not do route-refreshes automatically.

Check the communities listed in the BGP tables as you did earlier. Do you now see the private peer prefix with a community set on it?

Setting Community on the IXP Peer Routes

We now set the community on the prefixes we learn from our IXP peers via the Route Server. As with the previous example, we look for the prefix they are sending us, and tag it with the appropriate community. Here is a sample configuration as would be used by Groups 1 to 4 (Groups 5 to 8 would have a similar configuration but using IXP 2’s peering LAN). Note again that we are adding to the existing configuration (which includes the IXP-RS-in route-map and the inbound prefix-list for the IXP peer):

route-map IXP-RS-in permit 5
 description Tag prefixes from our IXP Peers
 set community X0:1300
!
router bgp X0
 address-family ipv4
  neighbor 100.127.1.254 route-map IXP-RS-in in
!
 address-family ipv6
  neighbor 2001:DB8:FFFF:1::FE route-map IXP-RS-in in
!

Once the route-map and the BGP neighbour configuration has been updated, remember to do the inbound route-refresh.

Check the communities listed in the BGP tables as you did earlier. Do you now see the IXP peer prefixes with a community set on them?

Updating EBGP Prefix-lists for Private and IXP RS Peers

We currently have prefix-lists applied to inbound on the peerings with our private peer and with the IXP Route Server. For this step, to give us more flexibility in future labs, we are going to move the prefix-list from being directly applied to the neighbour into the route-map that is applied to the neighbour.

The existing configuration on your Peering Router for those two peer types is shown below (this example is for Groups 1 to 4) - confirm that you have something similar (Groups 5 to 8 would see IXP 2’s peering LAN addresses instead):

router bgp X0
 address-family ipv4
  neighbor <ptp-v4> prefix-list ASY0-block in
  neighbor <ptp-v4> route-map private-peer-in in
  neighbor 100.127.1.254 prefix-list IXP-RS in
  neighbor 100.127.1.254 route-map IXP-RS-in in
!
 address-family ipv6
  neighbor <p2p-v6> prefix-list ASY0-v6block in  
  neighbor <ptp-v6> route-map private-peer-in in
  neighbor 2001:DB8:FFFF:1::FE prefix-list IXP-v6RS in
  neighbor 2001:DB8:FFFF:1::FE route-map IXP-RS-in in
!

We are going to remove the prefix-list as applied to the neighbour, and add it into the route-map used instead. Here is an example for the private-peer, where Y is the private-peer Group number:

route-map private-peer-in permit 5
 description Local pref for Private Peer
 match ip address prefix-list ASY0-block
 set local-preference 200
 set community X0:1100
!
route-map private-peerv6-in permit 5
 description Local pref for Private Peer
 match ipv6 address prefix-list ASY0-v6block
 set local-preference 200
 set community X0:1100
!
router bgp X0
 address-family ipv4
  no neighbor <ptp-v4> prefix-list ASY0-block in
 address-family ipv6
  no neighbor <p2p-v6> prefix-list ASY0-v6block in
!

And because we have now created an IPv6 version of the route-map, we need to update the entry under the BGP neighbour as well:

router bgp X0
 address-family ipv6
  neighbor <p2p-v6> route-map private-peerv6-in in
!

You can simply apply the above command - it will overwrite the existing route-map.

Do the same for the IXP RS peering session - the following example shows what Groups 5 to 8 would have to do:

router bgp X0
 address-family ipv4
  no neighbor 100.127.3.254 prefix-list IXP-RS in
 address-family ipv6
  no neighbor 2001:DB8:FFFE:1::FE prefix-list IXP-v6RS in
!
route-map IXP-RS-in permit 5
 description Local pref for IXP RS Peers
 match ip address prefix-list IXP-RS
 set local-preference 150
 set community X0:1300
!
route-map IXP-RSv6-in permit 5
 description Local pref for IXP RS Peers
 match ipv6 address prefix-list IXP-v6RS
 set local-preference 150
 set community X0:1300
!

And then we need to update the name of the route-map used for the IPv6 peering with the Route Server, for the same reason we did for the private peer earlier (Group 5 to 8 example):

router bgp X0
 address-family ipv6
  neighbor 2001:DB8:FFFE:1::FE route-map IXP-RSv6-in in
!

Once you have completed all this, carry out a route-refresh on the EBGP sessions on the Peering Router, and check that you are still receiving all the prefixes you expect to receive.

Updating Route Reflector Configuration

Our existing route reflector configuration (on the Core router) is set up with two types of neighbours:

standard BGP speaker which sees full routes (e.g. the Border router)
internal BGP speaker which sees only internal route (the Access and Peering routers)

The configuration for the internal BGP speaker included an as-path filter which only allowed locally originated routes to be passed on to the Access and Peering routers. Now that we have implemented communities for our AS, we can convert this to a community-based route-map too.

The current configuration on the Core router may look something like this:

ip as-path access-list 10 permit ^$
!
router bgp X0
 address-family ipv4
  neighbor ibgp-partial peer-group
  neighbor ibgp-partial description Local Routes only
  neighbor ibgp-partial remote-as X0
  neighbor ibgp-partial update-source loopback0
  neighbor ibgp-partial next-hop-self
  neighbor ibgp-partial password BGPlab
  neighbor ibgp-partial send-community
  neighbor ibgp-partial route-reflector-client
  neighbor ibgp-partial filter-list 10 out
!

with a similar configuration for IPv6.

We will now create a route-map called partial-iBGP to replace the as-path filter. Here is an example which only allows locally originated routes in our iBGP (we also allow customer independent address space as well):

route-map partial-iBGP permit 5
 description Prefixes for partial iBGP
 match community aggregate
 match community subnets
 match community customer-pi
!

And then we apply it to our existing route-reflector peer-group, like this:

router bgp X0
 address-family ipv4
  neighbor ibgp-partial route-map partial-iBGP out
  no neighbor ibgp-partial filter-list 10 out
!
 address-family ipv6
  neighbor ibgpv6-partial route-map partial-iBGP out
  no neighbor ibgpv6-partial filter-list 10 out
!

Once we have done this we can route-refresh our iBGP neighbours outbound, for both IPv4 and IPv6. Confirm on the other routers that you still see the internal routes you expect to see.

And once this has been done, remove the as-path filter:

no ip as-path access-list 10

We are now using communities to determine what prefixes the route-reflector clients receive - again this is more scalable than trying to maintain AS-PATH filters.

Outbound BGP Announcements Filtered by Community

The final step in this exercise is to update our outbound BGP filters to use BGP communities rather than the prefix-lists we set up earlier. Our inbound filters will remain using prefix-lists, as we want to control what our neighbours send to us. However, for outbound announcements, prefix-lists do not scale, as every time we need to announce a new prefix, we have to update the filters - which gets very tedious to maintain as the network grows larger.

The current BGP configuration on the Border router should be looking something like this:

ip community-list standard aggregate      permit X0:1000
ip community-list standard subnets        permit X0:1001
ip community-list standard customer-pi    permit X0:1005
ip community-list standard private-peer   permit X0:1100
ip community-list standard bilateral-peer permit X0:1200
ip community-list standard RS-peer        permit X0:1300
!
ip prefix-list ASX0-block permit 100.68.X.0/24
ipv6 prefix-list ASX0-v6block permit 2001:DB8:X::/48
!
ip prefix-list FULL-ROUTES permit 0.0.0.0/0 le 32
ipv6 prefix-list FULL-v6ROUTES permit ::/0 le 128
!
router bgp X0
 address-family ipv4
  neighbor <ptp-v4> remote-as <AS-N>
  neighbor <ptp-v4> description eBGP with TRANSIT N
  neighbor <ptp-v4> password BGPlab
  neighbor <ptp-v4> prefix-list ASX0-block out
  neighbor <ptp-v4> prefix-list FULL-ROUTES in
!
 address-family ipv6
  neighbor <ptp-v6> remote-as <AS-N>
  neighbor <ptp-v6> description eBGP with TRANSIT N
  neighbor <ptp-v6> password BGPlab
  neighbor <ptp-v6> prefix-list ASX0-v6block out
  neighbor <ptp-v6> prefix-list FULL-v6ROUTES in
!

and the Peering router (for Groups 5 to 8):

ip community-list standard aggregate      permit X0:1000
ip community-list standard subnets        permit X0:1001
ip community-list standard customer-pi    permit X0:1005
ip community-list standard private-peer   permit X0:1100
ip community-list standard bilateral-peer permit X0:1200
ip community-list standard RS-peer        permit X0:1300
!
ip prefix-list ASX0-block permit 100.68.X.0/24
ipv6 prefix-list ASX0-v6block permit 2001:DB8:X::/48
!
ip prefix-list ASY0-block permit 100.68.Y.0/24
ipv6 prefix-list ASY0-v6block permit 2001:DB8:Y::/48
!
ip prefix-list IXP-RS permit 100.68.5.0/24
ip prefix-list IXP-RS permit 100.68.6.0/24
ip prefix-list IXP-RS permit 100.68.7.0/24
ip prefix-list IXP-RS permit 100.68.8.0/24
!
ipv6 prefix-list IXP-v6RS permit 2001:DB8:5::/48
ipv6 prefix-list IXP-v6RS permit 2001:DB8:6::/48
ipv6 prefix-list IXP-v6RS permit 2001:DB8:7::/48
ipv6 prefix-list IXP-v6RS permit 2001:DB8:8::/48
!
route-map IXP-RS-in permit 5
 description Local pref for IXP Peer
 match ip address prefix-list IXP-RS
 set local-preference 150
 set community X0:1300
!
route-map IXP-RSv6-in permit 5
 description Local pref for IXP Peer
 match ipv6 address prefix-list IXP-v6RS
 set local-preference 150
 set community X0:1300
!
route-map private-peer-in permit 5
 description Local pref for Private Peer
 match ip address prefix-list ASY0-block
 set local-preference 200
 set community X0:1100
!
route-map private-peerv6-in permit 5
 description Local pref for Private Peer
 match ipv6 address prefix-list ASY0-v6block
 set local-preference 200
 set community X0:1100
!
router bgp X0
 address-family ipv4
  neighbor <ptp-v4> remote-as Y0
  neighbor <ptp-v4> description eBGP with ASY0
  neighbor <ptp-v4> password BGPlab
  neighbor <ptp-v4> prefix-list ASX0-block out
  neighbor <ptp-v4> route-map private-peer-in in
  neighbor 100.127.3.254 remote-as 65533
  neighbor 100.127.3.254 description eBGP with IXP RS
  neighbor 100.127.3.254 password ixp-rs
  neighbor 100.127.3.254 prefix-list ASX0-block out
  neighbor 100.127.3.254 route-map IXP-RS-in in
!
 address-family ipv6
  neighbor <ptp-v6> remote-as Y0
  neighbor <ptp-v6> description eBGP with ASY0
  neighbor <ptp-v6> password BGPlab
  neighbor <ptp-v6> prefix-list ASX0-v6block out
  neighbor <ptp-v6> route-map private-peerv6-in in
  neighbor 2001:DB8:FFFE:1::FE remote-as 65533
  neighbor 2001:DB8:FFFE:1::FE description eBGP with IXP RS
  neighbor 2001:DB8:FFFE:1::FE password ixp-rs
  neighbor 2001:DB8:FFFE:1::FE prefix-list ASX0-v6block out
  neighbor 2001:DB8:FFFE:1::FE route-map IXP-RSv6-in in
!

Our goal now is to replace the outbound prefix lists with a route-map which matches the community settings.

In this example, we want to replace the prefix lists called ASX0-block and ASX0-v6block:

ip prefix-list ASX0-block permit 100.68.X.0/24
ipv6 prefix-list ASX0-v6block permit 2001:DB8:X::/48

with route-maps for each of our three types of BGP neighbours. We will announce our address block community and any independent address space our customers bring - remember we defined those as community X0:1000 and X0:1005.

For our private peer we have:

route-map private-peer-out permit 5
 description Prefixes to our Private Peer
 match community aggregate
 match community customer-pi

For our IXP peers learned through the Route Server we have:

route-map IXP-RS-out permit 5
 description Prefixes to our IXP Peers
 match community aggregate
 match community customer-pi

For our upstream we create a new outbound route-map:

route-map Transit-out permit 5
 description Prefixes to our Transit Provider
 match community aggregate
 match community customer-pi

and then replace the relevant entries in the BGP configuration for the Border router (as in the sample below):

router bgp X0
 address-family ipv4
  neighbor <ptp-v4> route-map Transit-out out
  no neighbor <ptp-v4> prefix-list ASX0-block out
!
 address-family ipv6
  neighbor <ptp-v6> route-map Transit-out out
  no neighbor <ptp-v6> prefix-list ASX0-v6block out
!
no ip prefix-list ASX0-block permit 100.68.X.0/24
no ipv6 prefix-list ASX0-v6block permit 2001:DB8:X::/48
!

and Peering router (for Groups 1 to 4):

router bgp X0
 address-family ipv4
  neighbor <ptp-v4> route-map private-peer-out out
  no neighbor <ptp-v4> prefix-list ASX0-block out
  neighbor 100.127.1.254 route-map IXP-RS-out out
  no neighbor 100.127.1.254 prefix-list ASX0-block out
!
 address-family ipv6
  neighbor <ptp-v6> route-map private-peer-out out
  no neighbor <ptp-v6> prefix-list ASX0-v6block out
  neighbor 2001:DB8:FFFF:1::FE route-map IXP-RS-out out
  no neighbor 2001:DB8:FFFF:1::FE prefix-list ASX0-v6block out
!
no ip prefix-list ASX0-block permit 100.68.X.0/24
no ipv6 prefix-list ASX0-v6block permit 2001:DB8:X::/48
!

Note how we used the same route-map for both IPv4 and IPv6 BGP Peerings. This is because the route-map has no address family specific configuration in it - it is simply matching a BGP attribute that applies to both IPv4 and IPv6 address families.

Implement the route-refresh and then observe the changes to the BGP table - there should not be any change at all!

(Note that Cisco IOS does not send communities to eBGP neighbours unless that feature is turned on - we don’t need to do this here, as we are not using communities to signal anything to our neighbour networks - we are simply using them for our internal operational convenience.)

Now in future if we need to add any prefixes to our address announcements, we simply tag them with the correct community (ASN:1000) and that prefix will be automatically announced to all peers.

NOTE:

In the real operational Internet it is quite unlikely that anyone would use the identical outbound route-map for both transit, public and private peers which is why we gave each one different names here. It is more than likely that traffic engineering needs have to be considered, and different subnets being sent to upstreams, private and public peers, depending on the needs of customers. Which would mean the use of a route-map per peer (the more common case).

Conclusion

This lab has converted the policy implementation across the lab from a variety of prefix-lists and AS-PATH filters to using BGP communities. The only remaining prefix filters now are inbound on eBGP sessions. With prefix types in the network now identified by unique communities, it becomes simple for the network operator to scale the network infrastructure when adding more backbone routers, customers, peers and transit providers.