Fortigate ECMP with BGP

October 11, 2020·

It’s like clock work…around this time the seasons are changing..autumn colors are out, and the colder air reminds us that things are changing around here, embrace it! One thing that I’ve been working with is AWS Transit gateways, a common theme this year is all cloud! AWS Transit Gateways make it easier to move towards that!

AWS Transit gateways relatively new, (couple years old now) so instead of attaching to each AWS VPC environment independently we can connect to the transit gateway and attach VPCs to the gateway. If you enable BGP your routes will propagate. I’ve been working with ECMP, specifically with Fortigates.

There are two ways to have redundancy in a network when you have multiple paths to a destination. Similar routes that have a higher cost/weight then ones that don’t are only used when there is a problem with primary path. So you have a backup route on standby until the primary route fails. This could be a good option if the backup route isn’t the same bandwidth/media or really not equal to the primary link which is why different cost/weight is important to have.

However if you have multiple paths that were equal bandwidth/media we could/would want to use both of them at the same time so that traffic is load balanced across the network, this is called ECMP (Equal-cost multi-path routing)

Let’s walk though the steps on how to setup ECMP on a Fortigate using BGP. I’m already guessing you have an AWS account with an AWS transit gateway deployed that supports ECMP, as well as configured your customer gateway in AWS with BGP Autonomous system (AS) number.

In this example I have a Fortigate Firewall running version 6.4.2, I have set the BGP “AS” number to 65000 and the AWS “AS” number is 64512.

Amazon gives you a text file to download the configuration for these two tunnels but depending on your Fortigate setup we may have to tweak the file, (not a good idea to just paste it) In this example here is what I have for tunnel one:

# Configure Phase 1 of the tunnel, aka IKE  
config vpn ipsec phase1-interface  
edit "AWS-0615bcc6f-1"  
set dpd on-idle  

# Outside Interface to reach AWS  
set interface outside  

# External IP address using (RFC 5737) in this example
set local-gw 198.51.100.1  
set dhgrp 2  
set proposal aes128-sha1  
set keylife 28800  

# Remote Peer this is AWS Transit, using (RFC 5737) in example`  
set remote-gw 192.0.2.124  
set psksecret ryansrealm.com  
set dpd-retryinterval 10  
next  
end  
  
# Configure Phase 2 of the Tunnel, aka IPSEC  
config vpn ipsec phase2-interface  
edit "AWS-0615bcc6f-1"  
set phase1name "AWS-0615bcc6f-1"  
set proposal aes128-sha1  
set dhgrp 2  
set pfs enable  
set keylifeseconds 3600  
end

# Build the tunnel interface  
config system interface   
edit "AWS-0615bcc6f-1"  
set vdom "root"  
set ip 169.254.150.142 255.255.255.255  
set allowaccess ping  
set type tunnel  
set tcp-mss 1379  
set remote-ip 169.254.150.141/32  
set mtu enable  
set mtu 1427  
set interface "outside"  
next  
end

After the tunnel one is created, AWS gives us configuration for the second tunnel, basically the same configuration with some differences, here is tunnel two.

# Configure Phase 1 of the tunnel, aka IKE  
  
config vpn ipsec phase1-interface  
edit "AWS-0615bcc6f-2"  
set dpd on-idle  
# Outside Interface to reach AWS  
set interface outside  
# External IP address using (RFC 5737) in this example  
set local-gw 198.51.100.1  
set dhgrp 2  
set proposal aes128-sha1  
set keylife 28800  
# Remote Peer this is AWS Transit, using (RFC 5737) in example  
set remote-gw 203.0.113.25  
set psksecret ryansrealm.com  
set dpd-retryinterval 10  
next  
end  
  
# Configure Phase 2 of the Tunnel, aka IPSEC  
  
config vpn ipsec phase2-interface  
edit "AWS-0615bcc6f-2"  
set phase1name "AWS-0615bcc6f-2"  
set proposal aes128-sha1  
set dhgrp 2  
set pfs enable  
set keylifeseconds 3600  
end  
  
# Build the tunnel interface  
config system interface  
edit "AWS-0615bcc6f-2"  
set vdom "root"  
set ip 169.254.157.90 255.255.255.255  
set allowaccess ping  
set type tunnel  
set tcp-mss 1379  
set remote-ip 169.254.157.89/32  
set mtu enable  
set mtu 1427  
set interface "outside"  
next  
end

We now have two tunnels configured, these won’t show online until the Fortigate has a policy attached to them. So you could put firewall rules on each tunnel interface or better yet is to create a zone within the Fortigate and add the two tunnel interfaces into same zone. By creating a zone within Fortigate the same firewall rules apply to each interface instead of creating/copying rules to each tunnel interface. (This would be an example)

config system zone  
edit AWS-0615bcc6f  
set interface AWS-0615bcc6f-1  
set interface AWS-0615bcc6f-2  
set intrazone allow  
next  
end

Once those interfaces are either in a zone and in a Firewall policy you should be able to bring those tunnel interfaces online. The final thing is configure our Fortigate with BGP so that when you attach a VPC to the transit gateway that IP space gets advertised automatically. We also want to enable ECMP with BGP.

# Enable BGP on the Fortigate  
  
config router bgp  
# This is the number we set in AWS, so it should match what TG has  
set as 65000   
set router-id `198.51.100.1`  
  
# Since we are using "AS" Numbers, we tell the Fortigate to use    
# External Multipath with out it BGP will use one route at time  
# Active/Passive  
  
set ebgp-multipath enable  
config neighbor  
edit "169.254.157.89"  
set remote-as 64512  
next  
edit "169.254.150.141"  
set remote-as 64512  
next  
end

If we attached a VPC to the transit gateway, and in this example I attached a VPC range of 10.0.0.0/16 to the transit gateway. That IP range would show up in the Fortigate, if we look at the routing table via get router info routing-table all

FW01 # get router info routing-table all  
Codes: K - kernel, C - connected, S - static, R - RIP, B - BGP  
O - OSPF, IA - OSPF inter area  
N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2  
E1 - OSPF external type 1, E2 - OSPF external type 2  
i - IS-IS, L1 - IS-IS level-1, L2 - IS-IS level-2, ia - IS-IS inter area  
- candidate default  
  
Routing table for VRF=0  
S* 0.0.0.0/0 [5/0] via `198.51.100.`25, ppp1  
B 10.0.0.0/16 [20/100] via 169.254.150.141, AWS-0615bcc6f-1, 03:07:57  
              [20/100] via 169.254.157.89, AWS-0615bcc6f-2, 03:07:57  
C `198.51.100.1`/32 is directly connected, ppp1  
C 169.254.150.141/32 is directly connected, AWS-0615bcc6f-1  
C 169.254.150.142/32 is directly connected, AWS-0615bcc6f-1  
C 169.254.157.89/32 is directly connected, AWS-0615bcc6f-2  
C 169.254.157.90/32 is directly connected, AWS-0615bcc6f-2

We can see that 10.0.0.0/16 is available to us via the VPN interfaces, we have equal paths as well, traffic will load balance between each tunnel interface. BGP with ECMP is alive and active! … That’s all I got for this post, like always I hope this information is helpful, got questions? Comment below!

It's Always Changing Ten Years