In a previous post I detailed how to setup ECMP with multiple ISP connections.
One of the issues I faced was managing the ISP supplied modem when it is bridge mode. Bridge mode essentially disables routing, NAT and passes the traffic to the DHCP request coming from your actual router (in my case this is the Palo Alto firewall). The ISP supplied modems are cheap and don’t have an out of band management interface (with a separate routing table). There is no way to add a default gateway, short of hacking the device which voids warranty and you run the risk of bricking the modem. Heres what I did to solve this issue:
1. Find a enterprise grade router that can handle NAT and supports multiple routed ports:
Ubiquiti Edge Router X
2. Setup the Ubiquiti Edge Router X with eth0 the management interface (in your management subnet) and change the default password.
2. Create a network management subnet just for bridge mode modems. Can be any size you want, but I went with a /29 which gives me x 2 /30s and 4 useable IPv4 addresses. For this example I used 192.168.19.128/29. This subnet should be routable via your campus WAN service.
3. The modems I was dealing with are Technicolor TG789VAC, they have a built in protection where if the source ip is not in the same subnet as the modems IP address, you get an access denied error message on the nginx web server. I suspect it is IPtables. So I made the modems IP address in the /29 range. In this example we have set the modem to 192.168.19.130/30 (the first /30 in the /29 block.
4. Set your layer 3 device to route the subnet to the Ubiquiti Edge Max device. Incoming Cisco IOS-XE syntax:
ip route 192.168.19.128 255.255.255.248 10.2.0.10
5. The edge router is setup in pure routed mode. Each routed interface will correspond to a modem in bridge mode. Configure Eth1 to 192.168.19.129. Eth2 could use the next /30 if you had a second bridged modem on the same campus LAN.
6. Now here comes to magic; set the NAT rule for the interface to masquarade (which is Unix speak for PAT, in Cisco speak its called ‘overload’).
7. Now you can manage the modem again at http://192.168.19.130. This helps when a copper technology like VDSL requires a modem reboot or line speed / status troubleshooting.
8. Below is a network diagram detailing what we have done to solve this issue:
As a side note I use Ubiquiti Network Management System to manage all our UBNT devices. Free software and works really well (device backups, email alerts, rich HTML 5 management).
Update (03/05/20). Edge Router X also works great with NBN FTTP which uses IPoE and FTTC which uses PPPoE. I deployed an ER-X for a Fibre to the Curb installation this week. The uplink to the NBN NTD requires a 802.1q trunk port. The port facing your router needs to be set with port Vlan ID ‘2’. The ER-X supports dot1q and port vlans. For FTTP the port does not need trunking or a PVID, simply DHCP is fine to grab the IP assigned to the circuit from the ISP.