Compared to many of you my network is small and probably uninteresting, but it's mine, and lately I've been thinking I want to redesign our network routing. We have a core site and six edge sites, 5 here in the states and one overseas in Europe. About a year ago we finally switch from and old point-to-point WAN to an MPLS WAN provided by AT&T. This migration required us to use BGP on the edge routers which was really no big deal, but significantly impacted the implementation of our failover solution. After being unhappy for months with the current setup I'm considering switching to an all-BGP routing configuration. I'm interested in feedback if anyone out there has done this even in a fairly small network.
So with our previous point-to-point setup we used EIGRP and pretty much a flat network. Several of our sites had direct Internet connections and we used OpenVPN to create VPN tunnels between these sites for a "failover" network in the event of a primary link failure. These routes, along with the "default" route to the Internet for that site, were injected into the network by a Linux box running Quagga with OSPF and a neighbor Cisco router the redistributed the routes between OSPF and BGP. We also used the Linux box to monitor the local Internet connections.
This provided a great setup, if the Linux box detected that the local Internet connection was down, it would remove to default route, the router would know about default routes at other sites with higher metrics and the Internet would simply failover to using another sites connections. If the primary WAN link failed, those routes would be gone from the router, and the router would use the OSPF injected routers from the OpenVPN tunnel. Failover was reasonably quick (generally 30 seconds) and in a few cases sites ran for hours before they even noticed the primary link was down.
This became far more complicated when we switched to an MPLS WAN with BGP on the edge. We decided to dump EIGRP as a proprietary protcol and use only OSPF on the LAN and BGP on the WAN edge. Unfortunately, to achieve the desired failover results we needed to configure route redistribution between OSPF and BGP. This was once again easy enough, but during failover event we kept running into issues of routing loops because of our network topology in the core. We could easily use route tags to limit advertised routes, but as we fleshed out this configuration we started thinking that it seemed too complicated, there had to be an easier way.
After much thinking, reading, and discussion, we came to the conclusion that maybe the absolute best way to deal with this issue is to simply run BGP everywhere, LAN and WAN. With BGP we could easily use the local preference attribute to prefer a given path, and there would be no issues with complex route redistribution. In our head it seems crazy to run BGP in such a small network but, since we have to talk BGP with our WAN provider anyway, we though, why not? It seems like this would provide the simplest solution.
Is anyone else running BGP only for their entire network? We don't have any large, spread out campuses, generally a core which does all of the layer-3 routing, and an edge. That's it. I'll try to get some diagrams up soon.
Thursday, July 16. 2009 at 08:15 (Reply)
Thursday, February 25. 2010 at 13:56 (Reply)
Route redistribution ins't difficult. I'd be interested in hearing what parts you have problems with or where you feel the process is too complicated.
Also, from reading a few of your different articles, do you have anyone with a Professional level cert or higher (CCNX) on your team? If not, do you have a reason why?
Thursday, February 25. 2010 at 19:21 (Link) (Reply)
So, the above was simply me thinking out loud about whether an "internal" routing protocol is even required in most smaller environments. Certainly if you have a campus with many layer 3 devices then you'd want a routing protocol, but what I see at most smaller sites is an edge router, and a small layer 3 core, with may a handful of outlying layer 3 devices, but usually the access layer is still layer 2. Even with a layer 2 access, they'll still be running OSPF or EIGRP internally between the router and they're core layer-3 LAN device and then BGP on the edge for the MPLS provider. My question is, in this environment, why even bother with an internal routing protocol? If you think EIGRP or OSPF converge fast, just imagine how fast a static route converge! It's "instant" since there's no convergence at all.
Also, even though we opted to go with static routes internally, we actually technically still use "route redistribution". We simply redistribute the static routes into BGP, and use BGP for all peer connections between sites. BGP convergence time is a non-issue with this setup.
As far as having a "professional" level cert holder on our team, we currently do not. I have trained quite a number of network engineers in my 20 years in networking, and a good number of those have managed to achieve "professional" level Cisco, including several with CCIE level certs. Based on my track record of assisting others in preparing for these certs, I've never really felt the need to pursue one myself.
Monday, July 19. 2010 at 04:45 (Reply)