Hydra 1303


All News Items

NSX Design: BGP is the Better Choice

from the Hydra High Council Dec 3rd 2019

When implementing a dynamic routing protocol within NSX, many organizations use OSPF instead of BGP. The usual argument against BGP is complexity, and it's true that BGP has the potential to be a bit overwhelming with so many different knobs to control traffic. These controls (path attributes) are made for peering with other organizations that you need to route traffic through, but don't trust in the same way as your internal network. Manipulating the path attributes gives you the control. Big picture: BGP is providing routing and control.

But with NSX, we're really just using BGP just for the routing part. If we're not bothering with all of those additional controls, BGP turns out to be quite simple. Who knew? BGP has the option to ignore all of those control options and just use it to route, but it turns out that OSPF is a stickler for rules and if we ignore them, OSPF breaks. You want your NSX network design to be simple and it turns out it's easier to get there with BGP. Let's look at some examples.

A Flood of 7s

OSPF natively supports different Area types for optimization. With a physical Cisco router as an example, there can be standard areas, stub areas, not-so-stubby areas, totally stubby areas, and totally not-so-stubby areas. (If you're not familiar with Cisco, these types sound made up. They're not.) However, NSX only supports two types: standard and NSSA (not-so-stubby area).

The problem with NSSA is that they cause type 7 NSSA Link State Advertisements (LSAs) to be flooded from NSX into the physical network. The LSAs get messy quickly. The DLR would receive all LSAs if it is configured with a standard area, but if it's connected to an NSSA area, all type 5 External LSAs are filtered out (the key benefit in non-standard areas is the reduction of LSAs flooding through the area). However, despite the DLR filtering out the type 5s, the Edge will still see them and there is no special filtering at the Edge. The bottom line is that NSX ends up generating type 7 LSAs into the physical network, which means there is an extra LSA for the infrastructure folks to deal with.

Priority 0 Gets You Stuck

OSPF has a way to optimize multiaccess networks like Ethernet (vs point-to-point). On a point-to-point link neighbors exchange their LSAs. It's simple. But imagine in the physical world a switch with 10 OSPF routers attached. Each would try to form adjacencies with each other, creating a full mesh (or mess since it would create 45 connections all doing the same job). Instead, a Designated Router (DR) and a Backup Designated Router (BDR) are chosen to reduce the number of adjacencies and all the repeat chatter. Imagine having 9 other neighbors all telling you the same story. With a DR and BDR, only the DR is telling the story, so to speak. We can control which router is DR, BDR, or not participating in the election at all by setting a priority. In the physical world, you can opt a router out by setting the priority to 0. However, with NSX, setting a priority of 0 on a device running OSPF will prevent it from forming an adjacency with neighbors altogether. It gets stuck in a 2-way state and can cause countless hours of troubleshooting if you don't know about that behavior.

All Hell Breaks Loose

Not being able to set a priority of 0 for an NSX OSPF router affects your design in another way. If you are running a single OSPF area and have both NSX and physical routers, you must use priorities and you must ensure that the NSX router never becomes the DR. If it does, all hell breaks loose.

NSX-T

The bottom line is that OSPF doesn't reduce complexity in NSX, it increases it. Hard lessons have been learned in the field with this, so much that when VMware designed the next evolutionary step in NSX (going from NSX-V to NSX-T) that they don't even support OSPF to avoid the potential issues.

Boiling it Down

  • BGP running in NSX is child's play.
  • It's simple BGP with just routes. Nothing too fancy regarding those controls other than the ability to leverage prefix lists for filtering.
  • Unless there is a technical reason that our client wants OSPF, we always go with BGP, and by the way, there is no technical reason.
  • The proof is in VMware's decision to only support BGP as a dynamic routing protocol in NSX-T. Use it.