DEV Community

FirstPassLab
FirstPassLab

Posted on • Originally published at firstpasslab.com

Ditch the vPC Peer-Link: EVPN ESI Multi-Homing on Nexus 9000 with NX-OS 10.6.x (Config + Verification)

#networking #cisco #datacenter #tutorial

If you've ever wished you could multi-home a server to more than two leaf switches in your VXLAN EVPN fabric — or eliminate the vPC peer-link entirely — EVPN ESI multi-homing on NX-OS 10.6.x is the answer. It replaces Cisco's proprietary vPC control plane with standards-based BGP EVPN Type-1 and Type-4 routes, enabling multi-vendor interoperability and scaling beyond the traditional two-node limit.

This article walks through the architecture, a production-grade NX-OS configuration, and every verification command you need to confirm it's working.

Why ESI Multi-Homing Matters

vPC has served data center engineers well for over a decade. You pair two Nexus leaf switches, configure a vPC domain, and dual-home your servers. It works — but with well-known limitations:

  • Two-node limit: vPC is strictly a two-switch technology. You cannot multi-home a server to three or four leaf switches.
  • Peer-link dependency: The vPC peer-link must carry orphan traffic and synchronize MAC/ARP tables, adding complexity and consuming bandwidth.
  • Proprietary control plane: vPC uses a Cisco-specific mechanism (CFS over peer-keepalive and peer-link), which breaks multi-vendor interoperability.

EVPN multi-homing with ESI solves all three by moving the multi-homing intelligence into the EVPN control plane itself. Instead of a proprietary peer-link protocol, the leaf switches use BGP EVPN Type-1 (Ethernet Auto-Discovery) and Type-4 (Ethernet Segment) routes to coordinate forwarding, elect a Designated Forwarder (DF), and ensure loop-free traffic delivery.

Note: ESI multi-homing and vPC can coexist in the same NX-OS 10.6.x fabric. You can migrate incrementally — keep vPC for existing server connections and deploy ESI for new racks.

Core Concepts

Before diving into configuration, understand four key mechanisms:

Ethernet Segment Identifier (ESI)

An ESI is a 10-byte identifier that uniquely represents a multi-homed link bundle (e.g., a LAG connecting a server to two or more leaf switches). Every leaf switch participating in the same Ethernet Segment advertises the same ESI value via BGP EVPN, which is how remote VTEPs learn that multiple paths exist.

NX-OS 10.6.x supports both manually configured ESI values and auto-derived ESI values based on LACP system parameters. The auto-LACP approach eliminates the need to manually coordinate ESI values across leaf pairs.

EVPN Route Types 1 and 4

  • Type-1 (Ethernet Auto-Discovery per ES): Advertised by each leaf in the Ethernet Segment. Remote VTEPs use these to build a list of all VTEPs behind a given ESI, enabling aliasing (load balancing) and fast convergence on link failure.
  • Type-4 (Ethernet Segment Route): Used for Designated Forwarder (DF) election. All leaves in the ES exchange Type-4 routes, and a deterministic algorithm selects which leaf forwards BUM traffic per VLAN.

Designated Forwarder Election

When a broadcast or multicast frame arrives at the fabric, only one leaf per Ethernet Segment should forward it to the locally connected host. The DF election process ensures exactly one forwarder per VLAN per ES. NX-OS 10.6.x supports preference-based DF election.

Split Horizon and Aliasing

  • Split horizon prevents BUM traffic received from an ES member from being forwarded back to the same ES.
  • Aliasing allows remote VTEPs to load-balance unicast traffic across all leaves in an ES, even if the MAC was only learned on one of them.

Configuration: ESI Multi-Homing on NX-OS 10.6.x

We assume the underlay (OSPF or eBGP), NVE interface, and BGP EVPN overlay are already in place.

Step 1: Enable EVPN ESI Multi-Homing 

! Enable EVPN ESI multi-homing globally
nv overlay evpn
feature nv overlay
feature bgp
feature interface-vlan
feature vn-segment-vlan-based

evpn esi multihoming
  ethernet-segment 1
    identifier auto lacp
    designated-forwarder election type preference
      preference 32767
    route-target auto

! Associate the Ethernet Segment with a port-channel
interface port-channel10
  description ESI-to-Server-Rack1
  switchport
  switchport mode trunk
  switchport trunk allowed vlan 100,200
  ethernet-segment 1
  no shutdown
Enter fullscreen mode Exit fullscreen mode

This tells NX-OS to automatically derive the ESI from the LACP system ID and port-channel key. The preference 32767 sets this leaf as the preferred DF. On the partner leaf, configure the same Ethernet Segment with a lower preference (e.g., preference 16384).

Step 2: VXLAN Fabric Integration

Map VLANs to VNIs and configure anycast gateways:

vlan 100
  vn-segment 10100
vlan 200
  vn-segment 10200

fabric forwarding anycast-gateway-mac 0001.0001.0001

interface Vlan100
  no shutdown
  vrf member TENANT-1
  ip address 10.100.0.1/24
  fabric forwarding mode anycast-gateway

interface Vlan200
  no shutdown
  vrf member TENANT-1
  ip address 10.200.0.1/24
  fabric forwarding mode anycast-gateway

interface nve1
  no shutdown
  host-reachability protocol bgp
  source-interface loopback1
  member vni 10100
    ingress-replication protocol bgp
  member vni 10200
    ingress-replication protocol bgp
  member vni 50001 associate-vrf
Enter fullscreen mode Exit fullscreen mode

Step 3: BGP EVPN Overlay for ESI Routes

The BGP EVPN session to spine route reflectors carries Type-1 and Type-4 routes. Verify that extended communities are enabled:

router bgp 65001
  router-id 10.255.0.1
  neighbor 10.255.0.100
    remote-as 65001
    update-source loopback0
    address-family l2vpn evpn
      send-community extended
      send-community both
  neighbor 10.255.0.101
    remote-as 65001
    update-source loopback0
    address-family l2vpn evpn
      send-community extended
      send-community both

evpn
  vni 10100 l2
    rd auto
    route-target import auto
    route-target export auto
  vni 10200 l2
    rd auto
    route-target import auto
    route-target export auto
Enter fullscreen mode Exit fullscreen mode

Verification and Troubleshooting

Verify Ethernet Segment Status 

Leaf-1# show evpn esi
ESI: 0000.0000.0001.0001.0001
  Status: Up
  Interface: port-channel10
  DF election: Preference
  DF preference: 32767
  DF status: DF (elected)
  Peers:
    10.255.0.2 (preference 16384) - Non-DF
Enter fullscreen mode Exit fullscreen mode

Confirms Leaf-1 is elected as the Designated Forwarder. The partner leaf at 10.255.0.2 shows as Non-DF.

Verify BGP EVPN Type-1 and Type-4 Routes 

Leaf-1# show bgp l2vpn evpn route-type 1
BGP routing table information for VRF default, address family L2VPN EVPN
Route Distinguisher: 10.255.0.1:3
*>l[1]:[0000.0000.0001.0001.0001]:[0]/120
      10.255.0.1           100      0 i
*>i[1]:[0000.0000.0001.0001.0001]:[0]/120
      10.255.0.2           100      0 i

Leaf-1# show bgp l2vpn evpn route-type 4
BGP routing table information for VRF default, address family L2VPN EVPN
Route Distinguisher: 10.255.0.1:3
*>l[4]:[0000.0000.0001.0001.0001]:[10.255.0.1]/184
      10.255.0.1           100      0 i
*>i[4]:[0000.0000.0001.0001.0001]:[10.255.0.2]/184
      10.255.0.2           100      0 i
Enter fullscreen mode Exit fullscreen mode

Both local (*>l) and remote (*>i) Type-1 and Type-4 routes should appear with matching ESI values.

Verify Aliasing on Remote Leaves 

Remote-Leaf# show l2route evpn mac all | include 10100
Topology ID  Mac Address    Prod  Next Hop(s)
10100        aabb.cc00.0100 BGP   10.255.0.1, 10.255.0.2
Enter fullscreen mode Exit fullscreen mode

Two next-hop addresses confirms aliasing is working — unicast traffic gets load-balanced across both ESI member leaves.

Common Troubleshooting Scenarios

DF election not converging:

  • Check that both leaves have the same ESI configured (show evpn esi)
  • Verify Type-4 routes are being exchanged (show bgp l2vpn evpn route-type 4)
  • Confirm LACP is operational on both ends (show port-channel summary)

Duplicate BUM frames on the host:

  • DF election has failed and both leaves are forwarding BUM traffic
  • Verify designated-forwarder election type preference is configured consistently
  • Check for Type-4 route filtering on the spine route reflectors

MAC flapping on remote leaves:

  • Usually caused by an ESI mismatch — one leaf has the ESI configured while the other does not
  • Verify with show evpn esi on both leaves and ensure ESI values are identical

ESI vs. vPC: When to Use Each

Criteria vPC ESI Multi-Homing
Max leaf switches per group 2 2+ (standards allow more)
Control plane Proprietary (CFS) BGP EVPN (RFC 7432)
Multi-vendor support No Yes
Peer-link required Yes No
Maturity on NX-OS 10+ years NX-OS 10.6.x (new)
Incremental migration N/A Can coexist with vPC

For greenfield deployments on NX-OS 10.6.x, ESI multi-homing is the forward-looking choice. For brownfield environments with existing vPC domains, the coexistence capability lets you adopt ESI at your own pace.

Key Takeaways

  • EVPN ESI multi-homing replaces the proprietary vPC peer-link with standards-based BGP EVPN Type-1 and Type-4 routes, enabling multi-vendor interop and scaling beyond two-node pairs.
  • NX-OS 10.6.x supports both auto-LACP ESI derivation and manual ESI, with preference-based DF election for deterministic forwarding.
  • Aliasing ensures remote VTEPs load-balance across all ESI member leaves.
  • Coexistence with vPC makes incremental migration practical.
  • Key verification commands: show evpn esi, show bgp l2vpn evpn route-type 1, and show l2route evpn mac all.

Originally published at firstpasslab.com.

This article was adapted from original content with AI assistance. The technical configurations and verification outputs are based on NX-OS 10.6.x documentation and lab testing.

Top comments (0)