Page 1
Cisco NX-OS FCoE Configuration Guide for Nexus 7000 and MDS 9500 First Published: July 12, 2011 Last Modified: November 01, 2012 Americas Headquarters Cisco Systems, Inc. 170 West Tasman Drive San Jose, CA 95134-1706 http://www.cisco.com Tel: 408 526-4000 800 553-NETS (6387)
Page 2
HAVE BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGES. http:// Cisco and the Cisco logo are trademarks or registered trademarks of Cisco and/or its affiliates in the U.S. and other countries. To view a list of Cisco trademarks, go to this URL: www.cisco.com/go/trademarks .
FCoE and FIP FCoE Initiation Protocol FIP Virtual Link Instantiation FCoE Frame Format VLAN Tagging for FCoE Frames FIP Ethernet Frame Format DCBX Data Center Bridging Exchange Protocol Lossless Ethernet Cisco NX-OS FCoE Configuration Guide for Nexus 7000 and MDS 9500 OL-25809-02...
Page 4
Cisco MDS 9500: Guidelines and Limitations Configuring Virtual Interfaces Mapping a VSAN to a VLAN Creating a Virtual Fibre Channel Interface Creating a Virtual Fibre Channel Port Channel Interface Cisco NX-OS FCoE Configuration Guide for Nexus 7000 and MDS 9500 OL-25809-02...
Page 5
Cisco MDS 9500:Supported Features Guidelines and Limitations A P P E N D I X B Verified Limits for FCoE for the Cisco Nexus 7000 Series and Cisco MDS 9500 Cisco Nexus 7000: Guidelines and Limitations Cisco MDS 9500: Guidelines and Limitations...
Page 6
Contents Cisco NX-OS FCoE Configuration Guide for Nexus 7000 and MDS 9500 OL-25809-02...
Preface This preface describes the audience, organization, and conventions of the Cisco NX-OS FCoE Configuration Guide. It also provides information on how to obtain related documentation. • Audience, page vii • Document Conventions, page vii • Related Documentation, page viii •...
Caution or loss of data. Related Documentation The Cisco Nexus 7000 Series documentation is available at the following URL: http://www.cisco.com/en/us/products/ps9402/tsd_products_support_series_home.html The Cisco MDS 9500 Series documentation is available at the following URL: Cisco NX-OS FCoE Configuration Guide for Nexus 7000 and MDS 9500...
Obtaining Documentation and Submitting a Service Request For information on obtaining documentation, submitting a service request, and gathering additional information, see the monthly What's New in Cisco Product Documentation, which also lists all new and revised Cisco technical documentation, at: http://www.cisco.com/en/US/docs/general/whatsnew/whatsnew.html...
Page 10
Preface Obtaining Documentation and Submitting a Service Request Cisco NX-OS FCoE Configuration Guide for Nexus 7000 and MDS 9500 OL-25809-02...
The latest version of this document is available at the following Cisco website: http://www.cisco.com/en/US/products/ps9670/products_installation_and_configuration_guides_list.html To check for additional information about Cisco Nexus 7000 software, see the Cisco Nexus 7000 Series NX-OS Release Notes available at the following Cisco website: http://www.cisco.com/en/US/products/ps9670/prod_release_notes_list.html...
The latest version of this document is available at the following Cisco website: http://www.cisco.com/en/US/products/ps9670/products_installation_and_configuration_guides_list.html To check for additional information about Cisco MDS 9500 software, see the Cisco MDS 9000 Series NX-OS Release Notes available at the following Cisco website: http://www.cisco.com/en/US/products/ps9670/prod_release_notes_list.html...
Native Fibre Channel implements a lossless service at the transport layer using a buffer-to-buffer credit system. Ethernet links on Cisco NX-OS switches provide two mechanisms to ensure lossless transport for FCoE traffic: link-level flow control and priority flow control.
Bridge also encapsulates FC frames from a Fibre Channel network and forwards them to the FCoE network. Cisco MDS 9500 switches support Fibre Channel Bridge functions. The Cisco Nexus 7000 series switches do not offer native Fibre Channel ports and do not support FC Note Bridge functions.
The FCoE Initialization Protocol (FIP) allows the switch to discover and initialize FCoE-capable entities that are connected to an Ethernet LAN. Cisco NX-OS switches support the Converged Enhanced Ethernet Data Center Bridging Exchange (CEE-DCBX) protocol for T11-compliant Generation 2 CNAs.
VLAN. Frames that are not correctly tagged are discarded. Note You cannot map VLAN 1 or the native VLAN to an FCoE VSAN. The following switches do not support Pre-FIP on Generation 1 CNAs: Cisco NX-OS FCoE Configuration Guide for Nexus 7000 and MDS 9500 OL-25809-02...
(LLDP). DCBX end points exchange request and acknowledgment messages. For flexibility, parameters are coded in a type-length-value (TLV) format. Cisco NX-OS switches support the Converged Enhanced Ethernet Data Center Bridging Exchange (CEE-DCBX) is supported on all T11-compliant Generation 2 CNAs.
• FCoE software stack ◦ Runs on existing 10-Gigabit Ethernet adapters. The following Cisco NX-OS series and platforms support Generation 2 CNAs that use the FIP to exchange information about its available capabilities and to negotiate the configurable values with the switch: •...
Advanced Services License. For a complete explanation of the Cisco NX-OS licensing scheme and how to obtain and apply licenses, see the Cisco NX-OS Licensing Guide. Cisco NX-OS FCoE Configuration Guide for Nexus 7000 and MDS 9500 OL-25809-02...
• You cannot enable FCoE on default VLAN. • The QoS policy must be the same on all Cisco FCoE switches in the network. • Beginning with Cisco NX-OS Release 6.1, FCoE is supported on F2 and F2e Series modules.
• mgmt0_ip_address is the mgmt0 IPv4 address. • mgmt0_subnet_mask is the mgmt0 IPv4 netmask. • default_gateway is the IPv4 address of the default-gateway. For more information about VDC, see the Cisco Nexus 7000 Series NX-OS Virtual Device Context Configuration Guide. Cisco MDS 9500: Guidelines and Limitations •...
FCoE Configuring FCoE • In Cisco MDS NX-OS Release 5.2(x), you cannot install a FCoE module in a switch that is running DMM, SME, or IOA. Configuring FCoE Enabling FCoE Cisco Nexus 7000: Enabling FCoE You must install the FCoE feature set in the default VDC and enable dependent features in order to enable FCoE in a storage VDC.
You must create a storage VDC and allocate resources to finish enabling FCoE. Cisco MDS 9500: Enabling FCoE The Cisco MDS 9500 switch automatically enables the FCoE feature when an FCoE-capable module is present in the chassis. You must enable supporting features and the appropriate QoS policy for FCoE.
Page 24
Displays information about the enabled features. Example: switch(config-sys-qos)# show feature Step 6 copy running-config startup-config (Optional) Copies the running configuration to the startup configuration. Example: switch(config-sys-qos)# copy running-config startup-config Cisco NX-OS FCoE Configuration Guide for Nexus 7000 and MDS 9500 OL-25809-02...
The command must clean up all Example: of the configuration associated with the FCoE feature set. switch(config-vdc)# no feature-set fcoe Cisco NX-OS FCoE Configuration Guide for Nexus 7000 and MDS 9500 OL-25809-02...
The command must clean up all of the configuration Example: associated with the FCoE feature set. switch(config)# no feature-set fcoe Cisco NX-OS FCoE Configuration Guide for Nexus 7000 and MDS 9500 OL-25809-02...
For Cisco MDS 9500 switches, start at configure terminal below . Note Before You Begin For Cisco Nexus 7000 Series, you must be in the storage VDC to configure this feature. SUMMARY STEPS 1. (Cisco Nexus 7000) switchto vdc vdc-id type storage 2.
Example: default value. fcoe(config)# fcoe fcf-priority 42 This example shows how to configure the global fabric priority for a Cisco Nexus 7000 Series switch: switch# switchto vdc fcoe type storage fcoe# configure terminal fcoe(config)# fcoe fcf-priority 42...
8 seconds. The range is from 4 to 60 seconds. Example: fcoe(config)# fcoe fka-adv-period 8 fcoe# This example shows how to configure the advertisement interval for the fabric on a Cisco Nexus 7000 Series switch: switch# switchto vdc fcoe type storage fcoe# configure terminal fcoe(config)# fcoe fka-adv-period 42...
FCoE Disabling LAN Traffic on an FCoE Link This example shows how to configure the advertisement interval for the fabric on a Cisco MDS 9500 switch: switch# configure terminal switch(config)# fcoe fka-adv-period 42 Disabling LAN Traffic on an FCoE Link You can disable LAN traffic on an FCoE link.
This example shows how to display the FCoE settings for an interface. switch# show interface ethernet 1/37 fcoe Ethernet1/37 is FCoE UP vfc3 is Up FCID is 0x490100 PWWN is 21:00:00:1b:32:0a:e7:b8 MAC addr is 00:c0:dd:0e:5f:76 Cisco NX-OS FCoE Configuration Guide for Nexus 7000 and MDS 9500 OL-25809-02...
Notices), the Cisco Technical Services Newsletter, and Really Simple Syndication (RSS) Feeds. Access to most tools on the Cisco Support website requires a Cisco.com user ID and password. Cisco NX-OS FCoE Configuration Guide for Nexus 7000 and MDS 9500 OL-25809-02...
A virtual fabric (VF) port in an FCoE network acts as a fabric port that connects to a peripheral device (host or disk) operating as an N port. A VF port can be attached to only one N port. Cisco NX-OS FCoE Configuration Guide for Nexus 7000 and MDS 9500 OL-25809-02...
FCoE does not require a license. For a complete explanation of the Cisco NX-OS licensing scheme and how to obtain and apply licenses, see the Cisco NX-OS Licensing Guide. Cisco NX-OS FCoE Configuration Guide for Nexus 7000 and MDS 9500 OL-25809-02...
• You cannot enable FCoE on default VLAN. • The QoS policy must be the same on all Cisco FCoE switches in the network. • Beginning with Cisco NX-OS Release 6.1, FCoE is supported on F2 and F2e Series modules.
• SME does not support any FCoE connected devices including devices connected through the MDS FCoE linecard (DX-X9708-K9) • In Cisco MDS NX-OS Release 5.2(x), you cannot install a FCoE module in a switch that is running DMM, SME, or IOA.
(VSAN) in the SAN (for example, VLAN 1002 for VSAN 10, VLAN 1003 for VSAN 2, and so on). If you enable MST, you must use a separate Multiple Spanning Tree (MST) instance for FCoE VLANs. You must exit VLAN mode to execute the configured commands on the Cisco Nexus 7000 Series Switches. Note Before You Begin •...
This example shows how to map VLAN 200 to VSAN 200 on a Cisco MDS 9500 Series Switch: switch(config)# vlan 200 switch(config-vlan)# fcoe vsan 200 This example shows how to map VLAN 300 to VSAN 300 on a Cisco Nexus 7000 Series Switches: switch(config)# switchto vdc fcoe_vdc switch-fcoe_vdc# configure terminal...
Page 39
Displays information about the virtual Fibre Channel interfaces. Example: switch(config-if)# show interface vfc Step 6 (Optional) copy running-config startup-config Copies the running configuration to the startup configuration. Example: switch(config)# copy running-config startup-config Cisco NX-OS FCoE Configuration Guide for Nexus 7000 and MDS 9500 OL-25809-02...
Before You Begin • For the Cisco Nexus 7000 Series, ensure that you create the port channel interface before you create the virtual Fibre Channel port channel interface. • For the Cisco MDS 9500 switches, ensure that you create the Ethernet port channel interface before you create the virtual Fibre Channel port channel interface.
Fabric (VSAN) in the SAN (for example, VLAN 1002 for VSAN 1, VLAN 1003 for VSAN 2, and so on). If you enable MST, you must use a separate MST instance for FCoE VLANs. Before You Begin For Cisco Nexus 7000 Series Switches, ensure that you are in the storage VDC. SUMMARY STEPS 1. configure terminal 2.
Page 42
This example shows how to associate a virtual Fibre Channel interface to a VSAN: switch# configure terminal switch(config)# vsan database switch(config-vsan-db)# vsan 2 interface vfc 4 Cisco NX-OS FCoE Configuration Guide for Nexus 7000 and MDS 9500 OL-25809-02...
For Cisco MDS 9500 switches, start at Step 2, on page 31 Before You Begin For Cisco Nexus 7000 Series, you must be in the storage VDC to configure this feature. SUMMARY STEPS 1. (Cisco Nexus 7000) switchto vdc vdc-id type storage 2.
6. (Optional) Display the interface information for the virtual Fibre Channel interface. DETAILED STEPS Step 1 Enable the associated VLAN and map the VLAN to a VSAN. For Cisco Nexus 7000, ensure you are in the storage VDC. switch(config)# vlan 200 switch(config-vlan)# fcoe vsan 200...
Configuring FCoE VLANs and Virtual Interfaces Additional References for FCoE Step 3 Create a virtual Fibre Channel interface and bind it to a physical Ethernet interface. For Cisco Nexus 7000, ensure you are in the storage VDC. switch(config)# interface vfc 4...
Page 47
Notices), the Cisco Technical Services Newsletter, and Really Simple Syndication (RSS) Feeds. Access to most tools on the Cisco Support website requires a Cisco.com user ID and password. Cisco NX-OS FCoE Configuration Guide for Nexus 7000 and MDS 9500 OL-25809-02...
Page 48
Configuring FCoE VLANs and Virtual Interfaces Additional References for FCoE Cisco NX-OS FCoE Configuration Guide for Nexus 7000 and MDS 9500 OL-25809-02...
If the Ethernet VDC restarts or is suspended, any shared Ethernet ports are shut down in the corresponding VDC. These ports come up automatically once the Ethernet VDC is operational. Cisco NX-OS FCoE Configuration Guide for Nexus 7000 and MDS 9500 OL-25809-02...
• You cannot enable FCoE on default VLAN. • The QoS policy must be the same on all Cisco FCoE switches in the network. • Beginning with Cisco NX-OS Release 6.1, FCoE is supported on F2 and F2e Series modules.
• mgmt0_ip_address is the mgmt0 IPv4 address. • mgmt0_subnet_mask is the mgmt0 IPv4 netmask. • default_gateway is the IPv4 address of the default-gateway. For more information about VDC, see the Cisco Nexus 7000 Series NX-OS Virtual Device Context Configuration Guide. Cisco MDS 9500: Guidelines and Limitations •...
• SME does not support any FCoE connected devices including devices connected through the MDS FCoE linecard (DX-X9708-K9) • In Cisco MDS NX-OS Release 5.2(x), you cannot install a FCoE module in a switch that is running DMM, SME, or IOA.
Page 53
Switches to the storage VDC. Example: switch(config-vdc)# switchto vdc fcoe-vdc switch-fcoe-vdc# Step 11 (Optional) show feature-set Displays information about the feature-sets in this VDC. Example: switch-fcoe-vdc# show feature-set Cisco NX-OS FCoE Configuration Guide for Nexus 7000 and MDS 9500 OL-25809-02...
Displays information about the VLAN range allocated for FCoE. Example: switch(config-vdc)# show fcoe-vlan-range Step 5 copy running-config startup-config (Optional) Copies the running configuration to the startup configuration. Example: switch(config-vdc)# copy running-config startup-config Cisco NX-OS FCoE Configuration Guide for Nexus 7000 and MDS 9500 OL-25809-02...
8. (Optional) show vdc shared membership 9. (Optional) switchto vdc vdc-name 10. configure terminal 11. feature lldp 12. interface if-range 13. no shutdown 14. (Optional) show interface if-range 15. (Optional) copy running-config startup-config Cisco NX-OS FCoE Configuration Guide for Nexus 7000 and MDS 9500 OL-25809-02...
Page 57
Displays the interfaces that are shared for FCoE. Example: switch(config-vdc)# show vdc shared membership Step 9 switchto vdc vdc-name (Optional) Switches to the storage VDC. Example: switch(config-vdc)# switchto vdc fcoe-vdc switch-fcoe-vdc# Cisco NX-OS FCoE Configuration Guide for Nexus 7000 and MDS 9500 OL-25809-02...
!Create Storage VDC and allocate resources: switch(config-if)# install feature-set fcoe switch(config)# vdc fcoe_vdc type storage Cisco NX-OS FCoE Configuration Guide for Nexus 7000 and MDS 9500 OL-25809-02...
Page 59
!Switch to storage VDC and bring up the shared interface: switch(config-vdc)# switchto vdc fcoe_vdc switch-fcoe_vdc# configure terminal switch-fcoe_vdc(config)# feature lldp switch-fcoe_vdc(config)# interface ethernet 2/1 switch-fcoe_vdc(config-if)# no shutdown Ethernet 2/1 must be from an F-series module. Note Cisco NX-OS FCoE Configuration Guide for Nexus 7000 and MDS 9500 OL-25809-02...
Page 60
Storage VDC on Cisco Nexus 7000 Series Switches Example: Storage VDC Configuration Cisco NX-OS FCoE Configuration Guide for Nexus 7000 and MDS 9500 OL-25809-02...
Supports F2 Series modules. FCoE 6.1(1) Supports F2e Series modules (SFP+ only). Cisco MDS 9500:Supported Features Feature Release Feature Information 5.2(1) Supports the T11-compliant FIP. FCoE 5.2(1) This feature was introduced. Cisco NX-OS FCoE Configuration Guide for Nexus 7000 and MDS 9500 OL-25809-02...
Page 62
Feature History by Platform Cisco MDS 9500:Supported Features Cisco NX-OS FCoE Configuration Guide for Nexus 7000 and MDS 9500 OL-25809-02...
A P P E N D I X Guidelines and Limitations This chapter includes the following sections: • Verified Limits for FCoE for the Cisco Nexus 7000 Series and Cisco MDS 9500 , page 51 • Cisco Nexus 7000: Guidelines and Limitations, page 51 •...
Page 64
• mgmt0_ip_address is the mgmt0 IPv4 address. • mgmt0_subnet_mask is the mgmt0 IPv4 netmask. • default_gateway is the IPv4 address of the default-gateway. Cisco NX-OS FCoE Configuration Guide for Nexus 7000 and MDS 9500 OL-25809-02...
• SME does not support any FCoE connected devices including devices connected through the MDS FCoE linecard (DX-X9708-K9) • In Cisco MDS NX-OS Release 5.2(x), you cannot install a FCoE module in a switch that is running DMM, SME, or IOA.