Page 2
MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. USE, COPYING, AND DISTRIBUTION OF ANY DELL SOFTWARE DESCRIBED IN THIS PUBLICATION REQUIRES AN APPLICABLE SOFTWARE LICENSE. Dell, EMC, and other trademarks are trademarks of Dell Inc. or its subsidiaries. Other trademarks may be the property of their respective owners. Published in the USA.
CONTENTS Figures Tables Preface Chapter 1 Introduction Overview.....................12 VxFlex OS component requirements............12 VxFlex OS cluster components............12 Physical server requirements............13 Supported operating systems............13 VxFlex OS Gateway server requirements........13 Prerequisites....................14 Connectivity...................14 Java....................15 Hardware and operating systems........... 15 VxFlex OS packages...............15 Supported Internet browsers............
Page 4
Where to go from here................66 Chapter 9 Reference material DTK - Hardware Update Bootable ISO............68 Dell OpenManage DRAC Tools (RACADM)........68 Recommended BIOS and firmware settings........70 Troubleshooting the Hardware ISO..........73 Additional resources..............75 Hardware Configuration and Operating System Installation Guide - ESXi Servers...
FIGURES vSphere client, with Power off option selected............28 Networking pane - VMkernel NICs tab............... 29 Networking pane - Port groups tab................29 R640 and R740xd PCI slots, integrated NICs and BMC port locations......43 Hardware Configuration and Operating System Installation Guide - ESXi Servers...
Page 6
FIGURES Hardware Configuration and Operating System Installation Guide - ESXi Servers...
TABLES Server physical requirements..................13 Supported operating systems -VxFlex OS components..........13 VxFlex OS management IP network................37 VxFlex OS data IP network for Subnet #1..............38 VxFlex OS data IP network for Subnet #2..............38 R740xd server configuration for 10/25 GbE, 32xSAS, SFP+/SFP28......44 R740xd server configuration for 10/25GbE, 12xNVMe, SFP+/SFP28......
Page 8
TABLES Hardware Configuration and Operating System Installation Guide - ESXi Servers...
Preface As part of an effort to improve its product lines, Dell EMC periodically releases revisions of its software and hardware. Therefore, some functions described in this document might not be supported by all versions of the software or hardware currently in use.
Page 10
EMC products, go to Dell EMC Online Support at https://support.emc.com. Technical support Go to Dell EMC Online Support and click Service Center. You will see several options for contacting Dell EMC Technical Support. Note that to open a service request, you must have a valid support agreement. Contact your Dell EMC sales representative for details about obtaining a valid support agreement or with questions about your account.
CHAPTER 1 Introduction The following topics describe the prerequisites for configuring the hardware and the operating system in a Dell EMC VxFlex Ready Node environment. Overview......................12 VxFlex OS component requirements..............12 Prerequisites.......................14 Supported hardware configurations..............16 Introduction...
Introduction Overview To prepare a server to be part of a VxFlex Ready Node environment, you are required to perform the following steps. 1. Ensure that the environment meets the prerequisites. 2. Configure the hardware, including the BIOS, BMC (iDRAC) IP address, and ISO installation.
Introduction Physical server requirements Table 1 Server physical requirements Component Requirement Port specifications The following TCP ports are not used by any other application, and are open in the local firewall of the server: MDM: 6611 and 9011 SDS: 7072 (for multiple SDS, ports 7073-7076) VxFlex OS Gateway (includes REST Gateway, VxFlex OS Installation Manager, and SNMP trap sender): 80 and 443 Light Installation Agent (LIA): 9099...
Introduction Component Requirement You can change the default ports. For more information, see “Changing default ports” in the user documentation. Supported web browsers Internet Explorer 10, or later Firefox, version 42, or later Chrome, version 45, or later Java v1.8 (64-bit), build 149 or earlier. You can download previous versions from this link: http://www.oracle.com/technetwork/ java/javase/downloads/java-archive-javase8-2177648.html...
Introduction Ensure that you have either a VGA tool kit to allow console connection from a laptop computer to a server, or a computer screen and keyboard connection to the rack. Java These Java versions are required: Required version VxFlex OS GUI Java 1.8 latest update BMC (iDRAC) management client (Java Java 1.7 latest update...
Introduction Supported hardware configurations The supported hardware configurations are listed in the EMC Simple Support Matrix (ESSM) for your software version. Download the ESSM from https://elabnavigator.emc.com/eln/elnhome. Click Simple Support Matrices Storage and select the desired product and version. > The following points apply, as well: Boot drive - All configurations have two 120/240 GB BOSS boot drives (configured as RAID 1).
CHAPTER 2 Configuring the Hardware The following topics describe how to configure the hardware, set BMC (iDRAC) IP addresses, and map the ISO for servers in a VxFlex Ready Node environment. Set up the BMC (iDRAC) IP address and BIOS...........18 Verify the status of the system hardware, storage controller, and disks - 14G servers.......................
Configuring the Hardware Set up the BMC (iDRAC) IP address and BIOS Set up the BMC (iDRAC) IP address and set up or validate the BIOS on the VxFlex Ready Node servers. Before you begin Ensure that you have access to, or have the details for: The KVM console The server BMC (iDRAC) IP address The server BMC (iDRAC) subnet mask...
Page 19
Configuring the Hardware Enable IP IPv4 = Enabled Enable DHCP = Disabled Static IP Address = Static IP address Static Gateway = Gateway IP address Static Subnet Mask = Subnet mask IP address d. From the IPv6 Settings pane, configure the IPv6 parameter values for the BMC (iDRAC) port.
Procedure 1. From a browser, go to http:// <BMC/iDRAC_IP_address> . 2. In the DELL Console Login window, type the user name and password, and then click Login. The Dashboard displays the high-level status of all hardware devices in the System Health pane.
Procedure <iDRAC_IP_address> . 1. From your Internet browser, go to https:// 2. In the DELL Console Login window, type the user name and password, and click Login. 3. From the dashboard, click Launch Virtual Console to start a console session.
Updating the BIOS, firmware and settings VxFlex Ready Node deployments require specific versions of drivers, BIOS, and firmware that have been qualified by Dell EMC. If the servers do not have the correct versions, you must update them. A variety of factors can influence a mismatch between the required versions and the versions installed on the servers, such as firmware updates post server shipment, or a FRU replacement with a different firmware version than in the warehouse.
Configuring the Hardware Update the hardware using the iDRAC virtual console The iDRAC Virtual KVM console and Virtual CDROM features, provided by the iDRAC Enterprise license, eliminate the need for physical access to the VxFlex Ready Node servers. The Hardware ISO can be attached as a remote/Virtual CDROM image and configured to perform hardware updates to the VxFlex Ready Node firmware.
Page 24
12. For each VxFlex Ready Node server, after the updates are finalized, clear the iDRAC job queue using the iDRAC GUI: a. From your Internet browser, go to https:// <iDRAC_IP_address> . b. In the DELL Console Login window, type these credentials: username: root <password> (for Dell-supplied nodes) or password (for EMC- password: supplied nodes) c.
CHAPTER 3 Installing the Operating System The following topics describe the procedures for installing the operating system on a VxFlex Ready Node server. System requirements..................26 Driver support....................26 Rebuild the M.2 cards as a RAID device............. 26 Revert the AMS ESXi ISO Image to a native ESXi Image........27 Map the ISO file on a VxFlex Ready Node 14G server.........30...
Ensure that correct drivers are installed in your system. The best source for supported storage drivers is the Dell customized ISO for ESXi 6.5 U1. If this cannot be used, consult the VMware support matrix for Dell products on the VMware website.
Installing the Operating System 4. On the BOSS-S1 Configuration Utility screen, select Physical/Virtual Disk Information to verify the disk configuration. 5. On the BOSS-S1 Configuration Utility > Operation Menu screen, select Physical Disk Info as the Device Output Type, and then press Enter. The physical disk information should display two expansion card devices configured to the server.
Installing the Operating System 5. If a Remember Password popup appears, click Never remember password for this site. 6. If the Help us improve the VMware Host Client dialog appears, clear the Join CEIP check box, and then click OK. The vSphere client main screen appears.
Installing the Operating System 12. Right-click ScaleIO_Internal and select Remove. 13. In the confirmation dialog box, click Remove. 14. In the right pane, in the Vmkernel NICs tab, right-click vmk1 and select Remove. Figure 2 Networking pane - VMkernel NICs tab 15.
5. To verify the ISO file selection, click Virtual Media and view the ISO selection in the drop down list. 6. Reset the server: a. In the main DELL System Setup console window, select Power Reset > System (warm boot) to display the list of keyboard keys with related functionality.
Page 31
Installing the Operating System IP address of VMware vCenter server User name and password for accessing the vCenter with the vSphere Web Client Before deploying VxFlex OS systems, you must configure VMware security features. This procedure describes how to do so with the vSphere web client. Note Ensure that the remote connection is established within 1-2 minutes, otherwise, the lockdown mode is enabled on the ESXi servers, preventing you from subsequent...
Page 32
Installing the Operating System c. Using the Search ( ) field, filter the following security parameters: Set Security.AccountLockFailures = The parameter value defines the maximum number of failed login attempts allowed, before locking out the user's account. The zero (0) value disables account locking.
Page 33
Installing the Operating System Note The parameter option of Lockdown Mode must be Disabled. If it is not, Enable to Disable , click Edit change it. To change the parameter option from in the Lockdown Mode pane and change the parameter value to Disable .
Page 34
Installing the Operating System Hardware Configuration and Operating System Installation Guide - ESXi Servers...
CHAPTER 4 Network Architecture and Physical Connectivity The following topics provide networking requirements and connectivity information for VxFlex Ready Node. Networking connectivity architecture and cabling best practice (R640, R740XD servers)......................36 Network prerequisites..................36 Network Architecture and Physical Connectivity...
Network Architecture and Physical Connectivity Networking connectivity architecture and cabling best practice (R640, R740XD servers) This topic describes connectivity architecture, cabling best practice information, and cable connection examples from typical VxFlex Ready Node configurations in order to help you plan your network. Note If you are not familiar with VxFlex Ready Node system architecture, refer to the "Architecture"...
Network Architecture and Physical Connectivity Before you begin connecting the switches, ensure that they are installed and configured. Do not connect the switches to your organization's network until all installation and configuration activities are complete. Note Each node has at least four 10GbE or two 25 GbE ports. Two are used for internal network communications.
Network Architecture and Physical Connectivity Note If the AMS is not on the same subnet as the SVM management IP addresses, the Discovery server used as a proxy must reside on the same switch network broadcast domain (Physical or VLAN), in order to allow seamless detection and MGMT IP address configuration of the SVM and ESXi IP addresses of all nodes in the system.
Page 39
Network Architecture and Physical Connectivity Note The subnet mask and default gateway of the data networks cannot be modified after initial configuration. If you plan to have 1024 nodes in your VxFlex OS system, plan for a subnet allowing 1025 IP addresses for SVM, and 1024 IP addresses for hosts per data network.
Page 40
Network Architecture and Physical Connectivity Hardware Configuration and Operating System Installation Guide - ESXi Servers...
CHAPTER 5 Port Configuration The following topics describe the port designations for VxFlex Ready Node servers and provide instructions for configuring the ports. Determine the 2U1N node type................42 R640 and R740xd server NIC and BMC (iDRAC) port locations......42 VxFlex Ready Node R640 and R740xd port designations - ESXi......
Procedure 1. From your Internet browser, go to http:// <BMC/iDRAC_IP_address> . 2. In the DELL Console Login window, type the user name and password, and click Login. 3. In the navigation pane, select System Inventory Hardware inventory, and >...
Port Configuration Figure 4 R640 and R740xd PCI slots, integrated NICs and BMC port locations Integrated (iDRAC) NICs iDRAC Integrated (iDRAC) NICs Data cables are connected to two switches for high availability, via the nodes' 10 GbE, 25 GbE, or 100 GbE ports. Management interfaces are connected to a switch on a separate management network using onboard and BMC (iDRAC) ports.
Port Configuration If there are only 2*25GbE onboard ports, the left port is used for both the management and data network, and the right port for is used for data. Note The application\client network will have to run on those same two ports. If there are only 4*10G onboard ports: The left port is used for management The left port, and the third port from the left are used for VxFlex OS data...
Port Configuration Table 7 R740xd server configuration for 10/25GbE, 12xNVMe, SFP+/SFP28 Description VxFlex OS VxFlex OS VxFlex OS VxFlex VxFlex OS Not in Use (iDRAC) Data1 Client Client Managemen Monitoring Network Network Data2 t Network Port1 Port2 Port BMC (iDRAC) vmnic0 vmnic1 vmnic4...
Port Configuration Table 9 R740xd server configuration for 10GbE, 3xGPU (M60)/ 2xGPU (M10), SFP+ Description VxFlex OS VxFlex OS VxFlex OS VxFlex VxFlex OS Not in Use (iDRAC) Data1 Client Client Managemen Monitoring Network Network Data2 t Network Port1 Port2 Port BMC (iDRAC) vmnic0...
Port Configuration Table 10 R640 server configuration for 10/25GbE, 1CPU, SFP+/SFP28 (continued) Description VxFlex OS VxFlex OS VxFlex OS VxFlex VxFlex OS Not in Use (iDRAC) Data1 Client Client Managemen Monitoring Network Network Data2 t Network Port1 Port2 VMkernel port vmk2 User User...
Page 48
Port Configuration 4. Select Configure Management Network, and press Enter. 5. In the Configure Management Network window, select Network Adapters, and press Enter. 6. Using the Up/Down arrow keys, scroll to the appropriate vmnic for your hardware (the management port vmnic differs per server type: 1U1N, 2U1N Storage-Only, 2U1N Converged, etc.) 7.
CHAPTER 6 Installing the drivers The following topics contain information regarding VxFlex OS drivers. Install the VxFlex OS drivers................50 Installing the drivers...
Installing the drivers Install the VxFlex OS drivers It is recommended that you install all applicable VxFlex OS drivers in preparation for installing the VxFlex OS software. Procedure 1. Download the VxFlex OS driver Zip file from https://support.emc.com/ products/42216 to a Windows-based server. 2.
Page 51
Installing the drivers 10. After the system has rebooted, view the results of the previous steps: esxcli software profile get 11. Verify that the correct versions of drivers are running: a. Get the list of drivers currently installed on the server: esxcli software vib list b.
Page 52
Installing the drivers Hardware Configuration and Operating System Installation Guide - ESXi Servers...
CHAPTER 7 Preparing Disks The following topics describe how to prepare disks so that they can be added to VxFlex OS SDS devices. Adding disks to a VxFlex OS SDS............... 54 Verify the disk controller type................54 Enable PERCCLI on an ESXi server..............55 Create virtual devices with PERCCLI..............
Procedure <BMC/iDRAC_IP_address> . 1. From your Internet browser, go to http:// 2. In the DELL Console Login window, type the user name and password, and click Login. The Integrated Remote Access Controller screen appears. 3. In the navigation pane, select Storage Controllers.
Root access to the server on which PERCCLI is to be installed Access to the Internet to enable you to download the PERCCLI utility Procedure 1. Download PERCCLI from http://downloads.dell.com/FOLDER03747363M/1/ Vmware-NDS_Perccli_1-17-10.zip. 2. Extract the ZIP file. 3. Copy the installation file, vmware-esx-perccli-1.17.10.vib, to /tmp on the server.
Preparing Disks Create virtual devices with PERCCLI Create virtual devices (VDs) on disks using the PERCCLI utility. You need to perform this step before adding disks to a VxFlex OS SDS server in the VxFlex Ready Node system. Note If the server contains the HBA330 controller, skip this step. When creating virtual devices, note the following: You create one virtual device for each kind of disk, HDD or SSD.
Page 57
Preparing Disks Removing all the existing VDs from the node Setting up the controller card boot parameter Procedure 1. Log in to the node. 2. Display the disk information on the node: /opt/lsi/perccli/perccli /c0/eall/sall show Output similar to the following is displayed: The output shows the following: Enclosure ID (EID): Used in a later step when creating VDs Slot ID (SLT) of each drive: Used in a later step when creating VDs...
Preparing Disks 5. Set the controller card boot parameter: /opt/lsi/perccli/perccli /c0 set bootwithpinnedcache=on The boot parameter of the controller card is defined. Results You have verified that the disks are in the UGood (unconfigured, but good) state and that the controller card boot parameter is set to on. You can now create virtual devices.
Preparing Disks Create virtual devices for SSD with PERCCLI Use PERCCLI to create virtual devices (VDs) for SSDs on a VxFlex Ready Node server. Before you begin Ensure that the PERCCLI disk utility is installed. Before using PERCCLI, ensure that your working directory is /opt/lsi/perccli/. If it is not, run: cd /opt/lsi/perccli/ Use this procedure both for SSD for storage and SSD to be used to provide caching...
Page 60
Preparing Disks 2. Display the physical device (PD) information: /opt/lsi/perccli/perccli /c0/eall/sall show Output similar to the following is displayed: 3. Verify that all the devices have a virtual device created on them. In the Drive Information output, the device state displayed as "Onln" in the State column shows that the device has a virtual device created on it.
Page 61
Preparing Disks Cache = RWBD (for HDDs) After you finish Ensure virtual disk creation with PERCCLI...
Page 62
Preparing Disks Hardware Configuration and Operating System Installation Guide - ESXi Servers...
CHAPTER 8 Additional Configurations The following topics describe additional configurations that are required for the installation process. Install OpenManage Essentials................64 Activate the NVIDIA GPU on the server............. 64 Installing the iDRAC Service Module on an ESXi server........64 Where to go from here..................66 Additional Configurations...
Additional Configurations Install OpenManage Essentials Dell EMC OpenManage Essentials (OME) is a web-based console that simplifies hardware monitoring and firmware maintenance on VxFlex Ready Node servers. If your system does not use AMS, it is recommended that you install OME.
The file for the currently supported ESXi versions is: ISM-Dell-Web-3.1.0-960.VIB-ESX6i-Live_A00.zip Install the iSM using SSH Install the iDRAC Service Module (iSM) software on the ESXi host via SSH. Procedure 1. Download the ISM-Dell-Web-3.1.0-960.VIB-ESX6i-Live_A00.zip file from http://www.dell.com/support/home/us/en/19/drivers/driversdetails? driverId=FTGRN. 2. Unzip the file to a local directory on the system.
Additional Configurations Note If you are using the vSphere CLI on a Linux server, perform the command from any directory. 7. Install the iSM software: esxcli --server <host_IP> software vib install - d /var/log/ vmware/<iDRAC Service Module file> 8. At the prompt, type the root username and password of the ESXi host. The command output tells you whether the installation succeeded.
CHAPTER 9 Reference material This section contains additional information that may be required for the procedures described in this document. DTK - Hardware Update Bootable ISO............... 68 Reference material...
Guide. Update the hardware using remote RACADM You can install and execute the Dell RACADM tool from any management system with access to the iDRAC network. The remote RACADM command set is useful in this situation to mount and execute the Hardware ISO to a large number of VxFlex Ready Node servers.
Page 69
Do not reboot the VxFlex Ready Node server while the update process is being performed! The iDRAC will be reset several times during the update process. This causes the iDRAC virtual console viewer to close, virtual media to disconnect, and the Dell OpenManage DRAC Tools (RACADM)
Reference material iDRAC browser window to be unavailable for several minutes during each reset. The hardware update scripts will continue to run from RAM on the server. The update script will generate a log indicating whether each configuration and firmware flash is successful. 12.
The Hardware ISO runs a script that automatically forces the server to install the necessary firmware updates. This firmware is consistent with the qualified Dell EMC VxFlex Ready Node Driver and Firmware Matrix, located at https://support.emc.com/products/42216. Some of the firmware listed in the table is dependent on the VxFlex Ready Node hardware configuration.
Page 72
Reference material Table 12 Hardware ISO configuration settings (continued) Description Setting Value System Change Tracking BIOS.MiscSettings.InSystem Disabled Characterization CPU Virtualization Features BIOS.ProcSettings.ProcVirtu Enabled alization CPU Cores BIOS.ProcSettings.ProcCores All CPU X2APIC Mode BIOS.ProcSettings.ProcX2Api Enabled CPU Turbo Engagement BIOS.ProcSettings.Controlled Disabled Turbo System Power Profile BIOS.SysProfileSettings.SysP PerfOptimized rofile...
Reference material Troubleshooting the Hardware ISO This section describes troubleshooting procedures for problems you may encounter while using the Hardware ISO. Troubleshoot general iDRAC failures When problems occur with iDRAC or Lifecycle Controller jobs, you can delete all jobs with a single iDRAC command. All of the completed jobs, plus any orphaned pending jobs, are deleted, and the data manager service on the iDRAC is restarted.
Page 74
/bundleapplicationlogs/apply_components.log 3. You can also view the script for the Hardware ISO, which is useful in helping to identify and troubleshoot log entries: less /opt/dell/toolkit/systems/drm_files/apply_bundles.sh Results The script attempts to configure several boot order commands, regardless of the hardware configuration of the VxFlex Ready Node server. This allows the script to support multiple hardware platform configurations.
VxFlex Ready Node product, documentation, advisories, downloads, and white papers, visit https://support.emc.com/products/42216. Dell Lifecycle Controller (LC) With the launch of the Dell PowerEdge 13th-generation servers in September 2014, Dell enhanced embedded management without the need to install a software-based Additional resources...
Page 76
Dell OpenManage Deployment Toolkit (DTK) The Dell OpenManage Deployment Toolkit (DTK) includes a set of utilities, sample scripts, and sample configuration files that you can use to deploy and configure Dell systems. You can use the DTK to build script-based and RPM-based installation for deploying large number of systems on a pre-operating system environment in a reliable way, without changing their current deployment processes.