28th December 2017

VMware vSphere 6.5 Upgrade Considerations

I recently worked with a customer that had a well developed vSphere environment including;

vCenter 5.5 U3
vROPs 5.8.5
SRM 5.5.1
ESXi 5.5 U3

The reason I was working with them was to offer some advice for upgrading the environment up to vSphere 6.5.1, vROPs 6.6.1 and SRM 6.5.1.

This upgrade path is complicated by the need to keep the data held within SRM. Each of the components outside of SRM is capable of being upgraded directly from 5.5 U3 to 6.5.1… SRM can not, it must first go from 5.5.1 to 6.0 to 6.1.2 to 6.5.1.

Each of those versions of SRM has different compatibility requirements with other elements of the vSphere stack. So the requirements of SRM drive the upgrade sequence of the other components.

Read on for upgrade considerations for each of these components.

vCenter Server

vCenter Server 6.5 can manage a mixed environment
vCenter Server now supports an interchangeable mix of Windows and Linux versions
Supported vCenter database requirements
- Oracle 11g / Oracle 12c, PostgreSQL, MS SQL Server 2008 SP2 and earlier releases including versions of Microsoft SQL Server 2012 and 2014
- Validate from the VMware Compatibility List to verify database support
- vCenter Appliance no longer supports external databases.
In-place upgrade versus new installation
- Need to retain performance data and events?
- Preserve existing clusters, resource pools, folders, roles and permissions, registered virtual machine templates?
- Is existing vCenter Server configuration optimal or should it be replaced?
- Need to move to 64-bit platform
vSphere 6.5 consolidates external services to simplify the vCenter architecture
- All services are a part of either the Platform Services Controller or vCenter Server services
- The installer migrates data as appropriate to keep current settings on systems, even though the service functions location might have changed
The Platform Services Controller installation is a set of infrastructure services that include
- vCenter Single Sign-On
- License service
- Lookup Service
- VMware Directory Services
- VMware Certificate Authority
The vCenter Service installation provides the remaining vCenter supporting services including
- vCenter Server
- vSphere Web Client
- vSphere Auto Deploy
- vSphere ESXi Dump Collector
- vSphere Syslog Collector on Windows and vSphere Syslog Service for the vCenter Server Appliance
- vSphere Update Manager (Appliance Only)
vCenter Server provides two basic ways to configure services
- An embedded Platform Service Controller
- An external Platform Services Controller
When upgrading from vCenter 5.5, the architecture is determined by how the original configuration was installed, which cannot be changed during the upgrade
With vCenter 6.5, the deployment architecture can be changed after the upgrade
- Reconfigure command allows for embedded systems to be converted to external
- Repoint command allows for vCenter Server to be pointed at another Platform Services Controller (in the same SSO Domain only)
vSphere 6.5 can be installed by an in-place upgrade (sometimes referred to as a build-2-build upgrade).
vCenter 6.5 can be migrated from a Windows to Appliance platform with built in tools
The upgrade to vCenter Server includes a database schema upgrade along with the actual vCenter Server software
- The upgrade requires that vCenter Server is out of production long enough for the database and software to be upgraded
  - The time required to complete the upgrade is dependent on the size of the database. The upgrade normally takes between 30 minutes and 1 hour. This estimate does not include time for host re-connection after the upgrade
  - The database schema upgrade takes approximately 10-15 minutes of the total upgrade time
vSphere DRS does not work while the upgrade is in progress. VMware HA does work during the upgrade, because it is an agent on each ESXi host.
During the vCenter upgrade, downtime is not required for the ESXi hosts that vCenter Server is managing, or for virtual machines that are running on the hosts
A live migration, rather than an upgrade, can be done if vCenter availability is a concern
Upgrading to vCenter Server 6.5 can affect other software components in your data center
- Some existing data center components may not be supported by vSphere 6.5
  - Validate that VMware components in the environment support the upgrade. For example, VMware Horizon® with View and VMware vCloud® Automation Center™ have specific requirements for versioning of components
  - Third-party software.

vSphere Update Manager

vSphere Upgrade Manager is included in the vCenter Appliance as of vSphere 6.5
Upgrade the vSphere Update Manager in Step 1 because it can then be used to automate both host and virtual machine upgrades
Use the upgrade as an opportunity to assess current architecture and consider making any changes to align with VMware best practices, for example
- Separate the vSphere Update Manager patch repository from the vSphere Update Manager database
- Locate vSphere Update Manager near ESX/ESXi hosts to minimize network latency
- Observe recommended sizing guidelines
  - In small environments, all components can be on one host machine – vCenter Server, vCenter Single Sign-On, vCenter Server database, vSphere Update Manager server, and vSphere Update Manager database
  - For 30+ hosts or 300+ virtual machines, vCenter Server, vCenter Single Sign-On and vSphere Update Manager server can be installed on one system with each database instance installed on its own dedicated system
  - For 100+ hosts or 1000+ virtual machines, each component should be on its own system – vCenter Server, vCenter Single Sign-On, vCenter Server database, vSphere Update Manager server, and vSphere Update Manager database

vSphere vRealize Operations Manager

Download the right PAK file for the clusters you wish to upgrade. Notice that only the Virtual Appliance clusters use an OS Update PAK file.
Host name entries in the /etc/hosts of each node might be reset when applying the OS update PAK file for an update from vRealize Operations 6.0.x to version 6.1. You can manually update the hosts file after completing the software update.
It’s a good practice to create a snapshot of each node in a cluster before you update a vRealize Operations Manager cluster.
- Once the update is complete, you must delete the snapshot to avoid performance degradation
Before you install the PAK file, or upgrade your vRealize Operations Manager instance, clone any customized content to preserve it.
Customized content can include alert definitions, symptom definitions, recommendations, and views. Then, during the software update, you select the options named Install the PAK file even if it is already installedand Reset out-of-the-box content.
Version 6.2.1 vRealize Operations Manager update operation has a validation process. It is good practice to run the pre-update check and resolve any issues found.

vSphere Site Recovery Manager

In a SRM installation configured bidirectional protection, in which each site acts as the recovery site for the virtual machines on the other site, upgrade the most critical of the sites first.
Make a full backup of the Site Recovery Manager database by using the tools that the database software provides.
- Migration of data from an external database to the embedded database is not supported.
- Failure to back up the database results in the loss of all Site Recovery Manager data if the upgrade fails.
If you configured advanced settings in the existing installation, take a note of the settings that you configured in Site Recovery > Sites > Site > Manage > Advanced Settings in the vSphere Web Client.
The local and remote Platform Services Controller and vCenter Server instances must be running when you upgrade Site Recovery Manager.
Upgrade the Platform Services Controller, all vCenter Server components, and Site Recovery Manager on one site before you upgrade the Platform Services Controller, vCenter Server, and Site Recovery Manager on the other site.
Obtain the vCenter Single Sign-On administrator user name and password for both of the local and remote sites.
Obtain the user name and password for the Site Recovery Manager database, if you are not using the embedded database.
The Site Recovery Manager installer presents the SSL/TLS certificates of the vCenter Server components for validation when it runs. Obtain the necessary information to allow you validate the certificates for the Platform Services Controller instance on the local site and the Platform Services Controller and vCenter Server instances on the remote site.
If you use custom certificates, obtain an appropriate certificate file. Custom certificates must use at least the SHA1, or preferably SHA256, thumbprint algorithm.
Download the Site Recovery Manager installation file to a folder on the machines on which to upgrade Site Recovery Manager.
Verify that no reboot is pending on the Windows machine on which to install Site Recovery Manager Server. Pending reboots or running installations can cause the installation of Site Recovery Manager Server or the embedded Site Recovery Manager database to fail.
Verify that there are no pending clean-up operations on recovery plans and that there are no configuration issues for the virtual machines that Site Recovery Manager protects.
- All recovery plans are in the Ready state.
- The protection status of all of the protection groups is OK.
- The protection status of all of the individual virtual machines in the protection groups is OK.
- The recovery status of all of the protection groups is Ready.

ESXi Host Upgrades

The upgrade requires the host to be out of production for 30 to 60 minutes, depending on the method chosen
- Update Manager
  - Recommended upgrade option for environments using vCenter Server
  - Provides automated task management and coordination with functionality such as host maintenance mode, vSphere vMotion, and DRS for orderly evacuation of virtual machines from existing hosts
- Scripted Installation
  - Invokes the upgrade script to conduct unattended upgrade of multiple hosts
    - Script calls installer from CD, DVD USB, or by PXE-booting the installer
  - Requires script creation and configuration
- Interactive Update
  - Installer is run manually from CD, DVD, or USB to target the location of an existing ESXi 5.x /6.0 installation
  - Most appropriate method for a small number of hosts
- Migration Upgrade
  - This method is appropriate in environments where downtime cannot be taken to upgrade hosts
  - Allows for VMs to be systematically migrated off of old hosts and moved to new hosts for upgrade to occur with one of the other previous methods listed

Virtual Machines

Virtual machine hardware upgrade is not required, but VMware recommends it
Virtual machines with hardware versions earlier than version 13 can run on ESXi 6.5 hosts, but they will not have all the capabilities that are available in Virtual Machine Hardware Version 13
VMware Tools upgrade offers enhanced virtual machine usability and performance
Once hardware is upgraded, it cannot be run on older ESXi hosts. Verify that all hosts in the environment are upgraded prior to upgrading the hardware version.

VMFS

VMFS2 datastores are not accessible on ESXi 6.x hosts. They must be upgraded to VMFS3 to provide access
Existing VMFS3 datastores are fully supported and accessible on ESXi 6.x hosts, however new VMFS3 datastores cannot be created in an updated environment
VMFS6 datastores are created by default when using ESXi 6.5
Upgrading datastores to VMFS6 provides improved scalability and performance
VMFS upgrade is a one-way process
- After you have converted a VMFS3 or VMFS5 datastore to VMFS6, you cannot revert it back
- The only way to revert back is to remove the volume completely and recreate the volume, which can require significant downtime while the virtual machines are migrated off of storage. Note that this operation must be done with an older vCenter or ESXi host that still allows creation of the appropriate VMFS datastore version.

All the best

Simon