vCenter 5.1, Multisite SSO, Linked Mode, Custom SSL certificates, protected by vCenter HeartBeat. Part 2

In my previous post I talked about the requirements this customer had and the high level architecture and products used to fulfill their requirements. In this post I’ll walk through the installation of each component and protecting it with vCenter Heartbeat.

In each site we’ll need a total of 15 public IP’s and 10 channel IP’s, by public IP I’m referring to an IP address that is routable on your network that other machines can access the services by and by channel IP I’m referring to a non-routable VLAN like you setup for vMotion but this one is dedicated to vCenter Heartbeat channel (or heartbeat) traffic.

Create 5 Windows 2008 R2 VM’s in the primary site. For simplicity I’m going to refer to these VM’s as sql1, vcenter1, sso1, vcweb1, and vum1. Each of these servers will need 2 NIC’s, one on the public network and one on the channel network. Our clones (don’t create these yet!) for vCenter Heartbeat will be sql2, vcenter2, sso2, vcweb2, and vum2. And lastly our shared names will be sql, vcenter, sso, vcweb, and vum.

Starting with sql1 assign the IP to be used for the shared name and the IP to be used for management to the public NIC and disable DNS registration.
On the channel NIC assign the channel IP and disable DNS registration and netbios.
Rename the computer account to the shared name, join it to AD if it’s not already and reboot.
Delete any dynamic DNS records for this machine from DNS and add static records for the shared name and both sql servers management hostnames.

Repeat this process for sso1, vcenter1, vcweb1 and vum1.
Now we should have 5 VM’s with the windows hostnames of sql, vcenter, sso, vcweb, and vum.

For anyone who hasn’t used vCenter Heartbeat before the above may be a little confusing so I’ll give a brief explanation as to what we’re accomplishing here, for full details read the vCenter Heartbeat documentation. vCenter Heartbeat works by synchronizing the file system and registry between 2 servers, these can be a pair of VM’s (this is the method used here), a pair of physical servers or one virtual and one physical server and using a network filter driver to hide the shared IP address on the server who is not running the shared service.
Each of the two servers is accessible on the network by its management IP and windows hostname, in vCenter Heartbeat this is referred to as non-identical mode, which allows for maintenance tasks, such as patching, to be done on the secondary node. The shared service can be moved between servers using the vCenter Heartbeat management console. When this happens manually the services are shutdown on the primary server, the shared IP is hidden from the network using the filter driver on the primary server then the shared IP is exposed on the secondary server by the filter driver and the services are started on the secondary server. The same happens automatically if the server running the shared service fails, except in a situation like a blue screen where it’s not possible to shutdown the services.

Since we can’t do anything without SQL we’ll start there. If you haven’t already configure the VM with the number vCPU’s and memory as you would for a production SQL server.
Install SQL as you normally would , use an AD account with local admin rights on the server as the service account for the database service. Set the database service and system attendant services to auto when prompted in the install.

Shut Down the server and clone it to sql2 with no customizations, if possible place it in a different cluster and on different storage controllers or at least a different datastore.
Disconnect both NICs on sql2.
Boot both servers up.
On sql2 change the Public IP settings, leave the shared IP but change the management IP to the correct IP for this server.
Change the channel IP to the correct IP for this server.
Connect the Private NIC, leave the Public NIC disconnected.

Rename the adapters in windows on both servers, name the Public adapters Public and Channel adapters Channel on both servers.

Start the HeartBeat install on SQL1:

• Select Primary
• Select Second Server is Virtual
• Select LAN
• Select your channel adapter, Click the Add button and select your IP. Click the Add button below and enter the channel IP of the 2nd server.
• Choose your Public adapter, Click Add select the Shared IP and leave the default that both servers will use the same IP. Click Add choose your management IP, below click add and choose the 2nd servers management IP, click NO to the warning that the 2nd server can’t be reached this is normal since it’s adapter isn’t connected to the network.
• Ensure SQL is selected to protect and leave the credentials for the vCenter plug-in blank.
• Enter sql1 and sql2 when prompted for names to rename the computer accounts to.
• For the location to copy the configuration to create a HB folder on C:\ and then enter \\localhost\c$\HB.
• Finish the wizard by click Next as it completes each part of the install, reboot when prompted.

Once the first server is up move over to the 2nd server. You do not have Public network access, so copy the heartbeat install file by accessing the 1st servers private network i.e. \\Server1PrivateIP\c$.
Run the heartbeat install on SQL2:

• Select Secondary, when prompted for the directory that contains the configuration enter \\Server1PrivateIP\c$\HB
• Select your channel adapter
• Connect your Public adapter to the network before continuing
• Choose your public adapter and next through the rest of the install
• Reboot when prompted

In Active Directory, find the service account you created to run SQL, go to properties and security and click advanced, click add, enter the same service account and then click allow under Write All Properties. Repeat this for each on SQL server computer objects.

Once both servers are up, open up the manage server heartbeat application and navigate to Applications and Tasks. Click the Users button and enter your SQL Server service account information, now edit the two NFsetspn tasks, change the user account from local system to the account you just added. On the server currently running the SQL Services (should be the primary server) right click the NFsetspn task and choose run now and ensure it completes with an exit code of 0.

Move the SQL resources to the 2nd node and verify everything is working and then move them back to the primary.

Next up is SSO.

Using the provided DB scripts create your SQL SSO database and the required _dba and _user accounts, I change the DB and user accounts to be SSO and SSO_ instead of the default RSA.

Ensure you’ve set the shared IP and management IP on the Public NIC and the Channel IP on its NIC and that you have DNS records created for the shared name and both SSO servers.
If not done, rename the computer account to the shared name to be used by SSO. Create a service account in AD to run the SSO service and grant it local admin rights to the SSO server. Install SSO, choose to create a primary node for a new installation, again choose to create the primary node (do not pick basic).

Shut Down the SSO server and clone it to sso2 with no customizations, if possible place it in a different cluster and on different storage controllers or at least a different datastore.
Disconnect the both NICs on sso2.
Boot both servers up.
On sso2 change the Public IP settings, leave the shared IP but change the management IP to the correct IP for this server.
Change the Channel IP to the correct IP for this server.
Connect the Channel NIC, leave the Public NIC disconnected.

Rename all adapters, name the Pulbic adapters Public and Channel adapter Channel on both servers.

Start the HeartBeat install on SSO1:

• Select Primary
• Select Second Server is Virtual
• Select LAN
• Select your channel adapter, Click the Add button and select your IP. Click the Add button below and enter the channel IP of the 2nd server.
• Choose your Public adapter, Click Add select the Shared IP and leave the default that both servers will use the same IP. Click Add choose your management IP, below click add and choose the 2nd servers management IP, click NO to the warning that the 2nd server can’t be reached this is normal since it’s adapter isn’t connected to the network.
• Ensure vCenter Services is selected to protect and leave the credentials for the vCenter plug-in blank.
• Enter sso1 and sso2 when prompted for names to rename the computer accounts to.
• For the location to copy the configuration to create a HB folder on C:\ and then enter \\localhost\c$\HB.
• Finish the wizard by click Next as it completes each part of the install, reboot when prompted.

Once the first server is up move over to the 2nd server. You do not have Public network access, so copy the heartbeat install file by accessing the 1st servers private network i.e. \\Server1PrivateIP\c$.
Run the heartbeat install on SSO2:

• Select Secondary, when prompted for the directory that contains the configuration enter \\Server1PrivateIP\c$\HB
• Select your channel adapter
• Connect your Public adapter to the network before continuing
• Choose your public adapter and next through the rest of the install
• Reboot when prompted

Attempt to move SSO to the 2nd node and verify everything is working (at this point you can only look in the logs as there is nothing using SSO) and then move them back to the primary.

Now that we have SSO installed, you may think about replacing the self-signed SSL certs with ones issued from a CA. Don’t do it! If you do when you go to install the Inventory Service, vCenter and/or the Web Client the installs will hang while it’s trying to pull the SSL certificate down from the SSO server. In the primary site I did this and had to kill openssl and/or msiexec processes until the installer moved on. It’s much easier to install all the components then come back and replace their SSL certs, which I’ll cover in my next post.

With SSO installed and still using its self-signed certificate move onto the vCenter Inventory Service and vCenter Server installs.

Ensure you’ve set the shared IP and management IP on the Public NIC and the Private (Channel) IP on its NIC and that you have DNS records created for the shared name and both sso servers. If not done, rename the computer account to the shared name to be used by vCenter and the Inventory Service.

Install the vCenter Inventory Service, when prompted register it with the SSO server, be sure and use the shared name and not the hostname name of either SSO servers.

vCenter requires a DB on our SQL server HeartBeat instance we created earlier, create the database and a SQL account to use for vCenter to authenticate with. Give the new account DBO rights to the vCenter Database, and the MSDB Database. For other options on the creation of the SQL DB see the vCenter Install documentation.

If not already part of your standard build install the SQL Native Client for the version of SQL you’re running.
Create a 64bit DSN that points to the shared SQL name, not either windows hostname running HeartBeat, use the SQL account you created and change your default DB to the one you created for vCenter.

Install vCenter, you’ll be prompted for the DSN info created earlier as well as SSO information, fill in the necessary information. Since the Inventory Service is installed on the same server the vCenter install will auto detect it, ensure the name displayed is the shared name and not either hostname, do the same for the vCenter FQDN when prompted.

Once the install is completed Shut Down the vCenter server and clone it to vcenter2 with no customizations, if possible place it in a different cluster and on different storage controllers or at least a different datastore.

Disconnect the both NICs on vcenter2.
Boot both servers up.
On vcenter2 change the Public IP settings, leave the shared IP but change the management IP to the correct IP for this server.
Change the Channel IP to the correct IP for this server.
Connect the Channel NIC, leave the Public NIC disconnected.

Rename all adapters, name the Pulbic adapters Public and Channel adapter Channel on both servers.

Start the HeartBeat install on vcenter1:

• Select Primary
• Select Second Server is Virtual
• Select LAN
• Select your channel adapter, Click the Add button and select your IP. Click the Add button below and enter the channel IP of the 2nd server.
• Choose your Public adapter, Click Add select the Shared IP and leave the default that both servers will use the same IP. Click Add choose your management IP, below click add and choose the 2nd servers management IP, click NO to the warning that the 2nd server can’t be reached this is normal since it’s adapter isn’t connected to the network.
• Ensure vCenter Services is selected to protect enter the credentials for the vCenter plug-in.
• Enter vcenter1 and vcenter2 when prompted for names to rename the computer accounts to.
• For the location to copy the configuration to create a HB folder on C:\ and then enter \\localhost\c$\HB.
• Finish the wizard by click Next as it completes each part of the install, reboot when prompted.

Once the first server is up move over to the 2nd server. You do not have Public network access, so copy the heartbeat install file by accessing the 1st servers private network i.e. \\Server1PrivateIP\c$.
Run the heartbeat install on SSO2:

• Select Secondary, when prompted for the directory that contains the configuration enter \\Server1PrivateIP\c$\HB
• Select your channel adapter
• Connect your Public adapter to the network before continuing
• Choose your public adapter and next through the rest of the install
• Reboot when prompted

Once both servers are up, open up the manage server heartbeat application and navigate to Applications and Plug-Ins. Right click on the vCenter Plug-in and choose edit, enter an account that can logon to vCenter.

Move the vCenter Services to the 2nd node and verify everything is working and then move them back to the primary.

vCenter and the Inventory Service is now protected by vCenter Heartbeat.

By now I’m sure you see a pattern, finish out the installs by repeating the process for the Web Client and VUM servers.

If you are unable to keep the first and second instance of each server in separate clusters be sure to create anti-affinity rules on your cluster to ensure they don’t end up running on the same host.

Repeat this process for the secondary site with the following exceptions:

• When installing SSO you will choose to Join and existing SSO installation, then choose MultiSite and enter the FQDN and port number for the primary sites SSO instance (again used the shared name). Remember that MultiSite SSO does not perform replication so any changes you make in the primary site that you want reflected in the secondary site you must follow these instructions to accomplish.
• When installing vCenter choose to join a linked mode configuration and point the install to the primary site’s vCenter.

In part 3 in this series I’ll go through the process of replacing the self-signed certificates with ones generated by your CA. The KB Articles from VMware are fairly accurate but seem to be written from the perspective that all services live on a single server so there are some differences when installing the services on separate servers such as we did.