Let Lenovo XClarity Make Your Brownfield Blue (or Green)
In this blog post we are going to take a look at how Lenovo's XClarity management platform (Administrator and Controller) can sort out a mess in your datacenter in under 30 minutes. We will not spoon feed every single step, so you will be required to do some thinking.
Here is our situation:
We want to use VMware's Cloud Builder to deploy a VMware Cloud Foundation (VCF 3.10) management cluster.
What do we need? Four hosts.
What do we have? Four ThinkAgile VX Series hosts, on loan, that will be repurposed for lab use, but we only know the IP addresses…we do not have root credentials for any of the nodes.
What do we need to do? Regain management of the hosts, install the appropriate version of ESXi (6.7 update 3 - Lenovo custom image in our case), and add the hosts to our vCenter instance so we can proceed.
(And there will be a few in this process): Our hosts were managed by a previous XClarity Administrator and Controller that are way out of date and cannot be upgraded. In the process, we need to deploy a new XClarity Administrator that is current and use that instance to manage our hosts.
Step 1: Download the latest XClarity Administrator from Lenovo.
Step 2: Deploy XClarity Administrator in your vSphere environment.
During the process you will need network/hostname information and will need to run the initialization wizard after deployment to configure additional XClarity settings and accounts.
Step 3: Unmanage the servers in your old XClarity instance.
From XClarity Administrator: Hardware > Inventory (Servers) > Select desired objects to be unmanaged and select Unmanage.
Confirm and wait for the process to complete. When finished, shut down the old XClarity instance in vSphere.
Step 4: From the new XClarity Administrator instance, import the appropriate EXSi ISO from Provisioning > Manage OS Images and Import File.
In our case, we downloaded the ESXi 6.7 update 3 custom image from Lenovo. https://vmware.lenovo.com/content/2019_09/Lenovo_Custom_ISO/6.7u3/
The host(s) and XClarity Administrator must be able to talk. Our recommendation is to put them on the same Layer 2/subnet for this exercise. You can test connectivity from XClarity Administrator from Administration > Network > Test Connection.
Step 5: Now you need to add your old hosts to your new XClarity Administrator.
The best way to do this is with a manual add since you are going to need to "force" the management takeover. Even though we unmanaged the hosts from the old instance, they don't want to leave…just like your kids at a birthday party. Navigate to Hardware > Discover and Manage New Devices.
Select Manual Input and enter the known IP address of your old host and click OK.
In the Manage dialog box, leave Managed Authentication selected, but choose the Use Stored Credentials radio button.
Select Manage Stored Credentials and create a new credential with username USERID and password PASSW0RD (zero). This is the default account for the controller, and you will need this to manage your hosts since you don't know the root credentials. You only need to do this the first time…you will use this account for each additional host add.
Select the USERID - account in the dropdown and select the "Force Management" checkbox before clicking Manage. The force option ensures any of the previous management information is cleaned up.
Wait for the process to run and finish. This may take a few minutes.
You can close this dialog box and then repeat the process for your other hosts. After you have completed the process, click Dashboard in the upper left corner of the XClarity admin page.
You should see your 4 servers and you can select the number to see the list.
Step 6: Verify console connectivity.
The magic here is the administrative side of the configuration you just completed will now use the "USERID" account to console to the host. This is done by using a sys manager account under the hood and a result of the firmware on the bare metal hosts. It is essentially authenticating through XClarity. Test this by double-clicking on one of the hosts to bring up the server detail page.
Navigate to Actions > Launch > Management Controller Web Interface.
This will open in a new tab. Login with the admin credentials of your XClarity Administrator.
The first time this tab opens you will have to accept pop-ups. We recommend accepting all the pop-ups, then closing the tab and open again.
Here you can view the configuration and also launch a direct console to your host.
In the bottom left pane labeled Remote Console Preview, click the black box representing a console.
Adjust any settings as needed and select Launch Remote Console. This will open in another tab.
Repeat the process to verify you can console into your remaining hosts.
Step 7: Returning to XClarity Administrator, prepare to deploy ESXi Image by navigating to Provisioning > Deploy OS Images
Your server list (the ones you manually added) will be displayed. Ensure the Lenovo custom image you imported shows in the Image to Deploy column.
If any hosts are displaying a "Not Ready" status, view the note and correct any errors before proceeding.
Before deploying the image to the hosts, click All Actions > Global Settings and decide what makes sense for you in your lab (and eventually in your production deployment later). This is a great time to test options with no pain. We will be using static IP addressing versus DHCP which presents an interesting scenario that we will show you how to resolve later.
Enter the root credentials for your hosts.
Choose your IP assignment type and potential VLAN tagging.
The License Keys and Active Directory tabs here are for Windows servers only. Click OK to save.
Step 8: Deploy an ESXi image to your hosts.
Navigate back to Provisioning > Deploy OS Images if you are not still there.
Select the first host you would like to deploy. Theoretically, you could do them all at once. We chose to do one at a time and monitor as we went since this was a recovery mission.
Since we are doing manual IP deployment, we need to adjust our IP settings by clicking edit in the far-right column. XClarity is managing the host from the known IP we used to add to inventory, but we want to configure the address we want for our lab. We can also set the other network settings and update the hostname. You will need to do this for all of your hosts and could choose to do that now before moving on. Obviously if you are going to use the howitzer approach and do them all at once, you have to do this now regardless. Click OK to save on each entry.
With your server(s) still selected, choose All Actions > Deploy Images.
Verify XClarity still sees the right image and click deploy. This will take several minutes.
During the process, you can monitor by Navigating to Jobs > View all jobs.
You can also console to the host as you did previously to watch host boot up, etc.
Wait for the install to finish, then we can proceed with the next steps.
You can see from the top of the screenshot that the old IP on the BMC (controller) is still present. Recall we are consoling to the host via the magic of the XClarity Administrator, Controller and firmware. If you need to change the IP address on the BMC, you have to do that directly. The address space on the controllers should be different from IP space on the management VM Kernel port on the host. The cool part here is that once the controllers are set up, you don't ever have to touch them again unless you need to. Before we leave the console, let's prove we now have root access with our fresh install.
Step 9: (Optional) Correct the BMC configuration to the new host IP by navigating to Dashboard > Servers > "Number"
Double-click on the server in question (old .207 in our case) and navigate to Actions > Launch > Management Controller Web Interface.
Change any settings here if needed. If the controller address space was set up during the initial discovery and input of the hosts into XClarity Administrator, the controller address space should not need to be changed. You may want to update the host name to reflect the new name you just gave your host. Again…this is all out of band of vSphere traffic and the magic of XClarity Administrator, Controller and firmware!!
You will need to login to the controller again (new host). As a reminder, these are the XClarity admin account credentials.
From System Information and Settings, select the arrow next to BMC IP Address (or whatever setting may need to be updated/changed.)
Notice the previous settings persist. Adjust the hostname to the new value.
Click Apply and Apply. Close the tab.
Step 10: Let's now add rediscover the freshly installed host back into XClarity Administrator.
Navigate to Hardware > Discover and Manage New Devices and click Manual Input. Enter the IP address of your new host and click OK.
On the Manage dialog box, once again choose Managed Authentication (default) and Stored Credentials with the USERID account selected. Don't forget to click "Force".
Click Manage and wait for the import to finish. Click Close when finished.
Another great feature of the USERID account is if you have hundreds of servers (and you don't have root account uniformity) you don't have to worry about keeping up with all of that to manage your datacenter.
Navigate back to Dashboard > Servers > "Number". Your server may show pending - this is OK and will resolve in a few minutes as it is doing hardware inventory discovery.
Step 11: For those with OCD (speaking to present company), one last fix.
The "label" if you will in XClarity is simply that, and we only have all this update shenanigans because we started with existing values that we may or may not have created and we are re-using those components previously in inventory. Fret not…we will sort this final step.
Double-click the server name to launch the Server Summary Page.
Select Edit Properties and change the User-Defined Name and Save.
It will take a little time for everything to settle down and fully update. The XClarity polling interval is five minutes and just "works" in the meantime whether the display is totally correct or not.
Step 12: Finally, let's add the new host into our vCenter instance just to fully verify everything is working properly.
We are going to do this manually for now as future exercises and blogs will be leveraging Cloud Builder to deploy our VCF management domain, so we don't need host profiles or any settings for now: just a clean install of the proper ESXi build with a happy host.
From vCenter, right-click the desired cluster, add host and follow the prompts. And here we have it…
Get a snack and something to drink.
Now, go back up to Step 8 and repeat the process for the remainder of your hosts.
Now we have the VX Series hosts ready so we can deploy VCF.
This process was all done remotely. Recall we did not have the root credentials of our ESXi hosts, only an existing XClarity Administrator instance managing the old hosts we wanted to recover, re-image and migrate to a new XClarity Administrator instance.
Twelve easy steps with XClarity to make your Brownfield Blue (or Green).