To get started, go to this Dell link and download the version of OMSA for your client server that corresponds to the generation of the Dell server you have. If you're not sure which generation you have, you can tell by the middle characters in your server's model number. For example, a Dell R900 is a 10th generation server, R910 is 11th gen, and R920 is 12th gen. The file in question should be an .ISO around 3GB in size.
Don't download the ESXi package from the link I gave you above; at the time of this writing, they don't work. Only download the Windows Server .ISO file.
Once you've got your file downloaded, burn it to a DVD (or in our case, mount it in the virtual machine preferences). Then, go through the installer for OpenManage, choosing the default settings when prompted. This should give you the client portion of OpenManage, which you can access by going to https://yourserversIPaddress:1311 in the web browser window of the server you've installed it on. It also places a shortcut on the desktop for this as well.
Now that you've got the client, head to the Dell PowerEdge driver site and select your server model from the list. Once you've got your server model, go to the Drivers page for it, and change the server OS to VMware X.X. The file we're after is under the Systems Management section of the driver list, and should be called something along the lines of Dell OpenManage Server Administrator vSphere Installation Bundle (VIB). It should be a .ZIP file around 7MB in size. Download that, and Browse to the datastore on your VMware host in the vSphere Client. Add that file to the datastore (I added mine to the root of my datastore, it doesn't really matter where you put yours).
Now that we've got the files in place, you'll need to do two things on your ESXi host. Firstly, enable the SSH server on it if you haven't done so already. To do this from the vSphere Client, select your ESXi server in the list on the left, then go to the Configuration tab. Under the Software section, click on Security Profile. Now click on the Properties section of the Services list that appears (it's in the upper right). From here, find SSH in the list and click on the Options button. You can enable SSH from here.
Second on the list, you'll need to put your ESXi host into Maintenance Mode. This can be done from the ESXi hosts's summary page in the vSphere Client, and requires you to turn off all VMs on the server first. Once you have your host in Maintenance Mode, the fun can finally begin!
SSH into your VMware host with the root credentials, and run the following command:
esxcli software vib install -d "vmfs/volumes/datastorename/directoryname/patchfile.zip"
This takes a few minutes, but should close with an 'installation successful' message after a bit. Once this is done, you can run the below command to confirm the OMSA package is there; it should be at the top of the list.
esxcli software vib list
To wrap things up, reboot your server and take it out of maintenance mode. Now, from the OMSA client computer, open the OMSA web interface, and connect into your ESXi server with an admin-level account (root, in my case). You'll need to check the certificate checkbox on the bottom of the connection window if you're like me, and are using the self-signed cert that's installed by default with ESXi.
Questions/issues? Shoot me a message.