VMware Drivers 3.0

:!: To use this functionality you need to install the VMware Driver Addon

The VMware Drivers Addon enables the management of an OpenNebula cloud based on VMware ESX and/or VMware Server hypervisors. It uses libvirt to invoke the Virtual Infrastructure SOAP API exposed by the VMware hypervisors, and it entails a much better integration and performance than the java based drivers traditionally found in the OpenNebula distribution.

It features a simple installation process that will leverage the stability, performance and feature set of any existing VMware based OpenNebula cloud.

inlinetoc

Requirements

In order to install the VMware Drivers Addon, some software dependencies have to be met:

  • libvirt: LibVirt is used to access the VMware hypervisors , so it needs to be installed with ESX support. These drivers has been tested agains version 0.9, and is recommended to use versions >= 0.9
  • ESX, VMware Server: At least one VMware hypervisor needs to be installed. Further configuration for the DATASTORE is needed, and it is explained in the TM part of the Configuration section. Recommended versions are ESX/ESXi 3.5/4.x and VMware GSX 2.0.

Considerations & Limitations

The hook functionality is restricted to the local hooks, since there is no ssh connection to the VMware ESX hypervisors to perform remote scripts.

The OpenNebula front-end should also be NFS front-end. If the NFS server is in another separate server, the OpenNebula front-end needs to mount in /var/lib/one the same NFS export mounted in the ESX hypervisors as the DATASTORE. If using SQLite, this limitation should be taken into account.

Configuration

VMware Configuration

  • libvirt

To install libvirt with ESX support, download, untar the latest version and execute the following inside the libvirt directory

<xterm>

$ ./configure --with-esx
$ make
$ make install

</xterm>

  • ESX, VMware Server

A user needs to be created in the hypervisor (Users & Group tab in the VI Client) with the same UID and username as the oneadmin user executing OpenNebula in the front-end. Please remember to give full permissions to this user (Permissions tab).

OpenNebula Configuration

To install the VMware addon, download it here, untar and execute install.sh as the oneadmin user.

If the installer succeeds, it will print the needed sections that must be included in the /etc/one/oned.conf file to use the VMware Drivers Addon.

Follow these steps and restart the OpenNebula daemon to complete the installation:

1) OpenNebula main configuration file needs to be edit to include the VMware Drivers. The following must be added to /etc/one/oned.conf:

#-------------------------------------------------------------------------------
#  VMware Driver Addon Virtualization Driver Manager Configuration
#-------------------------------------------------------------------------------
VM_MAD = [
    name       = "vmm_vmware",
    executable = "one_vmm_sh",
    arguments  = "-t 15 -r 0 vmware",
    default    = "vmm_sh/vmm_sh_vmware.conf",
    type       = "vmware" ]

    
#-------------------------------------------------------------------------------
#  VMware Driver Addon Information Driver Manager Configuration
#-------------------------------------------------------------------------------
IM_MAD = [
    name       = "im_vmware",
    executable = "one_im_sh",
    arguments  = "-t 15 -r 0 vmware" ]
#-------------------------------------------------------------------------------

#-------------------------------------------------------------------------------
# VMware Driver Addon Transfer Manager Driver Configuration
#-------------------------------------------------------------------------------
TM_MAD = [
    name       = "tm_vmware",
    executable = "one_tm",
    arguments  = "tm_vmware/tm_vmware.conf" ]
#-------------------------------------------------------------------------------

2) The VMM and IM driver need to know the credentials of the ESX user in order to access the hypervisor. Please input this information in

/etc/one/vmwarerc

:!: Please be aware that the above rc file, in stark contrast with other rc files in OpenNebula, uses ruby syntax, therefore please input the values as ruby strings, i.e., between quotes.

:!: When using VMware Server, the connection string set under LIBVIRT_URI needs to have its prefix changed from esx to gsx.

3) To ensure the proper working of the Image Catalog, a script that sets the proper ownership of the file needs to be run after each VM gets executed. The installer already modifies the Image hook to perform this requirement, and a few extra configuration details needs to be done. First, sudo needs to be configured:

oneadmin ALL=NOPASSWD:/var/lib/one/remotes/hooks/fix_owner_perms.sh ""

Afterwards, edit /var/lib/one/remotes/hooks/fix_owner_perms.sh and set:

  • IMAGE_REPOSITORY_PATH: Must be set at the same value as in /etc/one/oned.conf

4) Specific needed configuration for each driver belonging to the VMware Drivers Addon follows:

  • Virtual Machine Manager (VMM)

The name of the DATASTORE (see the Transfer Manager section) needs to be set in

/etc/one/vmm_sh/vmm_sh_vmware.conf
  • Transfer Manager (TM)

The storage configuration needs the /var/lib/one folder to be exported as a NFS share, and this to be mounted as a DATASTORE in the VMware hypervisor. The name of the datastore needs to be set in the VMM configuration.

The Image Repository path in /etc/one/oned.conf must be empty (defaulting to /var/lib/one/images), or it must be pointing to a path contained in /var/lib/one, which is the folder exported as NFS. This will effectively deal with the special VMware storage requirements. If these requirements are meet, no further configuration is needed in the TM.

For the old, traditional method of setting manually the SOURCE attribute in the disk section of the VM template, a few things need to be considered. First, the .vmdk files need to reside inside /var/lib/one, in the Image Repository path (which needs to be set in the variable $IMAGE_REPO_PATH in /etc/one/oned.conf, even if the Image Catalog functionality is not used. Second, the descriptor .vmdk file needs to be renamed to disk.0, and the other flat .vmdk files needs to be left untouched (see the User's Guide section).

Usage

The VMware Driver Addons greatly improves the support of the VMware hypervisors with OpenNebula. Tight integration is achieved, among others, in the following areas:

  • Network

Networks can be defined in OpenNebula, and VMs can use leases from these networks, being FIXED or RANGED. The only requisite is to input the name of the VMware hypervisor defined vSwitch as the bridge of the Virtual Network in OpenNebula. The default name for the vSwitch in the ESX family is “VM Network”.

  • Storage

The Image Catalog introduced in OpenNebula v2.0 can also be used with the VMware Drivers Addon. In contrast with previous versions, no new commands are available from the VMware Addon. Instead, the main oneimage command can be used to register images in the Catalog. All the .vmdk files that conform a disk (they can be easily spotted, there is a main <name-of-the-image>.vmdk file, which must be renamed to disk.vmdk, and various <name-of-the-image-sXXX.vmdk flat files) should be placed in the same directory, with no more files than these. Afterwards, a image template needs to be written, using the vmware:// prefix for the PATH:

NAME=MyVMwareDisk
PATH=vmware:///absolute/path/to/disk/folder

Now is the time to register the image with oneimage create <path-to-image-template>. Then, the image can be used as described in the Virtual Machine Images guide.

The direct managing of files (traditional method) is also supported, but the following need to be considered: the source attribute of the disk section of the virtual machine template needs to be pointing to a directory inside /var/lib/one (see the Configuration section), containing both the .vmdk (that must be named disk.vmdk), and the flat files (which needs to have its filename untouched).

  • Virtualization

Following the two last sections, we can use a template for a VMware VM like:

NAME=myVMwareVM
DISK=[IMAGE_ID="7"]
NIC=[NETWORK="public"]

The onevm command will control the lifecycle of the VM as with other hypervisors natively supported by OpenNebula.

  • Physical Host Management

The physical hosts containing the VMware hypervisors need to be added with the appropriate VMware drivers (out-of-the-box TM drivers, for instance, won't provide the needed functionality). If the box running the VMware hypervisor is called, for instance, esx-host, the host would need to be registered with the following command:

$ onehost create esx-host im_vmware vmm_vmware tm_vmware

Tuning & Extending

The VMware drivers consists of three drivers, with their corresponding files:

  • VMM Driver
    • /usr/lib/one/mads/one_vmm_exec : generic VMM driver.
    • /usr/lib/one/mads/one_vmm_sh : local actions wrapper for VMM driver.
    • /var/lib/one/remotes/vmm/vmware : commands executed to perform actions.
  • IM Driver
    • /usr/lib/one/mads/one_im_exec : generic IM driver.
    • /usr/lib/one/mads/one_im_sh : local actions wrapper for IM driver.
    • /var/lib/one/remotes/im/vmware.d : vmware IM probes.
  • TM Driver
    • /usr/lib/one/mads/one_tm : generic TM driver.
    • /usr/lib/one/tm_commands : commands executed to perform transfer actions.

And the following driver configuration files:

  • VMM Driver
    • /etc/one/vmm_sh/vmm_sh_vmware.conf : This file is home for default values for domain definitions (in other words, OpenNebula templates). For example, if the user wants to set a default value for CPU requirements for all of their VMware domain definitions, simply edit the /etc/one/vmm_sh/vmm_sh_vmware.conf file and set a
  CPU=0.6

into it. Now, when defining a template to be sent to a VMware resource, the user has the choice of “forgetting” to set the CPU requirement, in which case it will default to 0.6.

It is generally a good idea to place defaults for the VMware-specific attributes, that is, attributes mandatory in the VMware driver that are not mandatory for other hypervisors. Non mandatory attributes for VMware but specific to them are also recommended to have a default.

  • /etc/one/vmm_sh/vmm_sh_vmwarerc : This file holds instructions to be executed before the actual driver load to perform specific tasks or to pass environmental variables to the driver. The syntax used for the former is plain shell script that will be evaluated before the driver execution. For the latter, the syntax is the familiar:
  ENVIRONMENT_VARIABLE=VALUE
  • TM Driver
    • In order to change the path of the image repository (as explained here), the REPO_NAME variable of /usr/lib/one/tm_commands/vmware/tm_ln.sh needs to be edited.
    • /etc/one/tm_vmware/tm_vmware.conf : This files contains the scripts tied to the different actions that the TM driver can deliver. You can here deactivate functionality like the DELETE action (this can be accomplished using the dummy tm driver, dummy/tm_dummy.sh) or change the default behavior.

More generic information about drivers: