FutureGrid Image Management for Cloud/HPC Infrastructures with OpenNebula

FutureGrid (FG) is a testbed providing users with grid, cloud, and high performance computing infrastructures. FG employs both virtualized and non-virtualized infrastructures. Within the FG project, we offer different IaaS frameworks as well as high performance computing infrastructures by allowing users to explore them as part of the FG testbed.

To ease the use of these infrastructures, as part of performance experiments, we have designed an image management framework, which allows us to create user defined software stacks based on abstract image management and uniform image registration. Consequently, users can create their own customized environments very easily. The complex processes of the underlying infrastructures are managed by our software tools and services. These software tools are not only able to manage images for IaaS frameworks, but they also allow the registration and deployment of images onto bare-metal by the user. This level of functionality is typically not offered in a HPC (high performance computing) infrastructure. Therefore, our approach changes the paradigm of administrator-controlled dynamic provisioning to user-controlled dynamic provisioning, which we also call raining. Thus, users obtain access to a testbed with the ability to manage state-of-the-art software stacks that would otherwise not be supported in typical compute centers. Security is also considered by vetting images before they are registered in a infrastructure. Figure 1 shows the architecture of the image management framework.

Figure 1. Image Management architecture

This framework defines the full life cycle of the images in FutureGrid. It involves the process of creating, customizing, storing, sharing, and registering images for different FG environments. To this end, we have several components to support the different tasks involved. First, we have an Image Generation tool that creates and customizes images according to user requirements (see Figure 2-a). The second component is the Image Repository, which is in charge of storing, cataloging and sharing images. The last component is an Image Registration tool, which prepares, uploads and registers images for specific environments, like HPC or different cloud frameworks (see Figure 2-b). It also decides if an image is secure enough to be registered or if it needs additional security tests.

Figure 2. Image Generation and Image Registration flow charts.

Within this framework, OpenNebula plays an essential role supporting the image creation process. As we can see in Figure 2-a, the image generation component is able to create images from scratch or by cloning images from our image repository. In case we generate an image from scratch, the image is created using the tools to bootstrap images provided by the different OSes, such as yum for CentOS and deboostrap for Ubuntu. To deal with different OSes and architectures, we use cloud technologies. Consequently, an image is created with all the user’s specified packages inside a VM instantiated on-demand by OpenNebula. Therefore, multiple users can create multiple images for different operating systems concurrently; obviously, this approach provides us with great flexibility, architecture independence, and high scalability.

More information in:

  • J. Diaz, G.v. Laszewski, F. Wang, and G. Fox. “Abstract Image Management and Universal Image Registration for Cloud and HPC Infrastructures”, IEEE Cloud 2012, Honolulu, Hawaii, June 2012.
  • FutureGrid Rain Software Documentation. http://futuregrid.github.com/rain
  • FutureGrid Portal. https://portal.futuregrid.org/