Three years ago, driven by the needs of some of our larger users, we incorporated support for Virtual Data Centers (vDCs) and multiple Zones into OpenNebula 3.0. Since that time, this innovative vDC functionality has helped many IT organizations to make the transition towards the next generation of cloud infrastructures supporting on-demand provisioning of multiple fully-isolated vDCs. Thanks to the feedback received by many of these organizations during the last years, we have improved this functionality and its integration with the rest of subsystems. This post describes the new cloud provisioning model based on vDCs that is brought by OpenNebula 4.6. The new model offers an integrated and comprehensive framework for resource allocation and isolation in federated data centers and hybrid cloud deployments.

The Infrastructure Perspective

Common large IT shops have multiple Data Centers (DCs), each one of them consisting of several physical Clusters of infrastructure resources (hosts, networks and storage). These Clusters could present different architectures and software/hardware execution environments to fulfill the needs of different workload profiles. Moreover, many organizations have access to external public clouds to build hybrid cloud scenarios where the private capacity of the Data Centers is supplemented with resources from external clouds to address peaks of demand. Sysadmins need a single comprehensive framework to dynamically allocate all these available resources to the multiple groups of users.

For example, you could have two Data Centers in different geographic locations, Europe and USA West Coast, and an agreement for cloudbursting with two cloud providers, Amazon and SoftLayer. Each Data Center runs its own full OpenNebula deployment.

Resources
The Organizational Perspective

Users are organized in Groups (also called Projects, Domains, Tenants…). A Group is an authorization boundary that can be seen as a business unit if you are considering it as private cloud or as a complete new company if it is public cloud. A powerful, configurable ACL system is needed to enable different authorization scenarios, from the definition of group Admins to the privileges of the users that can deploy virtual machines. Each Group can execute different types of workload profiles with different performance and security requirements.

For example, you can think Web Development, Human Resources, and Big Data Analysis as business units represented by Groups in OpenNebula.

Groups

The following are common enterprise use cases in large cloud computing deployments:

  • On-premise Private Clouds Serving Multiple Projects, Departments, Units or Organizations. On-premise private clouds in large organizations require powerful and flexible mechanisms to manage the access privileges to the virtual and physical infrastructure and to dynamically allocate the available resources. In these scenarios, the Cloud Administrator would define a Group for each Department, dynamically allocating resources according to their needs, and delegating the internal administration of the Group to the Department IT Administrator.
  • Cloud Providers Offering Virtual Private Cloud Computing. Cloud providers providing customers with a fully-configurable and isolated environment where they have full control and capacity to administer its users and resources. This combines a public cloud with the control usually seen in a personal private cloud system.

A New Cloud Provisioning Model Based on vDCs

A Group is simply a boundary, you need to populate resources into the Group which can be consumed by the users of the Group. These resources are obtained from Resource Providers that can be located in different Data Centers, ending up with the creation of a vDC. A Resource Provider is a Cluster of infrastructure resources (physical hosts, networks, datastores and external clouds).

For example, you could create three different vDCs:

  • BLUE: Allocation of (ClusterA@DC_West_Coast + Cloudbursting) to Web Development
  • RED: Allocation of (ClusterB@DC_West_Coast + ClusterA@DC_Europe + Cloudbursting) to Human Resources
  • GREEN: Allocation of (ClusterC@DC_West_Coast + ClusterB@DC_Europe) to Big Data Analysis

vdcs
A vDC is a fully-isolated virtual infrastructure environment where a Group of users, under the control of the vDC admin, can create and manage compute, storage and networking capacity. The users in the vDC, including the vDC administrator, would only see the virtual resources and not the underlying physical infrastructure. The physical resources allocated by the cloud administrator to the vDC can be completely dedicated to the vDC, providing isolation at the physical level too.

The privileges of the vDC users and the administrator regarding the operations over the virtual resources created by other users can be configured. In a typical scenario the vDC administrator can create virtual networks, upload and create images and templates, and monitor other users virtual resources, while the users can only instantiate virtual machines and virtual networks to create their services. The administrators of the vDC have full control over resources and can also create new users in the vDC.

UsersVDCs
Users can then access their vDC through any of the existing OpenNebula interfaces, such as the CLI, Sunstone, OCA, or the OCCI and AWS APIs. vDC administrators can manage their vDCs through the CLI or the vDC admin view in Sunstone. Cloud Administrators can manage the vDCs through the CLI or Sunstone.

The Cloud provisioning model based on vDCs enables an integrated, comprehensive framework to dynamically provision the infrastructure resources in large multi-datacenter environments to different customers, business units or groups. This brings several benefits:

  • Partitioning of cloud physical resources between Groups of users
  • Complete isolation of users, organizations or workloads
  • Allocation of Clusters with different levels of security, performance or high availability
  • Containers for the execution of software-defined data centers
  • Way of hiding physical resources from Group members
  • Simple federation, scalability and cloudbursting of private cloud infrastructures beyond a single cloud instance and data center

Want to Try?

The Beta release of OpenNebula 4.6 will be available in few days. In the meantime you can enjoy this screencast about partitioning clouds with vDCs.

We are looking forward to your feedback!.

OpenNebula 4.6 is slowly cooking, getting ready to get out of the oven and being spun around the certification carrousel. We’ve created a screencast to show on of the most interesting features that will be available in the next, shiny new release: the ability to manage Virtual Data Centers natively in OpenNebula, via Sunstone or the CLI.

First, let’s define some concepts. In OpenNebula, a Group (of users) is the authorization boundary. Authorization comes using ACLs  built in OpenNebula. ACLs model can be used to control who can manage the Group (that is, the Group admin) and who can deploy virtual machines. A Group can be seen as business unit if you are considering it as private cloud and complete new company if it is public cloud. You can think Human Resources, Marketing and Sales as business units represented by Groups in OpenNebula. Moreover, a Resource Provider is a OpenNebula Cluster of infrastructure resources (aggregation of physical hosts, virtual networks and datastores).

Combining the two above, OpenNebula can handle Virtual Data Centers (vDCs). vDCs are containers for the execution of virtual machines and a way of hiding physical resources from Group members. A Group is simply a boundary, you need to populate resources within the Group which can be consumed by the users of the Group. These resources are obtained from Resource Providers, ending up with the creation of a vDC by combining a Group and one or more Resource Providers. These Resource Providers can reside in other other datacenters, thus achieving a DC federation. But this a story for another screencast ;)

For other interesting screencasts, please take a look to the screencasts page.

This new feature is funded by Produban in the context of the Fund a Feature Program.

We’ve crafted this post to answer a recurring question we’ve been hearing lately, specially from organizations planning to build their own private cloud:

How do you compare OpenNebula with OpenStack?…

This is indeed a complex question. There is no single answer because open-source projects and technologies present several dimensions. But we are far from afraid to answer it: the short, tl;dr version would be that they represent two different open-source models. While OpenNebula is an open-source effort focused on user needs, OpenStack is a vendor-driven effort.

This is neither a question of one being better than the other, they simply represent different approaches. Let us compare both open source options based on the following criteria: internal organization, governance model, roadmap definition, contributor profile, target user, product, and market competition. Obviously this comparison is biased (no way around that), but we have tried to be as neutral as possible.

A. Different Open-source Project Models

Both projects release code under the liberal Apache 2.0 license, follow a transparent development process with a public roadmap, and have the same license agreement for new contributions. They present significant differences though, specially in:

  • Internal Organization. While OpenStack comprises many different subprojects (14 at the time of writing this post) aimed at building the different subsystems in a cloud infrastructure, OpenNebula offers a single integrated, comprehensive management platform for all cloud subsystems.
  • Governance Model. The main difference between both projects is in their governance model, mainly for the definition of the architecture, the release cycle and the roadmap. While OpenStack is controlled by a Foundation driven by vendors, OpenNebula follows a centralized, “Benevolent Dictator” approach. OpenNebula is managed by a single organization that focuses on the interest of the project and strategically leads it to ensure that meets users needs.
  • Roadmap Definition. OpenNebula roadmap is completely driven by users needs with features that meet real demands, and not features that result from an agreement among the different vendors participating in the management board of the project.
  • Contributor Profile. While in OpenNebula most of contributions come from the users of the software, the contributors to OpenStack are mostly vendors building their own OpenStack-based cloud product. Since we started OpenNebula six years ago, we wanted users to have a voice in the project and not to privilege contributors over users.

Now the question is,

Why is OpenNebula following a “Benevolent Dictator” management model?.

In our view, OpenStack is governed by a consortium of competitors, trying to create its own product or to provide compatibility for its particular device. The mixture of vendor motivations makes it increasingly difficult for a foundation to meet both the needs of the project and the monetization goals of each vendor. It is also interesting to remark that many of these vendors are also offering commercial products that directly compete with OpenStack components.

Traditionally, multi-vendor industrial consortiums are the best approach to commoditize a core component in the long term, mainly when there exists solid base software, but not to bring to market a complete enterprise-ready solution from scratch in the short term. In these situations the addition of more developers and members slows the project down, and the well-known Brooks law (The Mythical Man-Month) applies both at development and governance levels. OpenStack is reaching a point where the consensus based approach has limited the competitiveness of the project.

We believe that a centralized model with a strong individual leadership is the best way to quickly build a production-ready enterprise-class open-source product, mainly in the early stages of a fast growing market. Please do not pin this on us being control freaks; we do so because we want to create a great product and we want to take responsibility for the entire product and need to be responsive to our users. Benevolent dictator governance is the model followed by other successful open-source projects like Android or Linux Kernel, and, in our view, it is the most effective way to focus on engineering quality, to prioritize user needs, and also to ensure long term support.

The above reasons are the foundation of this claim: OpenNebula is made for users by users, OpenStack is made for vendors by vendors. This may seem like a daring statement, but we have been following this path for years, and haven’t observed anything that proves this wrong.

B. Different Cloud Models

Although there are as many ways to understand cloud computing as there are organizations planning to build a cloud, they mostly fall between two extreme cloud models:

  • Enterprise Cloud Model (Datacenter Virtualization): On one side, there are businesses that understand cloud as an extension of virtualization in the datacenter; hence looking for a VMware vCloud-like infrastructure automation tool to orchestrate and simplify the management of the virtualized resources.
  • Public Cloud Model (Infrastructure Provision): On the other side, there are businesses that understand cloud as an AWS-like cloud on-premise; hence looking for a provisioning tool to supply virtualized resources on-demand.

CloudModels

Although OpenStack now tries to be everything for everyone, it was created as an open-source effort to compete against Amazon Web Services (AWS). Therefore while OpenStack is addressing the Infrastructure Provision segment; OpenNebula better meets the needs of Enterprise Cloud Computing. Since both tools enable infrastructure cloud computing, there is some overlap in the features they provide. However, each cloud model presents different architectural constraints and requires specialized interfaces, management capabilities and integration support. OpenNebula and OpenStack serve different needs and implement completely different philosophies.

C. Different Product Views

OpenNebula is a single enterprise-ready open-source product, easy to install and operate, with a single installing and updating process, a one-stop community and a long-term commercial support. Any organization can use the open-source distribution to build a production cloud, and receive best-effort support through the community mailing list. Additionally, any organization can purchase commercial support directly from the developers. The important aspect is that we do not deliver enterprise editions of the software, we commercially support the community software.

Architectures

On the other hand, OpenStack comprises many subprojects with different levels maturity that require complex integration to achieve a functional cloud infrastructure. A growing number of components and subprojects is making even more difficult their integration and coordination, and the delivery of a single coherent solution. No update path is provided if you want to install a new version, and there is not commercial support. Any organization interested in using OpenStack, and requiring commercial support and enterprise maturity, is recommended (by the vendors running the project) to deploy any of the several enterprise distributions.

ValueChain

From a business perspective, OpenNebula does not compete with OpenStack but with the many existing vendor “stacks” based on OpenStack, mainly with those by HP, Red Hat and IBM. These enterprise-grade distributions incorporate different versions of the OpenStack components with extended features, custom enhancements and integrations that may erode their compatibility and interoperability. Moreover many of them include proprietary components and exhibit significant differences in the implementation of critical underlying functionality.

So the organization that chooses OpenStack is actually using proprietary software based on OpenStack, and is locked into that specific distribution given that the vendor only supports its own stack, not the community version. Even worse, there is no way to migrate to another vendor distribution. In other words, these distributions do not offer the main benefits of open-source: low-cost, no lock-in, flexibility and interoperability.

D. A Look To the Future

We expect OpenStack to further fragment into more vendor specific “stacks” with narrow test matrices and extended proprietary features that lock customers in and don’t interoperate well. OpenStack’s biggest success is marketing. These vendor “stacks” and cloud providers will continue marketing “OpenStack” as the primary and, in most cases only, differentiator.

However OpenStack penetration in the market is relatively small compared with the investment made by vendors and VCs. These vendor specific “stacks” are not only competing with OpenNebula, other open-source cloud management platforms like CloudStack and Eucalyptus, and proprietary incumbents, they are also competing between them and with the open source community itselfAll vendors claim they are the OpenStack leader because it’s a winner-take-all game. Only one of the OpenStack distributions will gain critical mass on public and private clouds. Red Hat, now the dominant contributor to OpenStack, is in our view the only plausible winner

Don’t get us wrong, OpenStack is an open-source project with excellent developers, and some of its components are great from a technology point of view. Because a single cloud management platform can not be all things to all people, we will see an open-source cloud space with several offerings focused on different environments and/or industries. This will be the natural evolution, the same happened in other markets. OpenNebula and OpenStack will coexist and, in some cases, work together in a broad open cloud ecosystem. In the meantime, we will continue with our focus on solving real user needs in innovative ways, and getting our users involved in a fully vendor-agnostic project.

 

Runtastic_logoAbout 10 months ago our small ops team had to design a new infrastructure solution to run the services needed for our ecosystem. Until then we ran everything on root servers at a huge german webhoster. We knew that we wanted to go for our own infrastructure, but still be able to burst our load peaks into the cloud. We played around with several solutions on the market and finally wanted to meet some people at CEBIT 2013 to give us some hints for our decision.

To be honest we didn’t really consider OpenNebula at that time, but since the only person we could get hold of during lunchtime at CEBIT was Tino Vazquez from C12G we started to. A week later we fired up our first testing environment and 4 month later our productive OpenNebula Cluster started its work.

Why did we choose OpenNebula? Most of the software we use at runtastic is opensource and we wanted to set on KVM for that reason instead of some proprietary solution.  So we actually were looking for some management solution on top of KVM in the first step. However OpenNebula fullfills our requirements in several other terms as well.We wanted something simple and stable. We are a small team and have a lot of different services to maintain. We didn’t want to have an additional spot we need to focus on. What we can say up to now: We use OpenNebula, thats it. No big issues, no big efforts in performance tuning. The infrastructure runs and runs and runs … Another thing was, that have a lot of ideas in extending OpenNebula (integrate LXC, trigger chef bootstrap from within OpenNebula, make use of Netapp cloning features, …). We know ruby. OpenNebula is written in ruby. Perfect match.

 

hardware_install

At the moment we have a pretty straight forward solution based on two Netapp 2240 providing NFS shared storage and 28 Cisco UCS Servers running KVM on Ubuntu 12.04. Everything is connected via 10G Cisco Nexus Switches. We still have bare metal hardware for our databases (MySQL, MongoDB and Cassandra) and the services needed for OpenNebula itself, but everything else, like webservers, mobile endpoints and backend services is running in our cloud. All in all we run about 280 quite big (compared to a typical AWS machine) VMs consuming about 200 cores, 4TB memory and more than 20000 IOPS in average.

 

Everything in our infrastructure, also OpenNebula and KVM is set up automatically with Chef, so for us it is very easy to add new nodes to our cluster, or create new datastores.

Next step will be to integrate the VMware vCloud based IAAS solution of our datacenter provider to cover load peaks.

 

We are happy to announce that next February 27 we will be giving a tutorial at the Open Cloud Forum event, that will take place at Cloud Expo Europe 2014, London.

open_cloud_forum

This hands-on tutorial will give an overview of how OpenNebula is used to build and operate private clouds. The target audience is devops and system administrators interested in deploying a private cloud solution, or in the integration of OpenNebula with other platform. The attendees will build, configure and operate their own OpenNebula cloud in their laptops, using two VirtualBox virtual machines.

Don’t miss this great conference, register now for free!

The OpenNebula Project is proud to announce the first OpenNebula TechDays, to be held in Ede, Netherlands, next 26th March. This event will be hosted by BIT.nl , an internet service provider with a strong relation with the open source community.

The agenda of this first TechDay includes a hands-on cloud installation and operation workshop. This workshop will give an overview of how OpenNebula is used to build and operate private clouds, and it is targeting devops and system administrators interested in deploying a private cloud solution. The attendees will build, configure and operate their own OpenNebula cloud. Moreover, the event includes presentations from OpenNebula community members and users.

If you are interested in giving a talk in the TechDays, let us know by email. The talk should be oriented towards sharing cloud use cases and deployment experiences, introducing new integrations and ecosystem developments and/or describing other related cloud open-source projects and tools. Also, there are sponsorship opportunities available. This is a not for profit event, all funds raised are used for OpenNebula promotion and for further OpenNebula TechDay Events. If interested, let us know.

Registrations are running via EventBrite with a nominal price of €25.00. Registration cost includes refreshments and lunch on the day. The number of seats is limited  to ensure there is plenty of opportunity for everyone to interact. We encourage everyone to register as early as possible.

This is the first event of a series of OpenNebula TechDays to be held all around the globe, stay tuned since we will announce shortly other events and one of them might be next door!

The OpenNebula Project announces the general availability of a Retina maintenance release, OpenNebula 4.4.1.

This is a maintenance release that fixes bugs reported by the community after OpenNebula 4.4 was released.

This release only includes bug fixes and is a recommended update for everyone running any 3.x or 4.x version of OpenNebula, whom for any particular reason do not want to upgrade their cloud manager to the latest available OpenNebula version.

This edition of the CentOS Dojo has been very intense. Besides being crowded with very interesting people and great conversations (as it’s customary in all Dojos) the hackathon went even better than we would have hoped. The following items were achieved:

  • CloudInit 0.7.4 100% supported by OpenNebula and CentOS . Big thanks to Sam Kottler for providing that package and assisting us with the process.
  • Initial set of systemd scripts for OpenNebula developed, will be published as soon as CentOS 7 is out.
  • OpenNebula-node-xen package developed. It will be added to the CentOS packages very shortly. Thanks to the Xen guys and to Johnny Hughes for his assistance with the kickstart file.

We also had the chance to meet new OpenNebula users, which  as usual provide great feedback and exciting comments. It is also worthy of mention the conversation we had with John Mark from the GlusterFS project, who besides providing excellent ideas and recommendations for Gluster, will work with us very shortly in an announcement!

dojo

The Fosdem has also been very exciting: interesting conversations around Cloud, interoperability, OpenNebula demos, storage solutions and new projects using OpenNebula that wil be announced very soon!

Big thanks to Karanbir Singh for organizing the Dojo and the hackathon, and for having us at the CentOS table of the Fosdem.

Stand by for a bunch of exciting announcements that have blossomed these past days!

February 3rd, 2014. The OpenNebula team is pleased to announce a beta release of AppMarket 2.0. This release includes important features that are a direct result of feedback from production deployments.

This release extends the AppMarket functionality by adding a new set of features that enables the management and processing of OVA files. A new component AppMarket Worker is introduced, which handles the OVA package treatment (download, unpack, OVF parsing) and image format conversion. This release also features a new API and a new AppMarket interface via Sunstone.

What’s New in AppMarket 2.0 Beta

In the following list you can check the highlights of AppMarket 2.0 Beta:

AppMarket

  • Multi-image Appliances: AppMarket Appliances can now have multiple disks, which creates new possibilities to upload and register more complex and feature complete Appliances.
  • Sunstone Import: users will be able to import registered Appliances to OpenNebula via Sunstone by using pre-filled Image and Template creation forms.
  • Sunstone Views: two new Sunstone views: an AppMarket *admin* view, that allows full control of the AppMarket, and a *user* view, that allows end users to import registered appliances.

AppMarket Worker

A new horizontally-scalable component that subscribes to the AppMarket and executes jobs. It enhances the AppMarket functionality by adding:

  • OVA processing: If a URL to an OVA package is supplied, the AppMarket will be able completely process it and integrate it to its repository. This involves: downloading and unpacking of the OVA package, parsing of the OVF file and creating a corresponding OpenNebula template.
  • Format conversion: appliances registered in a specific format, can be converted to a new format.
  • Extensibility: All the operations described above are implemented within the framework of an extensible driver engine, which allows further customization and integration by the administrators.

These new features will be only available interacting with AppMarket from an OpenNebula 4.6 deployment. If you want to test them before OpenNebula 4.6 is released, you can use the master branch of OpenNebula.

Acknowledgements

The new features introduced in the AppMarket 2.0 were funded by Produban in the context of the Fund a Feature Program.

appmarket