OpenNebula 5.4 – Enhancements in vCenter Integration

The development of OpenNebula 5.4 has already started, and the biggest focus for this release is improving the vCenter integration. In particular, two areas will be addressed, networking and storage, in order to bring the vCenter up to speed with the open source hypervisor, KVM. After this release there will be virtually no differences between managing VMs on VMware or on KVM.

Networking

The goal in the vCenter network integration is to be able to create Networks (ie Port Groups and Distributed Port Groups) from within OpenNebula, not just consume then. We will be using the same mechanism as in KVM, ie, configuring the underlying network dynamically as required by the Virtual Machines, including the management of VLAN IDs.

 

picture1

There is a clear benefit from this new feature, as it implies a great improvement to the provisioning model. Typical use cases involves administrators creating groups of users or even VDCs on a multi-tenancy environment, where network isolation is a must. Currently, the administrators are required to create beforehand either Port Groups or Distributed vSwitches in the vCenter in order to provide tenants with isolation.

The second step is a big enterprise, which will take more than just one release to accomplish, which  is to offer this functionality directly in OneFlow, with the ability to create new networks automatically whenever a service is launched. This will allow tenants to spin up network isolated multi-vm services without further management steps in vCenter or OpenNebula.

picture2

Storage

The aim here is to improve the vCenter storage drivers to be at the same level than the KVM counterpart, enabling:

  • System Datastores
  • Non persistent images

This again is a big enterprise that will take more than one release to accomplish.

This functionality will enable important features for the vCenter integration, like for instance the ability to allow the OpenNebula scheduler to intelligently choose the target system datastore (replacing Storage DRS!), support for storage quotas, disk resize both at boot and running time, and much more!

***

The Network and Storage import process will be greatly improved. Whenever a VM Template is imported, all the NICs and DISKs of the VM Template will be imported, and associated networks, datastores and images will be created.

Additionally, improvements in the driver efficiency will ensure the scalability of OpenNebula with very large scale infrastructures. The main focus will be time and memory efficiency, mainly in the monitoring operations.

OpenNebula roadmap is strictly based on your input. Thanks for your amazing feedback!

9 Years of OpenNebula!

nine

Today is the 9th anniversary of the founding of the OpenNebula open source project. We still remember the early days of OpenNebula, when we decided to start an open-source project to grow, distribute and support the software prototypes that we developed to support our research in virtualization management on distributed infrastructures. Yet here we are, nine years later, with one of the most successful projects in the open cloud ecosystem.

OpenNebula has grown into a widely-used user-driven project, with lots of examples of production use in academia, research and industry, over 100K downloads from the project repositories in the last year, more than 1,500 clouds connected to our marketplace, and large-scale infrastructures with tens of data centers and hundreds of thousands of cores. We are very proud of this growth as the only fully-open enterprise-ready management platform to build clouds.

None of it would be possible without your help, the hard work of thousands of users and contributors!

On behalf of the OpenNebula Team

Cloud TechDays 2017 – Call for Hosts

We are opening the Call for Hosts for the OpenNebula TechDays 2017!

The OpenNebula Cloud TechDays are day-long educational and networking events to learn about OpenNebula with a hands-on workshop on cloud installation and operation, and presentations from community members and users that focus on:

  • Sharing cloud use cases and deployment experiences
  • Introducing new integrations and developments
  • Describing other related cloud open-source projects and tools

The OpenNebula TechDay events are targeted at

  • Cloud architects
  • Data center admins
  • Systems admins and integrators
  • DevOps architects
  • Solutions architect

The emphasis is to deliver an education oriented event, with local speakers and users to come together to share stories from their experiences using OpenNebula.

OpenNebula TechDays started in March 2014 and we’ve celebrated already 22 TechDays in the Netherlands, Belgium, Spain, United States, Romania, Czech Republic, France, Canada, Malaysia, Bulgaria, Germany and Ireland. They have been hosted by organizations like:

  • BIT.nl
  • Transunion
  • HYVE
  • Microsoft
  • BlackBerry
  • Harvard University
  • Netways
  • and many others

Do you wish to become a TechDay host for 2017? It’s very easy, drop by the TechDays guidelines and form, and get in contact with us, we only require that you provide a room with enough capacity for the attendees and some essential materials (wifi, projector, etc…). Deadline for this call is December 16, 2017.

Thanks!

OpenNebula Newsletter – November 2016

Learn what kept the OpenNebula team busy this last month of November, the project activity, latest news, advances in the technology, community work and the dissemination efforts carried out.

You can check the material of the OpenNebulaConf 2016 on this blog post, including pics, slides and videos of the keynotes
We are looking for Cloud Developers and a Community Specialist to join our team, check this for more details.

Technology

Team is working hard on the next release, 5.4. An emphasis is given in vCenter integration, but also great advances are being worked upon on other areas:

  • Live disk resize operation, not only at boot time but while the VM is in running state
  • Full network management in vCenter, including the ability to create Networks and distributed vSwitches
  • Full storage management in vCenter, including resize and non-persistent images. This will effectively move the vCenter integration
  • Allow Folder definition for VMs in vCenter
  • Virtual Machine save for vCenter
  • Native HA for the OpenNebula controller
  • Affinity/Anti Affinity schedule options for VMs
  • Fine grain ADMIN and MANAGE VM operation assigment
  • More detailed history information, including user information
  • Many Sunstone goodies!

overcommitment2

…and many more features related with Sunstone, Database, upgrade path and more OpenNebula subsystems. We are working to make OpenNebula 5.4 the most useful and robust OpenNebula release to date!

uploadimage3

Also, an update of the VMware Cloud Reference Architecture was published. The update includes latest features in OpenNebula releases, like:

    • OpenNebula now allows to upload, clone and delete VMDK files.
    • VM importing workflow has been greatly improved through Sunstone, making it easier to import your existing workload into OpenNebula.
    • Resource pools defined in vCenter are supported by OpenNebula so available memory and CPU can be partitioned. When launching a VM from OpenNebula, a resource pool can be selected automatically or the user can choose one.
    • When a VM is instantiated from a VM Template, the datastore associated can be chosen. If DRS is enabled, then vCenter will pick the optimal Datastore to deploy the VM.

and much more!

Community

After the Conference hangover (both literally and figuratively), the community has been as active as ever.

We are restructuring the OpenNebula add-ons to keep track of those add-ons that are not longer maintained and we discourage the use of. If you are an add-on maintainer, give us a shot in the forum so we know about it.

As we always emphasize, it is very encouraging and important for an open source project to receive messages publicly like this.

Excellent summaries about OpenNebula, like the one that we recommend reading in this blog post, are a great way to spread the OpenNebula word. Kudos!

OpenNebula is a free and open source software solution for building clouds and for data centre virtualisation. It is based on open technologies and is distributed under the Apache License 2. OpenNebula has features for scalability, integration, security and accounting. It offers cloud users and administrators a choice of interfaces.

Another, and even more important, effort by our users can be seen in this blog post by Fuze, detailing their production use of OpenNebula and how it powers the infrastructure that boost their business. We are immensely proud and grateful for this information guys!

cloud_orchestration_v2

Currently, Fuze Private Cloud has a central OpenNebula deployment in a data center in the eastern part of the US. This fault tolerant OpenNebula deployment is configured to connect and manage Fuze data centers across US East/ West, Australia, Asia, and Europe.

By the way, we are taking care of the infrastructure that runs our web page, development portal, marketplace … we appreciate your patience. If you notice anything strange, let us know.

Outreach

The OpenNebulaConf 2016 has been a real success with awesome keynotes including usage of OpenNebula in Akamai and Telefonica. Also, it featured high quality tutorials and excellent workshops, which were very well received, as were the first day tutorials. Make sure that you attend next year to either the basic or the advanced one, they are a real bonus of the conference!

To sweeten your way to next year’s Conference, and to recap the contents of the different keynotes and lightning talks, we published a post with the material of 2016 OpenNebulaConf. Check out the YouTube channel, download the slides from the slideshare account or take a look to the pictures of the conference. Enjoy.

We will also be defining shortly the TechDays for 2017. Stay tuned if you are interested to host an OpenNebula event!

The OpenNebula team is growing at a fast pace. Are you looking to get into the cloud professionally? If you are looking to work in an agile environment to develop new features and integrations and participate in the upstream community, this is your chance. Check out the OpenNebula Systems vacancies if this is the case, we are currently interested in Cloud Developers and Community Specialists.

Remember that you can see slides and resources from past events in our Events page. We have also created a Slideshare account where you can see the slides from some of our recent presentations.

OpenNebulaConf 2016 Wrap-up

A couple of weeks ago, we celebrated OpenNebulaConf 2016, three days packed with interesting talks and great experiences exchange. It was a great success thanks to the support and participation of all of our community members, speakers, sponsors and friends.

If you missed the conference, now you have a chance to watch the talks in our YouTube channel, download the slides from the slideshare account or take a look to the pictures of the conference. Enjoy.

Update of VMware Cloud Reference Architecture

A year ago OpenNebula Systems published the VMware Cloud Reference Architecture, a blueprint to guide IT architects, consultants, administrators and field practitioners in the design and deployment of public and private clouds based on OpenNebula on top of VMware vCenter. This reference architecture is intended for organizations with existing VMware environments or expertise who want to limit changes to their underlying VMware infrastructure, but see benefits in a common provisioning layer via OpenNebula to control compute workloads and want to take a step toward liberating their stack from vendor lock-in.

Many things have changed since that document was published. This is a brief summary of what’s new and ready for you:

  • OpenNebula now allows to upload, clone and delete VMDK files.
  • VM importing workflow has been greatly improved through Sunstone, making it easier to import your existing workload into OpenNebula.
  • Resource pools defined in vCenter are supported by OpenNebula so available memory and CPU can be partitioned. When launching a VM from OpenNebula, a resource pool can be selected automatically or the user can choose one.
  • When a VM is instantiated from a VM Template, the datastore associated can be chosen. If DRS is enabled, then vCenter will pick the optimal Datastore to deploy the VM.
  • New disks can be hot-plugged and OpenNebula can be informed from erasing the VM disks if a shutdown or cancel operation is applied to a VM, so users won’t lose data accidentally.
  • Support for vCenter customization specifications, as a complementary alternative to contextualization.
  • Multi vCenter cluster can be now defined in a single VM Template definition.
  • Control how disks are managed in vCenter, through the KEEPS_DISKS_ON_DONE template variable which will help you to protect users data against accidental deletions.
  • Datastores in a Storage DRS can be used as individual datastores by OpenNebula.
  • A bandwidth limit per VM network interface can be applied. VM’s network usage information is now gathered from vCenter.
  • It’s possible to access the OneGate server from vCenter VMs since the onegate token is passed through to the VM.
  • And last but not least, cool features added to Sunstone: a smoother vCenter’s resource import, the Cloud View functionality has been extended, new tags for resources.

This blueprint has been created from the collective information and experiences from hundreds of users and cloud client engagements so your feedback is extremely valuable.

More features are continuously being added, OpenNebula is a project in constant evolution, so stay tuned and do not forget to send us your feedback!

How We Use OpenNebula at Fuze

fuze_logo
Fuze, headquartered in Cambridge, MA with additional locations in North America, Europe, Asia and Australia, is a global, cloud-based unified communications platform that empowers productivity and delivers insights across the enterprise through a single unified voice, video, and messaging application. Fuze allows the modern, mobile workforce to seamlessly communicate anytime, anywhere, across any device. With Fuze, customers have a single global carrier network leveraging a resilient private cloud infrastructure and QoS-engineered network to deliver the best enterprise-class IP PBX voice service available.

Given the company’s record momentum and growth, scaling the private cloud infrastructure to meet the global expansions and growth is a key to its customers’ satisfaction. In mid-2015, we deployed OpenNebula as part of the Fuze Private Cloud Management Stack to achieve a continuously reliable growing private cloud spanning Fuze global infrastructure.

Here’s a quick look at the key features of OpenNebula that powered Fuze Private Cloud:

  • Simple and lightweight deployment and easily extendable being open source.
  • Self-Serve capability through GUI (Sunstone), CLI and API.
  • Multi-tenancy built-in providing segregation among internal workloads.
  • Supporting Agile DevOps by abstracting underlying infrastructure and supporting mainstream hypervisors besides public cloud bursting.
  • Automated VM Orchestration via contextualization that personalize instances on instantiation time.
  • Service Orchestration provided by OneFlow and OneGate that support multi-tier applications using role based instances. Besides, OneFlow has the ability to apply auto-scaling and recovery rule support based on either generic or custom application attributes pushed via OneGate.

Currently, Fuze Private Cloud has a central OpenNebula deployment in a data center in the eastern part of the US. This fault tolerant OpenNebula deployment is configured to connect and manage Fuze data centers across US East/ West, Australia, Asia, and Europe.

cloud_orchestration_v2

VMware ESXi/vCenter 6.0 is the dominant virtualization technology used across its data centers backed by SAN-based storage and we also use Amazon Web Services. OpenNebula leverages the SOAP API offered by each DC vCenter to manage the entire global cloud infrastructure and present it to the different engineering teams as an effective single orchestration pane of glass. In addition, we connected OpenNebula to a few AWS Virtual Private Cloud (VPCs) with dedicated tenancy that it bursts into where applicable.

We are currently running OpenNebula 5.0 which brought in enhanced user experience through Sunstone and renamed lifecycle states along a redesigned Marketplace. We will upgrade to OpenNebula 5.2 soon to make use of the upgraded hybrid cloud drivers.

Overall, the OpenNebula project has taken Fuze Global Private Cloud to the next level and continues to be a fundamental factor for continuous innovation and customer satisfaction.

OpenNebula Newsletter – October 2016

The OpenNebula project activity this last month including the latest news, advances in the technology, the excellent community work and the dissemination efforts carried out.

The OpenNebulaConf 2016 happened this last week, and has been a total success! We’ll be uploading the video recording, slides and nice pics soon, stay tuned! We are looking for Cloud Engineers and Developers, as well as a Community Specialist, to join our team, check this for more details.

Technology

OpenNebula 5.2 Excession has officially hit the road this last October. This is a stable release, you can safely proceed now to upgrade your existing installations to enjoy the following improvements and new features:

  • New IPAM module, a general mechanism to integrate with different IPAM servers
  • Session management, with enhanced user authentication token allowing to maintain different sessions with the same user account working in different projects
  • Usability improvements in Sunstone and CLI
  • VM migration between clusters
  • Better VM fault tolerance hook, easier to implement fencing
  • More secure ssh drivers to avoid compromising the front-end if a hypervisor goes rogue
  • Improvements in ec2 hybrid drivers and vCenter drivers, optimizing for large scale deployments

labels_searches_sunstone
The team focus has now shifted to the next release, 5.4. Great functionality is planned, with an emphasis in vCenter integration. Expect great advances in this regard. Yes…we mean network management!. Also, full storage handling (non persistent images FTW!).

Community

Well, the community has been busy this month preparing their plane tickets and visit to Barcelona. It has been a blast at the Conf, and lots of interesting talks has shown us that OpenNebula is being used in amazing and crazy infrastructures around the world. We had the chance to host in our blog this quality post. This is an amazing work (thanks Marco!), which introduces Rancher and show how to launch OpenNebula Virtual Machines from the Rancher UI and provision them to run Docker compute hosts, which can then be used to run Docker containers. The post also comes with a promise!

That’s all, we will be back soon with another post about the integration of Rancher and OneFlow to deploy multi-tier services on OpenNebula clouds. Stay tuned!

opennebulamachinedriver
It is so good to know that old OpenNebula friends are also engaged in the community (like old but not forgotten OpenNebula contributors that showed up at the Conf)! For instance, Skymix is tinkering hard with OpenNebula with other technologies.

Very good news have arrived this last month from the community in the field of OpenNebula’s hybrid capabilities, extending the number of public clouds supported by OpenNebula to build hybrid cloud deployments. A new set of drivers developed by CSUC are now available as an add-on, enabling the hybrid extension of your OpenNebula cloud to any public cloud provider implementing the vCloud API. Check this exhaustive post for details. The software can be found in github

“With this addon, real hybrid architectures can use OpenNebula’s rich set of infrastructure management tools to manage cloud deployments across VCLOUD private, public and hosted cloud platforms.”

Outreach

It has been a pleasure! The OpenNebulaConf 2016 has been a real success, with awesome keynotes including usage of OpenNebula in Akamai and Telefonica, high quality tutorials and excellent workshops. We want to thank all the attendees, and especially the sponsors, for providing such a great atmosphere for the exchange of experiences, help to the newcomers and in general for being such a great community. It has been great to see you all, the team is looking forward to seeing you again next year!

The Conf had an educational twist that has definitely pleased the attendees. The workshops were very well received, and we believe this is the direction that we want to give the Conf, as it gives an extra value for the attendees, which not only see real world examples of OpenNebula usage but learns how to use the software, and how to tweak it. The first day tutorials were also well received, make sure that you attend next year to either the basic or the advanced one, they are a real bonus of the conference.

The OpenNebula team is growing at a fast pace. Are you looking to get into the cloud professionally? If you are looking to work in an agile environment to develop new features and integrations and participate in the upstream community, this is your chance. Check out the OpenNebula Systems vacancies if this is the case.

Remember that you can see slides and resources from past events in our Events page. We have also created a Slideshare account where you can see the slides from some of our recent presentations.

OpenNebula 5.2 ‘Excession’ is Out!

The OpenNebula project is proud to announce the availability of the stable release of OpenNebula 5.2 ‘Excession’. OpenNebula 5.2 (Excession) is the second release of the OpenNebula 5 series. A significant effort has been applied in this release to stabilize features introduced in 5.0 Wizard, while keeping an eye in implementing those features more demanded by the community. As usual almost every component of OpenNebula has been reviewed to target usability and functional improvements, trying to keep API changes to a minimum to avoid disrupting ecosystem components. Also, new components have been added to enhance the OpenNebula experience.hyperlinks_sunstoneOne important new module is the IPAM subsystem. In order to foster SDN integration, an important step is being able to integrate OpenNebula with existing IPAM modules, in those cases where outsourcing of IP management is required in the datacenter. Fitting in the OpenNebula architecture design principles, the IPAM subsystem interacts with IPAM servers using drivers, and as such a IPAM driver lets you delegate IP lease management to an external component. This way you can coordinate IP use with other virtual or bare metal servers in your datacenter. No default integration is provided, but rather to effectively use an external IPAM you need to develop four action scripts that hook on different points of the IP network/lease life-cycle.

Another great addition in Excession is the ability to use group bound tokens. The goal is to be able to use OpenNebula for different projects, which are identified with different groups. For instance, the same user can use OpenNebula for “WebDevelopment” project and a “BioResearch” one, for instance. This user can request a couple of tokens tied to each of these groups. Upon login with the “WebDevelopment” token, she will only be seeing resources from that particular project, and all new resources (VMs, images, networks) will be created within that group, isolating them from the “BioResearch” group. This feature is available both in the CLI and Sunstone, with helpers and dialogs to create, maintain and use the tokens.

All the OpenNebula drivers have been improved for robustness. For instance, a new default timeout (which is configurable) has been defined to identify hanging operations and kill crashed processes. In this regard, the EC2 drivers has also been thoroughly revisited, being updated to the v2 of the aws ruby gem, ensuring compatibility with all Amazon EC2 regions. Error handling has been improved as well in the EC2 driver, adding operation retries to circumvent those situations where the EC2 API is not consistent, and adding improved logging.

Sunstone is the face of OpenNebula for both administrators and users, and hence a constant target of enhancements to improve usability. Excession brings to the cloud table stabilized features that were introduced in the Wizard maintenance releases, like for instances advanced searches (that now are maintained regardless of tab switching), labels colors and ergonomics, improved vCenter dialogs and import tables (now with feature à la Gmail), hyperlinks to access resources displayed in the info tabs, and many other minor improvements.

There are many other improvements in 5.2:

  • revamped group mapping in LDAP authentication -now being dynamic mapping-
  • limiting bandwidth per VM network interface in both KVM and vCenter
  • rollback mechanism in failed migrate operations, significantly improved fault tolerant hook -to provide high availability at the VM level-,
  • improved SPICE support, improvements in ebtables and Open vSwitch drivers, multiple CLI improvements -i
  • … and much more.

As with previous releases, it is paramount to the project to help build and maintain robust private, hybrid and public clouds with OpenNebula, fixing reported bugs and improving general usability.

This OpenNebula release is named after the Ian M. Banks novel, a recommended read, as well as having a fitting slang meaning, “something so technologically superior that it appears as magic to the viewer.”. We are confident that OpenNebula, if not really appearing as magic, at least solves elegantly your IaaS needs.

OpenNebula 5.2 Excession is considered to be a stable release and as such, and update is available in production environments.

The OpenNebula project would like to thank the community members and users who have contributed to this software release by being active with the discussions, answering user questions, or providing patches for bugfixes, features and documentation.

The group token functionality and dynamic LDAP group mapping into OpenNebula groups were funded by BlackBerry in the context of the Fund a Feature Program.

Relevant Links

New OpenNebula VCLOUD driver: Building Hybrid Clouds with VMware cloud providers

Based in its definition, the “Hybrid Cloud Computing” is a model which combines the use of multiple Cloud services across different deployment models, including combining the use of services of public cloud services private cloud outside or inside organization / institution.

Most companies and organizations have not been born in the “cloud”, a situation that often causes the cloud resources are to be connected to traditional systems or applications with some criticality and are usually located in their own premises. This type of architecture is the most common where the keys to their success pass take into account aspects such as integration capabilities, hyper-converged management, etc.

Cloud bursting is always welcome!

Today we are one_vcloudsharing exciting news about the expansion of the number of public clouds supported by OpenNebula to build hybrid cloud deployments. As a result of the collaboration between OpenNebula and CSUC, a new addon to support VCLOUD providers has been added to OpenNebula catalogue.

“With this addon, real hybrid architectures can use OpenNebula’s rich set of infrastructure management tools to manage cloud deployments across VCLOUD private, public and hosted cloud platforms.”

 

The driver is developed for Opennebula 5.x and VCLOUD 5.5 version and is released today to be available for testing. The integration has been carried out using the ruby_vcloud_sdk, which interacts with the vCloud Director API, enabling a complete control of the lifecycle of Virtual Machines in a transparent way within an OpenNebula cloud. Thanks to these new addon, private resources can be easily supplemented with resources from external providers to meet fluctuating demands.

https://github.com/OpenNebula/addon-vcloud-driver

Description

This addon gives Opennebula the posibility to manage resources in VMware vCloud infraestructures. It includes virtualization and monitoring drivers.

This driver is based on vCenter Driver and uses a modified version of ruby_vcloud_sdk.

Alt text

Features

This addon has the following capabilities:

  • Deploy, stop, shutdown, reboot, save, suspend, resume and delete VM’s in the Virtual Data Centers hosted in vCloud.
  • Create, delete and revert snapshots of VM’s.
  • Change RAM and CPU values of VM.
  • It’s able to hot-attach and detach NICs to VM’s.
  • Automatized customization of the VMs instanciated.
  • Obtain monitoring information from the VDC, Datastore and VM’s.
  • In this development version we manage vApps with one VMs inside (A VM in OpenNebula equals a vApp with one VM in vCloud).
  • Each Virtual Data Center (VDC) in vCloud is managed as a Host in OpenNebula.
  • Import networks, hosts, templates and datastores hosted in vCloud using onevcloud script.

https://github.com/OpenNebula/addon-vcloud-driver

Need more information? You are welcome to use the OpenNebula community instruments to ask around (for instance, the forums tool is a good place to pose your questions) or reserve a seat to see details inside the next Open Cloud Free session in Barcelona (24/10 14:00h) https://www.eventbrite.com/e/open-cloud-free-session-inside-opennebulaconf-tickets-27753771277

As always, we value your feedback and contributions to this new feature!

Barcelona UserGroup Team –  www.cloudadmins.org