Agenda for Upcoming Cloud TechDays in USA & Canada

We are pleased to announce the agenda for the upcoming Cloud Technology Days that will be held in April in USA and Canada:

They feature a 4-hour hands-on tutorial in which you will learn how to install and configure an OpenNebula Cloud from scratch, and presentations from community members and users.

The event in Dallas, TX is hosted by Improving, a complete IT services firm, offering training, consulting, recruiting, and project services. The event is also sponsored by Digital Ocean, a cloud infrastructure provider focused on simplifying web infrastructure for software developers. In addition to the hands-on tutorial that will be held in the morning we will have the following talks during the afternoon:

  • Javier Fontán from OpenNebula Systems will talk about Customizing Virtual Machine Images and Docker Machine and Swarm on OpenNebula

 

The event in Toronto, Canada is hosted by Canada151 Data Centers, which provides carrier neutral, highly redundant Colocation and Disaster Recovery services from their West Toronto facility.  The event is also sponsored by Solgenia, a Cloud technology provider specializing in SaaS, PaaS and IaaS solutions. In addition to the hands-on tutorial that will be held in the morning we will have the following talks during the afternoon:

  • Khoder Shamy from Fuze (formerly ThinkingPhones) will share how they leverage OpenNebula open-source project in the rapidly growing global private infrastructure at Fuze
  • Javier Fontán from OpenNebula Systems will talk about Customizing Virtual Machine Images and Docker Machine and Swarm on OpenNebula
  • Presentation about Canada151 Data Centers
  • Varadarajan Narayanan from Wayz Infratek will talk about Hyperconvergence and OpenNebula
  • Presentation about Solgenia Private Cloud and Data Centers

Throughout the day, Canada151 will be conducting tours of the data center for interested parties.

 

The event in Cambridge, MA is hosted by Harvard Research Computing, which was established in 2007 as part of the Faculty of Arts & Sciences (FAS) Division of Science in Harvard University with the founding principle of facilitating the advancement of complex research by providing leading edge computing services. In addition to the hands-on tutorial that will be held in the morning we will have the following talks during the afternoon:

  • John Noss from Research Computing at Harvard will talk about their experience with OpenNebula in Research Computing
  • Dan Kelleher from Research Computing at Harvard will talk abouth the Corona Project
  • Roy Keene from Knight Point will talk about OpenNebula: The Integrator’s Story
  • Jaime Melis from OpenNebula Systems will talk about Docker Machine and Swarm on OpenNebula
  • Javier Fontán from OpenNebula Systems will talk about Customizing Virtual Machine Images

 

Many thanks to Improving, DigitalOceanCanada151 Data Centers, Solgenia and Research Computing at Harvard University for hosting and sponsoring these events.

Looking forward to seeing you there, these are three community events you cannot miss! Seats are limited, register ASAP!

OpenNebula Newsletter – March 2016

The OpenNebula monthly newsletter, with latest developments, events and future plans for the upcoming months from the OpenNebula project. Read this newsletter to keep up to date with your favorite Cloud Management Platform.

The OpenNebula Conf 2016 is due in October in Barcelona, Spain, and the Call for Speakers is now Open!. If you are a OpenNebula practitioner, user, architect, devop, admin or developer and have something to share, we welcome your submission. You might be also interested in taking a look at the sponsorship opportunities. Also check out the projected OpenNebula TechDays for this year in case one is close to your location. Upcoming TechDays in April will be held in Dallas,Toronto and Cambridge. Also, there is some (but just a few!) seats left for the OpenNebula course OpenNebula Systems is giving in Cambridge.

Technology

OpenNebula 5.0 is brewing and the team is closing new features and liking the overall feeling of the next major upgrade. We are set for a beta release of 5.0 this very same upcoming month of April, so stay tuned for the following:

  • revamped Marketplace, now a first class citizen in OpenNebula, to import/export images to and from any datastore (including the new vCenter datastores)
  • vCenter storage management, including datastore selection and disk hotplug, a full integration with the new Marketplace functionality. Share VMDKs among OpenNebula instances! Also, vCenter resource pool selection for deployed VMs will be possible.
  • fully integrated virtual router management (including default and transparent HA for routers). Link virtual networks together using a robust HA router, that will come out of the box in OpenNebula
  • revisited Sunstone style and interface changes (sneak peek), including an extension of the Cloud View available operations, to iron out the wrinkles in the user experience

There are more new bits and bobs, check the full list of feature in the development portal.
Continuing with the series of Docker integration posts, this past month this entry explained how to get Docker Swarms deployed with OpenNebula. The idea is to get Docker Swarm to build a cluster out of a collection of Docker Engine VMs deployed in OpenNebula with Docker Machine. Docker Swarm is native clustering for Docker, delivering many advantages, like scheduling, high availability, etc.

docker-swarm-opennebula
If you need professional support for your OpenNebula cloud, remember that OpenNebula Systems has extended the support services portfolio to include the Operating System of the virtualization nodes and the controller (which can be CentOS and/or Ubuntu), the hypervisor (libvirt and KVM), networking technologies as VXLAN/VLAN 802.1Q and Ceph as storage backend.

Community

The OpenNebula community never ceases to amaze us. Contributions to the common knowledge pool around OpenNebula deployments add huge value to the open source project. Several contributions were made this past month in this regard, let’s see the highlights.
A great addition is related to the LXCoNe addon with a very detailed and complete post. LXCoNE, a virtualization and monitoring driver for OpenNebula that comes as an add-on to provide OpenNebula with the ability to deploy LXC containers. It contributes to achieve better interoperability, performance and scalability in OpenNebula clouds. Right now, the driver is stable and ready for its release.
Also do not miss this post and contribution by TeleData, enabling the feed of monitoring data in the form of graphite metrics. Great work! And we blush with your words too!

In our opinion OpenNebula works like a charm. It is open and flexible. OpenNebula is one of the most mature, comprehensive and valuable Cloud-Stack available in the market

Overview (Zones + Clusters + Ressources)

And last but not least, our friends at LINBIT have contributed their OpenNebula driver for DRDB integration as an addon! This driver allows for highly available storage using DRBD9 + DRBD Manage in OpenNebula.

We are, as usual, thankful to have this wonderful community. Two thumbs up!

Outreach

The next OpenNebula Conference in Barcelona in October 2016 has already been announced . If you are willing to attend and can save now the date you can take advantage of a 40% discount in your Conf tickets. The Call for Speakers is now Open!. If you are a OpenNebula practitioner, user, architect, devop, admin or developer and have something to share, we welcome your submission.
More information is available from the event site. The Conf has already its first sponsor with StorPool. Welcome! Learn about the different sponsorship opportunities in the Conference web page. If you want to understand what all the OpenNebula Conference fuzz is about, check the last Conference material (talks, slides,pictures).
This March the OpenNebula team held a TechDay Madrid. A full house event with very interesting talks held by Rentalia in the crib that saw OpenNebula birth!

CdQ0wUOXIAAYm6r

If you are interested in participating in (or hosting) any of these TechDays agenda let us know. Besides the TechDays already celebrated (in Kuala Lumpur, Sofia and Madrid), this year member of the OpenNebula team will be participating in TechDays the following locations: Dallas, Toronto, Cambridge, Ede, Nuremberg and Dublin.
The Swiss Open Systems User Group would be gathering next 16th of June to discuss Cloud Computing. If you are around and want to learn more about Clouds and OpenNebula, drop by.
You may be interested in OpenNebula Systems training plans for 2016 for Europe and US. These courses are designed to train cloud administrators to properly operate an OpenNebula infrastructure. Please contact us if your would like to request training near you.
Members of the OpenNebula team will be present in the following events in upcoming months:

  • VMworld 2016 US, August 28 – September 1, Las Vegas (Mandalay Bay Hotel & Convention Center), Nevada, US.
  • VMworld 2016 Europe, October 17 – 20, Barcelona (Fira Barcelona Gran Via), Spain.

Remember that you can see slides and resources from past events in our Events page. We have also created a Slideshare account where you can see the slides from some of our recent presentations.

OpenNebula Newsletter – February 2016

Latest developments, events and future plans for the upcoming months from the OpenNebula project. Read this newsletter to keep up to date with your favorite Cloud Management Platform.

You might be interested in taking a look at the sponsorship opportunities for the next OpenNebula Conf 2016, due in October in Barcelona. Also it may be of your interest the projected OpenNebula TechDays for this year. Upcoming TechDays in March and April will be held in Madrid, Dallas and Toronto.

Technology

The OpenNebula team keeps working in OpenNebula 5.0, the next major upgrade that will incorporate a wealth of new features, keeping a smooth upgrade path. And no, this is no easy task folks. But we will make it happen.

OpenNebula 5.0 is a revolution around the corner. In a few weeks, a beta version will come out featuring the following highlights:

  • revamped Marketplace, now a first class citizen in OpenNebula, to import/export images to and from any datastore (including the new vCenter datastores)
  • vCenter storage management, including datastore selection and disk hotplug, a full integration with the new Marketplace functionality. Share VMDKs among OpenNebula instances!
  • fully integrated virtual router management (including default and transparent HA for routers). Link virtual networks together using a robust HA router, that will come out of the box in OpenNebula

5.0 will also feature a myriad of interface changes, including an extension of the Cloud View available operations, to iron out the wrinkles in the user experience and a long list of other features.

As you may have noticed, interest in Docker is surging. We see OpenNebula as the perfect IaaS support for Docker, and as such our strategy regarding integration of Docker ecosystem within OpenNebula is to follow a “container as a service” approach. This is containers within VMs, not replace KVM/ESX with containers to ensure we maintain strong multi-tenancy. First we want to integrate OpenNebula with Docker-Machine to be able to build a container environment within your cloud. This feature is very advanced, as you can see in blog post. The second step would be to integrate tightly with Docker Swarm to be able to create Swarms using the OneFlow component.

docker_arch

Community

Lots of exciting things are happening in the OpenNebula community. Collaborations, contributions, and yes! even praise. Let’s review the highlights.

LINBIT has developed a new set of drivers to give OpenNebula support for the excellent DRBD functionality. Check an informative demo in this video.

A new release of the Perl binding for the OpenNebula Cloud API (OCA) is also worth noting if Perl is your sysadmin scripting language of choice. No excuse to make all the chores automatically in your cloud infrastructure now!

A freshly brewed integration of OpenNebula with professional VDI platform has been announced by UDS Enterprise, to be available in the upcoming 2.0 release of their software.

Our friends at NodeWeavers never stop. Take a look at their new family addition. Cute is an understatement, and since it is based on OpenNebula, powerful another one.

World wide adoption of OpenNebula is a fact. If more proof is needed, this is an excellent one. OpenNebula has been translated to the Persian language! Check this out.

Overall, we are very proud of the OpenNebula community. OpenNebula won’t be half as good without you guys, keep it up!

Outreach

The next OpenNebula Conference in Barcelona in October 2016 has already been announced . If you are willing to attend and can save now the date you can take advantage of a 40% discount in your Conf tickets. More information is available from the event site. The Conf has already its first sponsor with StorPool. Welcome! Learn about the different sponsorship opportunities in the Conference web page. If you want to understand what all the OpenNebula Conference fuzz is about, check the last Conference material (talks, slides,pictures).

Two OpenNebula TechDays, from this year’s scheduled ones have already been hosted. The first on was the TechDay in Kuala Lumpur, at MIMOS, where we learned about the wide OpenNebula ecosystem that is fostering in Malaysia. We want to thank all the people at MIMOS who received us for making this possible. The second TechDay happened in Sofia, Bulgaria, hosted by StorPool. We also want to thank StorPool for the magnificent hosting and the excellent local promotion, which yielded an excellent number of attendees.

CcD9x9tUcAAWPuS

If you are interested in participating in (or hosting) any of these TechDays agenda let us know. The next TechDay will happen this month hosted by Rentalia, and it will be held in Madrid, Spain.

The Swiss Open Systems User Group would be gathering next 16th of June to discuss Cloud Computing. If you are around and want to learn more about Clouds and OpenNebula, drop by.

You may be interested in OpenNebula Systems training plans for 2016 for Europe and US. These courses are designed to train cloud administrators to properly operate an OpenNebula infrastructure. Please contact us if your would like to request training near you.

Members of the OpenNebula team will be present in the following events in upcoming months:

  • VMworld 2016 US, August 28 – September 1, Las Vegas (Mandalay Bay Hotel & Convention Center), Nevada, US.
  • VMworld 2016 Europe, October 17 – 20, Barcelona (Fira Barcelona Gran Via), Spain.

Remember that you can see slides and resources from past events in our Events page. We have also created a Slideshare account where you can see the slides from some of our recent presentations.

Upcoming Cloud TechDays Madrid and Dublin

Today we are announcing that we have just opened the call for speakers and registration to the following Cloud TechDays:

Send us an email at events@opennebula.org if you are interested in speaking at one of the TechDays and register as soon as possible if you are interested in participating, seats are limited!.

logo_rentalia

trinity-common-use

If you missed our last posts, these are the OpenNebula Cloud Tech Days that we have already announced for 2016:

For more information on past events, please visit the Cloud Technology Days page

TechDays

Please send us an email at events@opennebula.org if you are interested in hosting a TechDays event.

We look forward to your answers!

Upcoming Cloud TechDays Cambridge USA and Toronto CA

Last week we opened the call for speakers and registration to the OpenNebula Cloud TechDays in Kuala Lumpur (Malaysia), Sofia (Bulgaria), Dallas (TX, USA), Ede (The Netherlands) and Nuremberg (Germany). Today we are announcing that we just opened the call for speakers and registration to the following Cloud TechDays:

Send us an email at events@opennebula.org if you are interested in speaking at one of the TechDays and register as soon as possible if you are interested in participating, seats are limited!.

For more information on past events, please visit the Cloud Technology Days page

Please send us an email at events@opennebula.org if you are interested in hosting a TechDays event.

We look forward to your answers!

Upcoming Cloud TechDays in 2016

Besides our annual OpenNebula Conference in Barcelona (very early bird registration is now open), we are planning to organize Technology Day events in multiple cities globally during 2016. We have just published complete info about the following TechDays:

And in a few days we will publish the details of the TechDays we are organizing in:

  • Dublin, Ireland
  • Toronto, Canada
  • Cambridge, MA, USA
  • Madrid, Spain
  • San Francisco, CA, USA

TechDays

 

The OpenNebula TechDays are full day events to learn about OpenNebula with a hands-on cloud installation and operation workshop, and presentations from community members and users that will focus on:

  • Sharing cloud use cases and deployment experiences
  • Introducing new integrations and ecosystem developments
  • Describing other related cloud open-source projects and tools

Send us an email at events@opennebula.org if you are interested in speaking at one of the TechDays and register as soon as possible if you are interested in participating, seats are limited!.

For more information on past events, please visit the Cloud Technology Days page

Please send us an email at events@opennebula.org if you are interested in hosting a TechDays event.

We look forward to your answers!

OpenNebula Public Training for 2016

OpenNebula Systems has published the schedule for public classes in 2016 at its offices in Madrid, Spain, and Cambridge, MA, USA. This year OpenNebula Systems is expanding its public training services to the United States.

The course about OpenNebula Fundamentals: Cloud Operator and Architect uses a hands-on lab atmosphere to provide IT professionals with the skill-sets they need to install, configure and operate OpenNebula deployments. Additionally the program briefly addresses the integration of OpenNebula with other components in the data center.

You can contact OpenNebula Systems if your would like to request public training near you or private on-site training.

Automated Oversubscription and Dynamic Memory Elasticity for OpenNebula

In Cloud Management Platforms users typically deploy their VMs out of pre-defined templates that can specify a fixed amount of memory. Users may be able to customize the size of their VMs but they tend to overestimate the memory required for their applications. As an example, three 4 GB VMs with applications actually using 1 GB of RAM fit in a 12 GB node, but there is no more room left for additional VMs. Adjusting their memory to 1 GB VMs enables to deploy additional VMs on that node, as shown in the next figure.

CloudVamp

CloudVAMP is an open-source development that manages these situations for all the nodes in an OpenNebula Cloud deployment based on the KVM hypervisor. CloudVAMP monitors the memory usage of Virtual Machines (VMs) and dynamically changes the memory allocated to VMs by stealing the unused free memory from VMs. Then CloudVAMP enables OpenNebula to use that stolen memory, thus being able to increase the VM-per-node ratio. To prevent memory overload in the physical hosts, live migration is applied in order to accommodate the increasing memory demand by VMs across the OpenNebula Cloud.

Some Technical Details

CloudVAMP consists of three components:

  • Cloud Vertical Elasticity Manager (CVEM). An agent that analyzes the amount of memory actually needed by the VMs and dynamically updates the memory allocated to each of them, according to a set of customizable rules.
  • The Memory Reporter (MR). An agent that runs in the VMs and reports to the OpenNebula monitoring system the free, used memory and usage of the swap space, by the applications in the VM.
  • The Memory Oversubscription Granter (MOG). A system that informs OpenNebula about the amount of memory that can be oversubscribed from the hosts, to be taken into account by the OpenNebula scheduler.

CloudVAMP integrates with OpenNebula in several ways. The MR can be staged in the VMs using the contextualization mechanisms provided by OpenNebula or it can be pre-installed in the Virtual Machine Images. It contacts OneGate to report the memory usage in the VMs. The CVEM is installed as a daemon in the front-end node of the OpenNebula Cloud. Finally, the MOG is implemented as a new Information Manager in OpenNebula. The interaction with KVM is performed by means of LibVirt. Therefore, no modifications in the OpenNebula worker nodes are required. The interaction with the components is shown in the following figure:

cloudvamp-arch

Benefits of Using CloudVAMP

Deploying CloudVAMP in an OpenNebula Cloud enables to seamlessly allow OpenNebula to deploy more VMs per physical host, thus achieving increased server density. The memory usage of VMs is monitored in order to satisfy increased memory demands by the applications running in the VMs. The usage of live migration to redistribute the VMs without downtime is employed if necessary, without any user or sysadmin intervention. This enables an increased usage of the hardware platform that supports an OpenNebula Cloud.

In particular, at the GRyCAP research group we have integrated CloudVAMP in order to accommodate a larger number of incoming jobs from the ES-NGI (the Spanish National Grid Initiative) that are executed on a virtual elastic cluster deployed and managed by EC3 (Elastic Cloud Computing Cluster). The virtual cluster, deployed on top of our OpenNebula Cloud, is horizontally scaled whenever incoming jobs are received (i.e., deploying additional Worker Nodes (WNs)) and vertically scaled (i.e., adjusting the allocated memory to the VMs) in order to let OpenNebula deploy additional WNs in the same host, if necessary. Further details of this case study are available in CloudVAMP’s reference publication.

Availability

CloudVAMP has been developed by the GRyCAP research group at the Universitat Politècnica de València. It is available under the Apache 2.0 license at GitHub.

There is further information in CloudVAMP’s web page and in the corresponding publication:

Germán Moltó, Miguel Caballer, and Carlos de Alfonso. 2016. “Automatic Memory-Based Vertical Elasticity and Oversubscription on Cloud Platforms.” Future Generation Computer Systems 56: 1–10. http://linkinghub.elsevier.com/retrieve/pii/S0167739X15003155.

Contributions, feedback and issues are very much welcome.

OpenNebula TechDays 2016 – Call for Hosts

Besides our annual OpenNebula Conference, we are planning to organize Technology Day events in multiple cities globally during 2016.

The OpenNebula TechDays are full day events to learn about OpenNebula with a hands-on cloud installation and operation workshop, and presentations from community members and users that will focus on:

  • Sharing cloud use cases and deployment experiences
  • Introducing new integrations and ecosystem developments
  • Describing other related cloud open-source projects and tools

In the shorter term we would like to organize TechDays in USA (East and West coasts) and Europe, like we did during 2015:

For more information on past events, please visit the Cloud Technology Days page. These are not for profit events, all funds raised are rolled into the OpenNebula promo fund and will be used for further OpenNebula TechDay Events.

We look forward to your answers

OpenNebula Docker Driver and Datastore

ONEDock is a set of extensions for OpenNebula to use Docker containers as first-class entities, just as if they were lightweight Virtual Machines (VM). For that, Docker is configured to act as an hypervisor so that it behaves just as KVM or other hypervisors do in the context of OpenNebula.

The underlying idea is that when OpenNebula is asked for a VM, a Docker container will be deployed instead. In the context of OpenNebula, it is managed as if it was a VM, and the user will be able to use IP addresses to access to the container.

Docker Machine and similar projects deploy VMs in different Cloud Management Plattforms (e.g. OpenNebula, OpenStack) or commercial providers (like Amazon EC2), installing Docker on them. Afterwards, it is possible to deploy and manage Docker containers inside them, using the Docker client tools that communicate directly with the Docker services deployed inside the aforementioned VMs.

Instead, ONEDock takes a different approach by deploying Docker containers on top of bare-metal nodes, thus considering the containers as first-class citizens in OpenNebula. This allows to seamlessly integrate the benefits of Docker containers (quick deployment, limited overhead, availability of Docker images, etc.) in a Cloud Management Platform such as OpenNebula. On the other side, it provides new features for containers that are usually reserved for VMs (e.g. enhanced IP addressing, attachment of block devices, etc.).

ONEDock tries to adapt Docker semantics to the OpenNebula context. The workflow for a whole use-case is the following:

  1. An image is registered in a datastore of type ‘onedock’, by using the oneimage command.
  2. ONEDock will download the image from Docker Hub.
  3. A VM that uses an image registered in the ‘onedock’ datastore is requested.
  4. When the VM is scheduled, ONEDock will actually create a Docker container instead of the VM, and the container will be daemonized (e.g. kept alive).
  5. If the container has been connected to a network, it is possible to be accessed (e.g. using ssh or http).

The most prominent feature of ONEDock is that it does not introduce any API changes, therefore it does not modify the way of interacting with OpenNebula: It is possible to use the ONE CLI (i.e. oneimage, onevm, onetemplate), OpenNebula Sunstone, XML-RPC, etc. and keep the usual lifecycle for the VMs.

Technical details

ONEDock provides 4 components that need to be integrated into the OpenNebula deployment:

  • ONEDock Datastore, that enables to create a datastore that contains Docker images. It is self-managed in the sense that images are created as references that are automatically downloaded from Docker Hub.
  • ONEDock Transfer Manager, that stages the docker images that are in a Docker datastore into the virtualization hosts.
  • ONEDock Monitoring Driver, that monitors the virtualization hosts in the context of the Docker hypervisor.
  • ONEDock Virtual Machine Manager, that carries out the tasks related to the lifecycle of the Docker containers as if they were VMs.

These components have to be installed in the proper folders of a ONE frontend (i.e. /var/lib/remotes/) and activated in the oned.conf file. Therefore, no source code modifications of OpenNebula are required.

Once this has been done, it is possible to create datastores of type ‘onedock’ and virtualization hosts that use ‘onedock’ as the virtual machine manager.

The ONEDock datastore

In order to deploy a Docker container, a Docker image is required. When you run a container, Docker automatically retrieves the image from the Docker Hub repository.

To avoid that all virtualization hosts access Docker Hub, and as kind of cache, ONEDock supports a private registry installed in the OpenNebula front-end. Then, the references to the Docker images will point to the private Docker registry. ONEDock supports Docker registry v2.0.

The network in ONEDock

Docker containers are conceived to run applications, and so, it is common to find that ports are redirected to public ports in the machine that hosts the Docker container. ONEDock enhances this behaviour in order to expose all the ports of the container as it would happen in a VM. Therefore, you can run different services in different ports without the need of exposing them explicitly. The container will have an IP address where all the ports are available.

Testing ONEDock

ONEDock can be evaluated in a sandbox before deploying it in your on-premises Cloud. Easy as 1, 2, 3:

  1. Install Vagrant
  2. Spin up the vagrant VM, which will be automatically configured with ONE, ONEDock, the docker registry and all the needed components.
  3. Start creating Docker containers with the common ONE commands (i.e. onevm create, etc.)

Or

  1. Install LXC
  2. Create a testing container (using a self contained cli-utility that installs ONE, ONEDock, the docker registry and all the needed components).
  3. Start using ONE by issuing the common ONE commands (i.e. onevm create, etc.)

Getting ONEDock

ONEDock has been developed in the framework of the INDIGO-DataCloud (https://www.indigo-datacloud.eu) project under the Apache 2.0 license. You can get it from the public repository https://github.com/indigo-dc/onedock.

ONEDock is accepting contributions. You are invited to interact with us in the GitHub repository, by asking questions or opening new issues.