Agenda for the Upcoming Cloud TechDay in Ede, NL

Next week, on the 13th of May 2016, BIT.nl will host a new edition of an OpenNebula Cloud TechDay.

This TechDay will feature a 4-hour hands-on tutorial in which you will learn how to install and configure an OpenNebula Cloud from scratch. The presentations in the afternoon will be focused in Ceph. We want you to learn as much as possible about Ceph best-practices and how to use it with your OpenNenbula Cloud.

The agenda for the afternoon is:

  • Object scale out with Eternus CD10000 by Walter Graf and Frits de Kok from Fujitsu.
    Introduction to object storage, Ceph concepts and internals and how Fujitsu managed to overcome administrative challenges involved in running a Ceph cluster.
    Fujitsu and BIT will be giving a demo of Fujitsu CD10000 and OpenNebula.
  • Building the Dutch National Archive with Ceph by Wido den Hollander, Part of the Ceph Board.
    The Dutch National Archive has chosen Ceph to store their data on in Groningen, The Netherlands. Together with ODC Noord I’ve built the 8PB Ceph cluster running in a IPv6-only network. This talk will go in-depth in the design decisions made when building this cluster.
  • The OpenNebula Ceph Drivers by Jaime Melis from OpenNebula Systems.
    Overview of the Ceph Drivers. Configuration attributes, peculiarities. Everything you should know about before deploying your OpenNebula + Ceph cloud.
  • BIT’s experiences playing with Ceph and OpenNebula by Stefan Kooman from BIT.nl
    BIT has been running a Ceph test cluster for some time and will talk about their experiences so far. A live demo is planned where we will test Ceph’s ability to recover from failure.

Join this TechDay to learn about OpenNebula, the Cloud, Ceph and benefit from the expertise of the speakers!

fujitsu

bit_internet_tech_logo

Ceph_Logo_Standard_RGB_120411_fa

 

 

More information
Register for this Event

Providing Enterprise-Grade Infrastructure In The Cloud Age

CipherSpace Logo

 

 

 

The “Cloud” Need Not Be A Commodity

CipherSpace has a heritage of providing managed ICT infrastructure and services to clients who need customized “enterprise-quality” solutions on small to medium-sized business scales. Historically this demanded versatility, so that we could both provide each client with the right solutions for them and also with the expertise and judgment to guide them in the choice of their best solutions. As a consequence, we have used many approaches to virtualization over the years: FreeBSD Jails, Parallels, VMWare ESX/ESXi, Citrix XenServer, and Linux KVM, as well as working with others in non-production testing and evaluation. Because we have diverse clients with services in technically close proximity to each other, security has always been a critical focus of our work. We also have a strong preference for open source software, based not only on the obvious cost advantage but also for its adaptability and more transparent (and historically better) security. We’re not OSS purists in any sense, but openness is an important feature of many solutions we offer, as is the availability of commercial support for critical instances of critical tools, both for ourselves and our customers.

It became clear to us in 2011 that “Cloud” architectures were maturing in a way which made it imperative for us to create our own suite of solutions that would serve our clients differently and better than what they could get from the commodity sector. Read more

New DRBD Manage add-on for Highly Available storage

DRBD backed storage is now integrated into OpenNebula with the new DRBD Manage addon.

DRBD provides transparent, real-time replication of block devices without the need for specialty hardware. DRBD Manage is an administrative tool which facilitates easy Logical Volume Management (LVM) and configuration files for multi-node DRBD clusters.

With the DRBD Manage driver, create each new image for your virtual infrastructure as a DRBD volume. Volumes intelligently balance the load on your storage nodes. Alternatively, assign volumes to the specific nodes that you want in use. This is a simple scale-out storage solution, supporting the capability to add new nodes to your storage cluster at any time. This, combined with the flexibility of LVM, allows DRBD to keep up with your ever-increasing storage requirements.

DRBD 9 and DRBD Manage allow transferring Images to Virtualization hosts via the DRBD Transport protocol. This allows images to be available nearly instantly on host nodes without requiring them to have storage space available.

Below is a diagram showing a simple OpenNebula cluster using the DRBD Manage Driver. This cluster has a Front End, two storage nodes, and a single virtualization host. The host has two images attached to it via DRBD Transport. Both images are deployed to double redundancy and are being replicated in real time across both storage nodes. This means that the failure of a single storage node will not disrupt IO on the host. All nodes have a local copy of DRBD Manage’s control volume.

opennebula-diskless

Features

  • Data redundancy
  • Automatic fail-overs if a storage node fails
  • Database and high I/O application compatible
  • Transfers images over the network with DRBD Transport
  • Quickly attaches images to VMs
  • Fast image clones

NodeWeaver to Sponsor OpenNebulaConf 2016

OpenNebula Conf 2016 is getting closer and we would like to keep sharing with you the companies/projects that are sponsoring this year’s conference. Now it is time for NodeWeaver, as part of our Gold Sponsors.

nodeweaver-logo (1)
If you want to participate in OpenNebula Conf and meet NodeWeaver and other OpenNebula users, remember that you are still in time for getting a good price deal for tickets. Also, if your company is interested in sponsoring OpenNebulaConf 2016 there are still slots.

About NodeWeaver

NodeWeaver is a new kind of hyperconverged appliance-bringing together storage, networking and virtualization, completely self-managing and designed to be simple to operate and exceptionally reliable. We have chosen from the start to take advantage of the OpenNebula orchestrator as our main component, a choice driven by its simplicity, reliability and its open design and development process. Our choice has been demonstrated, over and over, an excellent one: OpenNebula simplicity has been extremely appreciated by our customers, and the modularity of its architecture substantially reduced the complexity of adapting and extending it. In three years of experimentation, and several mission critical deployments, OpenNebula has exceeded all our expectations.

Agenda for Upcoming Cloud TechDays in USA & Canada

We are pleased to announce the agenda for the upcoming Cloud Technology Days that will be held in April in USA and Canada:

They feature a 4-hour hands-on tutorial in which you will learn how to install and configure an OpenNebula Cloud from scratch, and presentations from community members and users.

The event in Dallas, TX is hosted by Improving, a complete IT services firm, offering training, consulting, recruiting, and project services. The event is also sponsored by Digital Ocean, a cloud infrastructure provider focused on simplifying web infrastructure for software developers. In addition to the hands-on tutorial that will be held in the morning we will have the following talks during the afternoon:

  • Javier Fontán from OpenNebula Systems will talk about Customizing Virtual Machine Images and Docker Machine and Swarm on OpenNebula

 

The event in Toronto, Canada is hosted by Canada151 Data Centers, which provides carrier neutral, highly redundant Colocation and Disaster Recovery services from their West Toronto facility.  The event is also sponsored by Solgenia, a Cloud technology provider specializing in SaaS, PaaS and IaaS solutions. In addition to the hands-on tutorial that will be held in the morning we will have the following talks during the afternoon:

  • Khoder Shamy from Fuze (formerly ThinkingPhones) will share how they leverage OpenNebula open-source project in the rapidly growing global private infrastructure at Fuze
  • Javier Fontán from OpenNebula Systems will talk about Customizing Virtual Machine Images and Docker Machine and Swarm on OpenNebula
  • Presentation about Canada151 Data Centers
  • Varadarajan Narayanan from Wayz Infratek will talk about Hyperconvergence and OpenNebula
  • Presentation about Solgenia Private Cloud and Data Centers

Throughout the day, Canada151 will be conducting tours of the data center for interested parties.

 

The event in Cambridge, MA is hosted by Harvard Research Computing, which was established in 2007 as part of the Faculty of Arts & Sciences (FAS) Division of Science in Harvard University with the founding principle of facilitating the advancement of complex research by providing leading edge computing services. In addition to the hands-on tutorial that will be held in the morning we will have the following talks during the afternoon:

  • John Noss from Research Computing at Harvard will talk about their experience with OpenNebula in Research Computing
  • Dan Kelleher from Research Computing at Harvard will talk abouth the Corona Project
  • Roy Keene from Knight Point will talk about OpenNebula: The Integrator’s Story
  • Jaime Melis from OpenNebula Systems will talk about Docker Machine and Swarm on OpenNebula
  • Javier Fontán from OpenNebula Systems will talk about Customizing Virtual Machine Images

 

Many thanks to Improving, DigitalOceanCanada151 Data Centers, Solgenia and Research Computing at Harvard University for hosting and sponsoring these events.

Looking forward to seeing you there, these are three community events you cannot miss! Seats are limited, register ASAP!

OpenNebula Newsletter – March 2016

The OpenNebula monthly newsletter, with latest developments, events and future plans for the upcoming months from the OpenNebula project. Read this newsletter to keep up to date with your favorite Cloud Management Platform.

The OpenNebula Conf 2016 is due in October in Barcelona, Spain, and the Call for Speakers is now Open!. If you are a OpenNebula practitioner, user, architect, devop, admin or developer and have something to share, we welcome your submission. You might be also interested in taking a look at the sponsorship opportunities. Also check out the projected OpenNebula TechDays for this year in case one is close to your location. Upcoming TechDays in April will be held in Dallas,Toronto and Cambridge. Also, there is some (but just a few!) seats left for the OpenNebula course OpenNebula Systems is giving in Cambridge.

Technology

OpenNebula 5.0 is brewing and the team is closing new features and liking the overall feeling of the next major upgrade. We are set for a beta release of 5.0 this very same upcoming month of April, so stay tuned for the following:

  • revamped Marketplace, now a first class citizen in OpenNebula, to import/export images to and from any datastore (including the new vCenter datastores)
  • vCenter storage management, including datastore selection and disk hotplug, a full integration with the new Marketplace functionality. Share VMDKs among OpenNebula instances! Also, vCenter resource pool selection for deployed VMs will be possible.
  • fully integrated virtual router management (including default and transparent HA for routers). Link virtual networks together using a robust HA router, that will come out of the box in OpenNebula
  • revisited Sunstone style and interface changes (sneak peek), including an extension of the Cloud View available operations, to iron out the wrinkles in the user experience

There are more new bits and bobs, check the full list of feature in the development portal.
Continuing with the series of Docker integration posts, this past month this entry explained how to get Docker Swarms deployed with OpenNebula. The idea is to get Docker Swarm to build a cluster out of a collection of Docker Engine VMs deployed in OpenNebula with Docker Machine. Docker Swarm is native clustering for Docker, delivering many advantages, like scheduling, high availability, etc.

docker-swarm-opennebula
If you need professional support for your OpenNebula cloud, remember that OpenNebula Systems has extended the support services portfolio to include the Operating System of the virtualization nodes and the controller (which can be CentOS and/or Ubuntu), the hypervisor (libvirt and KVM), networking technologies as VXLAN/VLAN 802.1Q and Ceph as storage backend.

Community

The OpenNebula community never ceases to amaze us. Contributions to the common knowledge pool around OpenNebula deployments add huge value to the open source project. Several contributions were made this past month in this regard, let’s see the highlights.
A great addition is related to the LXCoNe addon with a very detailed and complete post. LXCoNE, a virtualization and monitoring driver for OpenNebula that comes as an add-on to provide OpenNebula with the ability to deploy LXC containers. It contributes to achieve better interoperability, performance and scalability in OpenNebula clouds. Right now, the driver is stable and ready for its release.
Also do not miss this post and contribution by TeleData, enabling the feed of monitoring data in the form of graphite metrics. Great work! And we blush with your words too!

In our opinion OpenNebula works like a charm. It is open and flexible. OpenNebula is one of the most mature, comprehensive and valuable Cloud-Stack available in the market

Overview (Zones + Clusters + Ressources)

And last but not least, our friends at LINBIT have contributed their OpenNebula driver for DRDB integration as an addon! This driver allows for highly available storage using DRBD9 + DRBD Manage in OpenNebula.

We are, as usual, thankful to have this wonderful community. Two thumbs up!

Outreach

The next OpenNebula Conference in Barcelona in October 2016 has already been announced . If you are willing to attend and can save now the date you can take advantage of a 40% discount in your Conf tickets. The Call for Speakers is now Open!. If you are a OpenNebula practitioner, user, architect, devop, admin or developer and have something to share, we welcome your submission.
More information is available from the event site. The Conf has already its first sponsor with StorPool. Welcome! Learn about the different sponsorship opportunities in the Conference web page. If you want to understand what all the OpenNebula Conference fuzz is about, check the last Conference material (talks, slides,pictures).
This March the OpenNebula team held a TechDay Madrid. A full house event with very interesting talks held by Rentalia in the crib that saw OpenNebula birth!

CdQ0wUOXIAAYm6r

If you are interested in participating in (or hosting) any of these TechDays agenda let us know. Besides the TechDays already celebrated (in Kuala Lumpur, Sofia and Madrid), this year member of the OpenNebula team will be participating in TechDays the following locations: Dallas, Toronto, Cambridge, Ede, Nuremberg and Dublin.
The Swiss Open Systems User Group would be gathering next 16th of June to discuss Cloud Computing. If you are around and want to learn more about Clouds and OpenNebula, drop by.
You may be interested in OpenNebula Systems training plans for 2016 for Europe and US. These courses are designed to train cloud administrators to properly operate an OpenNebula infrastructure. Please contact us if your would like to request training near you.
Members of the OpenNebula team will be present in the following events in upcoming months:

  • VMworld 2016 US, August 28 – September 1, Las Vegas (Mandalay Bay Hotel & Convention Center), Nevada, US.
  • VMworld 2016 Europe, October 17 – 20, Barcelona (Fira Barcelona Gran Via), Spain.

Remember that you can see slides and resources from past events in our Events page. We have also created a Slideshare account where you can see the slides from some of our recent presentations.

OpenNebula Dashboards with Graphite and Grafana

We at TeleData are operating several OpenNebula instances in our datacenters and offer public Cloud services like IaaS, PaaS and scalable Hosting and ISP-Services for our business customers in Germany.

A professional 24/7 monitoring our whole infrastructure is a substantially element for our customers. In this case technical solutions have to offer the ability to drill down the hierarchy to identify bottlenecks and do some crucial capacity management. Every OpenNebula user has to know his workload and has to plan his resources. At TeleData we use a N+1 redundancy to cover host errors and so on.

With this strategy you cannot allocate 100% of your available ressources and you have to monitor your current allocations. Also the possibility to identify your most loaded VMM hosts or your most utilized vDCs could be a valuable information to do system and resource management the right way.

In this little guide we will try to get some monitoring data in form of graphite metrics for further, deeper and last but not least, long-term monitoring.

We think it was quite easy to pull out the information of OpenNebula. Therefore we developed a little script which collects the wanted metrics and sends them to Graphite.

First a small diagram for a quick overview:

Diagram OpenNebula - Graphite - Grafana

 

Lets start.
(This guide assumes that you are already familiar with Graphite and Grafana)

 

  1. Gathering the wanted Information
    As advised in the OpenNebula Forum [1] we used the XML-Output of the common tools “onehost” and “onegroup” in the OpenNebula Controller/Daemon, executed by Cronjob every 5 Minutes. The script is Ruby based and uses common Gems like “nokogiri” for XML operations.
  2.  Send out metrics to Graphite
    The script generates Graphite metrics out of the desired information and pushes them to your Graphite server. Graphite stores the metrics according to your storage schema (the Syntax is Frequency:Retention)
  3.  Display / Graph your metrics
    With your time series data in Graphite you could use shiny tools like „Grafana“ [2] or „Dashing“ [3] to create informative and quite impressive Dashboards for your OpenNebula OPS Team. With some templating (included in our JSON-Exports)  you can unchain the power of Grafana.

See some examples:

 

All work is published at Github: one-graphite
If there are any questions or issues feel free to add your comment at the OpenNebula Forum [1].

In our opinion OpenNebula works like a charm. It is open and flexible. OpenNebula is one of the most mature, comprehensive and valuable Cloud-Stack available in the market. We mentioned that before: http://opennebula.org/opennebula-at-teledata/

Because of the lack of complexity it`s simple to enhance and to add features.
Of course our Graphite cluster is also powered by OpenNebula, like many other Services.

Have fun!

 

Links:

[1] https://forum.opennebula.org/t/long-term-statistics-and-capacity-management-for-opennebula-clouds/1886/3
[2] http://grafana.org/
[3] http://shopify.github.io/dashing/

LXC Containers for OpenNebula

Operating-system-level virtualization, a new technology that has recently emerged and is being accepted into cloud infrastructures, has the advantage of providing better performance and scalability than other virtualization technologies such as Hardware-Assisted Virtual Machine (HVM).

containersLinuX Containers (LXC) allow the usage of this technology by creating containers that resemble complete isolated Linux virtual machines on the physical Linux machine, all this by sharing the kernel with the virtual portion of the system. A container is a virtual environment, with its own process and network space. LXC makes use of Linux kernel Control Groups and Namespaces to provide the isolation. Containers have their own view of the OS, the process ID space, the file system structure, and the network’s interfaces. Since they use kernel features, and there’s no emulation of hardware at all, the impact on performance is minimal. Starting up and shutting down containers, as well as creating or destroying them are fairly quick operations. There have been comparative studies, such as LXD vs KVM by Canonical, which show advantages of LXD systems over KVM. LXD is built on top of LXC and uses the same kernel’s features, so performance should be the same.

hyp2Nowadays public cloud Infrastructure as a Service (IaaS) providers, like Amazon, only offer application based on Docker containers deployed on virtual machines. Docker is designed to package a single application with all of its dependencies into a standardized unit for software development but not to create a virtual machine. Only a few systems offer IaaS on bare metal container infrastructures. Joyent, for instance, is able to put to use all of the virtualization advantages that OS provides.

However, on private cloud scenarios OS virtualization technology hasn’t had quite the acceptance it should. Private cloud managers, like OpenStack and Eucalyptus, don’t offer the support needed for this type of technology. OpenNebula, is a flexible cloud manager, which has gained very good reputation over the last few years. Therefore, to strengthen the OS virtualization support in this cloud manager could be a key strategic decision.

This is why LXCoNe was created. It is a virtualization and monitoring driver for OpenNebula that comes as an add-on to provide OpenNebula with the ability to deploy LXC containers. It contributes to achieve better interoperability, performance and scalability in OpenNebula clouds. Right now, the driver is stable and ready for its release. It is currently being used in the data center from Instituto Superior Politécnico José Antonio Echeverría in Cuba, with great results. The team is still working on adding some more features, which are shown next, to improve the driver.

Features and Limitations

The driver developed has several features such as:

  • Deployment of containers on File-Systems, Logical Volume Managers (LVM) and CEPH.
  • Attachment and detachment of network interface cards and disks, both before creating the container or while it’s on.
  • Monitoring containers and node’s resources usage.
  • Powering off, suspending, stoping, un-deploying and rebooting running containers.
  • Supports VNC.
  • Supports snapshots when using File-Systems.
  • Limits container’s RAM usage.

It lacks the following features, on which we are currently working:

  • Container’s CPU usage limitation.
  • Containers live migration.

Virtualization Solutions

OpenNebula was designed to be completely independent from underlying technologies. When the project started the only supported hypervisors were Xen and KVM, it was not thought out to support OS-level virtualization. This probably influenced the way OpenNebula managed physical and virtual resources. Because of this and due to the difference between the two types of virtualization technologies there are a few things to keep in mind when using the driver. These are:

Disks

When you successfully attach a hard drive, this will appear inside the container, in /media/<Disk_ID>. For detaching the hard drive it must be placed inside the container in the same path previously explained. It cannot be in use, otherwise it will purposely fail.

Network interfaces (NIC)

If you hot-attach a NIC, it will appear inside the container, but without any configurations. It will be up and ready to be used, contrary to what happens when you specify NICs in the template and then create the virtual machine. In this case, the NIC will appear set up and ready to be used, unless you specifically want it to appear otherwise.

Installation

Want to try? The drivers are part of the OpenNebula Add-on Catalog. Installation process is fully and simply explained in this guide.

Contributions, feedback and issues are very much welcome by interacting with us in the GitHub repository or writing a mail:

José Manuel de la Fé Herrero: jmdelafe92@gmail.com
Sergio Vega Gutiérrez: sergiojvg92@gmail.com

 

New Support Services Coverage by OpenNebula Systems

OpenNebula Systems has announced an extension of its support services aimed at those OpenNebula users in production environments based on KVM that need to support the whole stack. Hence, support can be extended to include the Operating System of the virtualization nodes and the controller (which can be CentOS and/or Ubuntu), the hypervisor (libvirt and KVM), networking technologies as VXLAN/VLAN 802.1Q and Ceph as storage backend.

The supported components are part of the Open Cloud Reference Architecture, created by OpenNebula Systems from the collective information and experiences from hundreds of users and cloud client engagements. This reference documents software products, configurations, and requirements of infrastructure platforms recommended for a smooth OpenNebula installation.

This new support coverage is offered as an Add-on over the traditional OpenNebula support.