Scientific Cloud Computing School in Beijing

An increasing number of Scientific Computing centres are beginning to provide their resources to users through the IaaS model, since it allows for an efficient management and an easy reallocation of the computational resources. Large experimental collaborations in the High Energy Physics community, which heavily depend on grid-based distributed computing, are also beginning to explore the Cloud paradigm, in order to integrate it in their complex computing and data models.

As an example, at the Torino computing centre of the Italian National Institute for Nuclear Physics (INFN) we are using OpenNebula since 2011 to provide computing resources to the ALICE experiment at the CERN Large Hadron Collider in Geneva, to the BESIII experiment at the Institute of High Energy Physics (IHEP) of the Chinese Academy of Science in Beijing, and to a growing number of other applications. Besides managing the site and participating to EU-funded R&D projects like INDIGO-DataCloud, we have an ongoing collaboration with the IHEP Computer Centre and the BESIII Collaboration to evolve the experiment’s computing from a Grid-based model to a Cloud-base one.

It is thus important to train a number of young researchers, spreading Cloud Computing knowledge across the physics computing community. It is in this context that we organised the 2015 BESIIICGEM Cloud Computing Summer School, aimed to provide a full introduction both to basic and more advanced concepts of Cloud Computing.

IHEP Main building

“BESIIICGEM” is an EU-funded project within the “Horizon 2020” Research and Innovation Framework Programme. Its primary aim is the design and construction of an innovative particle detector for BES-III, a high-energy physics experiment run by an international collaboration and hosted at IHEP in Beijing. Among its different Work Packages, dealing with the different demanding tasks needed to achieve such a goal, a specific activity is devoted to R&D in HPC and Cloud Computing. The school is one of its outreach and dissemination activities. The School is jointly organised by the BESIIICGEM Consortium, IHEP and INFN, and will be hosted by IHEP in Beijing from Monday, 7th to Friday, 11th September 2015.

The School will complement a more theoretical set of morning lectures  with a full hands-on approach in the afternoons, when each (small) group of students will deploy at the end of the week a working testbed, one for each group. The reference middleware stack of choice for the practical activities will be OpenNebula, chosen for its simplicity and our experience in its use for scientific computing use cases.

The School is aimed to Doctoral Students in the High Energy or Nuclear Physics fields or in the IT and Computing fields. Young researchers holding Post-Doctoral positions are encouraged to apply as well.

The detailed agenda of the school is being finalised; updated information can be easily found on the School’s web-site.

OpenNebula Newsletter – July 2015

This Newsletter contains the most worthy developments and events of the OpenNebula project and the community during this last month of July, and the plans for the upcoming months.

We want to remind our users that OpenNebula Systems, the company behind the OpenNebula Project and vOneCloud, has established a new subsidiary to oversee all business operations and to support the OpenNebula community in the U.S. If you are interested in supporting the OpenNebula community in the US, let us know.

Technology

The team has released the first Beta of OpenNebula 4.14 ‘Great A’Tuin’. We hope you enjoy using it as much as we did developing it. The new features and usability improvements present in this new release couldn’t have been without our community support and feedback. We want to thank you all for your contributions to make OpenNebula as mature and reliable as it is today. Every little helps!

One of the most interesting new features for cloud users and administrators is the ability to create and maintain a tree of VM disks snapshots, for Ceph an Qcow2 backends. Also important, the ability to save VMs into VM Templates for later use greatly improves the usability of OpenNebula based clouds, to which also adds the much better state management of VMs for recovery from failures.

snapbeta

There are many other improvements in 4.14, like a more flexible context definition of network attributes; the ability to import running VMs not launched by OpenNebula from all the supported hypervisors (including the hybrid ones, for instance now it is possible to manage through OpenNebula Azure, SoftLayer and EC2 VMs launched through their respective management portals);the possibility to cold attach disks and network interfaces to powered off machines (which complements the hot attach functionality); improvements in accounting to keep track of disk usage; better logging in several areas; the ability to pass scripts to VMs for guest OS customization; and many others. A great effort was put in this release to help build and maintain robust private, hybrid and public clouds with OpenNebula.

There are some features the team is still polishing and decided not to include in the Beta1- For instance another major feature is the ability to resize an existing disk for all the supported OpenNebula backends. If your VM needs more disk space than the one provided by the images used for its disk, you can now set a new size prior to instantiate the VM, OpenNebula will grow the disk and the guest OS will adapt the filesystem to the now bigger disk at boot time. The disk space won’t be an issue anymore. To support this feature, a new set of context packages has already been released so you can start preparing your images for disk resizing.

Another huge feature to support HPC oriented infrastructures is the consumption of raw GPU devices existing on a physical host from a Virtual Machine. A new type of consumable has been defined in OpenNebula and taken into account by the scheduler. VMs can now request a GPU, and if OpenNebula finds one free resource of type GPU available, it will set up the VM with PCI passthrough access to the GPU resource.

Channels to help people try out OpenNebula is a hassle-free way are key to foster adoption of OpenNebula. The OpenNebula project is well aware of this, so it has teamed up with Microsoft to offer a new OpenNebula Sandbox in the Azure Marketplace. Currently you can try out 4.12.1, but we will update as usual all the SandBoxes to Great A’Tuin as soon as the stable version is out.

Community

Summer is upon us, and this does not prevent the OpenNebula community to keep contributing to the rich ecosystem. Since the Beta1 is out, we want to make a call for feedback. Please install in a test environment Great A’Tuin, and let us know through the community forum. It is your feedback what makes OpenNebula user friendly, so thanks in advance!.

For the team working in the cloud trenches, comments like this one are like a blow of fresh air that keep us going, stronger, towards cloud excellence. Thanks!
Contributions to the OpenNebula ecosystem are always great news, to enable integrations with new components, develop new functionality and ease the day to day cloud operation. But if they are maintained such as this nodejs binding for the OpenNebula API, then we are talking about quality ecosystem. Excellent contribution.
Extensions to the OpenNebula contextualization system are also always good news, like this new contextualization package for Gentoo written as an Ansible playbook.

Another great contribution enables OpenNebula to manage LXC containers. LXC is a userspace interface for the Linux kernel containment features. It lets Linux users easily create and manage system or application containers. With this amazing contribution OpenNebula can deploy LXCs with the intrinsic improvements in performance. The new addon includes motorization and virtualization drivers, kudos to Sergio Vega and José Manuel de la Fé for their contribution.

Outreach

The upcoming third edition of the OpenNebulaConf will be held in Barcelona this October 2015. The Conference is a great opportunity to meetup and share your OpenNebula experiences with the community. Your company may be interested in the sponsorship opportunities for OpenNebulaConf 2015. If you are interested, you can apply with this form. StorPool just joined as Platinum Sponsor. We would like to take this opportunity to also thank our Platinum Sponsor PTisp; Gold Sponsors ungleich, Xen Server and NodeWeaver; and Silver Sponsors Runtastic and No Limit Network.

The agenda for the OpenNebulaConf 2015 is available, check out the high quality speakers. The agenda includes two keynote speakers:

July was a hectic month in terms of OpenNebula TechDays:

CJjj5w5UMAAkg2y

You can read more details about the presentations here. If you want to host a TechDay in your area, let us know.

An excellent event was also carried out in France, the SophiaConf2015. Check out the OpenNebula presentation given by members of TAS Group France on why they chose OpenNebula for their infrastructure, very interesting (warning: you need to understand french!).
During the following months, members of the OpenNebula team will be speaking in the following events:

If you are interested in receiving OpenNebula training, check the schedule for 2015 public classes at OpenNebula Headquarters. Next public OpenNebula training will take place next 15th of September in Madrid. Please contact us if your would like to request training near you.

Remember that you can see slides and resources from past events in our Events page. We have also created a Slideshare account where you can see the slides from some of our recent presentations.

StorPool to Sponsor OpenNebulaConf 2015

OpenNebula Conf 2015 is getting closer and we would like to keep sharing with you the companies/projects that are sponsoring this year’s conference. Now it is time for StorPool Storage, as part of our Platinum Sponsors. You can meet them in the booths area during the coffee and lunch breaks.

StorPool is intelligent storage software that runs on standard servers and builds scalable, high-performance storage system out of those servers. It focuses on the block-level storage and excels at it. OpenNebula is the preferred cloud management system for StorPool. It is simple, yet powerful and works very well.  StorPool is already integrated with OpenNebula. The integration is performed as a new datastore driver in OpenNebula. With StorPool, OpenNebula clouds get exceptional storage bandwidth, IOPS and low latency. This allows provisioning more VMs per host which increases utilization and ROI. Combining both products also allows for seamless scalability in capacity and performance as well as increased reliability.

At the event StorPool will demonstrate the joint solution, answer any questions and will help customers to improve the design of their cloud.  If you want to participate in OpenNebula Conf and meet StorPool and other OpenNebula users, remember that you are still in time for getting a good price deal for tickets. Also, if your company is interested in sponsoring OpenNebulaConf 2015 there are still slots available.

About StorPool Storage

StorPool is block-level storage software. It has advanced fully-distributed architecture and is arguably the fastest and most efficient block-storage software on the market today. StorPool is incredibly flexible and can be deployed in both converged setups (on compute nodes, alongside VMs and applications) or on separate storage nodes. More about StorPool at www.storpool.com or info@storpool.com

promo_banner

OpenNebula 4.14 ‘Great A’Tuin’ Beta 1 released!

The OpenNebula project is proud to announce the availability of OpenNebula 4.14 ‘Great A’Tuin’ Beta1. This release ships with several improvements in different subsystems and components. The Sunstone interface has been completely refactored, for maintenance and performance reasons. Expect major improvements in Sunstone from now on. Also, we are sure you will like the subtle changes in the look and feel.

sunsdash414

 

Several major features have been introduced in Great A’Tuin. One of the most interesting for cloud users and administrators is the ability to create and maintain a tree of VM disks snapshots. Now VM disks can be reverted to a previous state at any given time, and they are preserved in the image if it is persistent in the image datastore. For instance, you can attach a disk to a VM, create a snapshot, detach it and attach it to a new VM, and revert to a previous state. Very handy, for instance, to keep a working history of datablocks that can contain dockerized applications.

 

snaptree414

Also, in 4.14 snapshots are taken into account for quotas, accounting and showback, so cloud admins can keep track of disk usage in their infrastructure.

The ability to save VMs into VM Templates for later use is another feature that must be highlighted in this release. This new operation is accessible both from the cloud view and the admin Sunstone view. Of course, also from the command line interface.

One great improvement for cloud admins is a much better state management of VMs. It is now possible to recover VMs from failed state instructing OpenNebula to take the last action as success, to retry it or to make it fail gracefully, to recover for instance from failed migrations.

There are many other improvements in 4.14 (check the full list of changes in the development portal):

  • flexible context definition of network attributes
  • ability to import running VMs not launched by OpenNebula from all the supported hypervisors (including the hybrid ones, for instance now it is possible to manage through OpenNebula Azure, SoftLayer and EC2 VMs launched through their respective management portals)
  • the possibility to cold attach disks and network interfaces to powered off machines (which complement the hot attach functionality)
  • improvements in accounting to keep track of disk usage
  • better logging in several areas
  • the ability to pass scripts to VMs for guest OS customization

Overall, a great effort was put in this release to help build and maintain robust private, hybrid and public clouds with OpenNebula.

This OpenNebula release is named after Great A’Tuin,  the Giant Star Turtle (of the fictional species Chelys galactica) who travels through the Discworld universe’s space, carrying four giant elephants who in turn carry the Discworld. Allegedly, it is “the only turtle ever to feature on the Hertzsprung–Russell diagram.”

The OpenNebula team is now set to bug-fixing mode. Note that this is a beta release aimed at testers and developers to try these new features (not production environments) and send a more than welcomed feedback for the final release. There are a number of very interesting features that will make their appearance in the final release but not present in the Beta1, like for instance better support for Qcow2 for live snapshotting and GPU support for VMs.

Disk snapshots with Ceph backend was funded by Unity in the context of the Fund a Feature Program. Qcow2 snapshots implementation was funded by BIT.nl in the context of the Fund a Feature Program. GPU devices support was funded by SURFsara in the context of the Fund a Feature Program. Flexible network attributes definition in contextualization was funded by Université Catholique de Louvain in the context of the Fund a Feature Program.

Relevant Links

Recapitulation of the last 3 TechDays

On behalf of the whole OpenNebula Team we would like thank all the people who participated in the 3 wonderful TechDays that took place in the last weeks:

Many thanks to all the speakers for doing such a great job and sharing their ideas with the audience: Eliot Eshelman (Vice President at Microway, Inc), Varadarajan Narayanan (CEO at Megam), Alex Gorbachev (Intelligent Systems Services Inc), Jimmy Goffaux (Ingénieur Administration Système at IPPON HOSTING), Daniel Dehennin (Ac-Dijon), Guillaume Oberlé (Université de Strasbourg), Chris Johnston (Principal Platform Architect, Cloud Team at BlackBerry Limited) and Khoder Shamy (Principal Technical Lead, Cloud Team at BlackBerry Limited).

You can find the slides in the page for each TechDay.

We would like to give a special shout-out to the audience for all the 3 events, for being especially engaged and friendly. Quality conversations and lots of great networking. Thanks a lot!

Remember, if you want to bring the TechDay to your hometown, drop us a line and we will be happy to oblige!

See you next time!

Closed Agendas for Paris and Ontario TechDays

As you may know, next week (6th of July) members OpenNebula team will be on the road to participate in two very interesting Cloud Technology Days. TechDays are full day events to learn about OpenNebula with workshops, and presentations from community members. In both TechDays, core developers of OpenNebula will unveil and demo juicy features that will make their appearance in OpenNebula 4.14, Dark Horse.

The OpenNebula Cloud Tech Day on the 8th of July in Paris, France, is being organized by IPPON Hosting (thanks a lot!) in their premises. Check the agenda for more details, the highlights are presentations by Jimmy Goffaux from IPPON Hosting sharing their use of OpenNebula self-service interface and several other nifty features, and by Daniel Dehennin, who will let us know about the use of OpenNebula by two french ministries.

logo_ippon_hosting_HDIf you are in Paris next 8th and want to learn more about OpenNebula, heard community experiences and share your own, don’t think about it any longer!

The Ontario TechDay is being hosted by BlackBerry (kudos to you!) in one of their premises in Waterloo, ON, Canada. BlackBerry staff Chris Johnston, Principal Platform Architect in the Cloud Team; and Khoder Shamy, Principal Technical Lead in the Cloud Team, will present the use-case of OpenNebula at BlackBerry that facilitated building a large multi-regional and multi-tenant private cloud for internal service loads. See the full agenda for more details and the other contents of the TechDay.

BB_TechDayIf you are on Waterloo on the 10th, don’t think about it and register for this event to learn how OpenNebula is being used in a massive scale deployment. We hope to see you there!

OpenNebula Newsletter – June 2015

Check out what kept us busy this June. This Newsletter contains the most worthy developments and events of the OpenNebula project and the community during this last month, and the plans for the upcoming months.

We want to remind our users that OpenNebula Systems, the company behind the OpenNebula Project and vOneCloud, has established a new subsidiary to oversee all business operations and to support the OpenNebula community in the U.S. If you are interested in supporting the US OpenNebula community or contact us, let us know.

Technology

The OpenNebula team is not fooling around with the 4.14 release. We are settling on a code name for the release (do you like ‘Dark Horse’?), and meanwhile adding an impressive array of features. Probably the main highlight is the new snapshot management feature, that will let perform disk snapshots and maintain a history of them. The initial support would be only for Ceph (it is our storage backend of choice for medium to large deployments after all, read our reference architecture), and qcow2. OpenNebula will be aware of snapshot sizes, and allow to enforce quotas and also to report consumption via the accounting mechanisms. Pretty neat, is it not, hosting providers?

Another big highlight would be the inclusion of a new type of consumable, GPU. A VM would be able to consume GPUs, using PCI passthrough, to gain exclusive access to GPU PCI cards. Pretty neat, is it not, research institutions and supercomputing centers?

Sunstone is being reworking behind the scenes, and although final users will only notice minor changes in the GUI, rest assure that this is a very important feature for the future of Sunstone. Keep an eye on it!

Screen Shot 2015-06-30 at 13.34.52

Other features we are working on is image resizing (finally), system datastore flush for maintenance, import VMs capabilities for hybrid hosts, improved fault management with the inclusion of new VM state transitions (no VM left behind), improvements in the network context management, and many more. As mentioned before, we are not fooling around with Dark Horse, be sure to be ready for the beta version due at the end of July.

This month, a new set of contextualization packages was also released, with several nifty additionals like the ability to pass scripts directly to be executed at boot time, without having to create them first in the files datastore.

A new version of the drop in replacement for vCloud, vOneCloud, was also released. vOneCloud 1.6 comes with several improvements, like capacity resizing, resource pool confinement, improvements in VNC, and more. Make sure you update using the awesome one click updated.

Community

Keeping a healthy community is key for an open source project. The OpenNebula community is growing every day, contributing new pieces to an already rich ecosystem.
For instance, StorPool integration with OpenNebula is performed as a new datastore driver in OpenNebula. OpenNebula persistent and non-persistent images are stored in a StorPool cluster, and OpenNebula controls and monitors the StorPool cluster. An important contribution worth noted is on the works, promising to bring LCX support to OpenNebula.
User stories are also good addition to a project credibility. OpenNebula has been around for a while now, but we still are moved by our users from time to time when they decide it is time to announce to the world they use OpenNebula in their infrastructures. For instance, like Teledata did this month.

OpenNebula it turned out to be the most mature and proven Cloud-Stack available – lacking of unnecessary complexity or unneeded overhead.

And it is also nice to see how OpenNebula is being recognised as a solid alternative to commercial virtualization software. In this case, as an alternative to vCloud Director.
OpenNebula API is also being consumed by other components of the ecosystems to create amazing things like automated power management on opennebula based infrastructures, like the new release of CLUES.

Outreach

The upcoming third edition of the OpenNebulaConf will be held in Barcelona this October 2015. Your company may be interested in the sponsorship opportunities for OpenNebulaConf 2015.We already have a number of sponsors, so if you are interested let us know as soon as possible. As platinum sponsors, we have PTisp; as gold sponsors ungleich (check out the post announcing it), NodeWeaver (check out the announcement) and XenServer; and as silver sponsors Runtastic and no-limit-network (announced here).

The agenda for the OpenNebulaConf 2015 is available, check out the high quality speakers. The agenda includes two keynote speakers:

This month the OpenNebula project organized an OpenNebula TechDay in Cambridge Massachusetts , at the Microsoft New England R&D Center, organized by the HPC & GPU Supercomputing Group of Boston and sponsored by Microway.

CIrZZuZUYAACxGy

Next month, OpenNebula TechDays will be happening in Paris, France on the 8th of July, organized by IPPON Hosting. Two days after, on the 10th, another TechDay will happen in Ontario, Canada, this time organized by our long term users, BlackBerry. If you are interested in helping us organize a Cloud TechDay in your area, go ahead and contact us.

An excellent event developed in Bern last 16th of June, the Open Cloud Day. You can read here their article in ungleich blog, thanks guys for organizing this interesting event.
During the following months, members of the OpenNebula team will be speaking in the following events:

If you are interested in receiving OpenNebula training, check the schedule for 2015 public classes at OpenNebula Headquarters. Next public OpenNebula training will take place next 16th of June in Madrid. Please contact us if your would like to request training near you.

Remember that you can see slides and resources from past events in our Events page. We have also created a Slideshare account where you can see the slides from some of our recent presentations.

PTISP and OpenNebula, a Success Tale.

PTISP, one of the largest hosting provider and domain registrars in Portugal, is honoured to announce another strategic partnership, OpenNebula, the shared vision of an enterprise-class cloud data center management sealed this joint venture. The pursuit of a more efficient way to manage the companies IaaS cloud solution led PTISP to deploy OpenNebula in its infrastructure and commercial offer.

PTISP is a Portuguese based company that sets the trend of innovation and development. Established in 2001, PTISP is one of the biggest .PT registrars, managing two data centers in Portugal, with 700+ servers installed and 27.000+ websites hosted. PTISP is a reference company insuring the trust and loyalty of International and Portuguese companies from several areas using mainstream and custom solutions, insured by state of the art hardware, continuous improvement policies, and a multi-disciplinary tech team that is available 24/7/365 to their customers.

unnamed

OpenNebula ensures a enterprise-ready solution, widely used in research and industry (such as ESA, Unity, Akamai, BlackBerry, and many others), to build and manage clouds and virtualized data centers, and a powerful open platform for innovation, used in many international projects to research the challenges that arise in enterprise cloud and data center management.

PTISP chose OpenNebula due to its unique features, revealed to be a trustworthy partner, enabling a custom solution that integrated with all the multiple technologies that are a part of PTISP’s universe.

PTISP will be present as a Platinum Sponsor on the OpenNebulaConf 2015, next October at Barcelona. Drop by our booth to visit us and exchange all the experiences around OpenNebula and PTISP. Check out our CTO’s, Pedro Dias, presentation on our partnership and experience named “An OpenNebula Tale” that will take place on the 21st of October at Hall A, between 10:40 and 11:00.

Cloud TechDay in Waterloo, ON Canada, by BlackBerry

We are excited to announce that BlackBerry will host our first ever Cloud TechDay in Canada. The TechDay will be held on the 10th of July in the BlackBerry Offices in Waterloo, Ontario Canada.

BB_TechDay

The TechDay will be a full day event to learn about OpenNebula with a hands-on cloud installation and operation workshop, and presentations from community members and users. Chris Johnston, Principal Platform Architect in the Cloud Team at BlackBerry Limited, and Khoder Shamy, Principal Technical Lead in the Cloud Team at BlackBerry Limited, will present the use-case of OpenNebula at BlackBerry that facilitated building a large multi-regional and multi-tenant private cloud for internal service loads. See the Agenda for more details.

If you want to actively participate in this event, share your experience with OpenNebula or describe other related cloud open-source projects and tools, send us your talk proposal at events@opennebula.org.

The number of seats is limited to ensure there is plenty of opportunity for everyone to interact. We encourage everyone to register as early as possible.

Register for this Event

We hope to see you there, and a big thanks to BlackBerry!

Other Upcoming Cloud Techdays

OpenNebula Cloud Tech Day on the 29th of June in Cambridge, MA, at the Microsoft New England R&D Center, organized by the HPC & GPU Supercomputing Group of Boston and sponsored by Microway.

microsoft_logo

microway_logo

 

OpenNebula Cloud Tech Day on the 8th of July in Paris, France, organized by IPPON Hosting.

logo_ippon_hosting_HD

StorPool: A New Storage System Integrated with OpenNebula

Storage has always been the single most complex and expensive asset in any company that builds clouds, provides cloud services or manage its own infrastructure. Companies are in constant search of reliable, scalable, simple and affordable storage alternatives.

We are happy to announce that we have a new storage system which is now integrated and available to OpenNebula users: StorPool.

StorPool is block-level, distributed storage software. It runs on standard servers and builds a fast, scalable and reliable shared storage system out of these servers. StorPool is incredibly flexible and can be deployed in both converged setups (on compute nodes, alongside VMs and applications) or on separate storage-only servers.

StorPool was designed from scratch to deliver the fastest and most-efficient storage solution possible. It eliminates the need of traditional SANs and all-flash arrays (AFA). Because of it’s performance, flexibility and unmatched efficiency (taking minimal resources on the servers), it is preferred choice to other storage software (like Ceph), when it comes to building high-performance block-storage systems.

The integration between the two systems is performed as a new datastore driver in OpenNebula. OpenNebula persistent and non-persistent images (virtual disks) are stored in a StorPool cluster with the expected high level of reliability and performance. OpenNebula controls and monitors the StorPool cluster. For example, when you create a VM in OpenNebula, it instruct StorPool to create a new volume for the VM, which may be based on a ONE template. When a ONE template is being imported it goes directly into the StorPool image data store.StorPool data storage system

With StorPool, your OpenNebula cloud gets exceptional storage bandwidth, IOPS and latency, enabling you to provision many more VMs per host. Combining both products allows for seamless scalability in capacity and performance as well as increased reliability.

By deploying StorPool with OpenNebula companies get storage infrastructure that is:

Reliable – with no single points of failure, shared storage with solid data protection
Fast and scalable – having seamless scalability in capacity and performance
Intelligent and flexible – allowing online expansion, online changes, self-healing, in-service upgrades
Simple – all the ways – to buy, deploy, operate and expand, utilizing standard servers, drives and networking
Affordable – one of the most affordable alternatives on the market

To learm more about StorPool or get a free test version of the product visit: www.storpool.com or contact them at info@storpool.com.

storpool-logo-blue-mid