Federation of Supercomputing Facilities with OpenNebula

The Supercomputing Center of Galicia (CESGA) and the Supercomputing Center Foundation of Castilla y León (FCSCL) have built a federation of cloud infrastructures using the hybrid cloud computing functionality provided by OpenNebula. Both organizations have collaborated in order to execute an application to fight Malaria across both sites. This is a very interesting use case of cloud federation in the High Performance Computing field.

CERN Cloud Scaling to 16,000 VMs!

Last week at ISC Cloud 2010, Ulrich Schwickerath, from the CERN IT-PES/PS Group, presented the last benchmarking results of CERN’s OpenNebula cloud for batch processing. The batch computing farm makes a critical part of the CERN data centre. By making use of the new IaaS cloud, both the virtual machine provisioning system and the batch application itself have been tested extensively at large scale. The results show OpenNebula managing 16,000 virtual machines to support a virtualized computing cluster that executes 400,000 jobs.

Thanks to everyone who made OpenNebula 2.0 possible!

As you know, OpenNebula 2.0 is out the door, and we’re delighted with the response from our community. Besides the huge spike in downloads, we have received many congratulations over email and Twitter from many of our users, specially those who are deploying OpenNebula in very large infrastructures. However, we would like to take a moment to acknowledge that, if OpenNebula 2.0 has gotten so far, it is in no small measure due to the many users and organizations in our community who have actively contributed to OpenNebula.

So, besides our major and regular contributors, we want to thank the following people and organizations:

Community Contributors

We would like to acknowledge the work of many community members who contributed to the development of OpenNebula by actively participating on our mailing list or our IRC channel, answering user questions, or providing patches for bugfixes, features and documentation:

Sander Klous, Sebastien Goasguen, Ulrich Schwickerath, Jeroen Nijhof, Tiago Batista, Nils Dijk, Marlon Nerling, Alexandre Joseph, Ariel Garcia, Carsten Friedrich, Charles Loomis, Chris Usher, Claude Noshpitz, Damien Raude-Morvan, Enrique Alvarez, Florian Kruse, Frank Stutz, Gyula Csom, Jason Heiss, Johan Tordsson, John Dewey, Lakshan Perera, Marc-Elian Begin, Martin Kopta, Matt Lavin, Michael Coram, Neil M, Nuno Cardoso, Patrick Naubert, Petri Lammi, Rahul Chandra, Rangababu Chakravarthula, Shi Jin, Stefan Freitag, Tres Wong-Godfrey, Waheed Iqbal, and others who might have slipped through the cracks. An latest list can always be found in the OpenNebula development site.

The OpenNebula Ecosystem

The extensible and open architecture of OpenNebula has enabled various contributors to develop substantial pieces of software that, while not strictly a part of OpenNebula, can be used in combination with OpenNebula to provide additional functionality. This thriving ecosystem includes projects such as schedulers, additional drivers, and tools that enhance the OpenNebula user experience.

The following organizations have contributed to our ecosystem: CloudScaling, Telefonica I+D, C12G Labs, SARA Supercomputing Center, University of Chicago, Clemson University, and CRS4 Computing Center.

Cloud Computing R&D Projects

OpenNebula is the result of many years of research and the interaction with world leading industrial and academic organizations in cloud computing. OpenNebula is being enhanced in the context of several flagship international projects in cloud computing to address the requirements of business and research use cases from leading IT organizations across multiple industries. We would like to especially acknowledge these pioneering projects and their partners for their invaluable contributions.

  • RESERVOIR, main contributor with significant impact on OpenNebula’s design and innovative features, coordinated by IBM and with the participation of Telefonica I+D, University College of London, Umeå University, SAP AG, Thales Services SAS, Sun Microsystems Germany, ElsagDatamat S.p.A, DSA-Reasearch at UCM, CETIC, Universita Della Svizzera italiana, Universita degli Studio di Messina and the European Chapter of the Open Grid Forum.
  • StratusLab, coordinated by Centre National de la Recherche Scientifique and with the participation of DSA-Reasearch at UCM, Greek Research and Technology Network S.A., SixSq Sárl, Telefonica I+D and Trinity College Dublin.
  • BonFIRE coordinated by Atos Origin and with the participation of University of Edinburgh, SAP AG, Universitaet Stuttgart, FRAUNHOFER, Interdisciplinary Institute for Broadband Technology, DSA-Reasearch at UCM, Fundacio Privada I2CAT, Hewlett-Packard Limited, The 451 Group Limited, Technische Universitaet Berlin, IT-Innovation, and Institut National de Recherche en Informatique et en Automatique.
  • 4CaaSt coordinated by Telefónica I+D and with the participation of UPM, 2nd Quadrant Limited, BonitaSoft, Bull SAS, Ericsson GMBH, FlexiScale, France Telecom, Universitat St Gallen, ICCS/NTUA, Nokia Siemens Networks, SAP AG, Telecom Italia, DSA-Reasearch at UCM, Universitaet Stuutgart, UvT-EISS, and ZIB.

From all of us at OpenNebula: Thank you!

Microsoft to Bridge Azure with OpenNebula

At OGF30, after OpenNebula’s keynote, Dr. Daniel A. Reed, Corporate Vice President Extreme Computing Group & Technology Strategy and Policy at Microsoft, gave an excellent keynote speech about cloud and the future of research. Dr. Reed also presented the new EU-funded VENUS-C project, coordinated by Engineering and aimed at developing, testing and deploying an industry-quality Cloud Computing service for Europe. Microsoft announced that it will work with VENUS-C partners to implement a bridge between Windows Azure and OpenNebula.

We are very happy to announce this collaboration that will extend the current level of interoperability and portability provided by OpenNebula.

Announcing the release of OpenNebula 2.0!

The OpenNebula team is very proud to announce the release of a new stable version of the OpenNebula Toolkit! That’s right, OpenNebula 2.0 is now available for download.

OpenNebula 2.0 marks the end of a large and laborious release cycle and an important milestone for the project in terms of functionality, maturity and community engagement. Apart from improved scalability and robustness and many exciting new features, we have also opened the project QA & testing procedures and created a new add-on section for high-quality, third-party open-source components.

OpenNebula 2.0 includes a significant amount of changes and new features across many areas. The following is a selection of notable new functionality:

  • Image Repository. The Image Repository allows users to easily specify disk images from a catalog without worrying about low-level disk configuration attributes or block device mapping. Also, image access control is applied to the images registered in the repository, hence simplifying multi-user environments and image sharing. The traditional method of specifying VM disks is also fully supported. More info…
  • MySQL support. OpenNebula can now work with either a MySQL or SQLite database backend. The new MySQL support brings important performance and scalability improvements compared to the SQLite implementation. MySQL is the recommended option for large-scale clouds. More info…
  • Improved VMWare support. VMware drivers has been re-written to make use of all the features offered by OpenNebula, making the functionality of the drivers on par with the KVM and Xen ones. The new drivers uses the libvirt driver. More info…
  • Scalability Improvements. Several components of OpenNebula have been tuned to work with tens of thousands of VMs, including the redesign of the scheduler and adjustments to the monitoring modules.
  • Separate VM and Information actions. The specific operations performed when monitoring a host or performing an specific operation on a VM has been decoupled from the driver code base. Now it is even easier to tune your cloud. The number of connections to the node have also been reduced.
  • Cluster support. Physical hosts can now be grouped in logical clusters, so after a cluster is defined it can be used to set the placement of a VM. More info…
  • Accounting. New tools are provided to generate accounting reports for cloud users or physical hosts. More info…
  • Authorization & Authentication Drivers. The authorization and authentication processes in OpenNebula have been totally redesigned. Now they can be handled natively by the OpenNebula core, offering the same functionality as OpenNebula 1.4. Alternatively, the A&A processes can be handled by an external component that can be integrated with any A&A system or user maps (e.g. LDAP, Kerberos…). As an example OpenNebula 2.0 features a quota based authorization module and user authentication based on RSA keys. More info…
  • Quota Management. Allowing cloud administrators to set limits on cloud resources for users
  • LDAP Integration. Levering the new authorization & authentication framework, datacenters using LDAP for user management can integrate OpenNebula with it, without the need of re-creating the user data. More info…
  • Virtual Networking Improvements. OpenNebula 2.0 lets you define generic attributes associated to a Virtual Network (e.g. gateway, dns servers…) that can then be included in the context of a VM. This will let you easily configure VMs with multiple NICs. Additionally Virtual Networks can now be defined as public, and thus shared among multiple users.
  • Improved EC2 support. The EC2 Query server can now be used with the EC2 ecosystem. Supported third party applications include ElasticFox or EucaTools. More info…
  • Java Bindings for the OpenNebula Cloud API (OCA) OpenNebula 2.0 includes JAVA bindings that wraps the XMLRPC interface methods exposed by the core. More info…
  • Improved OCCI support. The OCCI implementation now follows a more coherent schema. The OCCI server also makes a more robust handling of incoming requests. More info…

For a complete list of changes, see this detailed changelog

The complete source tree for OpenNebula can be freely downloaded here. The compilation has been tested on the main linux distributions, please check the platform notes for more information. Also binary packages for Ubuntu 10.04, CentOS 5.5, Debian 5.0.6 and RHEL 5.5 are available for download. We will appreciate any feedback you have after trying the new version, and don’t forget you can report your bugs either at the development portal or through our mailing list.
Please refer to the OpenNebula 2.0 documentation guides to install and configure your system. More information about OpenNebula can be found at:

For a detailed list of bugs addressed by OpenNebula 2.0 check the development portal.

OpenNebula 2.0 is targeted at production environments and 1.4 installations are recommended to update. However, take into account that, starting with OpenNebula 2.0 Beta, a new DB schema was introduced. Thus, this new release will break compatibility with OpenNebula 1.4 (at the DB and API levels). Host, Networks or Virtual Machines defined in OpenNebula 1.4 installs will not longer work with this release, and must be recreated in the new system (we are working to make this process more seamless in future releases). OpenNebula 2.0 also includes new API calls and methods. However there have been no changes in those API calls already supported in OpenNebula 1.4. Applications using OCA (Ruby and Java bindings) are expected to work with OpenNebula 2.0.

Finally, we would like to thank all the people and organizations that have contributed to make OpenNebula the most flexible, scalable and feature rich Cloud Toolkit!

The OpenNebula Team

How big will OpenNebula 2.0 be?

OpenNebula 2.0 is right around the corner, and we were curious to see how much effort has gone into it. As it turns out, nearly double the effort we put into version 1.4, at least measured by the number of lines inserted/deleted/modified in our code repository:

Of course, lines of code are a notoriously imprecise metric, so take the above with a healthy dose of salt. Plus, the above doesn’t account for the additional effort we’ve put into quality assurance, preparing documentation, helping users on IRC and on the mailing list, etc.

In any case, we’re really happy with the 2.0 release cycle, and specially with all the great contributions and feedback we’ve gotten from our community. Looking at the above graph, we’re also amazed at how far OpenNebula has come, from a small technology preview that we initially just ran on a small cluster to a stable and well-tested piece of software capable of wrangling tens of thousands of virtual machines. Another interesting fact about the OpenNebula source code is the number of programming languages and technologies used: C++/C (63.02%), Ruby (21.89%), Java (6.19%), Shell Script (5.79%), Lex & Yacc (2.13%).

So stay tuned… before you know it, you’ll be getting a release announcement from us!

New OpenNebula QA & hudson.opennebula.org site!

Last months have been quite busy for the team, a bunch of new features has been added to OpenNebula core as well exciting new components. There has been however a huge amount of effort that may go unnoticed in the upcoming OpenNebula 2.0 release.  As part of this release we also wanted to open our internal QA & testing  procedures to the community… so OpenNebula QA was born!

Testing & QA of a cloud management tool is particularly challenging as it requires to verify the integration of multiple software components, from different hypervisors (Xen, VMware & KVM) and storage architectures (SSH, NFS) to operating systems (Ubuntu and CentOS) and DB backends (MySQL and sqlite). Just check the OpenNebula QA page for a description of the automated test suite we are using, or if you want to check how your favorite Cloud toolkit behaves go to hudson.opennebula.org!

The OpenNebula Team

Top Ten Reasons for Using OpenNebula

Here are the top ten reasons for using OpenNebula to build your cloud infrastructure:

  1. Most advanced and innovative functionality to build your private and public cloud
  2. Unique functionality for building hybrid cloud deployments to supplement your local infrastructure with computing capacity from external Clouds
  3. Hypervisor independence supporting orchestration of major hypervisors (Xen, KVM and VMware)
  4. Production-ready with highly scalable and efficient back-end tested to manage several tens of thousands of VMs on thousands of cores, and the features for security and accounting that many enterprise IT shops need for internal cloud adoption
  5. Unique capabilities for integration allowing the deployment of the cloud on your existing infrastructure environment
  6. Powerful interfaces for cloud administrators, users and integrators
  7. Cloud interoperability and portability providing cloud users and administrators with choice across most popular cloud interfaces, hypervisors and public clouds and with a flexible software that can be installed in any hardware and software combination
  8. Adopt and implement standards
  9. Fully open source (not open core), and open architecture and interfaces
  10. Stable and robust technology rigorously tested, and matured through an active and engaged community

Our sections Why OpenNebula? and Features elaborate on these reasons, and of course you always have our user mailing list if you have questions.

The OpenNebula Project

OpenNebula Participation in Upcoming Cloud Events

The OpenNebula Project is participating in many upcoming events in cloud computing.

  • Invited Talk about OpenNebula in Virtualized Service Platform Session at Internet of Services 2010, Brussels, Belgium, 19th October 2010
  • Keynote about  “Key Research Challenges in Cloud Computing” at the 3rd EU-Japan Symposium on Future Internet and New Generation Networks, Tampere, Finland, 20th October, 2010
  • Keynote about “OpenNebula: Leading World Innovation in Open-source Cloud Computing” at CloudComp 2010, Barcelona, Spain, 26th October 2010
  • Keynote about “OpenNebula: Leading Innovation in Cloud Computing Management” at OGF30/Grid2010, Brussels, Belgium, 27th October 2010
  • Invited Talk about “OpenNebula Cloud Case Studies” and Panel about “Your Organization’s Cloud Where to Start What to Do, What to Avoid” at ISC Cloud 2010, Frankfurt, Germany, 28th and 29th October 2010
  • Invited Talk about “Enterprise Usage of OpenNebula” at the 7th International Cloud Computing Expo, Santa Clara, U.S.A., 1st November 2010
  • Keynote about “OpenNebula: Leading World Innovation in Open-source Cloud Computing” at OW2 Annual Conference 2010, Paris, France, 24th November 2010
  • Tutorial about “OpenNebula: Leading Interoperability and Innovation in Cloud Computing Management” at CloudCom 2010, Indianapolis, USA, 2nd December 2010
  • Invited Talk about OpenNebula at the Telecom Cloud Information Days & Standards Workshop, Santa Clara, USA, 6th December
Would be great to meet you at these venues.  If you want to connect, please send an email to contact@opennebula.org
The OpenNebula Project

C12G Labs Delivers Enterprise Extensions to OpenNebula

C12G has just announced that the software extensions distributed in the Enterprise Edition of OpenNebula have been contributed back to the OpenNebula Project. These extensions were created to support customers and partners and to enhance the functionality and performance of OpenNebula in enterprise-class and very-large scale systems. The new open-source components include:

  • Enhanced VMware Adaptor that enables the management of an OpenNebula cloud based on VMware ESX, vCenter and/or VMware Server hypervisors
  • LDAP Authentication Module that permits permits users to have the same credentials as in LDAP, so effectively centralizing authentication
  • Accounting Toolset that visualizes and reports resource usage data, and allows their integration with chargeback and billing platforms

These are the first contributions to OpenNebula Addons that are available in the Addon Catalog. The OpenNebula Project endorses these extensions and supports them through the user mailing list. Moreover, the project ensures its full compatibility with current and upcoming releases of OpenNebula. This news confirms OpenNebula as fully open source cloud software, not being a feature or performance limited edition of an Enterprise version. OpenNebula Enterprise brings additional benefits of long term professional, integration and certification support services, and  regular updates and upgrades.

The OpenNebula Project