Written in conjunction with Tino Vazquez from the OpenNebula Project, and cross-posted on the OpenNebula blog.

Puppet is used for managing the infrastructure for many IaaS software packages, including Eucalyptus, OpenStack, and OpenNebula. OpenNebula is an IaaS manager which can not only manage a large amount of different virtualization and public cloud platforms, it can also emulate the API’s provided by EC2 and OCCI. It’s great for creating private and public clouds, as well as hybrids of the two.

Puppet Labs (or, more specifically, Ken Barber) has developed a powerful integration between OpenNebula and Puppet. The installation and configuration of OpenNebula be managed with this solution, and a virtualized infrastructure can be provisioned starting from bare metal using only Puppet code. The module for Puppet that integrates with OpenNebula can be downloaded from the Forge here: http://forge.puppetlabs.com/puppetlabs/opennebula

Installation and Configuration

The Puppet module contains several classes, types and providers for managing OpenNebula resources and installing OpenNebula.

The class ‘opennebula::controller’ is used for managing the main controller node for OpenNebula and is a simple class. An example usage would be:

class { "opennebula::controller":
  oneadmin_password => "mypassword",
}

This will configure the necessary parts for the main controller node, applying the necessary password for the primary ‘oneadmin’ user.

The class opennebula::node can be applied to nodes that will act as hypervisors. This class configures the necessary package and SSH authorization that is used by the SSH transfer, information and virtualization driver in OpenNebula.

The class itself really needs to know the location of its master, and uses stored configurations for shipping the necessary SSH keys across:

class { "opennebula::node":
  controller => "one1.mydomain.com",
}

The Sunstone GUI can be remotely managed using the ‘opennebula::sunstone’ class. The module also can manage the EC2 gateway using ‘opennebula::econe’.

Managing OpenNebula Resources

OpenNebula has many elements that can be managed on the command line:

• Hosts
• Images
• Virtual Networks
• Virtual Machines

What’s great about OpenNebula is that the same resources can be managed using their own GUI, namely ‘Sunstone’:

We provide some resource types through the Puppet OpenNebula module that allow managing these elements via Puppet as well. The detailed documentation for each of these is provided in the README file for the module, but let’s talk about one in particular: the onevm resource.

The onevm resource allows you to actually manage a virtual machine as if it was a Puppet resource. An example usage in Puppet would be:

onevm { "db1.vms.cloud.bob.sh":
  memory => "256",
  cpu => 1,
  vcpu => 1,
  os_arch => "x86_64",
  disks => [
    { image => "debian-wheezy-amd64",
      driver => "qcow2",
      target => "vda" }
  ],
  graphics_type => "vnc",
  graphics_listen => "0.0.0.0",
  context => {
    hostname => '$NAME',
    gateway => '$NETWORK[GATEWAY]',
    dns => '$NETWORK[DNS]',
    ip => '$NIC[IP]',
    files => '/var/lib/one/context/init.sh',
    target => "vdb",
  }
}

As you can see, this mirrors all of the options made available via the template when creating virtual machines using the command line or Sunstone GUI in OpenNebula:

Using Puppet provides just another capability for managing OpenNebula. Upon creation, the VM will be created just like any other VM and now appear when running ‘onevm list’ or viewing the list of virtual machines in Sunstone:

Managing Applications End-to-End

An end-to-end example to demonstrate the capabilities of this integration is the deployment of a sample pastie/pastebin application with redundant web servers:

The sample content to build such an infrastructure is located here: http://github.com/kbarber/puppet-onedemo

In this demo content we deploy the IaaS manager OpenNebula, correctly configured and including its dependencies like libvirt. We then use the newly installed virtualization engine to start a virtualized application consisting of web servers behind a load balancer.

Combining OpenNebula and Puppet allows you to achieve a fairly complete end-to-end architecture for rapid deployment within a private cloud infrastructure. The following diagram shows some of the necessary elements in such an end-to-end architecture:

Development Progress

Currently the puppetlabs-opennebula module is OpenNebula 2.2 specific, but we are looking to add OpenNebula 3.0 support once it becomes available in the distributions (such as Debian). If you like the idea of having Puppet manage OpenNebula for installation, configuration or for management we are looking for more code contributors, testers and users.

Bugs can be raised in the Puppet Redmine project for our public modules here.

And the code is available here: https://github.com/puppetlabs/puppetlabs-opennebula

Any help or comments are much appreciated, your feedback will be used to refine the integration and make it more functional. We are confident that this integration adds value to your IaaS and Private Cloud projects, and we hope you enjoy using it as much as we did implementing it.

Additional Resources

• More information on the Puppet/OpenNebula integration, in slides and video.

Puppet Labs was lucky enough to have two Google Summer of Code (GSOC) students working with us this past summer.  This was our first year working with the program and we got some results that the Puppet community will hopefully be able to start using soon.

One of our biggest challenges in working with the students was giving them consistent support while we dealt with our own rapidly changing environment that included a lot of new hires who also had to be trained, myself included.  I learned right along with the students as they figured out how to write puppet types and providers.  Thankfully, longer term community members such as Luke, James and Markus were there to answer questions and steer development.

Libvirt – Carla Araujo (Federal University of Campina Grande)
http://github.com/carlasouza/puppet-virt

Carla noticed a need that Distributed Systems Laboratory had for managing virtual machines with Puppet, so she set out to create a solution.  Her libvirt module has support for Xen and KVM virtual machines.  If that sounds like something you could use checkout her code and fire up some virtual instances.

Network Interfaces – William Van Hevelingen (Portland State University)
http://github.com/blkperl/puppet-network

William set out to fill a gap that many Puppet users have been wanting filled: managing network interfaces.  William had the benefit of being local to Portland where all the Puppet Labs developers work, so he was able to get some face time and pairing sessions on his project.  While he made great progress in his development, a bug in Facter and a difficult to parse Ubuntu interfaces file limited the scope of his solution.  However, William is continuing to work on the code past the deadline and continues to make progress.  If you are interested in his work and think you can help, feel free to take a look at his code and send him a message on github.

William had the following to say about his GSOC experience:

• I learned a lot about developing open source and working with an existing source repository
• I had a great time learning how to develop on Puppet and I plan to continue to contribute
• Puppet Labs was very helpful and supportive
• The Puppet community definitely encouraged and helped with the creation of my network module

Thanks to our GSOC students for their hard work this summer. We’re glad it was a positive experience and look forward to getting more Summer of Code students next year.

This week I had the opportunity to attend Ubuntu Developer Summit for the Maverick Meercat release. As you may know, Puppet is a component of the Ubuntu Cloud images for both EC2 and private cloud. What makes this even more exciting is the announcement that Dell will offer Ubuntu Cloud as a preinstall option on their servers. You may read this as, effectively, as “Puppet preinstalled on Dell servers”. We’re very happy about this. The included Puppet goes beyond mere package inclusion, and includes some nice work that Canonical engineers have done to make auto-signing and setup in the cloud even smoother. In coming releases, this will become even easier to use.

At UDS we discussed many great ideas, including ways to auto-scale puppetmaster in the cloud, provide auto-magical HA and distributed puppetmasters, further improving autosigning for EC2, improving the dpkg provider, and a new launchpad project to package Puppet content for things like database servers, mail servers, web servers, and LDAP — providing easy setup of server infrastructure out of the box. If you’re interested in following this activity, there are numerous blueprints in Launchpad to follow (just search for puppet). We’re also exploring similar things ourselves to make very large scale out and HA very easy to achieve as enterprises grow from small to large in very small amounts of time — a key feature provided by public and private cloud infrastructure.

In all, it’s an exciting time for Puppet, Ubuntu, EC2, and virtualized infrastructure. It’s going to be an great year.