Management of network devices is one of the exciting new features in Puppet Enterprise 2.0 and Puppet 2.7. In the initial release, support is limited to Cisco devices, but because Puppet is extensible via modules, we are able to build upon the existing framework and add support for F5 BIG-IP. Like most network appliances, installation of third party software is prohibited, which eliminates the ability to run an agent. Instead, Puppet takes advantage of F5 iControl API to interact and manage the device. F5 BIG-IP network appliances are capable of load balancing, SSL offloading, application monitoring, as well as many other advanced features, and now Puppet can manage these functionalities. Compared to traditional management methodologies and other third party tools that interact with F5, Puppet not only bridges the gap from deploying applications to bringing the service online to your customers, it also brings the unique benefit of the Puppet resource model to network devices. More specifically, the integration offers the ability to compare if a running configuration matches the desired configuration, and then enforce the changes once they’ve been reviewed.

In this blog post, we will step through the process of installing the F5 module, configuring connectivity, and writing a simple manifest to manage an F5 device with Puppet. If you are unfamiliar with BIG-IP devices, you may want to consult devcentral.f5.com for more information on F5 features such as iRules and iControl API.

In the following output, Puppet detects that the F5 device has the wrong iRule. We are running in simulation mode (--noop option) so Puppet only shows the changes that would be applied if we were enforcing the configuration.

$ puppet device --noop
...
notice: /Stage[main]//F5_rule[redirect_404]/definition: current_value: when HTTP_RESPONSE {
if { [HTTP::status] eq "404" } {
redirect to "http://www.puppetlabs.com/404/"
}
}, should be when HTTP_RESPONSE {
if { [HTTP::status] eq "404" } {
redirect to "http://www.puppetlabs.com/redirect/404/"
}
} (noop)
notice: Finished catalog run in 5.69 seconds

Now running Puppet with out the --noop option set, the iRules are changed to match our resource declaration.

$ puppet device
…
notice: /Stage[main]//F5_rule[redirect_404]/definition: definition changed when HTTP_RESPONSE {
if { [HTTP::status] eq "404" } {
redirect to "http://www.puppetlabs.com/404/"
}
}, to when HTTP_RESPONSE {
if { [HTTP::status] eq "404" } {
redirect to "http://www.puppetlabs.com/redirect/404/"
}
}
notice: Finished catalog run in 5.74 seconds

In addition to iRules, the initial release supports key features to configure the device certificate, manage applications pool/poolmember/virtualserver, and monitor application health. The module is published on the Puppet Forge, along with comprehensive documentation of supported F5 resources. The latest development release is available on GitHub. The ability to extend Puppet is not limited to network devices, and the commands shown later in this post to install the module is applicable for other Puppet modules available on the Puppet Forge and GitHub.

The puppet device command is a new application mode in Puppet intended to manage devices that can’t install Ruby/Puppet and run puppet agent. If you haven’t checked it out yet, I would review Brice’s introduction to network devices first. The high level overview of the entire communication process:

Devices are managed through an intermediate proxy system where Puppet agent is installed. The proxy system stores a certificate on behalf of the device. In the case of F5, it should have iControl gem installed, as well as the account information in device.conf to communicate with the device. The proxy connects to the Puppet master to retrieve the catalog on behalf of the F5 and applies changes as necessary.

F5 module installation

Before we get started with the installation process, there are two ways to install F5 module. First, via puppet-module tool which retrieves it from forge.puppetlabs.com (for stable releases of the module). The latest development release from GitHub is accessible via git. The instructions below are specific for Puppet Enterprise, but open source users can also install the module with some changes to the puppet module path. Puppet Enterprise currently ships with the puppet-module gem, and it’s freely available on rubygems.org. Eventually this will become a Puppet Face and turn into the command ‘puppet module’(expected in later release of 2.7). Onwards to the install process:

# Puppet Enterprise:
cd /etc/puppetlabs/puppet/modules
puppet-module install puppetlabs-f5

This should create a directory called f5. Older versions of puppet-module tool might create a puppetlabs-f5 directory in the modules directory—in that case, change it to f5.

Installing from GitHub:

# Puppet Enterprise:
cd /etc/puppetlabs/puppet/modules
git clone git@github.com:puppetlabs/puppetlabs-f5.git
ln -s puppetlabs-f5 f5

In Puppet 2.7, the transport between proxy agent and device supports telnet/ssh, however neither is suitable for F5 devices. Instead, we rely on F5’s iControl API. The iControl gem should be installed on both the master and the proxy system. This gem is available in the F5 module files directory.

# Puppet Enterprise:
/opt/puppet/bin/gem install /etc/puppetlabs/puppet/modules/f5/files/f5-icontrol-10.2.0.2.gem

Configuration and management

At this point we have the module installed, so we should configure connectivity for the device. The configuration for network devices by default are stored in /etc/puppet/device.conf:

[f5.puppetlabs.lan]
type f5
url https://username:password@f5.puppetlabs.lan/partition

[f5.dev.puppetlabs.lan]
type f5
url https://username:password@f5.dev.puppetlabs.lan/partition

You can also break down each device into it’s own configuration file such as /etc/puppet/f5_device1.conf, /etc/puppet/f5_device2.conf …, which is especially helpful if you wish to run against each device separately. In the square brackets is the device certificate name, and certificate management process is the same as puppet agent certs. The device type is f5, and the url is https instead of telnet/ssh. Because F5 supports different partitions we can optionally specify them at the end, and it will default to the ‘Common’ partition if it’s not provided. In the module, f5::config define resource type simplifies management of this configuration file on the proxy system:

f5::config { 'f5.puppetlabs.lan':
    username => 'admin',
    password => 'password',
    url      => 'f5.puppetlabs.lan',
    target   => '/etc/puppetlabs/puppet/device/f5.puppetlabs.lan.conf',
}

Once this configuration file is in place, we can initiate a puppet device run on the proxy server.

# execute on proxy server
$ puppet device --deviceconf /etc/puppetlabs/puppet/device/f5.puppetlabs.lan.conf

This should generate a certificate request on the master which should be signed:

# execute on puppet master
$ puppet cert -l
f5.puppetlabs.lan (2A:0C:A0:F8:C6:EE:EF:9B:B3:49:74:D1:27:31:1B:60)
$ puppet cert -s f5.puppetlabs.lan

At this point the master should have a node name f5.puppetlabs.lan in site.pp with the appropriate f5 resources:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34

node f5.puppetlabs.lan {
  f5_rule { 'redirect_404':
    ensure     => 'present',
    definition => 'when HTTP_RESPONSE {
if { [HTTP::status] eq "404" } {
redirect to "http://www.puppetlabs.com/redirect/404"
}
}',
  }
 
  f5_pool { 'webapp':
    ensure                          => 'present',
    action_on_service_down          => 'SERVICE_DOWN_ACTION_NONE',
    allow_nat_state                 => 'STATE_ENABLED',
    allow_snat_state                => 'STATE_ENABLED',
    lb_method                       => 'LB_METHOD_ROUND_ROBIN',
    member                          => {
      '10.10.0.1:80' => {'connection_limit' => '0',
                         'dynamic_ratio'    => '1',
                         'priority'         => '0',
                         'ratio'            => '1'},
      '10.10.0.2:80' => {'connection_limit' => '0', 
                         'dynamic_ratio'    => '1', 
                         'priority'         => '0',
                         'ratio'            => '1'},
      '10.10.0.3:80' => {'connection_limit' => '0',
                         'dynamic_ratio'    => '1',
                         'priority'         => '0',
                         'ratio'            => '1'}
    },
    minimum_active_member           => '1',
    minimum_up_member               => '0',
  }
}

When the puppet device command is executed again this will update the iRule and ensure the appropriate members are in the webapp pool:

$ puppet device --deviceconf /etc/puppetlabs/puppet/device/f5.puppetlabs.lan.conf

A limitation to watch out for in the current Puppet release is that ‘puppet apply’ and ‘puppet resource’ cannot modify network resources. However, we implemented a feature to allow puppet resource to query a F5 device. (For authors of types/providers, making changes to resources aren’t supported until apply_to_device in resources type are handled differently by puppet apply/resource commands). For now we use url facts to establish connectivity to specific F5 devices:

export RUBYLIB=/etc/puppetlabs/puppet/modules/f5/lib/
export FACTER_url=https://admin:password@f5.puppetlabs.lan/Common
puppet resource f5_rule

f5_rule { '_sys_https_redirect':
  ensure     => 'present',
  definition => '    when HTTP_REQUEST {
set host [HTTP::host]
HTTP::respond 302 Location "https://$host/"
}',
}
f5_rule { '_sys_auth_ssl_cc_ldap':
  ensure     => 'present',
  definition => '    when CLIENT_ACCEPTED {
set tmm_auth_ssl_cc_ldap_sid 0
set tmm_auth_ssl_cc_ldap_done 0
}
when CLIENTSSL_CLIENTCERT {
…

If you don’t have Puppet Enterprise 2.0 in your environment yet, you can either download the Learning Puppet VM, or Puppet Enterprise installation packages. F5 also provides F5 LTM Virtual Edition (VE) for trial on VMWare. Please report any issues or bugs to http://projects.puppetlabs.com/projects/modules/issues under the modules section.

Additional Resources

• Download the F5 module from the Puppet Forge
• Read the intro to managing network devices with Puppet: Puppet Network Device Management
• Learn how to use Puppet with the Learning Puppet VM or with a free ten node download of Puppet Enterprise

The first Puppet Triage-a-thon was a huge success. Thank you so much to everyone who attended and contributed! Some of our favorite stats from the event:

• We started the day with 2292 open tickets. Over the course of the day 565 tickets were triaged. We also closed 115 tickets. That’s almost 25% of the open tickets triaged and 5% closed!
• Seventeen community patches were submitted! (And many of these were merged during the day.) Several patches were from people who’ve never contributed before, which is awesome.
• We had about 50 people involved globally. Individuals and teams participated from Germany, Mexico, Australia, United Kingdom, France, and across the United States. And a big shout-out to the Atlanta Puppet Users group who had a team of five people!  We also had about 10 community members onsite in the Puppet Labs offices locally in Portland.

Our top contributors from the community were:

1. Dominic Cleal
2. Devon Peters
3. Patrick Otto
4. Oliver Hookins

They’ll be the recipients of $100 Amazon gift vouchers. The full list of participants is here, along with the tally of tickets triaged: http://triageathon.puppetlabs.com/

Everyone who contributed will receive a Puppet Labs t-shirt. If you participated you’ll receive an email shortly asking you for your preferred t-shirt size and where you’d like us to send it.

We’re going to continue to run these events in the future to keep the ticket database fresh and up-to-date.

Thanks again to everyone who came along and contributed and made it such a great success and an awesome amount of fun!

With the launch of Puppet Enterprise 2.0, it’s easy to get lost in the scale of what we’ve enabled sysadmins to do. Besides shipping a world-class configuration management tool, we now make it easy to create managed servers from thin air. In addition, we’re providing command line tools that take fresh OS installs directly to production-ready application servers with no manual intervention, monitoring and reporting for when your configuration drifts, and a web interface that allows you to inspect the current state of resources across your entire population—fast.

For all the things that we’ve done with our software, there’s a much longer list of things that we just didn’t have to do thanks to the open source community at large. So, we’d like to acknowledge some of the open source software that have helped make P.E. 2.0 a great product.

Batman

We made the decision early in the development cycle for live management that we wanted to build an application that was as lively and responsive as possible. Our search for an appropriate browser-based-site framework led us to several options, but Batman (who showed up fashionably late to the investigation) impressed us immediately.

Applications written against Batman were reasonably familiar to most of the live management team (including those who had little prior Javascript experience), but the real stars are the hard-working members of the Batman development team. From day one, the core developers were engaged, gracious, and actively involved in helping make our product the best it could be, even as we took Batman down roads they never expected anyone would. Our own Pieter van de Bruggen has made many contributions to Batman’s core.

Fog

When we were building our first pass of cloud provisioning, we began with the single target of Amazon’s EC2 cloud. Fog provided an accessible way to interface with that target (in addition to a number of future targets), so the decision felt fairly obvious up front. After building our first prototype, we discovered we had also built a tool for installing Puppet Enterprise on real hardware with only an SSH connection. With a few small tweaks, it could also work on Rackspace, OpenStack, and a large number of other providers (coming soon, we promise!). Combined with the developer’s responsiveness and guidance, Fog has been an amazing asset to us. At Puppet, Jeff McCune, Kelsey Hightower, Carl Caum, and Pieter van de Bruggen have made many contributions to Fog.

Sinatra

Sinatra’s not exactly an unknown in the Ruby web development community, but it has been fundamental to us in our live management development. With all of our presentation logic hidden away in a lightweight frontend application, we wanted to build the backend against a similarly lightweight framework. Sinatra gave us just the right balance between power and simplicity on a well-known, stable platform.

LESS

Less is a CSS abstraction tool, helping us make the Puppet Enterprise console CSS much lighter and more maintainable.

CoffeeScript

A browser-side language, compiling to Javascript, used extensively throughout live management. Batman.js is written in CoffeeScript.

ActiveMQ

A message queue, used by MCollective to distribute work to your entire infrastructure.

Puppet, Dashboard, MCollective, and Facter

Puppet Enterprise wasn’t developed in a vacuum: the core tools themselves are open source software, and have for years received significant contribution from the community outside our doors.

And so many others…

We don’t have the words or the space to properly say “thank you” to every project whose open source work has helped us reach this release. So let me just say once, to every contributor on every project that’s touched our work: thank you. Your work has not gone unnoticed nor unappreciated, and I hope that our own contributions to open source software can serve to pay the kindness forward.

It’s come time for us to pick a code name for our next release of Puppet. If you help develop Puppet you may be fairly familiar with our naming system, but for those who haven’t yet contributed to the open source project, we name each version of Puppet after a Muppet.

We use code names because it gives us the flexibility to decide the maturity of a release. By setting features at “Telly” we’ve not committed to “Telly” being a major release or a point release. The code name allows us to holistically review the growth of the project and decide which level of release is most appropriate.

This time we’d like you to pick the name. At the bottom of this post is a place for you to pick your favorite name(s) and send them to us. The Muppet Wiki has a comprehensive list of Muppets. Our only rule is that you pick a name we haven’t used yet that comes after the letter ‘T’ (in recent releases, we narrowed the field by picking names in alphabetical order). Polls close at the end of 2011.

For reference, here’s a table of previous (and therefore ineligible for this contest) code names:

We’re excited that Puppet has been chosen as the 2011 Best Open Source Configuration Management Tool by Linux Journal readers. This is the second year in a row Puppet has earned the honor, and we can’t thank our enthusiastic users and community enough.

From the Linux Journal article: “If you manage greater than zero servers, you will benefit from using a tool like Puppet.”

Related Content:

• Learning Puppet VM
• Join the Puppet Community
• Read the Puppet docs
• The Puppet Project Home

OVERVIEW

We have discovered a security vulnerability (“AltNames Vulnerability”) whereby a malicious attacker can impersonate the Puppet master using credentials from a Puppet agent node. This vulnerability cannot cross Puppet deployments, but it can allow an attacker with elevated privileges on one Puppet-managed node to gain control of any other Puppet-managed node within the same infrastructure.

All Puppet Enterprise deployments are vulnerable, and Puppet open source deployments may be, depending upon their site configuration.

We believe this to be a serious risk, and we have confirmed this with security experts outside of Puppet Labs.

You can immediately protect yourself from this vulnerability with the following steps:

1. Prevent your Puppet CA from issuing any more dangerous certificates
2. Create a new DNS entry for the Puppet master
3. Issue a new Puppet master certificate with its new DNS name
4. Configure all Puppet agent nodes to contact the Puppet master at its new name

After the threat has been mitigated, we recommend that users upgrade their version of Puppet open source or Puppet Enterprise and migrate to a new certificate authority for a more complete remediation.

Puppet Labs has released a toolkit to help users protect themselves immediately and to automate a complete remediation process. We’ve also written supporting documentation that includes a walk-through of the toolkit and describes other methods of remediation.

The toolkit and documentation are available for download here:

• CVE-2011-3872 AltNames Vulnerability – Remediation Toolkit & Documentation

Given the risk, Puppet Labs recommends customers and users remediate as quickly as their change management processes permit.  Regarding the time required to remediate, in internal tests a 20 node deployment required just over an hour to remediate from start-to-finish, with most of that time spent waiting between Puppet runs.

For users requiring additional help with remediation, our Customer Service Engineers are available via our #puppet IRC channel and Puppet Users Google Group:

• #puppet on IRC
• Puppet Users Google Group

Finally, Puppet Labs has prepared a detailed FAQ to answer any additional questions:

• CVE-2011-3872 AltNames Vulnerability – FAQ

Going forward, Puppet Labs will increase the scrutiny of and investment in the security of its products, and we look forward to hearing feedback on these projects from the community.

- The Puppet Labs Team

TECHNICAL DETAILS

We have discovered a security vulnerability which can cause Puppet’s certificate authority (CA) to issue Puppet agent certificates capable of impersonating the Puppet master. These certificates will remain dangerous even after reconfiguring the Puppet master, but can be made safe by reconfiguring all Puppet agent nodes. This issue affects all Puppet Enterprise versions and every version of Puppet open source since 0.24.0.

The bug

If the Puppet master’s “certdnsnames” setting contains anything when an agent certificate is signed, those “certdnsnames” values will be added to the certificate’s Subject Alternative Name field. (The expected behavior prior to discovery of this bug was that these Subject Alternative Name values would only be added to Puppet master certificates.)

How the bug creates a vulnerability

When Puppet agent connects to a server purporting to be its Puppet master, it checks its legitimacy by comparing the certificate’s Subject: CN and Subject Alternative Name fields to the value of its own “server” setting. Thus, to an agent that uses one of the master’s alternate DNS names in its “server” setting, any agent certificate issued by a CA with “certdnsnames” will appear to belong to the Puppet master.

How the vulnerability can be exploited

An attacker who has obtained an agent private key and a certificate created with an improper Subject Alternative Name field can combine them with a man-in-the-middle attack of their choice to impersonate a site’s Puppet master, and can serve arbitrary catalogs to any agent node that trusts those DNS alt names. As Puppet agent typically runs as root, this effectively allows the attacker to gain full control of nodes managed by Puppet.

Limitations on exploits

• Attacks on this vulnerability are not possible unless the “certdnsnames” setting has been activated on the Puppet master at some point during the lifetime of the current CA certificate. Note that all installations of Puppet Enterprise have used the “certdnsnames” setting for at least part of their lifecycle.
• Attacks on this vulnerability require a private key and a signed certificate with the Puppet master’s DNS alt names. The likelihood of an attacker obtaining a certificate and a private key varies depending on certificate signing policies and the types of machines managed by Puppet. An attacker is least likely to obtain them at sites where all certificates must be signed manually and are only issued to servers with no root access; an attacker is most likely to obtain them at sites where certificates are autosigned and user laptops are managed by Puppet.
• This vulnerability is not relevant if Puppet is being used in a masterless configuration where each node compiles its own catalog.
• Normal limitations on man-in-the-middle attacks apply.

The upgrades

Puppet

• 2.6.12 and 2.7.6

Puppet Enterprise Users

• Hotfix for PE 1.0/1.1
• Upgrade to PE 1.2.4 for 1.2

Distribution maintainers have been sent patches for all the versions of Puppet that are currently maintained in Fedora, EPEL, Debian, Ubuntu and Gentoo.

RubyGems has also been updated as part of our normal release process.

Additional Resources

• CVE-2011-3872 AltNames Vulnerability
• Puppet Labs Hotfixes
• Puppet Labs Security

The day after the party is always the hardest. As you brush your teeth, you remember all of the things you accomplished the night before: the people you met, the names you have forgotten, the number of extremely large Puppet t-shirts you somehow acquired, and two or three OSL mugs you carried across town back to your hotel. Perhaps those last few things only apply if you attended the Puppet Labs OSCON party last week.

Here at Puppet, we have our own list of accomplishments, and a lot of it has to do with you.

We are proud to announce that at this party:

• 784 participants entered the building
• Around $1500 of spirit, 3 kegs of beer, 40 lbs of ice, and 2000 sandwiches were consumed
• Thousands of memorable conversations took place—there is an old Chinese proverb that is roughly translated to this: a single conversation across the table with a wise person is worth a month’s study of books.

We had a party at Puppet Labs because we hold individuals like you in high regard—individuals who value curiosity, self-motivation, and adaptability. By hosting networking parties like the one you attended, we expose ourselves to a wealth of knowledge and inspiration. So thank you, for attending, socializing, drinking, and having a blast.

With that said, I am going to bluntly state that if you have the above-mentioned values and have a passion for what Puppet is all about, come work for us. We’ll have a grand time.

Moreover, attend PuppetConf in September so that we can keep having these conversations.

Lastly, thank you to OSUOSL (Oregon State University Open Source Lab) for their donation to the party. For more information or to donate, please visit their website.

Keep in touch, and see you soon.

(Our Pro Services Engineer Gary Larizza took the photo above and many more of the night; check them out!)

With Puppet we build infrastructures, piece by piece, manifest after manifest. We control how nodes are configured, what services they provide, how they are checked.
We manage where web applications stay and sometimes how they are built, tested, and deployed.

Puppet has a lot of knowledge about our systems. Every catalog we receive is an unique source of data that is exactly what brings our servers to the desired state. The more information we provide in our manifests about the system, the more we can things we can do with it. Puppi tries to bring this knowledge to the command line.

Present

Puppi was initially developed to standardize different procedures of web applications deployments and it has evolved into a shell command that provides handy and quick actions useful for the system administrator. Now it is stable enough to have reached version 1.0 (currently in RC state) the features initially planned are present, it’s used in production and no big changes are planned for this version. See the presentation held at The Puppet Camp Europe 2011 for some videos and further details.

Puppi’s most useful actions are:

• deploy: to manage the whole deploy workflow with a single keystroke
• check: to verify the general system’s health and specific checks on the application deployed
• log: to quickly tail some or all the known logs
• info: to show the output of a custom set of preconfigured commands
• rollback: to quickly rollback a deployed application

Puppi is currently entirely provided as a Puppet module, you include it and you have the whole Puppi functionality:

• the bash command /usr/sbin/puppi
• its configuration directory /etc/puppi, with plenty of files and dirs configured by Puppet defines like puppi::check, puppi::info etc
• a set of (customizable) defines that build deploy procedures, like puppi::project::maven that retrieves Java artifacts generated by Maven
• some general use native scripts that are used to accomplish the different steps of a deployment
• a set of defines to populate the output of Puppi actions
• some default content for Puppi info, log and check actions to make Puppi useful out of the box.

There are not other modules prerequisites, but for full functionality you need on the systems where you place Puppi these commands: wget, mail, rsync and the most common Nagios Plugins.
Note also that Puppi is entirely self-contained, it doesn’t need external services to run. Once deployed, all its actions are based on local files and data; checks are local, info scripts are local, deploy procedures need only the availability of the defined source to work.

How to Use

The Puppi module provides some deploy procedures that cover many typical scenarios.
For example, to retrieve a war from $source and deploy it in $deploy_root, keeping a copy for rollback and notifying $report_email, you need this:

 
puppi::project::war { "myapp":
    source           => "http://repo.example42.com/deploy/prod/myapp.war",
    deploy_root      => "/store/tomcat/myapp/webapps",
    report_email     => "sysadmins@example42.com",    
}

All the existing deploy procedure have a set of optional arguments that make them more flexible. Here a more complex case:

puppi::project::maven { "supersite":
    source           => "http://nexus.example42.com/nexus/content/repositories/releases/it/example42/supersite/",
    deploy_root      => "/usr/local/tomcat/supersite/webapps",
    config_suffix    => "cfg",
    config_root      => "/srv/htdocs/supersite",
    document_suffix  => "css",
    document_root    => "/srv/htdocs/supersite",
    firewall_src_ip  => $site ? {
        dr      => "192.168.101.1/30",
        main    => "192.168.1.1/30",
    },
    backup_retention => "3",
    init_script      => "tomcat",
    report_email     => "sysadmins@example42.com",
    enable           => "true",
}

This does the following actions in sequence:

1. Retrieves the maven-metadata.xml from $source
2. Blocks access from a loadbalancer IP
3. Backups the existing data for rollback operations
4. Deletes older backups (3 archives are kept, instead of the default 5)
5. Deploys the release war in $deploy_root
6. Unpacks a configurations tarball tagged with the Maven qualifier $config_suffix in $config_root
7. Unpacks a static files tarball tagged with the Maven qualifier $document_suffix in $document_root
8. Restarts tomcat and notifies via mail

All this can be triggered with the command “puppi deploy supersite” and if something fails you can “puppi rollback supersite”.

The above puppi::project::maven (or tar|war|list|dir|mysql..) defines build up the logic and the sequence of commands run in deployment and rollback operations using basic Puppi defines like puppi::project, puppi::deploy, puppi::rollback, puppi::init.
You can use the existing puppi::project::* procedures or build up your own ones, to manage special cases. The same bash scripts they use (“native scripts”, stored in puppi/files/scripts/) can be replaced by custom scripts, in whatever language.

The other Puppi actions require simpler constructs, for example you can manage a single check (we use Nagios plugins, as they are so common) with:

puppi::check { "Port_Apache":
    command  => "check_tcp -H ${fqdn} -p 80" ,
}

or insert more elaborated checks in your defines (for example when you create virtualhosts, using data you may already provide):

puppi::check { "Url_$name":
    enable   => $enable,
    command  => "check_http -I '${target}' -p '${port}' -u '${url}' -s '${pattern}'" ,
}

The logs to tail with Puppi log are defined by the puppi::log define (note that with a simple selector you can adapt the commands to run according the underlining OS):

puppi::log { "auth":
    description => "Users and authentication" ,
    log => $operatingsystem ? { 
        redhat => "/var/log/secure",
        darwin => "/var/log/secure.log",
        ubuntu => ["/var/log/user.log","/var/log/auth.log"],
}

Also in this case you can insert a puppi::log inside an existing define, using the data it inherently has:

puppi::log { "tomcat-${instance_name}":
    log => "${tomcat::params::storedir}/${instance_name}/logs/catalina.out"
}

You can manage the output of “puppi info network” with something like:

puppi::info { "network":
    description => "Network settings and stats" ,
    run         => [ "ifconfig" , "route -n" , "cat /etc/resolv.conf" , "netstat -natup | grep LISTEN" ],
}

or build more elaborated info subclasses using custom templates and specific data:

puppi::info::instance { "tomcat-${instance_name}":
    servicename => "tomcat-${instance_name}",
    processname => "${instance_name}",
    configdir   => "${tomcat::params::storedir}/${instance_name}/conf/",
    bindir      => "${tomcat::params::storedir}/${instance_name}/bin/",
    pidfile     => "${instance_rundir}/tomcat-${instance_name}.pid",
    datadir     => "${instance_path}/webapps",
    logdir      => "${instance_logdir}",
    httpport    => "${instance_httpport}",
    controlport => "${instance_controlport}",
    ajpport     => "${instance_ajpport}",
    description => "Info for ${instance_name} Tomcat instance" ,
}

Examples are endless, you can easily extend and customize the existing defines and you can integrate them in your modules according to the needs you have.

The Joys of Collectivism

Puppi’s purpose is not only to provide a command tool based on Puppet data, that helps the sysadmin to gather info, deploy applications, troubleshoot and check systems—it can be run manually from the local system, automatically via a cron job or triggered by a web interface. It can be used to summarize a set of common actions to be sudoed by non-privileged users, and it can be called by an agent of an orchestration tool.

Puppi enters into a new scale with the MCollective agent Puppi and the command mc-puppi: Whatever can be done locally with Puppi can be repeated on the whole infrastructure with the same syntax, using the power of MCollective. This becomes particularly interesting when your deploy procedures involve actions on different nodes, or when you need to quickly check the system’s health on a multitude of nodes. A command like “mc-puppi check” runs and shows the equivalent of ALL your Nagios checks on your WHOLE MCollective domain. There may be thousands; you have them in few seconds. I like to consider this a real-time distributed instant infrastructure test. Something like “mc-puppi info network” instead provides immediate overview of the network configuration and status of all your nodes, and if verbosity bothers you, just grep what you need.

A missing piece in the Puppi world is a web frontend that gathers the reports of the deployments, collects information about nodes, shows the results of local checks, and possibly lets users trigger deploy procedures via a central web console. The development of a web interface to Puppi, although planned since the beginning, has not yet started. The main reason is that it was considered a priority to have a stable command and a MCollective agent, another reason it’s time to make decisions about Puppi, and possibly these have to be shared.

Future

Puppet development is growing quickly. At the Puppet Camp Europe 2011 Luke presented Puppet Faces, and it’s clear that version 2.7 introduces us to a new era in Puppet evolution. There are some common points in Faces and Puppi: they both bring parts of Puppet to the CLI and they are expandable with actions. Actually, it just seems natural that Puppi’s future is to become a Puppet Face.
Now it’s a bash script that executes bundles of bash scripts, based on data more or less elegantly provided by Puppet modules. It works also on older Puppet versions (at least 0.25) so that it can be widely adopted and integrated in current layouts. The next version is probably going to be in ruby, directly use Puppet APIs and possibly be based on a more standardized modules data model. And of course, it is going to work only with Puppet 2.7 and later.
It may become something different, maybe even with a different name, but as far as I’m concerned it should keep the principles it’s based upon:

• Be based on Puppet data: Puppet knows the infrastructure, we want this knowledge and intelligence integrated in commands we run on the shell. The way this data is currently provided is not optimized (every piece of Puppi information is basically a Puppet resource (generally a file) and this is an overhead we should avoid), we could rely directly on the catalog, which seems the most natural source, but in order to do this I suspect some kind of standardization is needed at the module level
• Provide a simple single line standard command to run an application deployment (one keystroke to deploy them all)
• Provide useful actions that can be used from the CLI, an orchestrator agent or a web interface to show info, status, and working details on systems and applications (I would keep the check/info/log actions, as I’m finding them quite useful)
• It can be run manually or automatically, locally or from a central orchestrator and, possibly, also via a web interface

A Puppi webapp should let different users request, trigger, and view the results of a deploy, gather info from the systems (via rest?), and, in order to become an inventory frontend on steroids with as much detail on the system as users want, receive and show checks and possibly be able to search and correlate the large amount of data it could receive.

The truth is, development on the next Puppi and its web frontend is something that I would like to do in collaboration with Puppet Labs and interested members of the community.
Puppi 1.0 was done by me for a customer’s needs with the knowledge and the requirements I had at my disposal.
Puppi 2.0 (whatever the name and the shape) does not have immediate operative requirements; its design, how data is fed from modules, and how it’s integrated in Puppet should be discussed and shared.

Anyone interested? Leave a comment here, or email me directly.

Puppet 2.7.0 is available. This is a feature-based release for the Puppet project. This release incorporates several key new features, hundreds of bug fixes and enhancements, and a lot of input from the community. We went through four release candidates on Puppet 2.7.0 and appreciate all the feedback, bugs, questions and ideas that made Puppet 2.7.0.

First things first: You can download Puppet 2.7.0.

The release notes cover thoroughly what’s new, what’s changed and what’s improved with Puppet. I’ll highlight some of my favorite features, coming from a system management prospective. We’ve had a few blog articles already about some of the API enhancements, including Faces.

License Change

As previously mentioned, in 2.7.0 we changed the license of Puppet to the Apache Software License Version 2.0.

Ruby 1.9 Support

2.7.0 is the first version of Puppet to support Ruby 1.9. We’ve targeted 1.9.2 and higher versions. There are still a few known issues, but bugs are being aggresively fixed with Ruby 1.9 and it’s now a first-class citizen on our Continuous Integration system.

Deterministic Catalog Application

Previously, Puppet didn’t guarantee the application of a catalog of unrelated resources in any particular order. This meant that if you forgot to specify some important `before` or `require` relationship, a single catalog might work fine on eight nodes and then fail mysteriously on the ninth and tenth. This could be frustrating! Now it’s gone: Puppet will make sure that the same catalog will always be applied in the same order on every machine, and it’ll either succeed reliably or fail reliably. (This change will also be appearing in the final 2.6.x releases.)

Manage Network Devices

Puppet has new types for managing network hardware, and a new `puppet device` subcommand for applying these configurations. These are in their early stages and can currently only handle Cisco IOS-based hardware, but they’re the start of a big leap forward in Puppet’s ability to manage your entire infrastructure. Look a blog with more details of this functionality soon.

Deprecations

We’re starting the hourglass on a few older features:

• ‘puppet‘ as a synonym for ‘puppet apply‘ — Starting today, running `puppet my-manifest.pp` will issue a warning; you should start using `puppet apply` directly instead. Support for implicit invocation of puppet apply will be dropped in Puppet 2.8.
• Dynamic scope — We’ve started issuing warnings when variables or resource defaults are found via dynamic lookup. There’s more info and explanation in a guide on the docs site, but the short version is that you should start referencing variables with their qualified names instead of counting on dynamic scope. We hope to drop support for dynamic scope in Puppet 2.8.
• No more `–parseonly` option — This one’s already gone, because we used Faces to build a drop-in replacement: use `puppet parser validate [] [ ...]` instead.

Download Puppet 2.7.0