Wouldn’t it be nice if you never made a mistake or a typo again? Okay, that’s a slightly misleading offer. How about just never committing such gaffes in with your code? “How?” I hear you cry. With the illustrious Tim Sharpe’s puppet-lint and some version control derring-do!
Following on from Adrien’s wonderful post on syntax checking and writing tests, you have a way of checking the syntax and style guide from the command line. Using this approach, even on a mature (like a fine wine) Puppet code base, you can move towards having beautiful, aligned, ordered manifests—the type of manifests you’d be happy taking home to meet the family.
Puppet-lint takes the Puppet Labs Style Guide and places it nicely in a command line tool. Think of it as a programmatic PEP-8 for those from a Pythonic world. Now, where this comes in handy is when you’re using revision control (which is all the time, obviously!), because you can tie the two together. Yes, every time you try and commit your code to your RCS, it checks the files you’re committing, and makes sure you haven’t used a tab where there should be a space.
In Puppet Labs’ kickass Operations department, we’re ever so slightly keen on Git, not least due to the wonderful GitHub. RCSHub just doesn’t cut it for us these days.
With Git, as with many an RCS, you’re free to define hooks to do weird and wondrous things upon certain actions. This chapter on git hooks from Customizing Git explains them in detail. If you’re using Subversion, then this pre-commit documentation suggests what you can do on the server side to accomplish the same thing.
For this, the pre-commit hook is the one we want to utilise. I first make a bash script to get a list of the files that change in the commit. Then, the hook script goes through them one at a time, seeing if they’re Puppet manifests by name, and running puppet-lint on them as it goes. If any of the manifests fail linting, it exits there and then I may go fix them at my leisure!
#!/bin/bash
# Requires bash, as it uses the [[ ]] syntax.
#
# If it's puppet code, lint it up.
# I we don't have puppet-lint, so just exit and leave them be.
which puppet-lint >/dev/null 2>&1 || exit
# Variables goes hither
declare -a FILES
IFS="
"
FILES=$(git diff --cached --name-only --diff-filter=ACM )
for file in ${FILES[@]}
do
if [[ $file =~ \.*.pp\$ ]]
then
puppet-lint --with-filename "$file"
RC=$?
if [ $RC -ne 0 ]
then
exit $RC
fi
fi
done
exit 0I save that file to .git/hooks/pre-commit in my repository, give it a light sprinkling of ‘chmod +x’, a dash of ‘gem install puppet-lint’ and I am ready to roll.
Exhibit A, the Larch
So let’s see this bad-boy in use, I hear you cry! Let’s take a manifest that breaks all of the rules, and see what happens…
node 'hubert.humphrey.edu' {
user{ "mhunter":
managehome => true,
home => "/home/mhunter",
comment => 'Mark Hunter',
ensure => 'present'
}
}[enlil:puppetlabs-modules]% git commit hubert.pp
hubert.pp - WARNING: double quoted string containing no variables on line 3
hubert.pp - WARNING: double quoted string containing no variables on line 5
hubert.pp - WARNING: => on line isn't properly aligned for resource on line 5
hubert.pp - WARNING: => on line isn't properly aligned for resource on line 6
hubert.pp - WARNING: => on line isn't properly aligned for resource on line 7
hubert.pp - ERROR: two-space soft tabs not used on line 4
hubert.pp - ERROR: two-space soft tabs not used on line 5
hubert.pp - ERROR: two-space soft tabs not used on line 6
hubert.pp - ERROR: two-space soft tabs not used on line 7
hubert.pp - ERROR: two-space soft tabs not used on line 8
hubert.pp - ERROR: trailing whitespace found on line 2
hubert.pp - WARNING: ensure found on line but it's not the first attribute on line 7
Woah, that’s a lot of errors. By default, puppet-lint will let warnings through, but stop dead on errors, and we got them all! I’ll tidy this manifest up…
node 'hubert.humphrey.edu' {
user{ 'mhunter':
ensure => present,
managehome => true,
home => '/home/mhunter',
comment => 'Mark Hunter',
}
}Much more readable, and now when I try and commit it, I get:
[enlil:puppetlabs-modules]% git commit hubert.pp
[master 6020331] Add in a new pupil to the school!
1 file changed, 10 insertions(+)
create mode 100644 hubert.pp
Clean, and the best part is I don’t have to alter my workflow, I can just carry on editing with Vim, commit my work as normal, and just be kindly reminded when a manifest has things that need tidying up.
Learn More
Boundary is a network analysis tool that allows fast analysis and an ultra-fine resolution of data. To achieve this speed and resolution of data, Boundary employs software meters, installed on your application servers, that examine the header information of every packet and securely stream it to the Boundary service once a second. We’re excited to announce our Boundary module to install and manage Boundary network meters, developed in conjunction with the Boundary team and the Puppet community, has reached v1 and is released onto the Puppet Forge! The module also contains support for sending reports from Puppet to Boundary.
To use the module you need to download it from the Puppet Forge onto your Puppet master using the module tools:
# puppet module install puppetlabs-boundary
You can then use the module to install and manage the Boundary meters that monitor your network traffic and send the results to Boundary. To create a new meter you will need your Boundary ID and API key and create a `boundary_meter` resource like so:
boundary_meter { "db.example.com":
ensure => present,
id => "abcde12345",
apikey => "abcde12345",
tags => [ "production", "newyork", "mysql", "appcluster1" ],
}This resource will create a meter called “db.example.com” and add the tags production, newyork, mysql and appcluster1 (or the tags of your choice). After you run Puppet you will be able to see your meter in the Boundary console. We can also easily delete the meter by ensuring it is `absent`.
You can find the source code on GitHub and we welcome bug reports, ideas and features from the Puppet community. Thanks to Joe Williams, Jeff Hulten, W. Andrew Loe III, Alessandro Franceschi and Scott Smith for their contributions to the current module.
I am proud to announce the release of the next major production version of The Marionette Collective (MCollective). This release brings together almost a year’s worth of work and introduces great improvements in security, stability, platform support and new features in the core messaging layer.
The major areas of advance are below:
- Complete messaging protocol rewrite to enable direct style connectivity that would allow programs to bypass normal discovery instead using their own data sources
- An additional more robust messaging paradigm supporting a more assured addressing and delivery scheme
- Batched mode allowing users to address machines in small groups thus avoiding thundering herd and enabling more granular changes
- A more complete language for expressing discovery that includes and/or/not style queries across the infrastructure
- Improved Stomp connection security using normal industry standard Certificate Authority validated TLS
- New connector that uses ActiveMQ-specific features for better performance and scalability
- Security of the SSL and AES security plugins have been improved for tamper protection by middle men
- A message validity period has been introduced to lower the window of message replay attacks
- Better error handling and better logging for Stomp connections
- JSON output from the ‘rpc’ application
- Ability to pipe RPC requests into each other creating a chain of related RPC calls
- Better validations, better error handling and better documentation creation from the DDL
- Performance improvements in the CLI, more consistently formatted output of received data
- A Ruby GEM of the client is now made available on rubygems.org
- The rc script for Debian based systems have been improved to prevent duplicate daemons from running
- Built in packager for plugins into native OS packages—RedHat and Debian supported
- MS Windows Support
The full release notes for this release expands on key areas of the above list and you can download this release from our download area, Yum repository or our Apt repository.
Puppet Labs has been a busy place in the last month: we announced PuppetConf and released the very first seats for the Puppet Professional Certification Program (more details coming soon), we also announced our support for OpenStack and we announced PuppetDB, affectionately known around the office as project Greyskull. As we move ahead into May and June we hope to see you at some of the following events:
May 19 – Puppet Camp Los Angeles: Senior Developer Deepak Giridharagopal will introduce PuppetDB, and the Puppet projects team lead Daniel Pittman will share the Puppet roadmap for 2012.
May 21 – Find us at EMC world, we’ll be around and looking to find you. Follow us @puppetlabs to find out where we are.
We’ve announced new Puppet Camps for June and July:
- Southeast Asia in Kuala Lumpur on June 5th
- Sydney on June 8th (tickets are almost gone!)
- District of Columbia on June 19th
-
- Puppet Camp Boston on June 22
- Chicago on July 23rd
And don’t forget about our previously announced our European camps in Dublin, Ireland on July 6th and Geneva, Switzerland on July 11th.
After the end of July we’ll be working to produce an epic PuppetConf in San Francisco on September 27th and 28th. Register now to guarantee a seat. Like many of our Puppet Camps, PuppetConf is sure to sell out.
We’re also happy to offer you a 10% discount to Velocity on June 25-28. Just use code PUPPETLABS when you register and then come find us at booth #517 and check out presentations from James Turnbull and Luke Kanies at Velocity.
We’re looking forward to connecting with you this summer.
This week we’ll wrap up our coverage of the Puppet Module Tool (PMT) by exploring the install, upgrade, and uninstall commands. Lets jump right in.
PMT Install
The install command installs modules from the Forge using the full name of the module (author-module). We can install the “puppetlabs-apache” module like this:
Installing a specific version
We can also install a specific version of a module using the --version (-v) flag:
PMT now enforces SemVer when installing modules. If a module does not meet SemVer requirements it will fail to install.
Automatic dependencies resolution
One of the major enhancements to PMT is the ability to automatically resolve and install module dependencies. Once a manual process is now completely automated:
Notice that the install command grabs the requested module, puppetlabs-bacula, and its depenedencies: puppetlabs-mysql, puppetlabs-sqlite, and puppetlabs-stdlib.
PMT Upgrade
PMT also sports the ability to upgrade installed modules:
From the output we can see that we have upgraded the “puppetlabs-stdlib” module from version 2.2.1 to 2.3.1.
PMT Uninstall
Before this release of PMT, users had to manually remove modules from their module paths. While something like “rm -rf /etc/puppet/modules/stdlib” works, it’s not what we call user friendly. This version of the module tool greatly improves the experience around uninstalling modules.
Be sure to use the full module name when uninstalling modules or you’ll get the following error:
Notice the helpful error message suggesting that maybe you meant “puppetlabs-stdlib”, and in fact we did:
Look before you leap
The uninstall commands also checks for local changes and broken dependenices before uninstalling a module. Check out what happens if I try and uninstall the “puppetlabs-mysql” module:
PMT stops you with an error because uninstalling puppetlabs-mysql would leave the puppetlabs-bacula module with an unmet a dependency. You can by-pass this error by using the --force flag, but you can’t say we didn’t warn you!
Conclusion
PMT has been greatly enhanced to provide a much better experience when consuming modulew from the Puppet Forge. With the new list, search, install, upgrade, and uninstall commands managing modules couldn’t be easier.
Additional Resources