It’s been six months since I left you with part 1 of ‘Writing Great Modules’, a grab-bag of tips and patterns for writing better Puppet modules. It’s been a great time for Forge and Puppet modules. Forge has seen the formation of a dedicated team (including myself as product owner), several updates to the product, and has grown to contain nearly double the modules in the last six months. We’ve been really busy! I’ve finally gotten some time to write a follow-up post with more things I consider important for modules on the Forge. Enjoy!

Expressing SemVer dependencies — Depend on Other Modules

When I last wrote you, I wrote that everyone should version their modules based on the SemVer specification so that your release versions convey meaning to your users about what type of change went into a given update to your code.

It’s a great start and something you should get into the habit of doing, but you can take it one step further. Whenever you use another module as a dependency from your own module, you should express that relationship with semantic versioning.

All too often, I see someone express a dependency in their Modulefile like the following.

name 'ryan-mymodule'
dependency 'puppetlabs/stdlib', '>= 2.2.1'

This tells Puppet that any version of stdlib 2.2.1 or newer will work with your module. If the author of stdlib makes backwards incompatible changes but does so according to SemVer and releases those changes as version 3.0.0, users of your module may upgrade to 3.0.0 accidentally and break the functionality you were relying on. The author did the right thing, but your module was written to bring in the latest and greatest code.

Instead, you might want to restrict your module to depend on version 2 of the puppetlabs/stdlib module, accepting bug fixes and new features that don’t break backwards compatibility. The following expression in your Modulefile takes care of that for you.

name 'ryan-mymodule'
dependency 'puppetlabs/stdlib', '2.x'

Now, if puppetlabs/stdlib is following SemVer practices in new releases, your module continues to function with the promise you’ve set for it and you’re free to review major new versions of the modules you depend on at your leisure. Read more about publishing modules and expressing dependencies in the module publishing guide on docs.puppetlabs.com.

Collaborate on GitHub

If I search for Apache on the Puppet Forge, I get back at least seven results that appear to be modules for the httpd web server. Each could be completely different and approach the problem in different ways, but my guess is that most have approached things the same way and have repeated what an author had done before them.

Instead of taking your great idea straight into a brand-new module that will compete with a half-dozen others for the attention of Puppet users, consider collaborating with another author! This approach does take a little more effort, but should be well worth it given how little you’ll have to repeat. Plus, each time you do, an Octocat gets its wings.

About 85% of Puppet Forge modules are backed by a GitHub repository. To contribute to another module, all you need to do is fork the repository, make your changes, and submit a pull request. New to Git? That’s probably something you want to check off your list. Here are 10 tutorials that can help. It’s always nice to contact the author and let them know of your intent too, just to see if they’re into it and how they can help you.

Data/Code Separation

The release of Puppet 3 brings with it a new mechanism for finding data to use when you declare classes. Now, Puppet will look up your data in a several step process, depending on how you declare them. My example will cover the include-like behavior as it’s simpler to understand and use.

When you include a class in Puppet 3, data is looked up in the following manner (taken straight from the docs):

1. Use the override value from the declaration, if present (and if you’re using the resource-like class declaration, unlike my example).
2. Request a value from the external data source, using the key :: . (For example, to get the apache class’s version parameter, Puppet would search for apache::version.)
3. Use the default value.
4. Fail compilation with an error if no value can be found.

Number 2 is specifically important, as you no longer need to litter your manifests with hiera() function calls in order to use this useful hierarchical data lookup utility. It’s automatically installed, configured and consulted for data in Puppet 3. Let’s take a look at how this works with a simple module that simply creates a notify resource.

The following comes from a CentOS 6 VM on which I have Puppet 3.0 installed from yum.puppetlabs.com.

# /etc/puppet/modules/example/manifests/init.pp
class example (
  $notify_message = 'Hello, this is the default message'
){
  notify { $example::notify_message: }
}

If I simply declare that class (interactively through `puppet apply`), I get the default value of my parameterized class and everything works great.

[root@centos6 ~]# puppet apply -e 'include example'
Warning: Config file /etc/puppet/hiera.yaml not found, using Hiera defaults
Hello, this is the default message
/Stage[main]/Example/Notify[Hello, this is the default message]/message: defined 'message' as 'Hello, this is the default message'
Finished catalog run in 0.03 seconds

Now let’s say I downloaded this module from the Puppet Forge and I want to modify its behavior without modifying the module. All I need is Hiera! Again, Hiera is a useful utility for storing and looking up your Puppet-related data and it comes pre-installed with Puppet 3.0.

You may have noticed in the above output that I hadn’t configured a hiera.yaml file (Hiera’s configuration) yet. To correct this, I’ve dropped a simple configuration into /etc/puppet/hiera.yaml which describes which backends to use (you can have many!) and where to find the data, /etc/puppet/hieradata in our case.

---
:backends:
  - yaml
 
:logger: console
 
:hierarchy:
  - "%{operatingsystem}"
  - common
 
:yaml:
   :datadir: /etc/puppet/hieradata

This yaml document describes that I wish to have just one backend (again, you can have many!) and for that yaml backend, I expect to find yaml files in /etc/puppet/hieradata that match the lookup hierarchy I’ve specified. That hierarchy gets consulted in the following order.
The operatingsystem fact from Facter (like RedHat or Debian)
A static lookup called common
Note that we’re just specifying the term to lookup. The hierarchy applies to and of the backends. For the yaml backend, the .yaml extension is assumed, meaning that if we have common in the hierarchy, the yaml backend expects to find common.yaml. When using Facter facts as variables, make sure you respect case sensitivity! RedHat would become RedHat.yaml.

Let’s get started by building a common.yaml file which contains the data I want returned if I haven’t specified something specific to my agents operating system.

---
example::notify_message: "Hello, I'm from the common.yaml file, not the class default!"

Pretty simple, right? Pay special attention to the fully qualified variable name. Let’s see what happens now when I run Puppet with just the example class included.

[root@centos6 ~]# puppet apply -e 'include example'
Hello, I'm from the common.yaml file, not the class default! 
/Stage[main]/Example/Notify[Hello, I'm from the common.yaml file, not the class default! ]/message: defined 'message' as 'Hello, I'm from the common.yaml file, not the class default! '
Finished catalog run in 0.03 seconds

It’s magic! Well, no, it’s not. The new Puppet 3.0 data lookup requested data from Hiera first (instead of class defaults), Hiera consulted its configuration file for its lookup hierarchy and available backends and finally because there was no CentOS.yaml file, common.yaml was consulted. Ok, so let’s add that missing CentOS.yaml file which will be consulted higher than our common.yaml file.

---
example::notify_message: "Hello, because this is a CentOS system, this is your message!"

I placed that file in /etc/puppet/hieradata/CentOS.yaml and again, Puppet is run. Pay special attention to your case when creating your file. It should be named exactly like the output of `facter operatingsystem`, respecting case.

[root@centos6 ~]# puppet apply -e 'include example'
Hello, because this is a CentOS system, this is your message! 
/Stage[main]/Example/Notify[Hello, because this is a CentOS system, this is your message! ]/message: defined 'message' as 'Hello, because this is a CentOS system, this is your message! '
Finished catalog run in 0.03 seconds

Pretty cool, right? I hope the power of this pattern is evident to you. With this, module authors can create modules with class parameters that may or may not have a default value and as the consumer of that module, you can pass whatever value to those parameters that you choose with Hiera. The code (module) is separate from your data (Hiera). Hiera is extensible, supporting multiple backends that you can write in-house or consume from the community. Best of all, this pattern is backwards compatible with previous Puppet versions. Prior to 3.0, users can simply make parameterized class declarations (looks like a resource) to declare the necessary data.

I encourage everyone to at least try this out and experience what it can do for you. With that, allow me to leave you with one last thought.

Just Share Your Module on the Puppet Forge

I often see people explaining that they haven’t put their module on GitHub or on the Forge because they don’t feel that it’s “worthy.” The trouble seems to be that when you’re new to Puppet, you don’t feel that you’ve perfected your module writing skills enough to sit among the top contributors to the Forge.

I say, fiddlesticks! No one is perfect and no module is perfect. Take a look at a module you consider to be out of your league. See the release and/or commit history? That module took a long road to greatness and it was likely forged by many authors collaborating together over time. That’s how great modules are made. It’s all about iteration. Especially, if you’re automating something no one else has on the Forge, put it up there and point out where you’d like things improved. With any luck, someone will stumble on your module, get excited about the amount of work you’ve already put in, and help to make your module better, one commit at a time.

In summary,

1. Consider the practices I’ve written above
2. Share your module on the Forge
3. Be honest and direct in your Readme about what is missing or could be improved
4. Help your fellow community of module authors
5. Profit (or just feel good about your open-source contribution) :-)

Learn more:
• Contribute to the Puppet Forge
• Part I of Writing Great Modules
• Learn more about writing, installing & publishing Puppet modules
• Learn more about Hiera

Leave a Response