Welcome back to the module of the week! Previously we covered facter-dot-d. This week we’re switching our focus to the puppetlabs/stdlib data functions: getvar, loadyaml, and merge.

The data functions provide the ability to dynamically lookup data from Puppet namespaces, load data from external YAML files, and merge Hashes. These powerful features make it easy to separate Puppet code from data without the complexity of using an External Node Classifier or Custom Facts; leaving your manifests more readable and maintainable as a result.

Example Usage

In this brief tutorial I’m going to cover the loadyaml and merge functions to show you just how easy it is to work with external data:

The loadyaml function

As its name implies the loadyaml function allows us to load data from an external YAML file.

/etc/puppet/data/webservers/data.yaml

ports:
  http: 80
  https: 443
  tomcat: 8080
 
webdir: /var/www/html

/etc/puppet/modules/webservers/manifests/init.pp

class webservers {
  include 'stdlib'
 
  $data = loadyaml('/etc/puppet/data/webservers/data.yaml')
 
  $http_port  = $data[ports][http]   # => 80
  $https_port = $data[ports][https]  # => 443
  $webdir     = $data[webdir]        # => '/var/www/html'
 
  ...
}

Lets step through this:

• First we create a data file /etc/puppet/data/webservers/data.yaml
• Then we make a call to loadyaml with the full path to our data file
• Next loadyaml processes data.yaml and returns a Hash, which is then assigned to $data
• Finally we set the $http, $https, and $webdir variables by referencing the $data Hash

The merge function

The merge function allows us to merge two or more Hashes:

/etc/puppet/data/webservers/data.yaml

ports:
  http: 80
  https: 443
  tomcat: 8080
 
webdir: /var/www/html

/etc/puppet/data/database/data.yaml

ports:
  db: 5432

/etc/puppet/modules/webservers/manifests/init.pp

class webservers {
  include 'stdlib'
 
  $myhash         = { database_user => 'webdata', webdir => '/var/html' }
  $webserver_data = loadyaml('/etc/puppet/data/webservers/data.yaml')
  $database_data  = loadyaml('/etc/puppet/data/database/data.yaml')
 
  $data = merge($myhash, $webserver_data, $database_data)
 
  $http_port  = $data[ports][http]   # => 80
  $https_port = $data[ports][https]  # => 443
  $webdir     = $data[webdir]        # => '/var/www/html'
  $db_port    = $data[ports][db]     # => 5432
  ...
}

Here we are calling the merge function with three Hashes ($myhash, $webserver_data, and $database_data) which yields a single $data Hash.

Note, the merge function overrides duplicate keys with the argument farthest to the right “winning”. In the above example the webdir value provided by $myhash is overridden by the value provided by $webserver_data.

Conclusion

The data functions provided by puppetlabs/stdlib makes it easy to work with internal and external data. While great for simple use-cases, this mode of operation doesn’t scale. Once the number of external data files or Puppet modules grows, you’ll find your modules littered with lots of getvar, loadyaml, and merge function calls.

A better solution for more complex requirements is Hiera. While providing all the features of getvar, loadyaml, and merge, Hiera includes the ability to load data from virtually anywhere. Via Hiera’s pluggable backend interface, you’ll be able to pull data from external databases such as Redis, or JSON files; the possibilities are endless.

That wraps up our coverage of the puppetlabs/stdlib module. Hopefully you can now leverage some of its features in your own modules and have a sense of how robust Puppet modules can be.

Additional Resources

• Previous Modules of the Week
• Hiera – Lightweight Pluggable Hierarchical Database
• External Node Classifier
• Custom Facter Facts
• Puppet Forge – Module Repository

2 Comments

Leave a Response