Deploying Puppet in Client-Server, Standalone, and Massively Scaled Environments

A common misconception about Puppet is that it can only be used in client/server mode. Although this is the most common use case, it is actually just one of three common deployment practices.

Client/Server model:

Client/server deployment is the most common and feature rich way to run Puppet. For any environments where multiple hosts are managed, client/server deployment is usually the way to go.

This mode has two executables, puppetmasterd (server), and puppetd (client). Knowing the roles of these executables is important for understanding the differences between these three deployment practices.

  1. Client gathers local facts about its system using facter.
  2. Client initiates a request to the server requesting the latest version of its catalog(description of desired configuration state)
  3. Server compiles the configuration from source(manifests) into a catalog and returns it to the client.
  4. Client applies the catalog, resulting in configuration changes.

This deployment strategy has several advantages over the standalone puppet execution.

  • All configuration source is centrally stored and managed on the puppet server.
  • The client only receives the configuration information that it needs (and can't see the information that doesn't).
    - The client doesnt have access to the source code, just the compiled catalog that applies to it.
  • The server allows for more complicated management of nodes using an external node classifier (like the dashboard).
    - A node classifier allows assignment of classes and parameters to be handled by an external script. This allows information about how nodes are defined to be controlled by external data sources (think CMDB)
  • Uses certificates to ensure that only authorized clients can retrieve configuration.
  • Centralized function execution can simplify custom functions.

The main constraint of the client/server model of Puppet is that the server is CPU bound on catalog compilations. This can limit the scale at which puppet can be run on a single Puppet masters. The limiting factors of scale are:

  • Number of hosts per Puppet server.
  • Interval between client check-ins.

Many of these performance constraints have already been addressed in 0.25.x, and will continue to be improved upon as Puppet improves its ability to pre-compile catalogs

Single host:
One of the most overlooked features in Puppet is the puppet executable. This allows you to both compile and apply catalogs locally, removing the complexity of running in client/server mode.

This makes puppet a viable tool for application bootstrapping, even on one machine.

Executing:

#>puppet mymanifest.pp

will apply the configuration from the local source file mymanifest.pp to the local host. The puppet executable takes many of the same long options as puppet, namely:

  • --noop - allows you to see the effects of the compiled catalog without making any modifications to the local machine
  • --verbose, --debug - increased logging output
  • --modulepath - puppet standalone can use modules to organize and re-use code just like client server
  • --environment - standalone can also use multiple environments.

The puppet executable also reads configuration from the [puppet] section of puppet.conf

Running puppet on a single machine is not only a perfectly valid use case, it is also the best way to get started with the puppet language, and the best way to quickly develop and test new manifests.

Massively scalable deployments:

Some users have opted not to use the Puppet master at all for scaling reasons. Although we feel that we have addressed most of these scaling issues with performance improvements in .25.x, the PuppetMaster will always be the bottleneck in client/server mode.

For this deployment practice, users compile catalogs and apply configuration changes locally with the puppet executable. Code on all of the hosts is then kept in sync with a central repository using something like rsync.

This will always be the best way to scale Puppet deployments, but it suffers from a loss in security as well as ease of management.

As a follow up, I will give some examples of running puppet in standalone mode.

Comments

Scott

Scott

NIce post.

I've always found the idea of completely dropping puppetmasters, but see a few possible issues with it (most related to using storeconfigs):

- With no puppetmasters, you have to allow *every* server to connect to your database instance. Each one also must have a grant. This gets hairy without a wildcard grant.
- If you use async_storeconfigs, every server must hit the queue directly.
- How do you centralize reporting?
- Complex multi-VLAN environments could require pretty hairy ACLs even after solving most of these issues.

My solution to this problem is simple - sharding with horizontal scaling. I assign puppetmaster clusters logically (currently by site) and push manifest updates to them as I would update any other piece of production.

Duncan

Duncan

Thanks for the nice writeup. I came across it googling to find out exactly why an autoconfig system would even use a server, and now I know.

Leave a comment