CloudTweaks.com posted an article today that identified Puppet as one of the top 11 open source resources for cloud computing. The article originally appeared in GigaOM.

Puppet was specifically named because of Puppet’s strength in managing large numbers of machines and virtual machines:

Virtual servers are on the rise in cloud computing deployments, and Reductive Labs’ open-source software, built upon the legacy of the Cfengine system, is hugely respected by many system administrators for managing them. You can use it to manage large numbers of systems or virtual machines through automated routines, without having to do a lot of complex scripting.

Our community continues to use Puppet in a variety of ways to accomplish difficult tasks, whether that is managing virtual machines in the cloud, meeting security compliance requirements, or managing their corporate Mac OS X resources.

A common misconception about Puppet is that it can only be used in client/server mode. Although this is the most common use case, it is actually just one of three common deployment practices.

Client/Server model:

Client/server deployment is the most common and feature rich way to run Puppet. For any environments where multiple hosts are managed, client/server deployment is usually the way to go.

This mode has two executables, puppetmasterd (server), and puppetd (client). Knowing the roles of these executables is important for understanding the differences between these three deployment practices.

1. Client gathers local facts about its system using facter.
2. Client initiates a request to the server requesting the latest version of its catalog(description of desired configuration state)
3. Server compiles the configuration from source(manifests) into a catalog and returns it to the client.
4. Client applies the catalog, resulting in configuration changes.

This deployment strategy has several advantages over the standalone puppet execution.

• All configuration source is centrally stored and managed on the puppet server.
• The client only receives the configuration information that it needs (and can’t see the information that doesn’t).
  - The client doesnt have access to the source code, just the compiled catalog that applies to it.
• The server allows for more complicated management of nodes using an external node classifier (like the dashboard).
  - A node classifier allows assignment of classes and parameters to be handled by an external script. This allows information about how nodes are defined to be controlled by external data sources (think CMDB)
• Uses certificates to ensure that only authorized clients can retrieve configuration.
• Centralized function execution can simplify custom functions.

The main constraint of the client/server model of Puppet is that the server is CPU bound on catalog compilations. This can limit the scale at which puppet can be run on a single Puppet masters. The limiting factors of scale are:

• Number of hosts per Puppet server.
• Interval between client check-ins.

Many of these performance constraints have already been addressed in 0.25.x, and will continue to be improved upon as Puppet improves its ability to pre-compile catalogs

Single host:
One of the most overlooked features in Puppet is the puppet executable. This allows you to both compile and apply catalogs locally, removing the complexity of running in client/server mode.

This makes puppet a viable tool for application bootstrapping, even on one machine.

Executing:

#>puppet mymanifest.pp

will apply the configuration from the local source file mymanifest.pp to the local host. The puppet executable takes many of the same long options as puppet, namely:

• –noop – allows you to see the effects of the compiled catalog without making any modifications to the local machine
• –verbose, –debug – increased logging output
• –modulepath – puppet standalone can use modules to organize and re-use code just like client server
• –environment – standalone can also use multiple environments.

The puppet executable also reads configuration from the [puppet] section of puppet.conf

Running puppet on a single machine is not only a perfectly valid use case, it is also the best way to get started with the puppet language, and the best way to quickly develop and test new manifests.

Massively scalable deployments:

Some users have opted not to use the Puppet master at all for scaling reasons. Although we feel that we have addressed most of these scaling issues with performance improvements in .25.x, the PuppetMaster will always be the bottleneck in client/server mode.

For this deployment practice, users compile catalogs and apply configuration changes locally with the puppet executable. Code on all of the hosts is then kept in sync with a central repository using something like rsync.

This will always be the best way to scale Puppet deployments, but it suffers from a loss in security as well as ease of management.

As a follow up, I will give some examples of running puppet in standalone mode.

Came across this link with an interview of Lance Albertson of Oregon State University’s Open Source Lab talking about how they are managing their data center. Among the topics was a description of their reasons for switching from CFengine to Puppet. Here is what Lance had to say:

“OSUOSL uses Cfengine for systems management but hopes to migrate onto Puppet. What are the primary advantages of Puppet?

Albertson: For one, Puppet has a real community behind it instead of a single person who gets to decide what to include for patches. Additionally, Puppet’s primary goal is managing a large installation of servers instead of being a research project at a university (which Cfengine is). Also the primary contributors of Puppet just moved to Portland and have offered help to us if we switch, so that is an advantage.

From a technical point of view, Puppet offers more flexibility and an ability to actually use real code to deal with tasks. Cfengine has its own syntax language, but it’s not really suited for complex tasks. Puppet also offers some inheritance in its rules, so we could write a module that just does Apache configuration and have a host inherit that code. Then we can offer our module to the world without worrying about revealing security issues.”

Glad to see a good example of the strength of the Puppet community being a key factor in the decision companies make in their choice of tools. Thanks to R.I. Pienaar (@ripienaar) for the pointer to this article.

Reductive Labs is pleased to announce the availability of Puppet 0.25.2. The 0.25.2 release is a significant maintenance release in the 0.25.x branch and represents a significant amount of work from the Puppet community. Special thanks to everyone who contributed to the release and tested fixes – especially, but not limited to, Peter Meier (duritong), R.I.Pienaar (Volcane), Mark Plaskin, Dan Bode, Alan Harder, Ricky Zhou, Christian Hofstaedtler, Nigel Kersten, and especially Markus Roberts and Jesse Wolfe who worked around the clock to get the release out the door.

In this release we closed 123 open tickets, and while this release was primarily a bug fix release, we additionally increased our functionality in various environments (such as Solaris and HPUX), reduced our memory footprint and improved our error reporting.  We now pass 100% of the spec tests, have improved threading and lock contention (clearing up several causes of system hangs), improved certificate handling, improved symlink handling and improved passenger support.

The release is available at:

http://reductivelabs.com/downloads/puppet/puppet-0.25.2.tar.gz

http://reductivelabs.com/downloads/gems/puppet-0.25.2.gem

Thanks,

The Puppet Team at Reductive Labs

About Reductive Labs

Based in Portland, OR, Reductive Labs provides next-gen IT automation, enabling IT organizations to manage infrastructure as code so they can provide higher service levels with less staff. Reductive Lab’s flagship offering, Puppet, significantly improves the automation and management of IT operations for large scale, fast growing, transaction-driven businesses, which are facing unprecedented cost pressures, reduced headcount, and increased support demands. Puppet’s knowledge-based approach creates an audit trail that delivers a new level of transparency so that IT teams can implement change more consistently, accurately and rapidly over the lifecycle of the systems. For more information, please visit www.reductivelabs.com.

We’re going to take a tour of the newly released Puppet Dashboard web front-end.  Puppet Dashboard is (or will be) a web front end that keeps you informed and in control of everything going on in your Puppet ecosystem. It currently functions as a reporting dashboard and an external node repository and will soon do much more, including having better marketing copy.

Fundamentally, Dashboard lets you do two things: configure nodes using parameters, classes and groups for use as an external nodes tool; and monitor the status of nodes through real-time reporting and versioned change tracking. The main dashboard page shows you the status of recent Puppet runs, displays important information like pass/failure statistics, and alerts you of important failures, errors and unexpected events. Let’s take a look at how it does this.

Read the rest of this entry »

Puppet is included in a list of open source software being employed by larger businesses.

Gigaom’s list includes Puppet as a top resource.

REDUCTIVE LABS SECURES $2.0 MILLION FUNDING ROUND
LED BY TRUE VENTURES

Reductive Labs’ Puppet Helps Businesses Slash IT Expenses Through Improved Automation and Management of IT Operations

Portland, OR, Tuesday, June 23, 2009 — Reductive Labs, a provider of next-gen enterprise IT automation, announced today that it has closed its Series A round of funding in the amount of $2 million led by Silicon Valley-based, early stage investor True Ventures and other private investors. Reductive Labs will use the capital infusion to expand the company and further enhance its flagship offering – Puppet. Reductive Labs is pioneering a fundamentally different paradigm to IT management. Puppet helps businesses optimize IT operations, transforming what is often viewed as a cost center into a strategic advantage.

Over the past few decades, the world’s demand for computational resources has grown exponentially. However, the technology to manage these systems has not kept pace with this growth. Forrester estimates that there are currently over a billion PCs in use today, and projects that number will double by 2015. Scaling the system infrastructure, which enables researchers and businesses to solve problems and provide solutions, presents many challenges for traditional IT operations to keep up with the pace.

The global recession has only compounded this situation. Facing unprecedented cost pressures, businesses have required IT to reduce headcount yet still meet increased IT demands. Adding one more layer to “the Perfect Storm” forming in IT has been the growth in virtualization and cloud computing; both technologies offer a cost-effective way to expand storage, services and processing capacities without further cash outlays for new hardware infrastructure. However, virtualization and cloud computing can increase complexities and management expenses, as their adoption multiplies the number of machines and services running on virtual machines that still need to be configured. Gartner estimates worldwide cloud computing revenue will surpass $56.3 billion in 2009 and will likely skyrocket to $150 billion in 2013.

Reductive Labs is providing a breakthrough solution to help large scale, fast growing, transaction-driven businesses squarely address this real IT pain point. The world’s most leading-edge companies have selected and use Reductive Lab’s Puppet framework to help them automate and manage their IT departments, including Google, Digg, Twitter, New York Stock Exchange, Barclays Capital, Oracle, Sun, Red Hat, Harvard Law, and Stanford.

“We are impressed with the vision, commitment and passion that the Puppet founding team exhibits as well as the growing market opportunity at hand. Reductive Labs has brought to the market a truly disruptive solution, as evidenced by its adoption within the open source community and from some of the market’s leading-edge companies,” noted Phil Black, Co-founder, True Ventures.

Manage Entire IT Infrastructure as Code
Reductive Labs’ Puppet is an open source software framework to automate infrastructure. Puppet has fundamentally changed how companies can configure, provision, manage and scale their IT infrastructure using software tools rather than IT staff. In doing so, Puppet helps IT departments do more work and provide higher service levels with less staff.  Knowledge is the driving philosophy behind the Puppet framework. Puppet’s simple, declarative-based language enables engineers to programmatically encode semantics about ‘why’ systems are configured a particular way. Instead of just ‘what’ or procedures about ‘how’ services are configured, systems are built and managed with code, utilizing all the recognized best practice tools and processes for software development.

By managing IT infrastructure as code, Puppet creates an audit trail, showing what systems are running, the history of all the work done on the system, and the policies that Puppet is carrying out. This new level of transparency can effectively tear down the wall of confusion between development and operations teams, who now have improved insight into what is being done and why. As a result, IT departments are better able to support the company’s requirements and implement change consistently, accurately and more rapidly.

Puppet’s automation capabilities also alleviate management complexities and expenses introduced by cloud computing and virtualization. Puppet can manage and configure the virtual machine lifecycle at the operating system level more effectively than machine image-based management approaches. IT departments can also leverage Puppet to flexibly move services back and forth from the cloud to behind the company’s firewall.

“We’re thrilled to have True Ventures’ endorsement, investment and involvement in our company. We felt their entrepreneurial spirit and philosophy best matched up with what we wanted in an ideal venture partner.  The funding will enable us to enhance Puppet’s capabilities while helping more companies transform their IT operations into a strategic competitive advantage,” noted Luke Kanies, CEO of Reductive Labs.

About True Ventures

Based in the Silicon Valley, with offices in Palo Alto, CA, San Francisco, CA, and Great Falls, VA, True Ventures invests in promising entrepreneurs at the earliest stages in the highest-growth segments of the technology market. The partners at True Ventures have started over ten companies as founders, and the venture firm is designed by entrepreneurs for entrepreneurs. The firm clearly understands both opportunities and challenges in the earliest stage of development and provides young companies with a powerful, seasoned partner. True Ventures has raised two venture capital funds to date and manages $375 million in limited partner capital. For more information, please visit www.trueventures.com.

About Reductive Labs

Based in Portland, OR, Reductive Labs provides next-gen IT automation, enabling IT organizations to manage infrastructure as code so they can provide higher service levels with less staff. Reductive Lab’s flagship offering, Puppet, significantly improves the automation and management of IT operations for large scale, fast growing, transaction-driven businesses, which are facing unprecedented cost pressures, reduced headcount, and increased support demands. Puppet’s knowledge-based approach creates an audit trail that delivers a new level of transparency so that IT teams can implement change more consistently, accurately and rapidly over the lifecycle of the systems. For more information, please visit www.reductivelabs.com.

An interview with James Turnbull about our favorite Open Source configuration management framework.

James came for the configuration management and now he’s a lifer.

Matt Asay makes a compelling case…