Since starting at Puppet Labs four months ago, many friends have asked me why. “Puppet is a systems tool, right? Isn’t it all command line? Where’s the user experience? Don’t you usually work on the web? Are you designing their website?”

To the contrary, I’m happy to say that for the first time since the Internet was invented I’m working at a company whose web site doesn’t need much of my attention. We have fantastic people working on it already, and I think it’s quite good.

There’s a lot of user experience in Puppet if you look. That will be an easier argument to make if we have a good idea of what user experience is.

What is User Experience?

Back in the day, much of this was called Human Computer Interaction, or HCI. It is the theory and practice of how people interact with computers. Computers at the time were slow-moving and without much rich user interaction, and this started as academic work—a meeting point between psychology and computer science.

As we got faster at building software and interaction became richer, we began to apply HCI principles to one-off projects. “How can this corporate or consulting project succeed?” was the question. Usability became well-established as a discipline and method.

With the Internet, richer interaction is possible not just with a product but with a company. Perception of the company matters, including installing and using the product, filing bugs and receiving responses, updating software, finding and reading documentation, and interacting with the community. The path of the customer through this process is user experience.

So wait, is this just tarted-up brand psychology? No, because the heart of user experience is a deep interaction with software, and the user’s need to get something done.

The desire for, and techniques of, measuring much of this come from usability. How many people succeeded or failed? How many were frustrated, how many had positive or negative impressions of the product?

These metrics are all useful, but mine is simpler: Joy. My metric is joy.

I want people to look forward to using Puppet. I want people to enjoy using Puppet not because it has no bugs or prevents mistakes, but because it has a personality. A good tool is something you can relate to because you understand how and why it works. Enabling that emotional connection and building a framework around it is what I call user experience.

Is that ambitious? Absolutely, but good goals always should be.

What is User Experience at Puppet?

Certainly a lot of the work in Puppet is done by one computer talking to another, but all of that work is initiated and mediated by a human being. Without a human operator Puppet has nothing to do, and every interaction with each component of Puppet is user experience.

The Command Line

My first answer to my friends about working at Puppet is that for 25 years I’ve been a victim of bad command line tools and now it’s payback time. It’s been a great pleasure over the last few months to help create a first-class command line experience.

The command line is different from the GUI in fundamental ways. There’s very little context, no affordances, and interaction mostly relies on memory and low-density text display. Interaction principles are the same, though, and a successful user experience has the same properties regardless of the interface used to accomplish it.

For instance, we should be careful and deliberate about presenting help on the command line. We can’t use the GUI trick of contextual help; most help must be asked for explicitly. How do we tell our users that help is available? How much detail do we show at once? We’ve thought about this a lot for our new Faces commands.

The GUI

The graphical user interface is what most people think about when they hear “user experience” or “interface design.” Puppet’s GUIs today are the Forge and Dashboard.

The Forge is a place where Puppet users can find and share Puppet modules. It keeps accumulating useful modules, but has not seen much development from us in the last year. We have big plans, and will be talking about those soon in public.

Dashboard is a reporting tool and external node classifier. Its reporting features have improved, and we expect to ship version 1.2 in the near future with improved summary reporting and useful at-a-glance views of your nodes.

The DSL, or Puppet Language

Puppet’s DSL, also called the Puppet Language, is how Puppet users model the desired state of their systems. For example, this is the clearest and most compact way we’ve come up with for defining the properties we care about for one file:

    file { '/var/log/syslog':
      ensure => present,
      mode   => '0644',
    }

There are many user experience considerations here. This looks a little like Ruby code, but in fact it isn’t. Is the potential confusion to a new user worth the other benefits? What are those benefits and how do we weigh them?

For another example, there’s ticket #7599, discussing whether the trailing commas after “present” and “0644″ should be mandatory. Technical considerations aside, what would be the effect of making these optional, or even of prohibiting their use? Given that they are now required, would there be a migration costs for existing users? Would it be worth it?

The External API

An API is a structured way of issuing commands to a remote server and receiving responses. It’s a common way for web-based (and other) applications to communicate with each other. REST is one way of defining an API. Many Puppet commands are addressable via API.

Most user experience concerns here concern consistency and documentation. If a DELETE command works in one context it should work in another. Documentation should be clear and easy to access.

Not all of Puppet has an API, and not all of the APIs are as good as we would like. This is important to us and we will continue to improve.

The Internal API

The Internal API is a way of using Ruby both to define resources and to interact with Puppet internals. We have good documentation about how to define resources with the Ruby API. Our own Daniel Pittman has been writing a
series
of
posts
about our new Faces API and how to interact with it. These are a great introduction to manipulating Puppet’s internals.

The Future

It’s a mistake to think of this work as magic or guru-driven design. In fact its most important component is connection with the real users of our products. Puppet Camp was fantastic for this; our upcoming PuppetConf (watch for more information Monday) will be another great opportunity. We’re also starting more robust in-house usability testing for new work.

Next up in this space: What is user experience on the CLI? This will be a detailed look at user interaction on the command line, and extraction of useful principles to inform our work.

“DevOps” is an oft-discussed term—new discipline or just hype; movement or toolset? Whether you’re a DevOps devotee or detractor, we want to know what the state of DevOps is in your organization. Become a part of the spirited debate by sharing your thoughts in the first Puppet Labs DevOps survey!

The survey takes no longer than ten minutes to complete, and participation enters you into a drawing for some fabulous prizes. Three participants will receive $100 Amazon gift cards, and 25 lucky folks will win snazzy new Puppet t-shirts. And of course, all participants will get to reap the benefits of the survey results.

We’re hoping to hear from as many people as possible, so please encourage other interested folks to take the survey. Share it on facebook, tweet about it, or just bring it up around the water cooler.

Take the Survey

This week we released Puppet 2.6.8, the latest maintenance release in our 2.6 series. This release addresses several issues users have had with previous Puppet 2.6 releases, as well as adding in a couple of nifty new features:

• (#2311) The deprecated darwinports provider has been replaced by a new MacPorts provider. This allows Puppet users managing Mac OS X systems to specify UNIX packages in their manifests to be built, installed, and managed by MacPorts.
• (#4844) Puppet now features a Shell provider for the Exec type that allows users to specify inline shell code within a manifest, rather than referring to an external shell script file. The previous functionality of the Exec provider has been transferred to a POSIX provider for the Exec type.
• (#5670) In previous 2.6 versions, a bug caused failing resources to notify other resources that depended on them, causing them to be trigged erroneously. As of 2.6.8 this behavior has been fixed, and failing resources will not cause refreshes in any resources than depend on them.
• (#5477) Previously, if a catalog referred to a file that did not exist when it was compiled, it would not detect the file’s creation or any subsequent changes to it. This caused particular problems if the site.pp manifest did not exist yet when a puppetmaster’s catalog was compiled, requiring the master to be restarted. The fix we have implemented in 2.6.8 resolves this issue, so that puppetmasters will detect the creation of a site.pp file.

As most of you realize by now, Puppet 2.7 was released under the Apache 2.0 license instead of under the GPL, and Facter has already been released under the Apache license. My goal in this post is to explain why, and what effects you might expect to see as a result.

We’ve been talking about the possibility of this change for about two years, but it was only in the last six months that it’s been solidified as the right plan. For the vast majority of people, this change won’t affect you at all—Puppet is still open source, and under one of the most open licenses available. For a few of you, however, this license change will make it easier to embed Puppet into your software, ship it as part of a solution you’re building, or contribute code to it.

Our goal at Puppet Labs, and my goal since I started the company and the project, has been to have Puppet everywhere. I want every OS to ship with Puppet, I want every appliance to use Puppet, and I want every device that can’t run Puppet to speak its language and still integrate with it. Heck, I want to replace the package manifest formats with Puppet’s language. I’m a big believer in open source, but for strictly practical reasons. Puppet always had to be open source, because I couldn’t get the kind of ubiquity that I wanted with a purely commercial product—too much of our infrastructure is already open, and too many sysadmins understand the risks of locking yourself into software you can’t control and have no visibility into. Also, when I started the project I had no reputation and, um, no money; open source was a great way to bootstrap the company and project with very little expense.

With the GPL, realizing this goal of ubiquity was very difficult—some companies are comfortable with the GPL and have no problems including software released under it, but plenty of other companies are dreadfully afraid of the potential for it to force releasing of other code, whether or not that fear is rational. With Apache, these companies can focus on whether Puppet will make their solution better, and not worry about whether they’ll have to make significant business changes in order to use it. You won’t see me making arguments about freedom here (and you probably won’t have much luck engaging in such a conversation with me separately), but you have already seen that my practical perspective on this drives toward open source as much as anything else.

To me, the choice between GPL and Apache was never about which one is more free. It has always been about which one can best accomplish the goals of the project, and possibly which can do the best job of helping me fund those goals. In the end, choosing GPL means more companies that want to partner with us have to pay us, else their software must also be open, while choosing Apache means anyone can use, embed, and extend Puppet without having to worry about it affecting any other software. In other words, the choice between GPL and Apache for us as a company largely comes down to the GPL enabling fewer partnerships but some number of which that pay us, while Apache enables far more partnerships but few (if any) that pay us.

Puppet is primarily meant to be directly used by sysadmins. Switching to Apache, and hopefully seeing far more integrations and embedding, seems like a good trade-off. We give up business we might never have in exchange for relationships right now. Thus, the goal of ubiquity feels a bit closer.

I know this argument doesn’t persuade all of you, and I’ve already exchanged emails with one person who is convinced that this change means Puppet is no longer free software, but it is my sincere hope that we can quickly get back to writing and releasing software rather than talking about licenses. And, of course, it’s also my sincere hope that we see far more people embedding and integrating with Puppet.

As always, please contact me directly at luke@puppetlabs.com if you have any questions or concerns.

Puppet Faces is one of the features I am proudest to deliver in the new Puppet 2.7.0 release. (Well, the RC series heading toward release, anyway.) They represent a new API for creating subcommands, as well as extending existing subcommands to add extra capabilities‚ without having to modify the supplied source code.

In this post, and the rest of the series, I am going to dive right on down deep into the technical details of Faces and how they can be used to program (and reprogram) a whole bunch of Puppet.

The key capabilities of Faces that make them important are:

• Faces provide a defined, documented, robust API to the internals of Puppet.
• They let you have that same robust, documented power for your extensions.
• Faces have solid backward compatibility baked in to their core.
• Faces give you access to a whole mess of capabilities for free.
• You get a command line interface to your Face for free…
• …and we have a whole lot more coming down the pipe, too.

Faces are all over backward compatibility!

While we were building the Faces system we put a whole lot of thought into how we can make it easy for you to extend Puppet ‚ and how to make sure that we don’t break your extensions when we need to change part of that API.

One of the first things you will run across is version numbers. When you refer to a Face, either to use it, or to extend it, you need to specify what version you are talking about:

    facts  = Puppet::Face[:facts, '0.0.1']
    status = Puppet::Face[:status, '0.0.1']

On the plus side, a Face is just a regular Ruby object with a whole bunch of behaviour and convention poured on top, so you can assign them to a local variable and then call it any number of times. So, don’t worry about having to type out that version number more than once in your code, eh.

Those version numbers show up when you are looking at writing or extending Faces, too; building up the face, or adding an action, requires you to specify what version of the Face it is built on:

    Puppet::Face.define(:node, '0.0.1') do
      action :example do
        option "--[no-]override"
        option "--fruit FLAVOUR"
        when_invoked |a, b, c, options|
          # Your code goes here...
        end
      end
    end

Now, the obvious question is why? Why do we force you to declare the version of the Face you are extending here? Our answer is pretty simple: because it means that we can change the current version of the face and things that you write will just keep working, unchanged.

I think that is one of the finest features we are delivering, a guarantee of backward compatibility in your extensions. Internally, we implement this by running your action in the context of the Face it was defined on: if you expect the 0.0.1 API we deliver, and things just work.

Obviously, if you want to use cool new features in a newer version of the API you have to migrate your code to handle any other changes … but at least the basics still work as you expected. Unless you are really clever…

    Puppet::Face.define(:example, '1.0.0') do
      action :wicked_smart do
        # documentation omitted for brevity...
        when_invoked do |alpha, beta, options|
          # This totally works, but don't tell anyone I told you.
          advanced_example = Puppet::Face[:example, '2.0.0']
          advanced_example.new_feature current_action_name(alpha, beta)
 
          # You can even do this, if you want ^W need to: ;)
          current_action_name(alpha, beta)
          advanced_example.current_action_name(beta, alpha)
        end
      end
    end

Anyway, that aside put aside for now, versioning is a huge part of why I think that Faces are cool: it is universal, it is powerful, and it lets us keep things working.

Our promise about versions

Any promise goes two ways, and versioning Faces like this is no exception. We don’t treat it lightly: we promise that in return for the extra little bit of work it takes to use the versioned API that we will try our level best to keep old API versions around and working for as long as possible.

(They might, heck, will‚ end up rewritten to just call up to the newer version of the API transparently, but you don’t need to care. We will keep the external API compatible so your code just works‚ the whole time.)

Which is pretty cool and all, but that isn’t it: we also promise to use our powers for evil, not just for good. Now that things you write are good and stable with versioning, we can release newer versions of the API sooner.

That means that you should see less time between a great idea and getting to use it. (Plus, you can reliably work out if it isn’t there on a machine, and fall back in your own code.)

Faces Versioning FAQ

Do I really have to specify a version number?

Technically, yes, but. We have two ways to do something smarter than giving the full version number. One is the ability to specify only part of the version number, and have the rest match “any version”:

• “1.0.1″ will only match exactly version 1.0.1.
• “1.0″ will match version 1.0.1, and also 1.0.2, but not 1.1.0.
• “1″ will match anything with the major version 1.

We also offer, but do not encourage, the ability to refer to the :current version of a particular face, which will return the latest “officially blessed” version of the face found in your Ruby load path.

We don’t really recommend using it, though: saying “use :current” is the same as saying “please break when the API changes”. I don’t know about you, but that usually isn’t what I want to say when I write code ‚ even throw-away scripts. (I guess mostly because I noticed that I almost never get to throw those ones away, while the ones I expect to last? Pffft, gone.)

So, I would encourage you to always specify at least the major number of your version. It means less surprises, and less headaches, for you in the long run.

What do your version numbers mean?

The great thing about standards is that there are so many to choose from, as the saying goes, and version numbers are no different. Lots of people have their own ideas about what they mean, how they work, and how you should sort them.

Here at Puppet Labs we love inventing a new standard as much as the next programmer, but this time sense prevailed: we picked up the Semantic Versioning Specification and used that to define our Face versioning. (Plus, it was invented by one of the folks behind GitHub, and they are wicked-smart and all.)

Specifically, each digit has a defined meaning:

• digit one, major: incremented for a backwards-incompatible API change.
• digit two, minor: incremented for a backwards-compatible API change.
• digit three, patch: incremented when change does not touch the API.

Not that you need to worry too much: if you want something simple, just use 1.0.0, 2.0.0, and so forth in your code, and pretend the rest of the digits don’t actually exist.

Puppet Labs and Eucalyptus Systems are partnering up to make data center infrastructure management in the cloud experience easier. Announced today, the Eucalyptus-Puppet Cloud integration kit vastly increases the speed and ease for users to configure, provision, and manage Eucalyptus on-premise IaaS clouds with Puppet. Learn how the best utilize these resources in our joint webinar with Eucalyptus tomorrow: Increasing the Speed and Agility of Cloud and Data Center Infrastructure Management with Puppet and Eucalyptus.

The webinar will feature co-founder Teyo Tyree on the Puppet side of things, and will cover:

• Overview of the new administration challenges and solutions for configuring, provisioning and managing IaaS clouds in the data center
• Description of how Data Center Administrators are configuring, provisioning and managing with Puppet and the advantages they are realizing
• Puppet-Eucalyptus use case demo: See how cloud and Data Center Administrators work together using Puppet and Eucalyptus

Register for the Webinar

The skies are clear and blue, and I am headed to Amsterdam for Puppet Camp. This is my second time organizing camp, and despite selling out of tickets a bit early things seem to be on track. I am looking forward to arriving and getting started—I’ve been so caught up in planning PuppetConf the last few weeks that I almost forgot about actually attending Camp.

I’ve received a lot of questions concerning Puppet Camp and PuppetCon, so I want to take a moment to clear up the differences. PuppetConf will be on September 22nd and 23rd in Portland, Oregon. It’s not meant to replace Camp. Instead, it’s a new event in our conference series. PuppetConf will be based in Portland and happen annually. This year we’re expecting around three hundred in attendance, doubling Puppet Camp’s numbers. We’re going to offer four tracks: Puppet 101, Developing Puppet, a Cloud topic, and a DevOps topic.

Puppet Camp will stick around, but will become a more regional one-day event occurring several times a year. I suspect we’ll offer one in the bay over the summer, and one in Boston after PuppetCon, but don’t hold me to that. In 2012 I want to aim for five camps—I’m thinking San Francisco, New York, Austin, Berlin, and London; if I feel really ambitious we might shoot for Bangalore. The goal is for these events to be community-focused. Camps will promote local user partnering and will continue to be unconferences.

Anyhow, this Camp is sure to be a righteous, good time. I am looking forward to meeting everyone. Be sure to chat with us during the conference on twitter using #puppetcamp.

Version 1.1 of Puppet Dashboard was recently released, fixing a number of bugs and broadening the feature set. If you’re already a Dashboard user, you’ll certainly find this a worthwhile upgrade. If you’re not, then now is a great time to check it out. Here’s why:

Inventory service integration. Working in beautiful harmony with the recently-released inventory service feature of Puppet, Dashboard can now query and report facts across all your nodes. When viewing an individual node, its current fact information will be retrieved via the inventory service for your perusal.

Also of note is the “Custom query” function, found in the left-hand node navigation. This page allows you to specify conditions on particular facts and search for all nodes matching those conditions. For instance, this can be used to easily find all your Debian nodes that haven’t yet been upgraded to Puppet 2.6.

Filebucket integration. Continuing in the vein of interconnectivity with Puppet, Dashboard is now filebucket-aware. When the checksum of a file is mentioned in a report, Dashboard will offer to display the bucketed contents. Similarly, when a file’s content changes, you will have the option to view a diff.

Improved report support. Reports in the Puppet 2.6 format are now fully supported, in addition to inspect reports created by the new “puppet inspect” application. The page for a Puppet 2.6 report provides resource-level summaries, including information about the previous and current values of any changed properties. Viewing an inspect report is quite similar, except the information shown represents a snapshot of the current state of the system at the time of the report.

File search. Using the new file search page, you can easily find which of your nodes have a particular version of an important file. Simply provide the file name and desired checksum, and receive a list of which nodes have the content you want, and which don’t. This feature searches the latest inspect report for your nodes, so you will need to periodically run “puppet inspect” for this functionality to work.

This release is available for download, and documentation is coming—preliminary documentation is available here.

We are excited to be a program development sponsor for The Ada Initiative, a non-profit committed to increasing the involvement and involvement opportunities of women in open technology and culture. The cause is near and dear to our heart, as CEO Luke sums up:

The open source community needs to recruit and retain more women. Sponsoring the Ada Initiative is an easy way for any organization to make a difference. It also demonstrates our company’s commitment to a a positive work environment for women and our conviction that equal opportunity in technology in open source is critical for its long term success.”

About the Ada Initiative

The Ada Initiative is a non-profit organization dedicated to increasing participation of women in open technology and culture, founded by long-time women in open technology activists and programmers Valerie Aurora and Mary Gardiner. The Ada Initiative is named for Countess Ada Lovelace, widely recognized as the world’s first computer programmer. The Ada Initiative partners with organizations and communities to increase the participation of women in ways that shape the technology, such as software design and development, writing for Wikipedia, and community leadership.

Swisstopo is Switzerland’s national cartography agency, providing customers with both physical and digital maps, many of which are also available to mobile phones with a GPS. Wanting to transition their infrastructure to the cloud, Swisstopo used Puppet to streamline their servers and increase uptime.

Hanspeter Christ, Deputy Head of the Federal Spatial Data Infrastructure (FSDI) within Swisstopo explains the benefits of Puppet:

“The biggest benefit from our investments is not from moving to the cloud, but more that we have ‘Puppetized’ our servers. We have become more efficient in provisioning servers so we can easily get 100 of them up and running in a very short time.”

To learn more, read the rest of the Swisstopo case study