Design is an integral part of the way we build software at Puppet Labs. More specifically, we strive to answer a very simple question: what does the user need? This isn’t always an easy question to answer, but we’ve been happy with our success in doing so. User empathy is our conduit for user satisfaction.

In real-life terms, what does design at Puppet look like? Let me show you a project that just finished its initial design phase: Puppet Sites.

What do users need?

Puppet admins want to spend their time managing nodes that are under Puppet control. They don’t want to spend their time adding nodes to their deployments. On the contrary, their goal is to start managing new nodes as quickly as possible. We formalize this need in the following way:

As a Puppet admin I want to easily add new nodes to my deployment so that I can start managing them as quickly as possible.

How do we we satisfy user needs?

The current workflow for adding nodes to a deployment is as follows:

1. Configure puppet.conf with the location of the master/CA
2. Perform an agent run on the node
3. Login to the CA
4. Sign the node’s certificate
5. Perform another agent run on the node

The problem with the existing workflow is that it appears to have nothing to do with adding new nodes to a deployment and everything to do with signing certificates. There’s a disconnect between the workflow and the user need.

To fix this, we did three things:

1. We introduced the concept of a site. A site is a service that owns the list of nodes in your deployment, the authentication mechanism for adding new nodes to your deployment, and the configuration of Puppet services in your deployment.
2. We changed the semantics of the workflow, so that it directly addresses the user need.
3. We eliminated the overhead of signing into the CA and manually signing node certificates. There are secure ways to do this that don’t involve user interaction over SSH.

The workflow now resembles the following:

1. Login to the site host
2. Generate a pre-shared key
3. Join a node to the site using the pre-shared key
4. Repeat step 3 for every node you want to add to the site

We now have a workflow to fulfill the user’s goal. But notice that we’re still not talking about sites in technical terms. This workflow could apply equally well to a Puppet Face, a REST API, or a Dashboard plugin.

How do users interact with our workflows?

It’s all well and good to produce a general workflow for addressing a user need, but eventually you have to tie it to a real user interface. For the initial release of Puppet Sites, we decided to focus on two user interfaces: a Puppet Face and a REST API.

A designer is nothing without regular interaction with customers. While designing user interaction, we stayed engaged with our internal operations and professional services teams. After several rounds of feedback, the design of the Puppet Face for the workflow given above resembles the following:

node02$ ssh admin@site02.domain.com
Last login: Mon May  7 18:15:43 2012
site02$ mount /media/usbdisk
site02$ puppet site generate key > /media/usbdisk/site.key
site02$ umount /media/usbdisk
site02$ exit
node02$ mount /media/usbdisk
node02$ puppet node join site02.domain.com < /media/usbdisk/site.key
Trying to add node02.domain.com to the site at site02.domain.com...
 Use `puppet site status node02.domain.com` to confirm success
 To stop waiting for the command to complete, press Ctrl-C.
   The command will still complete in the background.
Added node02.domain.com to the site at site02.domain.com

What do users think?

We strive to make our design process as agile as our engineering process. We specifically avoid a waterfall model, where the design is completed in an ivory tower, then handed over the wall for implementation. Instead, our designers sanity check their design with internal and external users, talk to architects and engineers to make sure they aren’t violating the laws of physics, and stay involved in the project until it ships, making course corrections as necessary.

The Puppet Sites project is still underway, but we’ve already discovered mistakes made early in the process, and corrected them. Each group of users has been vital to this iterative process.

Our internal users made us aware that omitting a pre-shared key system was a poor idea. By themselves, the two other fixes—changing semantics and introducing sites—weren’t sufficient to solve the user goal of “quickly” adding nodes to their deployments.

Our architects made us aware of a “law of physics” deal-breaker in the design, where users could have passed their pre-shared key as a flag to puppet node join. This would have exposed the key to sniffer processes on the host machine, and represented an unacceptable security risk.

Our external users helped us realize that the concept of a “site” is foggy at best and they didn’t really understand the problem it was designed to solve. Also, they were concerned that we were deprecating existing functionality without providing a drop-in replacement. We’ve been working to ameliorate these concerns by formulating clearer stories around the problems that sites solve.

Design at Puppet Labs

Of course, Puppet Sites does far more than just provide admins with an easy way to add new nodes to their deployments. For the sake of brevity, I’ve only highlighted this one user story. However, having a site also helps Puppet admins who want to…

• …get a list of all the nodes in their deployment with a single command, so they don’t have to trawl multiple services to get this information.
• …centrally manage the configuration of all nodes in their deployments, so they don’t have to manually manage puppet.conf on each node.
• …access information about Puppet services from their manifests, so they don’t have to hardcode the location of their services into their manifests.

You can expect Puppet Sites to be available soon, as a point release for Puppet 3.0.

In summary, the UX team at Puppet produces three main artifacts: user stories, workflows, and wireframes.

• User stories allow us to talk about the needs of our users in a non-technical, goal-oriented way.
• Workflows describe the tasks users go through to achieve their goals.
• Wireframes are mockups of the actual systems that users use to complete these tasks.

Not all of our projects follow this process exactly, but, in general, the above holds true. User story to workflow to wireframe. Rinse and repeat as necessary. That’s how we do design at Puppet Labs.

Learn More

• What is User Experience in Puppet?
• Looking Forward to 2012: Design, Big Data in the Infrastructure, and DevOps

This week I took on the role of CTO at Puppet Labs, and started reflecting on the awesome journey that led me here.

It was 2006, and I was scrambling to make it onto the last bus back to San Francisco from the Apple WWDC Beer Bash down in Cupertino. I’d been to quite a few Beer Bashes and knew the drill: forget the lineup for the campus store, just concentrate on finding beer and the few Apple employees who could fix the OpenDirectory bugs that were making my life hell. Both objectives were completed, leaving me only minutes to avoid having to spend way too much money getting a taxi back up to the city.

Fatefully, I found a seat next to this intensely opinionated sysadmin, Jeff McCune (later to become one of the first pro services guys at Puppet Labs, and now one of our core developers). He recognized me from my WWDC presentation that year and started grilling me about how I ran my university campus, particularly the file-based configuration management system I used, Radmind, and the hacked up framework I’d put in place to try to manage higher level objects than mere files.

He’d been to a talk by Luke Kanies (now the CEO of Puppet Labs) at LISA and was very excited about this guy who had built a tool that worked the way sysadmins actually needed to work with a pragmatic, model-based approach. Even more importantly though, Luke was serious about fostering adoption, and had helped Jeff write some useful extensions for Puppet. Jeff had already gotten religion about idempotent, declarative approaches for sysadmins, and spent pretty much all of the bus ride bending my ear about this new project called “Puppet” and how it was going to change the world of operations.

After WWDC, I flew back to Australia, fully intending to try out this magical Puppet project, but got distracted by the day to day life of running campus IT operations on a shoestring budget for users who were academics and artists.

I was even more distracted a few months later when one of the MacEnterprise community members came out of lurking and told me I should apply for a role at Google in Mountain View. Several of the toughest interviews of my life followed, and within a couple of months, I was moving my young family to the other side of the world to run Mac Operations at Google HQ.

It was clear that tools like Radmind simply weren’t going to work at Google for the many thousands of corporate Macs. Opinionated engineers who demanded a high degree of customization, immense growth, globally distributed offices and a very small team meant that it was completely insane to even think about trying the old methods of file-based config management of the entire system.

We needed a better and more sustainable way, a solution that gave us higher levels of abstraction with meaningful entities such as users, groups, services and packages, and that didn’t require you manage the entire machine.

Jeff and I had kept in contact, and he was presenting on Puppet at WWDC that year. I popped up to San Francisco with some of my coworkers, and made sure we turned up to his talk.

10 minutes into his presentation we were getting pretty excited, and we started experimenting over VPN. By the time Jeff finished his talk, we had a working Puppet master back at Google managing the contents and permissions of a few critical files in /etc, and knew we had a great match.

As it turned out, the Mac deployment was such a rapid success that one of the Linux Ops team started a skunkworks project to manage the internal Linux distro with Puppet, as there had been a few failed CFEngine attempts. This worked so well that Puppet eventually managed all the Google corporate Mac and Linux desktops, laptops and servers.

Puppet was a much younger project in those days. We were building a lot of custom Puppet extensions for Mac OS X that went back into the core, and were having to scale Puppet to manage tens of thousands of nodes, so I spent a lot of time on the mailing lists and IRC channels brainstorming with Luke and the community. I quickly fell in love with the community. It was full of thoughtful sysadmins, people who were frustrated with the unreliable state of operations tools, and knew there was a better way out there than continually reinventing arcane bash/ssh frameworks.

We have some great technology with Puppet, but one of our greatest strengths is our outstanding community.

I ended up at the first ever Puppet Camp, San Francisco, 2009. It was small, but was one of the most exhilarating conferences I’ve ever been to. I love looking back at those photos and seeing how many of that group are now part of the Puppet Labs team. Dan Bode, James Turnbull, Ben Hughes, Gary Larizza, Michael Stahnke, Carl Caum, Deepak Giridharagopal (Little known fact: his last name is actually Tamil for “Grid Computing”).

That’s an awesome group of people to end up working with, let alone all the other great people we have here at Puppet Labs.

I had an amazing couple of years at Google, surrounded by super sharp minds and working on truly interesting operations problems at a scale greater than anything I’d ever touched before, but I was starting to look enviously at friends who left for early stage startups and the breadth of knowledge they were acquiring. Luke had poked me a couple of times about coming to work for him, but I didn’t seriously consider it until late 2010 when he, Teyo and James made a much more concerted effort.

“You’re opinionated about Puppet. Want to put your money where your mouth is?”

One visit to Portland and I knew I wanted to live in this awesome food, beer, and cycling-obsessed city full of people following obscure passions. I jumped ship from Google and we moved north, where I dived headfirst into being responsible for Product at a very quickly growing startup.

It’s been an immense 18 months. We started with our first commercial release, Puppet Enterprise 1.0, and followed that up with several great releases, all solving real problems for real users. We’ve brought on the open source MCollective and Hiera projects from the incomparable RI Pienaar, released Puppet 2.7.0, and grown at an amazing pace. We’ve grown from 2 events a year to 15, including the incredibly successful PuppetConf `11 and are building up to an even bigger PuppetConf this year. Nothing like startup speed to quicken the blood.

From the original 20 odd folks I started with in the tiny office in the seedy and urine-drenched Old Town to our shiny digs in the Pearl, with over 80 employees. From a distinct lack of in-house beverages to decent espresso and delicious local beer. From a company that knew the user experience was critical, to one with a growing UX/Design department headed up by Randall of the impenetrable Gandalf gaze.

I love this company. I love what we’ve done already to change the face of operations, I love the ambition we have to change it even more, and I especially love the people I get to do it with.

I’m thrilled to take on the role of CTO, and to concentrate on fostering our culture of technical innovation so that we continue to build applications and platforms that truly advance the state of IT infrastructure. The world of operations is undergoing radical change right now. The cloud, pervasive virtualization, corporate adoption of FOSS, BYOD, IaaS, PaaS and SaaS are all forcing sysadmins to be truly agile and adaptive. Some of the brightest people in our industry work in operations, and it’s going to be incredible to see what they come up with when IT automation gives them space to concentrate on genuinely important matters.