FAQ: Puppet Labs & MCollective

What is the announcement?

Puppet Labs is acquiring the intellectual property of MCollective, an open source project also known as The Marionette Collective. The project’s founder, R.I. Pienaar, is joining the development team at Puppet Labs and will continue to maintain his office in the United Kingdom.

What is MCollective?

MCollective is a framework to build server orchestration or parallel job execution systems. It enables real-time discovery of network resources and can select which resources to affect based on configuration data from leading systems management platforms, including Puppet. With MCollective a systems administrator or IT team can:

• Interact with small to very large clusters of servers
• Use a broadcast paradigm for request distribution. All servers get all requests at the same time, requests have filters attached and only servers matching the filter will act on requests. There is no central asset database to go out of sync, the network is the only source of truth.
• Break free from ever more complex naming conventions for hostnames as a means of identity. Use a very rich set of meta data provided by each machine to address them. Meta data comes from Puppet, Chef, Facter, Ohai or plugins you provide yourself.
• Use simple command line tools to call remote agents.
• Write custom reports about your infrastructure.
• Manage packages, services and other common components using contributions from the community.
• Write simple RPC style agents, clients and Web UIs in an easy to understand language – Ruby
• Customize the system using MCollectives pluggable architecture
• Leverage the rich authentication and authorization models available via middleware, including fine-grained
• Authentication, Authorization and Auditing of requests.

• Re-use the ability of middleware to do clustering, routing and network isolation to realize secure and scalable setups.

MCollective works with many leading message queues including ActiveMQ and RabbitMQ.

You can learn more about MCollective at here. You can also check out screencasts, an online demo or the various plug-ins.

What is Puppet?

Puppet is an open source platform that helps systems administrators and developers automate the configuration, deployment and management of IT infrastructure. Puppet makes it easy to define the desired state of your systems, keep those systems in that state or rapidly modify them for changing requirements. With Puppet your infrastructure can now work for you rather than the other way around. Puppet improves the performance of your IT infrastructure by maximizing scalability and minimizing waste on repetitive, error-prone tasks to define and manage your systems.

Get a Puppet Labs logo

What is the history of MCollective?

The foundation for MCollective was created by R.I. Pienaar 2005. As a systems administrator Pienaar experienced first hand the numerous tasks that need to be performed or orchestrated in a programmatic way on a cluster of severs. He originally conceived of MCollective as a way to simplify this process through real-time discovery of network resources and a framework for sequencing commands on large and diverse collections of servers.

MCollective works with a variety of configuration management solutions. As a very active and prominent member of the Puppet community, most of Pienaar’s work with MCollective has been in environments that also rely on Puppet, the configuration management platform created by Puppet Labs.

What does MCollective bring to Puppet Labs?

The deal accelerates Puppet Labs’ delivery of important multi-server and multi-datacenter orchestration to the thousand of enterprises that use Puppet’s model-driven framework for systems management. Combining MCollective with Puppet adds an important time element to the way systems are managed. It also improves how Puppet can work with managing applications on existing systems. MCollective’s real-time discovery capabilities enhance the Puppet Dashboard, simplifying how users schedule complex sequences of activities using data available from the Puppet platform. The deal also brings Pienaar’s significant experience and thinking related to message queues and real-time discovery of complex enterprise networks.

Download large image

Download large image

What does Puppet Labs bring to MCollective?

This enhanced relationship will leverage additional resources for the MCollective project, accelerating its development, and further integrating the project with the Puppet platform. MCollective customers will be able to use Puppet Labs professional services, obtain training and support and tighter integration into the tools they already use – Puppet and Puppet Dashboard.

Puppet is a widely adopted open source project and in July 2001 Puppet Labs raised $5 million in a Series B round led by Kleiner Perkins Caufield & Byers. The investment was focused on building Puppet Labs’ engineering and development capabilities.

Who will benefit from using both systems together?

Any organization running Puppet on large pools of machines has concerns about how to manage deployments so that upgrades happen on a subset of the machines at a time. By using both systems together an organization can perform key tasks in a sequenced manner, limiting any interruptions to their service.

When will new product features and integration improvements be released?

The road map for MCollective is currently being revised. The initial road map includes the following:

• Q4 2010 – Orchestrate Puppet runs using Dashboard classifications. View run status of Puppet clients via MCollective.
• Q1 2011 – Generic interface to MCollective Agents in Puppet Dashboard. Puppet REST interface integration with MCollective. Scheduled and background MCollective action invocation
• Q2 2011 – Simplified cross node dependency orchestration. MCollective support for non-Ruby agents and clients.

Download large image

What are the financial terms of the deal?

As a privately-held company Puppet Labs is not disclosing the specific terms of the deal.

Potential Quotes

“Puppet Labs has a proven record of selling into the enterprise market, running successful training programs and have a very strong development team, professional services and management. These combine to make Puppet Labs a good fit that will serve future customer needs but also ensure the continued development of MCollective.” – R.I. Pienaar

“Puppet has always focused on the problem of configuration management, but solving that doesn’t make all of your problems go away. MCollective targets the next layer of change control and orchestration in a small but powerful tool that already integrates well with Puppet. With both Puppet and MCollective, our customers and users have a unified solution that scales from small departments and one-off installations to large server farms running tens of thousands of servers.” – Luke Kanies

“We will continue to grow Puppet’s functionality and scale as our customers needs grow. This deal demonstrates our commitment to both open source and defining the next generation data center stack.” – Luke Kanies