Check out Part 1 of this post here.
Using MCollective to Query Your Nodes
MCollective is an awesome solution for accessing information from all of your nodes very quickly. MCollective has a facter plugin that can pull fact information out of facter, but there’s a method that Jordan Sissel devised for exporting all of your facter facts (as well as all of the variables in the puppet scope) to a YAML file (specifically, /etc/mcollective/facts.yaml). Setting this up is pretty trivial (it’s a single-line inline template), and the benefits are exponential; not only is this MUCH faster than querying facter, but, as I mentioned, it pulls all of your variables out of the puppet scope in which you declared the /etc/mcollective/facts.yaml file. Follow the instructions in that link and let’s setup the /etc/mcollective/facts.yaml file.
Once you’ve set this up, let’s make sure MCollective is able to read from the file. This can be set in your /etc/mcollective/server.cfg config file with the following:
factsource = yaml
plugin.yaml = /etc/mcollective/facts.yaml |
Next, we need to install a plugin so MCollective can handle YAML files as a factsource. The yaml_facts.rb file is included as a plugin with the MCollective source, but it can also be downloaded from the PuppetLabs Github site. Drop this file into your MCollective library’s facts folder (usually /usr/libexec/mcollective/mcollective/facts), ensure that the factsource is set in your server.cfg config file, and restart MCollective to test it out. You can execute “mc-inventory (hostname)” to test and make sure that facts are being reported properly.
With MCollective up and running and reading facts directly from a YAML file, let’s run a sample query with “mc-facts warranty_out” and see what we get:
Report for fact: warranty_out
No found 38 times
Yes found 75 times
Finished processing 113 / 113 hosts in 1047.24 ms |
If you remember, this fact tells us if our machine’s warranty has expired. In this test with 113 nodes, you can see that 75 nodes have expired warranties and 38 nodes are currently under warranty. Next, let’s look at the expiration date for our warranties with “mc-facts warranty_end”:
Report for fact: warranty_end
2011.02.23 found 4 times
2011.04.30 found 1 times
2011.07.14 found 23 times
2011.07.15 found 3 times
2011.07.16 found 2 times
2011.07.17 found 2 times
2011.07.19 found 16 times
2011.09.16 found 2 times
2012.09.15 found 5 times
Expired found 123 times
Finished processing 183 / 183 hosts in 3098.34 ms |
Here’s a larger sampling of nodes with several warranty expiration dates. This is all great information, but wouldn’t it be nice to see exactly WHICH nodes are corresponding to these dates. You can do that by running “mc-facts warranty_end -v”. The -v argument will list nodes corresponding to each fact underneath that fact in your output:
2012.09.15 found 4 times
berry-shawstaff-shel
deville-staff-wes
kluding-staff-shel
solomon-staff-shel |
You can repeat this command with ANY fact. Since I am using the YAML solution as a factsource, it’s also exporting variables from my puppet scope. I have several variables associated with packages that are to be installed, and these package reside in the same scope as the /etc/mcollective/facts.yaml file declaration, so I can also query these variables with “mc-facts (variablename)”
Report for fact: firefox
firefox3.6.11.dmg found 218 times
Finished processing 247 / 247 hosts in 3027.37 ms |
NOTE: Pay attention to the scope where you declare the /etc/mcollective/facts.yaml file. If this file is declared within the scope of its own class, you won’t see variables OUTSIDE its scope. You can do creative things to change the scope of this file if you wish (possibly using puppet-concat), but I’ll leave that up to you.
What About Offline Nodes?
MCollective works great for nodes that are online and connecting to your middleware solution (ActiveMQ, RabbitMQ, etc.), but what about machines such as laptops (or machines that have hardware failures) that might not be online? Fortunately, your puppet master server keeps a YAML store of fact data for all nodes that have ever connected to it. Located in your puppet master’s $vardir/yaml/facts directory, there’s a YAML file (identified by $certname) for every node. Writing a script to traverse the directory and query the nodes is pretty trivial—you can even create an MCollective Agent to do the task for you. Let’s look at a ruby script I created to search the YAML store:
#!/usr/bin/env ruby
#
# File: yaml_store_search.rb
#
# Description: A script to search through your puppet master's YAML store
# for a specific node's data.
require 'getoptlong'
require 'puppet'
Puppet.settings.parse
if not ARGV[0]
puts "You must have an argument. Use --help for more information."
exit(1)
end
# Parse the arguments here and print the help prompt if any incorrect arguments
# are being used.
opts = GetoptLong.new(
[ '--hostname', '-n', GetoptLong::REQUIRED_ARGUMENT],
[ '--certname', '-c', GetoptLong::REQUIRED_ARGUMENT],
[ '--help', '-h', GetoptLong::NO_ARGUMENT]
)
begin
opts.each do |opt, arg|
case opt
when '--hostname'
$hostname = arg
when '--certname'
$certname = arg
when '--help'
puts <<-EOF
yaml_store_search.rb [OPTION] value
-n, --hostname:
Search for a specific hostname in the YAML store. Requires a hostname argument
-c, --certname:
Search for a specific certname in the YAML store. Requires a certname argument
-h, --help:
Display this help text
EOF
end
end
rescue GetoptLong::InvalidOption, GetoptLong::AmbigousOption
puts <<-EOF
-n, --hostname:
Search for a specific hostname in the YAML store. Requires a hostname argument
-c, --certname:
Search for a specific certname in the YAML store. Requires a certname argument
-h, --help:
Display this help text
EOF
exit(1)
end
# Determine whether we're doing a lookup via hostname or certname
searchfield = 'hostname' and arg = $hostname if $hostname
searchfield = 'certname' and arg = $certname if $certname
# Iterate through your puppet YAML store using the $vardir from
# your puppet.conf settings. If we find the YAML file with a
# matching certname or hostname, break out of the loop.
Dir.glob("#{Puppet[:vardir]}/yaml/facts/*") {|file|
$tempfile = YAML::load_file(file).values
if $tempfile[searchfield] == arg
$found_file = true
break
end
}
# Output the list of facts.
if $tempfile
$tempfile.each_pair {|key, value|
puts "#{key} = #{value}"
}
end |
This script will gather the value of your $vardir and will search the $vardir/yaml/facts directory in search of a node by way of its hostname or certname. If the node was found, it will output every fact that it has kept about that node. This script will ONLY work on puppet masters as they’re the only machines who keep this YAML store.
Since we’re talking about MCollective, let’s look at an MCollective Agent that will execute this code on your puppet master. Here’s the sample agent I created:
module MCollective
module Agent
class Yaml_store "yaml_store.rb",
:description => "A conduit to search your puppet master's YAML store.",
:author => "Gary Larizza ",
:license => "Apache License, Version 2.0",
:version => "1.0",
:url => "http://glarizza.posterous.com",
:timeout => 3
# Search action: This action will check for the YAML file of a specified certname or hostname.
# => If the file exists, it will output the contents of the YAML file. If it
# => doesn't exist, you will receive an error message.
# Variables:
# => fact => The fact for which we're checking a value
# => value => The value for which we're searching
# Calling:
# => Run the fact with 'mc-rpc --agent yaml_store --action search --arg hostname=lab01-hsimaclab-hhs -v'
# => or with 'mc-rpc --agent yaml_store --action search --arg certname=lab01-hsimaclab-hhs -v'
# => or with 'mc-rpc --agent yaml_store --action search --arg certname=lab01-hsimaclab-hhs --arg fact=warranty_end -v'
#
action "search" do
if request.include?(:hostname)
if request.include?(:fact)
searchfield = 'hostname'
arg = request[:hostname]
reply[:facts] = get_specific_fact(searchfield, arg, request[:fact])
else
searchfield = 'hostname'
arg = request[:hostname]
reply[:facts] = get_all_facts(searchfield, arg)
end
elsif request.include?(:certname)
if request.include?(:fact)
searchfield = 'certname'
arg = request[:certname]
reply[:facts] = get_specific_fact(searchfield, arg, request[:fact])
else
searchfield = 'certname'
arg = request[:certname]
reply[:facts] = get_all_facts(searchfield, arg)
end
end
end # Action end
def get_all_facts(searchfield, arg)
Dir.glob("#{Puppet[:vardir]}/yaml/facts/*") {|file|
$tempfile = YAML::load_file(file).values
if $tempfile[searchfield] == arg
$found_file = true
break
end
}
if $found_file
$tempfile.each_pair{|key, value|
puts "#{key} = #{value}"
}
else
puts "That #{searchfield} was not found."
end
end # Def end
def get_specific_fact(searchfield, arg, thefact)
Dir.glob("#{Puppet[:vardir]}/yaml/facts/*") {|file|
$tempfile = YAML::load_file(file).values
if $tempfile[searchfield] == arg
$found_file = true
break
end
}
if $found_file
if $tempfile[thefact]
puts "#{thefact} = #{$tempfile[thefact]}"
else
puts "#{thefact} is not a valid fact, or wasn't found."
end
else
puts "That #{searchfield} was not found."
end
end # Def end
end # Class Etc_facts end
end # Module Agent end
end # Module MCollective end |
This agent will accept an argument of a node’s hostname or certname and also, optionally, a specific fact whose value we would want to inspect. I’ve also created a DDL file that will parse the output of this agent properly. Here’s that DDL file:
metadata :name => "Yaml Store",
:description => "A conduit to search your puppet master's YAML store",
:author => "Gary Larizza ",
:license => "Apache License, Version 2.0",
:version => "1.0",
:url => "http://glarizza.posterous.com",
:timeout => 60
action "search", :description => "Retrieves Facter facts for specified nodes" do
display :always
input :hostname,
:prompt => "Hostname",
:description => "The hostname of the node for which we're getting facts",
:type => :string,
:optional => true,
:validation => '(.*?)',
:maxlength => 230
input :certname,
:prompt => "The Puppet Certname Variable",
:description => "The certname of the node for which we're getting facts",
:type => :string,
:optional => true,
:validation => '(.*?)',
:maxlength => 230
input :fact,
:prompt => "A Searchable Facter Fact",
:description => "The facter fact for which we want a value",
:type => :string,
:optional => true,
:validation => '(.*?)',
:maxlength => 230
output 'facts',
:description => "The Facter facts you want displayed",
:display_as => "Fact Information"
end |
In order for this agent to work, your puppet master must also be running MCollective and the agent and DDL file must be located in your MCollective library’s agent folder (typically /usr/libexec/mcollective/mcollective/agent). Once you’ve dropped these files into the agent folder, you’ll need to restart MCollective for it to work. Here’s the syntax you will use to run it:
mc-rpc --agent yaml_store --action search --arg hostname=machine_hostname
mc-rpc --agent yaml_store --action search --arg certname=machine_certname
mc-rpc --agent yaml_store --action search --arg hostname=machine_hostname --arg fact=facter_fact
mc-rpc --agent yaml_store --action search --arg certname=machine_hostname --arg fact=facter_fact
# Or Simplified:
mc-rpc yaml_store search hostname=machine_hostname
mc-rpc yaml_store search certname=machine_certname fact=facter_fact |
You can run this command from ANY MCollective client so long as the agent is installed on your puppet master server. The catch is that if you want the client to have the output parsed correctly, they will need to have ONLY the DDL file dropped into their MCollective library’s agent folder (the agent file itself is only needed on the puppet master server). If the client DOESN’T have the DDL file installed, they can simply append “-v” at the end of the above commands to see output from the reply hash that is used to communicate with the DDL file.
This agent will return either the entire list of facts for your specified hostname/certname or, optionally, the value of a specific fact. This is useful if your machine has malfunctioned (or is offline) and you quickly want to access whether the warranty has expired or not. Being able to access it from an MCollective client who has access to mc-rpc is handy if you’re not currently on your work computer. This access, however, is entirely up to you.
In Conclusion
If you’re familiar with MCollective and Puppet (and you have a functional understanding of Ruby), it’s pretty easy to write custom Facter facts and MCollective agents to automate simple tasks. The process of maintaining an inventory can be particularly troublesome, but these scripts should give you near-immediate access to some of the most important facts about your Macs. Like I mentioned in the opening, most vendors will provide you with a conduit to access similar information to facts I’ve presented here for my Macs. All it will take is a bit of tweaking to adapt my code for your use. Keep pulling those strings!
Gary Larizza is the Director of Technology for a small K-12 School district in Northern Ohio where he routinely breaks systems in the name of advancement. When he’s not trying to automate himself out of a job through the use of Puppet and Ruby, he enjoys seeking out awesome open source tools that will make Mac Management easier. He’s been using Puppet in the Education sector for the past two years on hardware that would make Linux admins cry (namely, G5 Xserves and an iMac) and loves to share the knowledge he’s gained with anyone who buys him a wheat beer (Weihenstephaner FTW). All of his notes are on his Posterous blog, and he will be speaking at the 2011 Penn State University Macadmins Conference if you’d prefer to chat with him in-person.
