Creating Puppet types and providers is easy...

James Turnbull included in Blog Puppet open source

2010-02-01 1391 words 7 minutes

Contents

Puppet types are used to manage individual configuration items. Puppet has a package type, a service type, a user type, etc. Each type has providers. Each provider handles the management of that configuration on a different platform or tool, for example the package type has aptitude, yum, RPM, and DMG providers (amongst 22 others - what is wrong with people that they need to invent new packaging systems… but I digress). There are a lot of types, in fact I think Puppet covers a pretty good spectrum of configuration items that need to be managed. I don’t know of anything in particular that is missing that I can’t live without. But there are little gaps that are annoying, I’d like network and firewall types for example, but creating both these types in a generic enough way to support multiple platforms would be, IMHO, a non- trivial problem. Another gap is VCS/DVCS management. A lot of people use source code in repositories to do things with (including install stuff from you bad people - package things … it’s healthier). Puppet currently relies on creating and removing these repositories with the exec type (which executes scripts or binaries), for example:

exec { "svn co http://core.svn.wordpress.org/trunk/ /var/www/wp":
    creates => "/var/www/wp",
}

This is a bit ugly and it’d be a lot easier to write a Puppet type to manage repositories. But Puppet types and providers are written in Ruby and really, really complex and hard to develop. Right? Right? No. No, they are not… and I’m going to create a simple type and provider to show you. :) Here’s a very (very!) simple Puppet type, called repo, for managing repositories. I’ve created providers for SVN and Git as examples also. The first part of the repo type is the type itself - these are usually stored in lib/puppet/type or distributed via modules (see the PluginsInModules page in the Puppet wiki). I’ll create a file called repo.rb.

$ touch repo.rb

And then populate the file:

Puppet::Type.newtype(:repo) do
    @doc = "Manage repos"

    ensurable

    newparam(:source) do
        desc "The repo source"

        validate do |value|
            if value =~ /^git/
                resource[:provider] = :git
            else
                resource[:provider] = :svn
            end
        end

        isnamevar

    end

    newparam(:path) do
        desc "Destination path"

        validate do |value|
            unless value =~ /^\/[a-z0-9]+/
                raise ArgumentError , "%s is not a valid file path" % value
            end
        end
    end
end

So - pretty simple. We create a block Puppet::Type.newtype(:repo) do that creates a new type, which we’ve called repo. Inside the block we’ve got a @doc string. This is the documentation for the type. Add whatever level of detail and examples in here that is required. We’ve also got the ensurable statement. Ensurable provides some “automagic” that creates a basic ensure property. Puppet types use the ensure property to determine the state of a configuration item.

service { "sshd":
    ensure => present,
}

The ensurable statement tells Puppet to expect three methods: create, destroy and exists? in our provider. These methods, allow, respectively:

A command to create the resource
A command to delete the resource, and
A command to check for the existence of the resource

All we then need to do is specify these methods and their contents and Puppet creates the supporting infrastructure around them but more on this when we look at our providers. Next, we’ve defined a new parameter - this one called source.

    newparam(:source) do
        desc "The repo source"

        validate do |value|
            if value =~ /^git/
                resource[:provider] = :git
            else
                resource[:provider] = :svn
            end
        end

        isnamevar
    end

The source parameter will tell the repo type where to go to retrieve/clone/checkout our source repository. In this parameter we’re also using a hook called validate. Normally used to check the value for appropriateness here we’re using it to take a guess at what provider to use. Our code says, if the source parameter starts with git then use the Git provider, if not default to the Subversion provider. This is obviously fairly crude as a default and we can override this by defining the provider attribute in our resources:

provider => git,

We’ve also used another piece of Puppet automagic, isnamevar, to make this parameter the “name” variable for this type. In Puppet-speak, the value of this parameter is used as the name of the resource. (Types have two kinds of values - properties and parameters. Properties “do things”. They tell us HOW the provider works. We’ve only defined one property, ensure, by using the ensurable statement. Parameters are more like variables, they contain information relevant to configuring the resource the type manages rather than “doing things”.) Finally, we’ve defined another parameter, path.

    newparam(:path) do
        desc "Destination path"

        validate do |value|
            unless value =~ /^\/[a-z0-9]+/
                raise ArgumentError , "%s is not a valid file path" % value
            end
        end
    end

This is a variable value that specifies where the repo type should put the cloned/checked-out repository. In this parameter we’ve again used the validate hook to create a block that checks the value for appropriateness. Here we’re just checking, very crudely, to make sure it looks like the destination path is a valid fully-qualified file path. We could also use this validation for the source parameter to confirm a valid source URL/location was being provided. (You can also use another hook called munge to adjust the value of the parameter rather than validating it before passing it to the provider.) And that is it for the type. Next, we need to create a provider for our type. Let’s start with a Subversion provider like so:

require 'fileutils'

Puppet::Type.type(:repo).provide(:svn) do
    desc "SVN Support"

    commands :svncmd => "svn"
    commands :svnadmin => "svnadmin"

    def create
        svncmd "checkout", resource[:name], resource[:path]
    end

    def destroy
        FileUtils.rm_rf resource[:path]
    end

    def exists?
        File.directory? resource[:path]
    end
end

Up front we’ve required the fileutils library, which we’re going to use a method from. Next, we’ve defined the provider as a block:

Puppet::Type.type(:repo).provide(:svn) do

We tell Puppet that this is a provider called svn for the type called repo. Then we use a desc method that allows us to add some documentation to our provider. Next, we define the commands that this provider will use, here the svn and svnadmin binaries, to manipulate our resource’s configuration.

    commands :svncmd => "svn"
    commands :svnadmin => "svnadmin"

Puppet uses these commands to determine if the provider is appropriate to use on a client, if Puppet can’t find these commands in the local path then it will disable the provider. Next, we’ve defined three methods - create, destroy and exists?. Sounds familiar? Yep, these are the methods that the ensurable statement expects to find in the provider: The create method ensures our resource is created. It uses the svn command to create a repository with a source of resource[:name] (remember the source parameter in our type is also the name variable of the type - we could also specify resource[:source] here too) and a destination of resource[:path] (the value of the path attribute). The delete method ensures the deletion of the resource. In this case, it deletes the directory and files specified in the resource[:path] parameter. Lastly, the exists? method checks to see if the resource exists. Its operation is pretty simple and closely linked with the value of the ensure attribute in the resource:

If exists? is false and ensure is present, then create method will be called.
If exists? is true and ensure is set to absent, then the destroy method will be called.

In this case the exists? method checks if there is already a directory at the location specified in the resource[:path] parameter. So, let’s put all this together and create a resource with our new type. I’ve assumed you’ve already distributed your type and providers to Puppet. We can then create a resource like:

repo { "wp":
    source => "http://core.svn.wordpress.org/trunk/",
    path => "/var/www/wp",
    ensure => present,
}

Simple eh? We specify a repo resource, the source we wish to check out or clone from, the destination path and the ensure attribute (present or absent) and that’s it. You can see the complete code for this type and its providers at my Puppet repository on GitHub. It’s obviously very basic but should be easy to extend to provide additional capabilities (and currently has no tests - my bad). You can find further documentation (in a lot more detail!) on creating your own types and providers at the Puppet wiki.