Library (TODO)

This documents the standard library of components that batou provides.

Managing files and directories


The File component has been developed with Puppet‘s file type in mind. It accepts a very similar parameter set and has almost identical features.

You can use it to manage files, directories, and symlinks, and you can specify content (literally or as Jinja templates). You can also manage the Unix attributes and control whether leading directories should be managed or not.

The most basic usage is simply:

self += File('myfile')

This example creates a file at work/mycomponent/myfile, taking the contents from a file of the same name in the component’s directory (i.e. components/mycomponent/myfile). By default, the source file is run through Jinja, with the file’s parent component made available as component.

class batou.lib.File(path)

Creates a file. The main parameter for File is the target path. A File instance has an attribute path containing the full, absolute path to the resulting file.

File accepts the following additional parameters:


Filename of the source file to be used as the File’s content (absolute path or relative to the component’s directory). [Default: same as target path]


Literal file contents as a string.


Process file contents as Jinja template. [Default: True]


Object to make available as component to the Jinja template. [Default: File’s parent component]


Dict of additional arguments to make available to the Jinja template.


Encoding for the file contents [Default: utf-8]


Unix owner username.


Unix group name.


Unix permission mode (octal number, e.g. 0o755)


Create leading directories that were given in the target path. [Default: False]


Type of object to be created: ‘file’, ‘directory’, or ‘symlink’. This is useful for complex situations (e.g. creating a symlink with special ownership), for simple situations it’s probably more readable to use Directory or Symlink.

Source of symlink (for ensure = ‘smylink’)

XXX explain detailed factoring of Presence and Content

class batou.lib.BinaryFile(path)

Subclass of batou.lib.File. Creates a non-template binary file.


class batou.lib.file.Directory(path)

Creates a directory. The main parameter is the target path.


Path to a source directory whose contents are to be synchronized to the target path (uses rsync internally).


List of file names or patterns that should not be synchronized to the target path (passed to rsync as --exclude argument, see the rsync documentation for details).

Creates a symlink at target by linking to source.

Removing files

Removal of obsolete things is a difficult topic in the convergence paradigm. If in the past we created a file foo, but now it is not used anymore, the code that originally said, “please manage foo”, will not be there anymore. This means that nobody knows that the file foo that is still lying around on the production system is not actually in use anymore. In most cases, a few stray files do not matter, but in case they do, the deployment code has to explicitly state that something should not be present anymore.

class batou.lib.file.Purge(pattern)

Ensures that a set of files (given as a glob pattern) does not exist.

Extracting archive files

batou can extract archive files in Tar, Zip, and DMG (on OSX target machines) format:

class batou.lib.archive.Extract(archive)

The main parameter is the archive filename (relative to the component’s directory). The archive format is determined according to the file name extension (‘.tar’, ‘.tar.gz’, ‘.tgz’, ‘.tar.bz2’, ‘.tar.xz’ use tar, ‘.zip’ uses unzip and ‘.dmg’ uses hdiutil). The following additional parameters are supported:


Target directory to extract the archive into. Directory is created if it does not exist (compare create_target_dir). [Default: base name of the archive file]


Extract into the directory given in target. Set to False to extract directly into the work directory. [Default: True]


Only for tar archives: number of directories contained in the archive to strip off (see the tar documentation for details) [Default: 0]

VFS mapping (TBD)

XXX writeme

Managing python installations


The basic building block for Python-based components is creation of virtualenvs (to separate package installations from each other):

self += VirtualEnv('2.7')
class batou.lib.python.VirtualEnv(version)

Creates a virtualenv for the given Python version in the working directory of the parent component. (Requires that pythonX.Y is in the PATH)


Full path to the Python executable to create the virtualenv for (default: pythonX.Y based on the version attribute).

batou downloads a compatible version of virtualenv (depending on the Python version you need) to ensure everything works as expected and to avoid problems with incompatibilities or unexpected behaviours of whatever version might be installed already on the system. (virtualenv base installations are shared by all components for creating new virtualenvs, it is installed to work/.virtualenv).

Installing packages

Python packages are installed from a package index such as PyPI. batou uses pip or easy_install for this purpose (but that actually is an implementation detail and depends on the specifics of the Python and virtualenv version).

Packages must be added to a virtual environment.

venv = VirtualEnv('2.7')
self += venv
venv += Package('Sphinx', version='1.1.3')
class batou.lib.python.Package(package)

Install the Python package with the given name into the virtualenv of the parent component. Using Package requires that it is added to a VirtualEnv instance.


The version of the package to install (required).


List of options that are passed to pip/easy_install on the command line.

[Default: depends on the Python/virtualenv version in use]


Verify that the package is installed by trying to import it (more precisely, the first component of its dotted name). This is a stopgap against, but should be pretty safe to disable if it causes trouble for specific packages (distribute is a notable example, since it installs a Python module named setuptools).

[Default: True]


A timeout (in seconds) that the installer should use to limit stalling network activity.

Only works when using pip.

[Default: equal to the environment’s timeout setting]


Whether only the package itself or its dependencies should be installed.

[Default: True]


batou has in-depth support for managing installations that use buildout. It automatically wraps them in a virtualenv, installs the appropriate buildout version, and takes care of running buildout whenever changes to configuration files makes it necessary. A typical usage example:

self += Buildout(python='2.7', version='2.2', setuptools='1.0',
                 additional_config=[Directory('profiles', source='profiles')])
class batou.lib.buildout.Buildout

Manage a buildout installation


Python version (required)


Full path to the python executable to create the virtualenv for (used instead of pythonX.Y).


Version of zc.buildout to install (required)


Version of setuptools to install into the virtualenv (must be appropriate to the buildout version, e.g. since 2.2 buildout requires setuptools, but some versions before that required distribute) (required)


Version of distribute to install into the virtualenv. Mutually exclusive with setuptools, of course.


If a different configuration file name than buildout.cfg should be used, pass in a File or Component instance.


Optional list of component instances (e.g. File or Directory) that contain further configuration files (so Buildout knows when running buildout is needed).

Downloads and VCS checkouts

Downloading files

batou supports downloading files via HTTP(S) or FTP, for example:

self += Download(

Download from the given URL (uses urllib or requests internally).


Keyword arguments to pass to requests get method, e.g. to support authentication.


Checksum of the file to be verified (required). Must be given in the format algorithm:value, where algorithm must be a function of the hashlib stdlib module.


Filename to save the download as. [Default: last component of the URL]


self += Clone('', revision='tip')
class batou.lib.mercurial.Clone(url)

Clone a Mercurial repository from the given URL.


Which revision to clone. At least one of revision or branch is required. If both are given, revision will be used.


The name of a branch to clone. At least one of revision or branch is required. If both are given, branch will be overridden by revision. A clone of a named branch will be updated to the most recent upstream revision of the branch whenever batou is run.


Path to clone into (Default: workdir of parent component)


Whether to update the clone with incoming changesets (Default: True). Leaving clones of source code unchanged is often desirable during development.


self += Clone('', revision='HEAD')
class batou.lib.git.Clone(url)

Clone a Git repository from the given URL.


Path to clone into (Default: workdir of parent component)


Update the clone on each batou run. If False, the repository is cloned once and then never updated again. [Default: False]


git.Clone does not support specifying a revision yet.


self += Checkout('', revision='130345')
class batou.lib.svn.Checkout(url)

Check out a Subversion repository from the given URL.


Which revision to check out (required)


Path to clone into (Default: workdir of parent component)

Building software

batou has some support for downloading and compiling software packages, aka the configure-make-make install (CMMI) dance. Example usage:

self += Build(
class batou.lib.cmmi.Build(url)

Download archive from url, extract it and run CMMI on it.

Checksum for download (see for details)

Path to use as prefix for the installation (passed to configure --prefix) [Default: workdir of parent component]


String of additional arguments to pass to configure.


Dict of variables to add to the environment during all CMMI invocations.