Component Python API

This part of the documentation covers all the interfaces of batou you can use to develop your deployments.

Component

class batou.component.Component(namevar=None, **kw)

A component that models configuration and can apply it.

Use sub-classes of Component to create custom components.

The constructor takes one un-named argument which is assigned to the attribute set by the namevar class attribute.

The remaining keyword arguments are set as object attributes.

If a component is used as a sub-component (via +=), then the constructor arguments sets the object attributes. If a component is used directly from an environment (becoming a root component) then the constructor is called without arguments and overrides from the environment and the secrets are set through an internal mechanism.

namevar = None

The namevar attribute specifies the attribute name of the first unnamed argument passed to the constructor.

This helps making components more readable by providing one “natural” first argument:

class File(Component):

    namevar = 'filename'

    def configure(self):
       assert self.filename

class Something(Component):

    def configure(self):
        self += File('nginx.conf')

workdir: str = None

(readonly) The workdir attribute is set by batou when a component before a component is configured and defaults to <root>/work/<componentname>. Built-in components treat all relative destination paths as relative to the work directory.

During verify() and apply() batou automatically switches the current working directory to this.

property defdir

(readonly) The definition directory

(where the component.py lives).

Built-in components treat all path names of source files as relative to the definition directory.

property host

(readonly) The Host object this component is configured for.

property environment

(readonly) The Environment object this component is configured for.

property root

(readonly) The RootComponent object this component is configured for.

configure()

Configure the component by computing target state and declaring sub-components.

This is the “declarative” part of batou – consider a rather functional approach to implementing this.

Perform as much preparatory computation of target state as possible so that batou can perform as many checks as possible before starting to modify target systems.

Sub-components are added to this component by using the += syntax:

class MyComponent(Component):

    def configure(self):
        self += File('asdf')
        self += File('bsdf')
        self += File('csdf')

The order that sub-components will be worked on is given by the order of the += assignments.

Warning

configure must not change state on the target systems and should only interact with the outside system in certain situations. It is not guaranteed whether this method will be called on the host running the master command, or on any number of the target systems.

configure can be called by batou multiple times (with re-initialized attributes) for batou to automatically discover correct order.

Consequently, it is wise to keep computational overhead low to ensure fast deployments.

Warning

Using functions from :py:module:random can cause your configuration to be come non-convergent and thus cause unnecessary, repeated updates. If you like to use a random number, make sure you seed the random number generator with a predictable value, that may be stored in the environment overrides or secrets.

verify()

Verify whether this component has been deployed correctly or needs to be updated.

Raise the batou.UpdateNeeded exception if the desired target state is not reached. Use the assert_*() methods (see below) to check for typical conditions and raise this exception comfortably.

This method is run exactly once on the target system when batou has entered the deployment phase. The working directory is automatically switched to the workdir.

update()

Update the deployment of this component.

update is called when verify has raised the :py:class:UpdateNeeded exception.

When implementing update you can assume that the target state has not been reached but you are not guaranteed to find a clean environment. You need to take appropriate action to move whatever state you find to the state you want.

We recommend two best practices to have your components be reliable, convergent, and fast:

1. Create a clean temporary state before applying new state. (But be careful if you manage stateful things like database directories or running processes.)

2. If update and verify become too complicated then split your component into smaller components that can implement the verify/update cycle in a simpler fashion.

   verify and update should usually not be longer than a few lines of code.

last_updated()

When this component was last updated, given as a timestamp ( seconds since epoch in local time on the system).

You can implement this, optionally, to help other components that depend on this component to determine whether they should update themselves or not.

__enter__()

Enter the component’s context.

Components are context managers: the context is entered before calling verify and left after calling update (or after verify if no update was needed).

This can be used to perform potentially expensive or stateful setup and teardown actions, like mounting volumes.

See Python’s context manager documentation if you want to know more about this mechanism.

__exit__(type, value, tb)

Exit the component’s context.

log(message, *args)

Log a message to console during deployment.

The message is %-substituted with *args, if it is put out, and prefixed with the hostname automatically.

Use this message to add additional status to the deployment output, i.e. “Deploying Version X”.

Note

During configure() log messages are not put out immediately but only after the configure phase is done, because configure() is called multiple times. Only the logs of the last call are put out.

In verify() and update() messages are put out immediately.

__add__(component)

Add a new sub-component.

This will also automatically prepare the added component if it hasn’t been prepared yet. Could have been prepared if it was configured in the context of a different component.

__or__(component)

Prepare a component in the context of this component but do not add it to the sub components.

This allows executing ‘configure’ in the context of this component

provide(key, value)

Provide a resource.

Parameters:

• key (str) – They key under which the resource is provided.

• value (object) –

  The value of the resource.

  Resource values can be of any type. Typically you can pass around component objects or individual configuration values, like network addresses, or similar.

require(key, host=None, strict=True, reverse=False, dirty=False)

Require a resource.

Parameters:

• key (str) – The key under which the resource was provided.

• host (object) – The host object that the provided resource belongs to.

• strict (bool) – If true, then it is an error if no resources were provided given the required key.

• reverse (bool) – By default a component that requires another one also depends on the one that provides a resource. If reverse is set to True then this dependency is reversed and the component that provides a resource depends on the component requiring it.

• dirty (bool) –

  When a component requires a resource then it will normally be configured again when another component is configured later that changes the list of resources that were required.

  Under very special circumstances it may be necessary to not get reconfigured when the required resource changes to break cycles in dependencies. Use with highest caution as this can cause your components to have incomplete configuration.

Returns:

The matching list of resources that were provided.

Return type:

list

Note

Calling require may cause an internal exception to be raised that you must not catch: batou uses this as a signal that this component’s configuration is incomplete and keeps track of the desired resource key. If another component later provides this resource then this component’s configure will be run again, causing require to complete successfully.

require_one(key, host=None, strict=True, reverse=False, dirty=False)

Require a resource, returning a scalar.

For the parameters, see require().

Returns:

The matching resource that was provided.

Return type:

object

This version returns a single value instead of a list. Also, if the number of potential results is not exactly one, then an error will be raised (which you should not catch). batou will notify you of this as being an inconsistent configuration.

assert_cmd(*args, **kw)

Assert that given command returns successfully, raise UpdateNeeded otherwise.

For details about the command arguments and what a successful execution means, see batou.component.Component.cmd().

assert_file_is_current(reference, requirements=[], **kw)

Assert that the file given by the reference pathname has been created or updated after the given list of requirement file names, raise UpdateNeeded otherwise.

Parameters:

• reference (str) – The file path you want to check for being current.

• requirements (list) – The list of filenames you want to check against.

• kw (dict) – Arguments that are passed through to last_update which can be used to use different time stamps than st_mtime. See batou.lib.file.File.last_updated() for possible values.

Returns:

None, if reference is as new or newer as all requirements.

Raises:

UpdateNeeded – if the reference file is older than any of the requirements.

assert_component_is_current(requirements=[], **kw)

Assert that this component has been updated more recently than the components specified in the requirements, raise UpdateNeeded otherwise.

Parameters:

• requirements (list) – The list of components you want to check against.

• kw (dict) – Arguments that are passed through to each last_update call. The semantics depend on the components’ implementations.

Returns:

None, if this component is as new or newer as all requirements.

Raises:

UpdateNeeded – if this component is older than any of the requirements.

The age of a component is determined by calling last_updated on this and each requirement component.

assert_no_subcomponent_changes()

Assert that, during this run of batou, non of this components’ sub-components have required an update.

Returns:

None, if none if this components’ sub-components have required an update during this run of batou.

Raises:

UpdateNeeded – if any of this components’ sub-components have required an update during this run of batou.

Note

Using this change indicator can be unreliable if you fail to perform your update correctly. It is likely that when later resuming an aborted deployment this change won’t be triggered again.

assert_no_changes()

Assert that, during this run of batou, neither this component nor any of its sub-components have required an update.

Returns:

None, if neither this component nor any of its sub-components have required an update during this run of batou.

Raises:

UpdateNeeded – if this component or any of its sub-components have required an update during this run of batou.

Note

Using this change indicator can be unreliable if you fail to perform your update correctly. It is likely that when later resuming an aborted deployment this change won’t be triggered again.

cmd(cmd, silent=False, ignore_returncode=False, communicate=True, env=None, expand=True)

Perform a (shell) command.

Use this to interact with the target system during verify, update, __enter__, or __exit__.

Warning

Do not use this during configure.

Parameters:

• cmd (str) – The command you want to execute including all arguments. This will be parsed by the system shell, so be careful of quoting.

• silent (bool) – whether output should be shown in the case of errors.

• ignore_returncode (bool) – If true, do not raise an exception if the return code of the command indicates failure.

• communicate (bool) – If True, call communicate() and wait for the process to finish, and process the return code. If False start the process and return the Popen object after starting the process. You are then responsible for communicating, processing, and terminating the process yourself.

• expand (bool) – Treat the cmd as a template and process it through Jinja2 in the context of this component.

• env (dict) – Extends environment variables with given ones.

Returns:

(stdout, stderr) if communicate is True, otherwise the Popen process is returned.

Raises:

CmdExecutionError – if return code indicated failure and ignore_returncode was not set.

map(path)

Perform a VFS mapping on the given path.

If the environment has VFS mapping configured, compute the new path based on the mapping.

Whenever you get a path name from the outside (i.e. environment overrides or from the constructor) or use absolute paths in your configuration, you should call map as early as possible during configure. If you are using :py:class:batou.component.Attribute for constructor arguments or overrides, then you can specify map on the attribute to avoid having to map this yourself.

You should rely on other components to do the same, so if you pass a path to another component’s constructor, you do not have to call map yourself.

touch(filename)

Built-in equivalent of the touch UNIX command.

Use during verify, update, __enter__, or __exit__, to interact with the target system.

Warning

Do not use during configure.

expand(string, component=None, **kw)

Expand the given string in the context of this component.

When computing configuration data, you can perform inline template expansions of strings. This is an alternative to Python’s built-in string templates, to keep your inline configuration in sync with the external file templating based on Jinja2.

The following variables are available in the template context:

• host: The host object this component is configured for.

• environment: The environment object this component is configured for.

• component: The component object this component is configured for.

• batou_generated_header: A string that can be used to mark

  generated files.

Parameters:

• string (unicode) – The string you want to be expanded as a Jinja2 template.

• component (batou.component.Component) – By default this self. To perform the template expansion in the context of another component you can pass it through this argument (or call the other component’s expand).

• kw (dict) – Additional keyword arguments are passed into the template’s context as global names.

Returns:

the expanded template.

Return type:

unicode

template(filename, component=None)

Expand the given file in the context of this component.

Instead of using the File component to expand templates, you can expand a file and receive a unicode string (instead of directly rendering the file to a target location).

The following variables are available in the template context:

• host: The host object this component is configured for.

• environment: The environment object this component is configured

  for.

• component: The component object this component is configured for.

• batou_generated_header: A string that can be used to mark

  generated files.

Parameters:

• filename (str) – The file you want to expand. The filename is not mapped by this function. Map the filename before calling template if needed.

• component (batou.component.Component) – By default this self. To perform the template expansion in the context of another component you can pass it through this argument (or call the other component’s expand).

• kw (dict) – Additional keyword arguments are passed into the template’s context as global names.

Returns:

the expanded template.

Return type:

unicode

chdir(path)

Change the working directory.

Use this to interact with the target system during verify, update, __enter__, or __exit__.

Warning

Do not use this during configure.

The given path can be absolute or relative to the current working directory. No mapping is performed.

This is a context mapper, so you can change the path temporarily and automatically switch back:

def update(self):
    with self.chdir('/tmp'):
        self.touch('asdf')

class batou.component.HookComponent(namevar=None, **kw)

A component that provides itself as a resource.

configure()

Configure the component by computing target state and declaring sub-components.

This is the “declarative” part of batou – consider a rather functional approach to implementing this.

Perform as much preparatory computation of target state as possible so that batou can perform as many checks as possible before starting to modify target systems.

Sub-components are added to this component by using the += syntax:

class MyComponent(Component):

    def configure(self):
        self += File('asdf')
        self += File('bsdf')
        self += File('csdf')

The order that sub-components will be worked on is given by the order of the += assignments.

Warning

configure must not change state on the target systems and should only interact with the outside system in certain situations. It is not guaranteed whether this method will be called on the host running the master command, or on any number of the target systems.

configure can be called by batou multiple times (with re-initialized attributes) for batou to automatically discover correct order.

Consequently, it is wise to keep computational overhead low to ensure fast deployments.

Warning

Using functions from :py:module:random can cause your configuration to be come non-convergent and thus cause unnecessary, repeated updates. If you like to use a random number, make sure you seed the random number generator with a predictable value, that may be stored in the environment overrides or secrets.

class batou.component.RootComponent(name, environment, host, features, ignore, factory, defdir, workdir, overrides=None)

Wrapper to manage top-level components assigned to hosts in an environment.

Root components have a name and determine the initial working directory of the sub-components.

batou.component.platform(name, component)

Class decorator to register a component class as a platform-component for the given platform and component.

batou.component.handle_event(event, scope)

Attribute (TODO)

class batou.component.Attribute(conversion=<class 'str'>, default=<object object>, expand=True, map=False)

An attribute descriptor is used to provide:

• declare overrideability for components

• provide type-conversion from overrides that are strings

• provide a default.

Conversion can be given as a string to indicate a built-in conversion:

literal - interprets the string as a Python literal list - interpretes the string as a comma separated list

If conversion is a callable the callable will be used for the conversion, when the value is read from config file. On a setattr the conversion is not applied.

The obj is expected to be a Component so that ‘expand’ can be accessed.

Parameters:

• conversion (str, callable) – A conversion callable which takes one parameter or a string for built-in conversion (literal or list). This function is used for strings from config files.

• default – The default value for the Attribute. When a

ConfigString value is passed then it will expanded, mapped, and passed through the conversion function, depending on the other arguments. :type default: None

Parameters:

• expand (bool) – Expand the config string in the context of this component.

• map (bool) – Perform a VFS mapping on the config string.

from_config_string(obj, value)

Perform expansion, mapping and conversion after another.

Host (TODO)

class batou.host.Host(name, environment, config={})

Environment (TODO)

class batou.environment.Environment(name, timeout=None, platform=None, basedir='.', provision_rebuild=False, check_and_predict_local=False)

An environment assigns components to hosts and provides environment-specific configuration for components.