automating web application deployments
batou is a BSD-licensed utility, written in Python, to configure development and production environments for web applications.
Deployments are hard and complicated. Tools like Docker, Puppet, chef, and others exist that try to solve this problem once and for all. However, they usually need you to change your workflow or toolchain massively while still missing important steps.
batou makes deployments more bearable without requiring developers to change their applications. It provides a “one command” approach that should never need additional wrapper scripts.
As a developer all you ever have to run after cloning or updating your project is:
$ git clone https://github.com/myorg/myproject $ cd myproject $ ./batou deploy dev
To run a production deployment all you should ever have to run is:
$ cd my-project $ git pull $ ./batou deploy prod
Writing a deployment with batou is a two step process:
Step 1: Model your application’s configuration¶
With our component model you write a configuration specification in Python based on a simple API. Components make configuration convergent and idempotent and using Python lets you perform any computation you need. The component model is recursive, so you can refactor complicated components into simpler ones without breaking your setup.
Here is an example application model that installs a Python package into a VirtualEnv and asks Supervisor to run it:
from batou.component import Component from batou.lib.python import VirtualEnv, Package from batou.lib.supervisor import Program class MyApp(Component): def configure(self): venv = VirtualEnv('3.5') self += venv venv += Package('myapp') self += Program('myapp', command='bin/myapp')
Step 2: Fit your model to your environments¶
Your model from step 1 is abstract: it does not mention the names of the servers you deploy to.
By describing an environment you tell batou how your abstract model should actually be applied: on your local development machine, to a vagrant setup, or on servers on the network.
Here’s an environment specification that sets up an application on multiple hosts and provides an override for the publicly visible address.
[environment] host_domain = fcio.net [host:host01] components = nginx, haproxy, varnish [host:host02] components = myapp [host:host03] components = myapp [host:host04] components = postgresql [component:nginx] server_name = staging.example.com
Run the same command to deploy locally, to Vagrant, or to remote clusters.
Use different versions of batou in different projects. batou automatically ensures everyone uses the correct version in each project and updates when needed.
Check before deploying whether your configuration is internally consistent and consistent with what has been deployed before.
Predict changes and predict what further changes will be triggered.
Convergent, idempotent components are fast to deploy.
Resume partial deployments where they were aborted.
Store database passwords, SSH keys, SSL certificates or other secret data with on the-fly decryption. Manage access to secrets per environment and user.
Use Jinja2 templates to easily create dynamic configuration.
Dynamically connect services during deployments and track their dependencies.
Few run-time requirements on your servers: only Python 2.7 and SSH are needed.
Use pre-defined components to manage files, python environments, supervisor, cronjobs, and more.
Writing your own components is easy and you can use additional Python package dependencies.
This part of the documentation, begins with some background information about batou, then focuses on the basic parts needed on a regular basis when creating and maintaining increasingly complex deployments with batou. The last part of the narrative documentation covers selected topics in depth.
- Foundations for creating and maintaining deployments
- Create a new project
- Writing a component configuration
- Local environments
- Vagrant environments
- Remote environments
- Overriding configuration per environment
- Templating from files
- Storing secrets as encrypted overrides
- Storing additional secrets as separate files
- Using version control to ensure consistent deployments
- Downloading and building software
- Managing Python environments with VirtualEnv and Pip
- Managing Python environments with zc.buildout
- Registering programs with supervisor
- Working with network addresses
- Registering and discovering services
- Checking a deployment configuration before running it
- Predicting the changes a deployment will cause
- Updating batou in an existing project
- In-depth topics
- Writing a custom component (TODO)
- Debugging batou runs
- Using 3rd party libraries within batou
- Multiple components in a single component.py (TODO)
- Skipping individual hosts or components when deploying (TODO)
- Events (TODO)
- Using bundle transfers if the repository server is not reachable from your remote server (TODO)
- Timeout (TODO)
- VFS mapping for development (TODO)
- VFS mapping with explicit rewrite rules (TODO)
- Extended service discovery options (TODO)
- Platform-specific components
- Host-specific data
- DNS overrides
- context manager (TODO)
- last_updated (TODO)
- workdir overriding (TODO)
- Importing components from a different component.py
- Documentation for batou secrets with age.
Reference: Command line interface¶
If you are looking for information on what commands the batou CLI provides then this is for you.
Reference: Component Library¶
This is the list of components that batou provides – builtin and through the batou_ext package:
- Managing files and directories
- Downloads and VCS checkouts
- Building software
- Managing python installations
- Managing services
Reference: Python API¶
If you are looking for information on a specific function, class or method, this part of the documentation is for you.
If you want to contribute to the project, this part of the documentation is for you.
batou itself is released “as is”. You can also report bugs to our bugtracker.
We also have a matrix channel: #batou:matrix.org.
We will be happy to give you commercial support for batou: feature implementation, bug fixing, consulting, or education.
To get in touch, send an email to firstname.lastname@example.org and we’ll be happy to discuss your requirements.
- 2.4rc2 (unreleased)
- 2.4rc1 (2023-09-12)
- 2.4a4 (2023-07-25)
- 2.4a3 (2023-07-17)
- 2.4a2 (2023-07-07)
- 2.4a1 (2023-06-27)
- 2.3 (2023-01-24)
- 2.3b6 (2022-12-09)
- 2.3b5 (2022-09-21)
- 2.3b4 (2022-08-22)
- 2.3b3 (2021-11-30)
- 2.3b2 (2021-10-05)
- 2.3b1 (2021-05-21)
- 2.2.4 (2021-02-11)
- 2.2.3 (2021-01-20)
- 2.2.2 (2020-12-14)
- 2.2.1 (2020-12-14)
- 2.2 (2020-12-10)
- 2.1 (2020-09-09)
- 2.0 (2020-07-15)
- 2.0b14 (2020-06-25)
- 2.0b13 (2020-06-25)
- 2.0b12 (2020-05-13)
- 2.0b11 (2020-05-13)
- 2.0b10 (2020-05-11)
- 2.0b9 (2020-05-09)
- 2.0b8 (2020-05-08)
- 2.0b7 (2020-05-07)
- 2.0b6 (2020-04-24)
Presentations from conferences: