In-depth topics¶
Writing a custom component (TODO)¶
Debugging batou runs¶
Using a debugger¶
batou comes with remote-pdb
pre-installed. When running on Python 3.7+ [1] you can use breakpoint() to
drop into the debugger. You need telnet or netcat to connect to the
hostname and port displayed.
If you are using the default configuration, call:
$ nc -C 127.0.0.1 4444
or
$ telnet 127.0.0.1 4444
If you are debugging a remote deployment you should create a port forward beforehand like this:
$ ssh -L 4444:localhost:4444 my.remote.host.dev
You are able to configure hostname and port. For details see the documentation of remote-pdb. This works both for local and remote deployments. The environment variables for host and port are propagated to the remote host.
Example shell session for debugging using a custom port:
$ REMOTE_PDB_PORT=4445 ./batou deploy dev
batou/2.3b2.dev0 (cpython 3.6.15-final0, Darwin 20.6.0 x86_64)
============================= Preparing ============================
main: Loading environment `dev`...
main: Verifying repository ...
main: Loading secrets ...
============================= Connecting ... =======================
localhost: Connecting via local (1/1)
============================= Configuring model ... ================
RemotePdb session open at 127.0.0.1:4445, waiting for connection ...
Example for a second terminal where the opened port gets connected to:
$ nc -C 127.0.0.1 4445
> /.../components/supervisor/component.py(32)configure()
-> if self.host.platform in ('gocept.net', 'fcio.net'):
(Pdb)
Using “print debugging”¶
Inside a component you can use
self.log("output")to print to the console.You can call
batouwith the-dflag to enable debugging output during the deployment run.
Using 3rd party libraries within batou¶
Sometimes, when writing custom components, you may need additional Python packages, for example to configure databases by connecting directly to their SQL interface instead of using their command line clients.
You can use additional Python packages by adding a requirements.txt file to your batou project repository:
$ tree
.
├── batou
├── components
│ └── myapp
│ └── component.py
├── environments
│ └── local
│ └── environment.cfg
└── requirements.txt
sqlalchemy
The next time when you call batou the dependencies will be automatically updated. When deploying then the requirements will also be installed on the remote hosts.
$ ./batou
Installing sqlalchemy
usage: batou [-h] [-d] {deploy,secrets,init,update} ...
Note
batou already provides a number of packages that it depends on. If you create contradicting requirements then this may lead to batou failing. You will see pip complaining in that case.
Multiple components in a single component.py (TODO)¶
Skipping individual hosts or components when deploying (TODO)¶
Events (TODO)¶
Using bundle transfers if the repository server is not reachable from your remote server (TODO)¶
Timeout (TODO)¶
VFS mapping for development (TODO)¶
VFS mapping with explicit rewrite rules (TODO)¶
Extended service discovery options (TODO)¶
Platform-specific components¶
New in version 1.4.
Platform-specific components allow to customize behavior depending on the system or “platform” the target system runs as. Examples:
Production system on Gentoo, local development on Ubuntu, or
All VMs on Ubuntu but Oracle is being run with RedHat.
To define a platform specific aspects, you use the platform class decorator. Example:
import batou.component
import batou.lib.file
class Test(batou.component.Component):
def configure(self):
self += batou.lib.file.File('base-component')
@batou.component.platform('nixos', Test)
class TestNixos(batou.component.Component):
def configure(self):
self += batou.lib.file.File('i-am-nixos')
@batou.component.platform('ubuntu', Test)
class TestUbuntu(batou.component.Component):
def configure(self):
self += batou.lib.file.File('i-am-ubuntu')
The platform is then defined in the environment:
[environment]
platform = default-platform
[host:nixos]
# Host specifc override:
platform = nixos
components = test
[host:ubuntu]
# Host specifc override:
platform = ubuntu
components = test
Host-specific data¶
New in version 1.5.
Host-specific data allows to set environment dependent data for a certain host. It looks like this in an environment configuration:
[host:myhost00]
components = test
data-alias = nice-alias.for.my.host.example.com
In a component you can access all data attributes via the host’s data dictionary:
def configure(self):
alias = self.host.data['alias']
The data- prefix was chosen in resemblance of the HTML standard.
DNS overrides¶
New in version 1.6
When migrating services automatic DNS lookup of IP addresses to listen on can be cumbersome. You want to deploy the service before the DNS changes become active. This is where DNS overrides can help.
The DNS overrides short circuit the resolving completely for the given host names.
Example:
[environment]
...
[resolver]
www.example.com =
3.2.1.4
::2
Whenever batou configuration (i.e. batou.utils.Address) looks up www.example.com it will result in the addresses 3.2.1.4 and ::2.
The overrides support IPv4 and IPv6. You should only set one IP address per type for each host name.
Note
You cannot override the addresses of the configured hosts. The SSH connection will always use genuine name resolving.
context manager (TODO)¶
last_updated (TODO)¶
prepare, |=, component._ (TODO)¶
workdir overriding (TODO)¶
Importing components from a different component.py¶
The component configuration in the ./components folder is not a Python
package: it has no __init__.py and should not have one. That’s why is not
possible import a component into another one:
# This will not work
from components.nginx.component import MyAddress
In some rare circumstances it might be necessary to have this kind of import. There are two options:
Import from
batou.c.Create an extension module which can be imported.
Import from batou.c¶
There is a special module batou.c which is dynamically populated with all
the classes in all component.py files.
Given the following tree:
$ tree
.
├── batou
├── components
│ └── myapp
│ └── component.py
│ └── myconfig
│ └── component.py
├── environments
│ └── local
│ └── environment.cfg
├── requirements.lock
└── requirements.txt
If myconfig/component.py looks like this:
from batou.component import Component
class MyAddress(Component):
"""Some custom address."""
ipv4 = "0.0.0.0"
myapp/component.py could import MyAddress like this:
from batou.component import Component
import batou.c
class MyApp(Component):
"""Some custom address."""
def configure(self):
"""Configure MyApp."""
self.address = batou.c.myconfig.MyAddress(ipv4="127.0.0.1")
self += self.address
self.ip = self.address.ipv4
This way, importing is only possible and reasonable inside methods of the
component, which are executed after initial configuration such as configure
(). It cannot be used to create an attribute on a class or import a base
class for the current component file.
Caution
The components are loaded alphabetically, which can be an issue for the import.
Create an extension module¶
Another option to share code between different component files is to create a
custom extension module. This can either be a separate repository like
batou_ext or batou_scm but for a light weight start it can be
included in the deployment repository. Have a look at the setup.py and
other files for inspiration.
$ tree
.
├── batou
├── batou_myapp
│ ├── setup.py
│ └── src
│ └── batou_myapp
│ └── utils.py
├── components
│ └── myapp
│ └── component.py
├── environments
│ └── local
│ └── environment.cfg
├── requirements.lock
└── requirements.txt
# batou_myapp/src/batou_myapp/utils.py
from batou.component import Component
class MyAddress(Component):
"""Some custom address."""
ipv4 = "0.0.0.0"
class MyAppBase(Component):
"""Base component for all apps."""
# myapp/component.py
from batou.component import Component
from batou_myapp.utils import MyAppBase
from batou_myapp.utils import MyAddress
class MyApp(MyAppBase):
"""Some custom address."""
address = MyAddress(ipv4="127.0.0.1")
def configure(self):
"""Configure MyApp."""
self.ip = self.address.ipv4
The requirements.txt has to be adapted to include the new extension module
as requirement to be installed directly from source.
# requirements.txt
batou==2.3b2
-e ./batou_myapp