mini cl

A suite of minimal cluster management tools, meant for the situation where a small, tightly knit group of people needs to manage a small number of homogeneous machines for a specific project.

The intent is to minimize infrastructural overhead, making it possible to control a small cluster of machines using just a centralized git repository for coordination, but nevertheless offering a reasonably decent set of features.

The suite provides a minimal base system, on top of which you can setup your services, possibly taking advantage of the default configuration management system that minicl uses (or something else entirely, if you so prefer).

Features

The base system offers a number of boring but useful features which were deemed necessary for the basic operation of a distributed service:

• it offers a configuration management system to manage your services. This is Slack, a simple and effective solution with a very low barrier to entry (it's basically a glorified rsync, with pre- and post-install scripts);

• firewall autoconfiguration to allow simple IP-based network ACLs based on your cluster config;

• centralized control of users and credentials;

• basic monitoring infrastructure.

It is expected that the cluster will use a DNS domain that you control. One entry per host will need to be created under that domain, plus an additional entry for the special monitor host.

Usage

minicl uses a centralized git repository to distribute the cluster configuration. Configuration is split into two separate parts:

• a cluster configuration file, which configures global attributes of the setup, and some specific features of the base system.
• the specific configuration for your services: minicl uses slack to manage the base system, so you can start defining new slack roles right away.

Setup your repository

Set up a git repository that will host your cluster configuration. Unless you are just testing things, this repository should only be available to the admins via SSH, as it will contain private credentials for your cluster.

To begin, create the repository locally:

$ git init myrepo && cd myrepo

This will be the root of your slack repository, which means that it will need to have a subdirectory named roles where the actual service roles are defined:

$ mkdir roles

The minicl base system is itself a slack role, which will need to be part of this repository as well. This is done by importing the minicl repository using a git submodule:

$ git submodule add https://git.autistici.org/ale/minicl.git roles/base

The role must be called base. On top of that, you are free to define your own roles and add them to the roles directory.

The next step is to set up the local pre-commit git hook, which is used to verify that your cluster configuration file is valid before you push it to the repository:

$ cp roles/base/pre-commit-hook .git/hooks/pre-commit

You will now need to create the cluster configuration file, which should be saved in the file cluster.json in the top-level directory of the repository. Check the Configuration section for details on what it should contain.

Once this is done, commit all changes with:

$ git add cluster.json deploy.key
$ git commit -m 'initial commit'
$ git push

Configuration

Cluster configuration file

The cluster.json configuration file contains global attributes of the cluster that are required for its operation. It is a key/value dictionary encoded in JSON format.

The following top-level attributes are understood:

• domain: the domain that defines the cluster identity. All names will be based on this domain. It should be a real DNS domain that you control, as it will be necessary to create records for the hosts there.
• admin_email: all root email will be sent to this address, which can not be left empty.
• ca_url: (optional) a URL for a X509 Certification Authority certificate that should be installed system-wide, for example for access to SSL-protected services (including git itself).
• smtp: an object specifying the SMTP relay parameters. Email from hosts in the cluster is sent out using authenticated SMTP with the credentials specified here. The following attributes must be specified:
  • relay: host name of the SMTP relay
  • username
  • password
• users: the list of users that should be present on each host. minicl will distribute the authentication credentials of all users listed here on every cluster host (no further level of ACL is provided). Each user should have the following attributes:
  • uid: numeric UID for this user. Pick them so that they are unique and do not overlap with system users.
  • passwd: password, encoded with the system's crypt() method. These will end up directly in the shadow file.
  • ssh_key: public SSH key for this user. This can be an array, if the user has more than one SSH key. Each user must have at least one SSH key, as password-based remote access is disabled.
• hosts: list of hosts that are part of the cluster. This is an object whose keys are the short names (i.e. minus domain) of the hosts, and whose values are objects with the following attributes:
  • ip: one or more public IP addresses for this host (IPv4 or IPv6).
  • roles: Slack roles assigned to this host. The list should contain the base role.

Here is an example of a valid, two-host cluster configuration:

{
    "domain": "example.com",
    "admin_email": "cluster-alerts@example.com",
    "users": {
        "foo": {
            "uid": 1001,
            "passwd": "$6$NSl9Lz3f$...",
            "ssh_key": "ssh-rsa AAAAB3Nz..."
        }
    },
    "hosts": {
        "host1": {
            "ip": "1.2.3.4",
        },
        "host2": {
            "ip": "2.3.4.5",
        }
    },
    "smtp": {
        "relay": "smtp-relay.example.com",
        "username": "clustersmtp",
        "password": "..."
    }
}

This defines two hosts (host1 and host2), and a user named foo with administrative access to all machines.

Deploy key

Since the primary configuration repository is supposed to be protected by SSH access, the hosts should be provided with the necessary credentials in the form of a SSH key.

The private key should be stored in the repository itself, in a file named deploy.key in its top-level directory.

Keeping up with upstream changes

If in the future you'll want to update the minicl codebase, you can run the following command:

$ git submodule update --remote

This will result in a change of roles/base that you will need to commit.

Installing a server

Adding a host to the cluster is relatively straightforward. Minicl won't attempt to wipe an existing installation, so we are going to assume that the server is a Debian system that has been just installed, it is currently running no services, but the hardware and networking have been properly set up already.

In order to bootstrap this new server, it will need two pieces of information:

• the URL of the configuration repository;
• a way to get access to it, either via SSH agent forwarding of an administrator's credentials, or by manually copying the deploy key on the server.

Once you have those, here's the one-liner for those who like to live dangerously (to be run as the root user):

$ wget --no-check-certificate -O- \
    https://git.autistici.org/ale/minicl/raw/master/install.sh | sh \
    && minicl-bootstrap --repo=$REPO

The first command will set up the minicl software suite, while the second will perform the initial setup and install the cluster configuration.

You can use the --deploy-key option to minicl-bootstrap if you have manually copied the deploy key to the new host.

At the end of the process, minicl-bootstrap will print out the JSON snippet that you should add to the host entry in your cluster.json file.