ai3 Test Suite

We need to be able to test arbitrary combinations of float / the ai3 configuration / individual service releases, so we've created a simple framework to programmatically generate test environments.

Furthermore, since the ai3 configuration is very large, and we're generally concerned about minimizing the time it takes to set up a test environment, we have created a number of "base" test configurations which only include specific subsets of the ai3 services.

To reduce the parameter space, the inventory always consists of 3 hosts, one in the frontend group, two in backend.

This test framework combines:

• a specific float version
• test data

as an alternative to using git submodules, which are cumbersome to manage programatically. Instead, repositories are cloned every time (though specific branches might be requested via the FLOAT_BRANCH and TESTDATA_BRANCH environment variables).

A consequence of this setup is that the float repository path will change on every invocation, as repositories are cloned to a temporary directory, so we have to edit the test skeleton YAML files to point at the float location. These YAML files contain the special variables $FLOAT_DIR and $CONFIG_DIR which are replaced by the mkenv script before invoking "float create-env". The consequence is that the environment configurations in the env directory can't be used directly with "float create-env".

Setup iniziale

Per testare l'ambiente locale di test di ai3 (Vagrant), bisogna prima installare alcuni prerequisiti. Anzitutto serve virtualenv, per es. su una macchina Debian:

$ sudo apt install python3-virtualenv

Dopodiché bisogna creare una directory venv con un'installazione locale di Python (per evitare di "sporcare" il sistema con le dipendenze della suite di test):

$ virtualenv --python python3 venv

Infine bisogna installare la test suite dentro questo ambiente Python temporaneo:

$ ./venv/bin/python3 setup.py develop

Test

Per lanciare i test, bisogna passare il file di configurazione ai3 di Ansible (quello cioè passato come inventory) attraverso la variabile di ambiente TEST_CONF. Questo punta la test suite agli host giusti, relativamente ad una particolare configurazione usata con Ansible. Per esempio, per testare l'ambiente locale Vagrant di default:

$ TEST_CONF=../../conf/config-vagrant.yml ./venv/bin/nosetests -v .

L'integrazione con 'float' avviene tramite un playbook apposito (ai3-test):

$ ANSIBLE_STDOUT_CALLBACK=minimal ../../float/float run ../../config/playbooks/ai3-test.yml