radio.ai

The RadioAI service aims to provide a reliable, fault-tolerant Icecast streaming network for audio and video. It provides all the necessary components to ensure that the traffic from the source to the clients is uninterrupted, even in face of high load or server crashes.

It is a full-stack service, meaning that it includes its own DNS and HTTP servers, for full control of the request flow.

RadioAI works by using etcd to coordinate the various nodes and store the global mount configuration. The intended target is a set of homogeneous servers (or virtual machines) dedicated to this purpose. RadioAI also needs a dedicated DNS domain (or a delegation for a subdomain).

Installation

The simplest installation method is probably to use the pre-built Debian packages (only available for amd64 at the moment), by placing this line in /etc/apt/sources.list.d/radioai.list:

deb http://www.incal.net/ale/debian radioai/

And then running:

$ sudo apt-get update
$ sudo apt-get install etcd radioai

This will install and start the necessary jobs (which will initially fail due to the missing configuration).

Edit /etc/default/radioai and set, at least, the DOMAIN variable to what you've assigned to the cluster. The jobs will automatically start as soon as the configuration is saved.

Full cluster install procedure

This assumes that you have an existing domain name (here example.com) that you control, and that you will run the cluster under a sub-domain (radio.example.com). Follow these steps to bootstrap a new streaming cluster:

1. Make sure that, on each of your servers, the output of hostname -f is the fully-qualified hostname of the machine, and that it resolves to its public IP (possibly using /etc/hosts).

2. On every server, run the above-mentioned steps to set up the APT repository and install (do not configure) the etcd and radioai packages.

3. Pick one of your servers and add a delegation for radio.example.com to it. For instance, with bind:

   radio  IN   NS  3600  machine1.example.com.

4. On machine1, edit /etc/default/etcd and set BOOTSTRAP=1. Once you save the file, the etcd daemon will start with an empty database.

5. On machine1, edit /etc/default/radioai and set DOMAIN=radio.example.com. This will start the radiod and redirectord daemons, and you will be able to serve DNS records for the radio.example.com zone. Check with:

   $ ping -c1 radio.example.com

   This should send a ping to machine1.

6. Set up all other machines, setting ETCD_SERVER=etcd.radio.example.com in /etc/default/etcd and DOMAIN=radio.example.com in /etc/default/radioai.

Securing etcd

In a production cluster, you will want to limit access to the etcd daemons so that only the other nodes can connect to it. While it is possible to do this with firewall rules, the dynamic membership of the cluster may make this difficult. We suggest using instead etcd's support for X509 client authentication, together with a tool to manage an online CA (such as autoca). This way, enrolling a new machine in the cluster only requires generating a new client certificate, and no other configuration.

Install the CA certificate in /etc/radioai/etcd_ca.pem, the client certificate in /etc/radioai/etcd_client.pem and its private key in /etc/radioai/etcd_client.key, and the clients will connect to etcd using SSL authentication.

Operation

In order to create a new stream (mount, in the Icecast terminology), assuming you are running RadioAI on the example.com domain:

1. On any node, run:

   $ radioctl create-mount /path/to/mount.ogg

   this will output the username and password used to authenticate the source. Take note of them.

   The cluster will be automatically reconfigured with the new mount in a few seconds at most.

2. Configure the source, using the username/password provided in the previous step, and point it at the following URL:

   http://stream.example.com/path/to/mount.ogg

3. Tell your users to listen to the stream at:

   http://stream.example.com/path/to/mount.ogg.m3u

DNS zone delegation

Since we can't modify the DNS glue records for the zone delegation in real-time, we have to restrict slightly the assumptions on the availability of nodes in the cluster: you have to assume that at least N of your nodes will be partially available at any one time (i.e. at least one of a chosen subset of N servers will be reachable). The number N should be fairly low, say 3. Then, you can use those 3 servers as the nameservers for zone delegation, and the other nodes are free to have dynamic membership.