autoradio Tuning Guide
This document attempts to provide a high-level overview of the trade-offs involved in tuning the free parameters of an autoradio cluster. While autoradio works with the default settings out of the box in testing environments, most real-world deployments will require some tuning.
Etcd
The default settings for etcd are tuned for a local (LAN) network environment. In the case of a geographically distributed cluster, the default timeouts are so low that it's unlikely that a consensus will ever be reached. You'll want to set both the peer heartbeat interval and the election timeout to higher values. A reasonable value for the heartbeat interval is 5x to 10x the maximum inter-node latency in your cluster, while the election timeout should be at least 3 times the heartbeat interval.
With our etcd package, you can set these values in
/etc/default/etcd (values are milliseconds):
DAEMON_OPTS="--heartbeat-interval=1000 --election-timeout=3000"Increasing the etcd timeouts causes a related increase in the time required to reach consensus and elect a new etcd master in case of node failure. It is advisable that the radiod master election ttl is set to a value greater than the etcd peer election timeout.
Radiod timeouts
Similar considerations, with respect to latency, apply to the presence
and master-election protocols that are run by autoradio itself. These
are controlled by radiod's --heartbeat and --master-election-ttl
command-line flags. For these time values, though, there are further
considerations to be made: