Skip to content
Snippets Groups Projects
README.md 3.59 KiB
Newer Older
ale's avatar
ale committed
build-deb
===

ale's avatar
ale committed
A set of Docker images used to build Debian packages and push them to
a remote repository, for multiple target distributions and
architectures.

The resulting images are huge (>500MB), but can be successfully used
to build packages and push them to a remote repository (which must be
ale's avatar
ale committed
using [urepo](https://git.autistici.org/ai3/tools/urepo)).
ale's avatar
ale committed

ale's avatar
ale committed
A base [Gitlab CI configuration template](ci-common.yml) is provided,
ale's avatar
ale committed
to be included in other projects' *.gitlab-ci.yml* files.
ale's avatar
ale committed

For package upload, the following environment variables should be set:

* `REPOSITORY_SSH_PRIVATE_KEY` is the private SSH key to use for authentication
ale's avatar
ale committed
* `REPOSITORY_URI` is the repository target hostname (or user@hostname)
ale's avatar
ale committed

By default, the CI scripts are set up to upload packages to different
repositories based on the Debian release they've been built for
(repositories float/*codename*), but it is possible to override the
repository name autodetection with another environment variable:

* `REPOSITORY_NAME` is the urepo repository name

ale's avatar
ale committed
# Usage

In order to maintain centralized control over the package build
process for a large number of projects, we make use of Gitlab CI's
*include* functionality.

At the very minimum, a new project should set up its *.gitlab-ci.yml*
file like the following:

```yaml
godog's avatar
godog committed
include: "https://git.autistici.org/ai3/build-deb/raw/master/ci-common.yml"
ale's avatar
ale committed
```

The default template already includes the *test* (empty, no jobs
defined here by default), *build* and *release* stages, so you can
directly add jobs to them:
ale's avatar
ale committed

```yaml
godog's avatar
godog committed
include: "https://git.autistici.org/ai3/build-deb/raw/master/ci-common.yml"
ale's avatar
ale committed

test:
  stage: test
  script: tox -e py3
ale's avatar
ale committed
```

# Release policy

We have a few knobs to tweak in order to facilitate the process of
following the current Debian stable release when it changes. These
are:

* The ability to build packages for different releases at once, and
  upload them to different repositories. At any given moment, we
  shouldn't need more than two (either *stable* and *nextstable*, or
  *oldstable* and *stable*).
* The ability to decide which release build should cause pipelines to
  fail, i.e., which releases are "important" and which aren't. Using
  this we can do gentle phased roll-ins of new releases (or phase-outs
  of old ones).

We can identify a few stages we can adopt to follow the Debian stable
release upgrade process:

* In the "normal" stage, we're not particularly interested yet in
  targeting the next stable release, so *ci-common.yml* will only
  include a build on the stable release.
* When the next testing release starts to stabilize, we add it to
  *ci-common.yml* in optional (no pipeline failure) mode, to see which
  packages will need work without interrupting their release workflows.
* When we start experimenting with the new "next" stable release, we
  remove the optional bit on those builds, and fix the packages that
  are broken in the new release. Once that is done, we are ready to
  upgrade the servers once the next Debian release becomes *stable*.
* After all servers are upgraded, we can start to drop the old
  build. Depending on whether there are other users of the packages or
  not, we can set the optional bit on the old builds and keep them
  around for a while, before returning to stage 1.

# Alternative scripts

Most packages should include *ci-common.yml*, but there are other
scripts for packages that have specific, custom requirements:

* *ci-backports.yml*, identical to *ci-common.yml* but includes
  backports. Mostly used by Go packages to overcome issues with the
  Buster Go version being too old.
* *ci-buster.yml*, only builds packages for Debian Buster