Autistici.org website

This repository contains the source code for the autistici.org website.

Contents are written in Markdown, and rendered to static HTML at deployment time.

Overview

To summarize quickly how the website is structured: it is a multi-language site, with the page language determined by the source file extension (.lang.md, where lang is the two-letter ISO code). As mentioned, pages are written in plain Markdown. If a page has some associated content, such as images, the best practice is to save those images under a new subdirectory of static/img/

• src/ contains the site sources
• static/img/ is for images, one subdirectory per associated content page

The pages are rendered using Bootstrap 5, with some Javascript functionality to support quick search and autocompletion:

• ui/templates/ has the HTML templates (one for the documents, and one for the search page)
• ui/src/ contains the Javascript code and CSS customizations, which are then compiled and minified using Webpack

Making changes

The website contents are located in the src subdirectory. Pages are written in Markdown, with a small header providing page metadata (mostly the page title).

The header is YAML-encoded, and it must be separated from the page content with a line containing only four dashes (----). For example:

title: About
----
Contents of the about page.

When editing the website sources, it is a good idea to test the results locally, before submitting the changes. In order to do so, there are some setup steps you will need to perform on your system.

Requirements

Docker is required to build the website. Since the result is a self-contained standalone web server container with no further dependencies, that makes it pretty easy to test locally too.

Testing changes for correctness

There are a few static checks that you can run after modifying the website. They will check for broken internal links, bad syntax, and other things. You can invoke the lint script that runs these checks with:

$ ./scripts/lint.sh

If this scripts does not output anything, the tests have passed successfully.

It is a good idea to set up the linter script as a git pre-commit hook, to make it impossible to commit a change that does not pass these tests successfully:

$ cat >.git/hooks/pre-commit <<EOF
#!/bin/sh
exec ./scripts/lint.sh
EOF
$ chmod +x .git/hooks/pre-commit

How to build the website

Simply run, from the top-level directory:

$ docker build -t ai-website .

The resulting HTML pages will be found in the public directory.

How to run a test webserver

Once you've built a container image (see How to build the website above), you can just run it locally with:

$ docker run --rm -p 8080:8080 ai-website

the website should then be visible at http://localhost:8080/.

Embedding external data sources

It is possible to use dynamically-generated data in the templates by placing a script in the data.d/ directory. The script must be executable, and its name should not contain dots. Its output must be valid JSON.

Data from these scripts will be collected in a single dictionary in the file data.json: the data returned by each script will be present with a key named as the script.

Files ending in .md_in will be preprocessed by the template engine (using the Go html/template syntax) before being rendered into HTML.

As an example of this technique, you can check out:

• data.d/dns
• src/docs/web/domains.en.md_in

How the search functionality works

Even though the contents are hierarchically structured in directories and subdirectories, the site's presentation has no explicit, navigable hierarchical structure (because we never found a good way to do so). This is compensated by relatively fancy search functionality, which:

• works well with multiple languages, having an understanding of the language the query is made in
• is fast to respond to simple queries thanks to a pre-loaded local list of page titles
• complements results with a reliable search engine on the server

The website uses typeahead.js to provide immediate search results as-you-type. The server-side search engine is sitesearch, contained in the ai/webtools repository.

Site build pipeline

In order to build the website, a number of steps are required:

• The static assets that are part of the site style are compiled with Webpack, to generate minimized and minified Javascript and CSS (removing all the code we're not using). Webpack also modifies the site templates, to inject the correct links to the generated assets (which contain a hash, for cache-related reasons).
• The site's HTML pages are built using a minimalistic "static site generator" (gostatic).
• The original site contents are indexed by the search engine, generating the search index.
• The static assets and the HTML pages are pre-compressed with both GZip and Brotli, so that we can serve them quickly without using any CPU time and with extremely cache-friendly headers.