ai3 issueshttps://git.autistici.org/groups/ai3/-/issues2018-01-04T17:23:44Zhttps://git.autistici.org/ai3/tools/suexec-sandbox/-/issues/1Signal handling2018-01-04T17:23:44ZaleSignal handlingSince we're forking a child (and waiting for it to terminate), we should forward signals to the monitored process. At least SIGTERM, which mod_fcgid is going to use to ask us to terminate.Since we're forking a child (and waiting for it to terminate), we should forward signals to the monitored process. At least SIGTERM, which mod_fcgid is going to use to ask us to terminate.https://git.autistici.org/ai3/tools/suexec-sandbox/-/issues/2Proper mount setup2018-01-04T17:26:28ZaleProper mount setupRight now the sandbox code can only do basic rw chroot. We'd like something a bit more sophisticated, where the / is mounted read-only, and the document root is mounted read-write (and /tmp is noexec on a tmpfs, perhaps). So, something c...Right now the sandbox code can only do basic rw chroot. We'd like something a bit more sophisticated, where the / is mounted read-only, and the document root is mounted read-write (and /tmp is noexec on a tmpfs, perhaps). So, something closer to the systemd ReadOnlyDirectories / ReadWriteDirectories model, as an example.
Obviously this will need some changes on the configuration side too, to support more elaborate setups.https://git.autistici.org/ai3/float/-/issues/5Improve container networking2021-04-27T10:09:16ZaleImprove container networkingRight now we simply use "docker --network=host" and manage network overlays separately. It would be nice to support more advanced container networking configurations, in particular, a closer integration between net-overlays and the conta...Right now we simply use "docker --network=host" and manage network overlays separately. It would be nice to support more advanced container networking configurations, in particular, a closer integration between net-overlays and the container scheduling itself.
More specifically here's a possible outcome:
* containers are assigned their own IPs
* net_overlay assigns a subnet to a host, not just a single IP
* container IPs are picked out of private network ranges
There are a few challenges:
* the current service discovery layer assigns IPs to service instances. Multiple containers within a service should use separate ports on the same IP (and should be visible to each other as 'localhost'). Maybe we can do something with "docker network create", or we can bind the docker bridge and the vpn interface later somehow.
* ...https://git.autistici.org/ai3/float/-/issues/14Idea: transparent sharding of user-keyed SSO-enabled services2019-10-25T19:37:43ZaleIdea: transparent sharding of user-keyed SSO-enabled servicesCurrently we support sharding by publishing shard-specific URLs (e.g. https://2.webmail.my.domain). This is a very simple and efficient approach, but it has a few disadvantages:
* the sharding structure is exposed publicly
* people migh...Currently we support sharding by publishing shard-specific URLs (e.g. https://2.webmail.my.domain). This is a very simple and efficient approach, but it has a few disadvantages:
* the sharding structure is exposed publicly
* people might bookmark links etc. which become invalid on re-sharding
In order to support partitioned services directly in the HTTP router we have to solve the following problem: given a HTTP request, figure out which shard it should be sent to. In the general case of a complex service (where the answer isn't just in the URL itself) this is a complex problem, but the situation is different for user-partitioned, SSO-enabled services:
* the sharding key is also the username (or can be derived from it)
* the HTTP router has access to the SSO token (for this to be the case we would need to standardize all applications on using the same cookie name for SSO, but that's doable)
in this case, the HTTP router itself can look at the SSO token and route the request accordingly.
This will incur a performance overhead, as finding the backend from the username might require an RPC (a LDAP lookup, for instance), but this can be mitigated with a short-term cache. The implementation would require a new HTTP proxy layer (the alternative of writing a pile of LUA into nginx itself is not very appealing) co-hosted with nginx, like the sso-proxy. Such a proxy:
* would not perform SSO authentication itself (the backend application should do that)
* in fact it might not even validate the SSO token, just look at it
* in pseudo-code, its decision algorithm might look like this:
* unauthenticated request?
* send to a random backend (handles things like /sso_login etc)
* authenticated request?
* find backend from SSO username
* send to that backend
This would allow us to implement the above-mentioned "webmail" service like this:
* public URL is just https://webmail.my.domain
* we simply need to provide a username->backend lookup functionhttps://git.autistici.org/ai3/float/-/issues/40It is hard to tell which process belongs to which container from ps2019-05-11T09:18:55ZgodogIt is hard to tell which process belongs to which container from psFor example in the output below the container name (or better the systemd unit name) isn't mentioned anywhere:
```
root 16562 2.7 0.6 713840 6448 ? Ssl 10:24 13:13 /usr/bin/containerd
root 19863 0.0 0.1 10740 131...For example in the output below the container name (or better the systemd unit name) isn't mentioned anywhere:
```
root 16562 2.7 0.6 713840 6448 ? Ssl 10:24 13:13 /usr/bin/containerd
root 19863 0.0 0.1 10740 1312 ? Sl 10:27 0:04 \_ containerd-shim -namespace moby -workdir /var/lib/containerd/io.containerd.runtime.v1.linux/moby/7948083fb7690b5154238ec2dfa2c332247ad4b05cfbc59
6f80cb0e2b641ff44 -address /run/containerd/containerd.sock -containerd-binary /usr/bin/containerd -runtime-root /var/run/docker/runtime-runc
docker-+ 19881 0.0 0.0 4288 0 ? Ss 10:27 0:00 | \_ /bin/sh -c /usr/bin/memcached -vv -m ${MEM:-64} -p ${PORT:-11211} ${ENABLE_SASL:+-S}
docker-+ 19905 0.1 0.0 327252 0 ? Sl 10:27 0:45 | \_ /usr/bin/memcached -vv -m 64 -p 11212
root 23375 0.0 0.1 9396 1568 ? Sl 10:45 0:03 \_ containerd-shim -namespace moby -workdir /var/lib/containerd/io.containerd.runtime.v1.linux/moby/fdd79fbd67ea703489738eb9120c23cdbc1f918227314b3
0b7b5c90f1d490b97 -address /run/containerd/containerd.sock -containerd-binary /usr/bin/containerd -runtime-root /var/run/docker/runtime-runc
docker-+ 23413 1.2 3.0 1343960 30648 ? Ssl 10:45 5:51 | \_ /usr/share/kibana/bin/../node/bin/node --no-warnings /usr/share/kibana/bin/../src/cli serve --config /etc/kibana/kibana.yml --quiet
root 7837 0.0 0.1 9332 1384 ? Sl 10:49 0:03 \_ containerd-shim -namespace moby -workdir /var/lib/containerd/io.containerd.runtime.v1.linux/moby/61d73f72397b863b184101bce521ca57328c8b21069bb38
764963e88a3a72658 -address /run/containerd/containerd.sock -containerd-binary /usr/bin/containerd -runtime-root /var/run/docker/runtime-runc
docker-+ 7859 0.0 0.0 62288 580 ? Ss 10:49 0:00 | \_ /usr/bin/python3 /usr/local/bin/chaperone
docker-+ 8268 0.2 0.7 137040 7980 ? Sl 10:49 0:57 | \_ /usr/bin/apache_exporter -scrape_uri http://127.0.0.1:8084/server-status/?auto -telemetry.address :8184
docker-+ 8327 0.0 0.0 324660 296 ? Ss 10:49 0:07 | \_ php-fpm: master process (/etc/php/7.0/fpm/php-fpm.conf)
docker-+ 8334 0.0 0.0 324660 172 ? S 10:49 0:00 | | \_ php-fpm: pool www
docker-+ 8335 0.0 0.0 324660 172 ? S 10:49 0:00 | | \_ php-fpm: pool www
docker-+ 8336 0.0 0.0 324660 172 ? S 10:49 0:00 | | \_ php-fpm: pool www
docker-+ 8329 0.0 0.1 99276 1268 ? S 10:49 0:04 | \_ /usr/sbin/apache2 -DFOREGROUND
docker-+ 8355 0.0 0.0 25388 0 ? S 10:49 0:00 | \_ /usr/bin/logger -t apache -p local3 info
docker-+ 8358 0.2 0.0 1306132 272 ? Sl 10:49 1:12 | \_ /usr/sbin/apache2 -DFOREGROUND
docker-+ 8359 0.2 0.0 1306132 268 ? Sl 10:49 1:12 | \_ /usr/sbin/apache2 -DFOREGROUND
root 9408 0.0 0.1 9332 1552 ? Sl 10:50 0:04 \_ containerd-shim -namespace moby -workdir /var/lib/containerd/io.containerd.runtime.v1.linux/moby/33e612efc89a9a54edf63b5346469ebd39534819a2c197752e6038cb51b7b66c -address /run/containerd/containerd.sock -containerd-binary /usr/bin/containerd -runtime-root /var/run/docker/runtime-runc
root 9428 0.0 0.1 62292 1192 ? Ss 10:50 0:00 \_ /usr/bin/python3 /usr/local/bin/chaperone
root 9453 0.2 0.6 210772 6632 ? Sl 10:50 0:57 \_ /usr/bin/apache_exporter -scrape_uri http://127.0.0.1:8083/server-status/?auto -telemetry.address :8183
root 9462 0.0 0.1 89444 1588 ? S 10:50 0:05 \_ /usr/sbin/apache2 -DFOREGROUND
root 9494 0.0 0.0 25388 28 ? S 10:50 0:01 \_ /usr/bin/logger -p local3 info -t apache
www-data 9495 0.0 0.0 89168 768 ? S 10:50 0:02 \_ /usr/sbin/apache2 -DFOREGROUND
www-data 9596 0.2 0.0 1296292 704 ? Sl 10:50 1:12 \_ /usr/sbin/apache2 -DFOREGROUND
www-data 9597 0.2 0.0 1296292 624 ? Sl 10:50 1:13 \_ /usr/sbin/apache2 -DFOREGROUND
```https://git.autistici.org/ai3/tools/float-debug-proxy/-/issues/1Use some packaging/publishing other than gitlab artifacts for binaries2018-12-23T20:04:02ZgodogUse some packaging/publishing other than gitlab artifacts for binarieshttps://git.autistici.org/ai3/tools/runcron/-/issues/1Send run stats metrics on syslog as structured logging2018-12-29T14:16:54ZgodogSend run stats metrics on syslog as structured loggingIt would be useful for auditing purposes if `runcron` would send its events (start/stop/etc) on syslog as structured fields, thus showing up for later analysis.It would be useful for auditing purposes if `runcron` would send its events (start/stop/etc) on syslog as structured fields, thus showing up for later analysis.https://git.autistici.org/ai3/docker/mailman/-/issues/3VERP2019-01-10T10:53:09ZaleVERPMaybe enable VERP wherever possibleMaybe enable VERP wherever possiblehttps://git.autistici.org/ai3/docker/roundcube/-/issues/4Quando scade SSO, roundcube non e' in grado di recuperare la sessione2019-01-24T09:46:26ZaleQuando scade SSO, roundcube non e' in grado di recuperare la sessioneCome riprodurre il problema:
* loggarsi nella webmail
* aspettare che scada il token SSO
* ricaricare la pagina
si ottiene la pagina di login di roundcube (horror! questa non dovrebbe mai apparire), e il messaggio "Your session is inva...Come riprodurre il problema:
* loggarsi nella webmail
* aspettare che scada il token SSO
* ricaricare la pagina
si ottiene la pagina di login di roundcube (horror! questa non dovrebbe mai apparire), e il messaggio "Your session is invalid or expired."https://git.autistici.org/ai3/config/-/issues/2postfix-delivery bounce per unknown recipient2019-05-25T13:22:24Zalepostfix-delivery bounce per unknown recipientL'istanza postfix-delivery attualmente non e' davvero configurata come delivery-only, il default transport e' smtp quindi provera' a consegnare all'esterno recipient sconosciuti etc.L'istanza postfix-delivery attualmente non e' davvero configurata come delivery-only, il default transport e' smtp quindi provera' a consegnare all'esterno recipient sconosciuti etc.https://git.autistici.org/ai3/tools/acmeserver/-/issues/2acmeserver force cert regeneration at runtime2019-06-09T15:36:50Zgodogacmeserver force cert regeneration at runtimewhile acmeserver is running it'd be nice to have a way to force trying to renew a certificatewhile acmeserver is running it'd be nice to have a way to force trying to renew a certificatehttps://git.autistici.org/ai3/float/-/issues/50Impossible to tell which backend the web request was routed to2021-02-04T20:19:57ZgodogImpossible to tell which backend the web request was routed toDue to anonymization all backends are 0.0.0.0, thus it is only possible to tell which apache instance should serve the request.Due to anonymization all backends are 0.0.0.0, thus it is only possible to tell which apache instance should serve the request.https://git.autistici.org/ai3/float/-/issues/53Expose internal HTTP endpoints through the sso-proxy2019-10-25T19:37:22ZaleExpose internal HTTP endpoints through the sso-proxyMost services with HTTP endpoints these days also have debug information etc. and it would be useful to be able to access it externally as administrators. This is doable but it's going to require care, as it would primarily rely on split...Most services with HTTP endpoints these days also have debug information etc. and it would be useful to be able to access it externally as administrators. This is doable but it's going to require care, as it would primarily rely on split DNS techniques, so we'd have to be careful to maintain strict separation of internal and external lookups (right now *float* does not control resolv.conf).
Steps for implementation:
* [ ] generate DNS zones for *domain*
* at first add them on top of /etc/hosts and do not modify host.conf
* make it so there are separate internal and external zones:
* the internal zone should match what currently is in /etc/hosts
* the external zone should point all names at the frontend hosts
* [ ] set up NGINX sso-proxy entries for all service backends
* these would match the *shard*.*service*.*domain* structure, without the port
* [ ] set up ACME entries for all these names
* one single certificate for all of them? one per service, with shards as subjectAltNames? a wildcard?
Alternatives to consider:
* perhaps we should simply create sharded public_endpoints manually for internal services with debug APIs? less magic, more manual work.https://git.autistici.org/ai3/tools/replds/-/issues/2Package updates do not restart replds@ instances2019-08-17T08:10:50ZalePackage updates do not restart replds@ instancesWe're probably missing a PartOf= or a WantedBy= in the systemd unit? Otherwise it could be related to the fact that in debian/rules we do not auto-setup the primary service (so who knows what happens on upgrade)?We're probably missing a PartOf= or a WantedBy= in the systemd unit? Otherwise it could be related to the fact that in debian/rules we do not auto-setup the primary service (so who knows what happens on upgrade)?https://git.autistici.org/ai3/tools/zonetool/-/issues/2Needs a --diff mode2019-08-21T09:29:32ZaleNeeds a --diff modeIt would be nice to be able to see the diff in the output zone files (might require stable sorting of those) (might imply --dry-run).It would be nice to be able to see the diff in the output zone files (might require stable sorting of those) (might imply --dry-run).https://git.autistici.org/ai3/tools/replds/-/issues/3Readonly follower instances2019-08-28T21:27:56ZaleReadonly follower instancesWe need a mode ("readonly") where instances do not participate in the sharing protocol, and do not check local disk for updates. Instead, they will simply periodically pull information from the primary peers.We need a mode ("readonly") where instances do not participate in the sharing protocol, and do not check local disk for updates. Instead, they will simply periodically pull information from the primary peers.https://git.autistici.org/ai3/tools/yarascan/-/issues/1Add links to web UI from email notifications2019-12-31T15:32:46ZgodogAdd links to web UI from email notificationsIt'd be nice if the notification email links back to the web UI e.g. to the site in question.It'd be nice if the notification email links back to the web UI e.g. to the site in question.https://git.autistici.org/ai3/docker/rt4/-/issues/13pulire il database dagli utenti2020-01-29T21:35:06Zputropulire il database dagli utentiil db andrebbe analizzato per capire cosa salva esattamente degli utenti (solo la mail ?),
ed eventualmente pulito se possibile, una volta che i ticket di un utente sono stati eliminati.il db andrebbe analizzato per capire cosa salva esattamente degli utenti (solo la mail ?),
ed eventualmente pulito se possibile, una volta che i ticket di un utente sono stati eliminati.putroputrohttps://git.autistici.org/ai3/tools/cgroups-exporter/-/issues/1Metrics for instantiated units are not reported2020-02-11T12:31:45ZgodogMetrics for instantiated units are not reportedIt looks like for instantiated (some?) units (i.e. using the `@` notation) are not reported:
```
$ curl -s localhost:3909/metrics | grep -i @
cgroup_blkio_bytes{mode="read",service="ifup@eno1.service",slice="system.slice"} 0
cgroup_blki...It looks like for instantiated (some?) units (i.e. using the `@` notation) are not reported:
```
$ curl -s localhost:3909/metrics | grep -i @
cgroup_blkio_bytes{mode="read",service="ifup@eno1.service",slice="system.slice"} 0
cgroup_blkio_bytes{mode="read",service="user@0.service",slice="user.slice/user-0.slice"} 0
cgroup_blkio_bytes{mode="write",service="ifup@eno1.service",slice="system.slice"} 0
cgroup_blkio_bytes{mode="write",service="user@0.service",slice="user.slice/user-0.slice"} 0
cgroup_cpu_usage{mode="system",service="ifup@eno1.service",slice="system.slice"} 0
cgroup_cpu_usage{mode="system",service="user@0.service",slice="user.slice/user-0.slice"} 0.03
cgroup_cpu_usage{mode="user",service="ifup@eno1.service",slice="system.slice"} 0
cgroup_cpu_usage{mode="user",service="user@0.service",slice="user.slice/user-0.slice"} 0.01
cgroup_memory_usage{service="ifup@eno1.service",slice="system.slice"} 0
cgroup_memory_usage{service="user@0.service",slice="user.slice/user-0.slice"} 1.380352e+06
```
Although `replds@acme` for example should be there:
```
$ systemctl status replds@acme
* replds@acme.service - Replicated file repository manager
Loaded: loaded (/lib/systemd/system/replds@.service; enabled; vendor preset: enabled)
Drop-In: /etc/systemd/system/replds@acme.service.d
`-group.conf
Active: active (running) since Sat 2019-10-26 19:15:25 UTC; 2 months 23 days ago
Main PID: 10778 (replds)
CGroup: /system.slice/system-replds.slice/replds@acme.service
`-10778 /usr/bin/replds --config /etc/replds/acme.yml server
```https://git.autistici.org/ai3/accountserver/-/issues/11Validation needs to take into account the requester's identity2020-04-08T09:01:39ZaleValidation needs to take into account the requester's identitySample scenario: admins should be allowed to create email accounts on all domains, including some which should not be freely available to all users. To support this case, accountserver needs to know *who* made the request, and change the...Sample scenario: admins should be allowed to create email accounts on all domains, including some which should not be freely available to all users. To support this case, accountserver needs to know *who* made the request, and change the validation results according to the identity.