PIES Studio 1.9 ยท Documentation

Disk and retention

A PIES Studio server builds the applications you deploy on itself. Each deployment builds two container images, and Docker keeps a build cache, the previous images of every upgrade and the logs of every container. Without retention, that disk fills up within days on a busy server, and then every build fails.

The installer sets up retention for you. This page explains what it keeps, what it removes, and what to do when the disk is still filling up.

What the installer sets up

WhatWhereEffect
Disk guard/usr/local/sbin/pies-disk-guard, run every 15 minutes by the pies-disk-guard.timer systemd timer (root's crontab where there is no systemd)Removes build leftovers, see below
Container log rotation, PIES servicesdocker-compose.ymlAt most 3 log files of 50 MB per service
Container log rotation, deployed apps/etc/docker/daemon.json (written only when the file does not exist yet)At most 3 log files of 50 MB per container
System journal limit/etc/systemd/journald.conf.d/pies.confThe journal uses at most 1 GB and leaves 2 GB free

If /etc/docker/daemon.json already existed, the installer leaves it alone and warns you when it has no log-opts. Add these lines to it, then restart Docker at a quiet time:


{
  "log-driver": "json-file",
  "log-opts": { "max-size": "50m", "max-file": "3" }
}

A Docker restart restarts every container, including your deployed apps. Log rotation applies to containers created after the restart.

What the disk guard removes

Every run:

The guard never removes a container, a volume (databases, files, generated previews) or an image that any container uses. Stopped and scheduled-off apps keep their images.

Disk thresholds

Disk usedWhat happens
Below 80%Normal retention only
80%The whole build cache is cleared, by the disk guard and before each new deployment build. Before each preview build, expired previews are removed first.
90%New app deployments and new preview builds are refused with "the server's disk is N% full and new builds stop at 90%". Running apps and previews are not affected.

To change the thresholds, set DISK_CLEAN_PERCENT and DISK_REFUSE_PERCENT on the deployer service (deployments) and the loki service (previews), and PIES_DISK_CLEAN_PERCENT, PIES_DISK_REFUSE_PERCENT and PIES_BUILD_CACHE_KEEP for the disk guard (for example in a systemd drop-in for pies-disk-guard.service).

Checking it


systemctl list-timers pies-disk-guard        # when it last ran and runs next
journalctl -u pies-disk-guard -n 20          # what it did
sudo pies-disk-guard --dry-run               # what it would remove now, without removing it
cat /var/lib/pies/disk-status.json           # last reading: used_percent, state ok / warn / refusing
docker system df                             # images, containers, volumes, build cache

When the disk keeps filling up

  1. Run sudo pies-disk-guard --dry-run and docker system df.
  2. Deleted apps still running. docker ps lists deployed-<id>-web and deployed-<id>-core for every deployed app. If one belongs to an app you have already deleted in Studio, delete its deployment again from the application's Deployments tab. Upgrade to the latest 1.9 build: earlier builds did not remove the containers of an internal deployment when its application was deleted.
  3. Database growth. docker system df -v shows each volume's size. Databases are never cleaned automatically. Archive or delete data from inside the application.
  4. The disk is simply too small. See Sizing. Allow at least 5 GB for each app you keep deployed, on top of the platform.

Never run docker system prune -a or docker volume prune on a PIES server. They remove the images of stopped apps and can remove database volumes.