Disk and retention

A PIES Studio server builds the applications you deploy on itself. Each deployment builds two container images, and Docker keeps a build cache, the previous images of every upgrade and the logs of every container. Without retention, that disk fills up within days on a busy server, and then every build fails.

The installer sets up retention for you. This page explains what it keeps, what it removes, and what to do when the disk is still filling up.

What the installer sets up

What	Where	Effect
Disk guard	`/usr/local/sbin/pies-disk-guard`, run every 15 minutes by the `pies-disk-guard.timer` systemd timer (root's crontab where there is no systemd)	Removes build leftovers, see below
Container log rotation, PIES services	`docker-compose.yml`	At most 3 log files of 50 MB per service
Container log rotation, deployed apps	`/etc/docker/daemon.json` (written only when the file does not exist yet)	At most 3 log files of 50 MB per container
System journal limit	`/etc/systemd/journald.conf.d/pies.conf`	The journal uses at most 1 GB and leaves 2 GB free

If /etc/docker/daemon.json already existed, the installer leaves it alone and warns you when it has no log-opts. Add these lines to it, then restart Docker at a quiet time:


{
  "log-driver": "json-file",
  "log-opts": { "max-size": "50m", "max-file": "3" }
}

A Docker restart restarts every container, including your deployed apps. Log rotation applies to containers created after the restart.

What the disk guard removes

Every run:

Build cache beyond 5 GB.
Rollback images from upgrades: only the newest one per service is kept.
Untagged images older than a day.
Images of deleted apps: a deployed app's image that no container uses, running or stopped, and that is older than a day.
Build folders of deployed apps: node_modules and .next, once idle for 6 hours. The built image already holds the app, and the next deploy recreates them.
Generated app code: the guard asks the preview service to remove the installed packages and build output of idle generated-code folders right away (see below), instead of waiting for its next pass.

The guard never removes a container, a volume (databases, files, generated previews) or an image that any container uses. Stopped and scheduled-off apps keep their images.

Generated app code

Every Build, Preview and Export writes the app's generated source code into a folder on the server. The preview service removes these folders on its own, every few minutes:

Expired previews: the folder of a preview nobody has opened recently.
Installed packages and build output: once a folder has been idle for 6 hours, its node_modules, .next, .angular and dist folders are removed and the source code is kept. The next Build or Preview of that app regenerates the code and installs the packages again, so it takes a little longer than usual.
Idle folders: a folder that no build has touched for 3 days.
Too many folders for one user: when one user has more than 20 folders, the least recently used ones are removed until they are back at 20.
Low free disk: when the disk has less than 10 GB free, the least recently used folders are removed, oldest first, until it is back above 10 GB.

A folder used in the last hour is never removed by these last two rules.

A folder is never removed while a build, preview or export is using it, and the folder of a deployment is kept. Removing a folder is safe: the next Build, Preview or Export of that app generates its code again from the app definition. Deployed apps keep running, because their images are already built.

To change the idle period, set LOKI_GENERATED_RETENTION_DAYS on the loki service (default 3; 0 turns idle removal off). To change when installed packages and build output are removed, set LOKI_GENERATED_STRIP_HOURS (default 6; 0 turns it off). To change the other two limits, set LOKI_GENERATED_MAX_PER_OWNER (folders per user, default 20) and LOKI_GENERATED_MIN_FREE_GB (free disk to keep, in GB, default 10) on the same service. Each removal is logged with a [REAP] line in the loki service's log.

Disk Usage page

Administrators can see this in Studio under Admin > Disk Usage: the number of generated-code folders, how many are in use, free disk against the 10 GB floor, when packages were last removed, and a table of folders per app.

Measure sizes adds up the size of every folder. On a busy server this can take a few minutes.
Reclaim now removes the installed packages and build output of every folder idle for longer than the strip period (6 hours by default) and shows how many folders it cleaned and how much space it freed.
Refresh reads the figures again.

Disk thresholds

Disk used	What happens
Below 80%	Normal retention only
80%	The whole build cache is cleared, by the disk guard and before each new deployment build. Before each preview build, expired previews are removed first.
90%	New app deployments and new preview builds are refused with "the server's disk is N% full and new builds stop at 90%". Running apps and previews are not affected.

To change the thresholds, set DISK_CLEAN_PERCENT and DISK_REFUSE_PERCENT on the deployer service (deployments) and the loki service (previews), and PIES_DISK_CLEAN_PERCENT, PIES_DISK_REFUSE_PERCENT and PIES_BUILD_CACHE_KEEP for the disk guard (for example in a systemd drop-in for pies-disk-guard.service).

Checking it


systemctl list-timers pies-disk-guard        # when it last ran and runs next
journalctl -u pies-disk-guard -n 20          # what it did
sudo pies-disk-guard --dry-run               # what it would remove now, without removing it
cat /var/lib/pies/disk-status.json           # last reading: used_percent, state ok / warn / refusing
docker system df                             # images, containers, volumes, build cache

When the disk keeps filling up

Run sudo pies-disk-guard --dry-run and docker system df.
Deleted apps still running. docker ps lists deployed-<id>-web and deployed-<id>-core for every deployed app. If one belongs to an app you have already deleted in Studio, delete its deployment again from the application's Deployments tab. Upgrade to the latest 1.9 build: earlier builds did not remove the containers of an internal deployment when its application was deleted.
Database growth. docker system df -v shows each volume's size. Databases are never cleaned automatically. Archive or delete data from inside the application.
The disk is simply too small. See Sizing. Allow at least 5 GB for each app you keep deployed, on top of the platform.

Never run docker system prune -a or docker volume prune on a PIES server. They remove the images of stopped apps and can remove database volumes.