feat: raise the memory ceilings and add supaspin upgrade

The two logging services were being killed by the kernel in a loop on instance 1
(supa1.didgibot.com): vector 297 times, analytics 36 times, both exit code 137
(out of memory). The old ceilings sat right on top of what the services normally
use, so any backlog crossed them.

Measured across the three running instances on 2026-09-22:

- analytics 612 / 694 / 662 MiB against a 768 MiB ceiling  -> 1536 MiB
- vector    287 / 290 MiB against 512 MiB                  -> 1024 MiB
- realtime  269 / 287 / 300 MiB against 384 MiB            ->  768 MiB
- kong      260 / 269 / 265 MiB against 512 MiB            ->  768 MiB
- studio 203, storage 187, rest 162 MiB against 256 MiB    ->  512 MiB
- db        163 / 175 / 273 MiB against 2048 MiB           -> unchanged

Limits only. The requests stay at 128 MiB (512 for db): the server is one machine
shared with the production workloads and already has 80% of its memory reserved,
so raising the requests would leave production nothing to schedule into. Raising a
limit reserves nothing - it only moves the point at which the kernel kills.

Per-instance totals go from 5.875 GiB to 9.5 GiB of limit; reserved memory is
unchanged at 1.875 GiB.

New numbers only reach instances created afterwards, so this also adds
`sudo supaspin upgrade [N]`: it reads back the values Helm stored at install
time, replaces the memory block in them and applies the chart again. Pods are
replaced service by service; databases, uploads, certificates and .env files are
untouched. Without a number it does every instance and reports which ones failed.
Also available from the menu.

Fixes a rollout that could never finish, found by the first upgrade: Studio binds
port 3001 (3002, 3003) on the server itself so it can be reached over an SSH
tunnel, and one port can be held by one pod at a time. The default rollout starts
the replacement pod before stopping the old one, so the replacement never got the
port and stayed Pending for ever. The Studio deployment now stops the old pod
first. Postgres was never affected: a StatefulSet already replaces its pod that
way.

The values file that carries the instance's secrets and the post-renderer
arguments are now built by shared helpers used by both create and upgrade.

Tests: 113 pass (12 new). Version 1.1.2 -> 1.2.0 in VERSION, the VERSION constant
and README.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit was merged in pull request #5.
This commit is contained in:
Igor
2026-09-22 21:50:14 +03:00
co-authored by Claude Opus 5
parent b86863e6f0
commit 1d49234400
4 changed files with 351 additions and 49 deletions
+36 -18
View File
@@ -1,6 +1,6 @@
# supaspin
Version 1.1.2
Version 1.2.0
Create, list, show and delete throwaway [Supabase](https://supabase.com) instances on a
single-node [microk8s](https://microk8s.io) server, from a terminal menu.
@@ -17,7 +17,7 @@ endangering them. Nothing in it is specific to any one app.
| Postgres | `<LAN IP>:544N` (instance 1 → 5441, 2 → 5442, …), bound on the LAN address only |
| Studio (admin UI) | `http://<LAN IP>:300N` (3001, 3002, …), not on the public hostname |
| Disk | fixed-size ext4 image files, mounted at boot: **db 2 GB, storage 1 GB, every other volume 256 MB** |
| Memory | limits add up to **5.875 GiB** per instance, requests to 1.875 GiB (table below) |
| Memory | limits add up to **9.5 GiB** per instance, requests to 1.875 GiB (table below) |
| Priority | pods run at priority -100 (`supaspin-low`); when the server runs short of memory they are evicted first and never evict anyone else |
| Secrets | generated fresh per instance: JWT secret, anon and service keys, Postgres password, Studio password, and the internal keys |
| Uploads | stored in the instance's own storage image (no MinIO); deleted with the instance |
@@ -93,6 +93,7 @@ Supabase Helm chart (0.8.0, Postgres 17.6.1.136) once into `/var/lib/supaspin/ch
| Show instance | the `.env` (secrets hidden unless you ask), the tunnel command, pod status and recent events |
| Delete instance | asks you to type the number; removes everything (see below) |
| Copy instance values into another .env file | puts an instance's `.env` values into your app's `.env`, on this server or on another computer (see below) |
| Apply current memory caps to running instances | puts this version's memory limits on instances that already exist, replacing their pods one service at a time; databases, uploads and `.env` files are not touched |
| Mail settings | view and change the saved default mail settings |
| Settings | change any first-run setting |
@@ -106,9 +107,10 @@ sudo supaspin env N # the full .env (contains secrets)
sudo supaspin tunnel N # the tunnel commands, step by step
sudo supaspin push-env N TARGET [--port P] [--key FILE] [--dry-run] # see "Copying an instance's values"
sudo supaspin delete N [--yes]
sudo supaspin upgrade [N] # apply this version's memory caps to running instances
sudo supaspin mail # change the saved default mail settings
sudo supaspin setup # change settings
supaspin --version # prints "supaspin 1.1.2"
supaspin --version # prints "supaspin 1.2.0"
```
### Connecting your app
@@ -209,22 +211,38 @@ run `supaspin delete N` to clean up.
## Memory per instance
| Service | Request | Limit |
|---|---|---|
| db | 512 MiB | 2048 MiB |
| analytics | 128 MiB | 768 MiB |
| kong | 128 MiB | 512 MiB |
| realtime | 128 MiB | 384 MiB |
| vector | 128 MiB | 512 MiB |
| auth, functions, imgproxy, meta, rest, storage, studio | 128 MiB each | 256 MiB each |
| **Total** | **1.875 GiB** | **5.875 GiB** |
| Service | Request | Limit | Highest use measured |
|---|---|---|---|
| db | 512 MiB | 2048 MiB | 273 MiB |
| analytics | 128 MiB | 1536 MiB | 694 MiB |
| vector | 128 MiB | 1024 MiB | 290 MiB |
| kong | 128 MiB | 768 MiB | 269 MiB |
| realtime | 128 MiB | 768 MiB | 300 MiB |
| auth, functions, imgproxy, meta, rest, storage, studio | 128 MiB each | 512 MiB each | 203 MiB (studio) |
| **Total** | **1.875 GiB** | **9.5 GiB** | |
Most limits are half of what a production setup would use. Analytics, realtime and vector are higher:
at half of production they were killed for running out of memory in a loop (measured on a fresh
instance: analytics peaks at 624 MiB, realtime at 280 MiB). Vector is the log collector: it reads the
log files of every pod on the server, not only its own instance's, so its memory need follows the
server's total log volume; at 256 MiB it was killed in a loop on one instance and used 89% on another. Image resizing (imgproxy) reads the
uploads from the storage volume, so it has no image of its own.
The request is what the scheduler sets aside for a service and can never be given to anything else.
The limit is the ceiling the kernel kills the container at. They do different jobs here.
The requests are deliberately small: the server is one machine shared with the production
workloads, and reserving what three instances really use would leave production nothing to start
in. The limits are roughly twice the highest use measured across three running instances on
2026-09-22 (right-hand column above).
The earlier limits sat right on top of normal use and the kernel kept killing the two logging
services. Analytics (Logflare, which stores the logs Studio shows) used 612, 694 and 662 MiB of a
768 MiB ceiling and was killed 36 times on one instance. Vector, the log collector that reads the
log files of every pod on the server and not only its own instance's, was killed 297 times on that
same instance, where a backlog it could not deliver kept growing in its memory. The two feed each
other: analytics dies, vector cannot deliver, vector fills up and dies, and on restart it replays
the backlog into analytics. Realtime at 300 MiB of 384 and studio at 203 MiB of 256 were the next
in line. Image resizing (imgproxy) reads the uploads from the storage volume, so it has no image of
its own.
Changing the numbers in this table only affects instances created afterwards. To put them on
instances that already run, use `sudo supaspin upgrade`: it reads back the values Helm stored when
the instance was installed, replaces the memory block in them and applies the chart again, so the
pods are replaced service by service and nothing else about the instance changes.
## Troubleshooting