feat: raise the memory ceilings and add supaspin upgrade
The two logging services were being killed by the kernel in a loop on instance 1 (supa1.didgibot.com): vector 297 times, analytics 36 times, both exit code 137 (out of memory). The old ceilings sat right on top of what the services normally use, so any backlog crossed them. Measured across the three running instances on 2026-09-22: - analytics 612 / 694 / 662 MiB against a 768 MiB ceiling -> 1536 MiB - vector 287 / 290 MiB against 512 MiB -> 1024 MiB - realtime 269 / 287 / 300 MiB against 384 MiB -> 768 MiB - kong 260 / 269 / 265 MiB against 512 MiB -> 768 MiB - studio 203, storage 187, rest 162 MiB against 256 MiB -> 512 MiB - db 163 / 175 / 273 MiB against 2048 MiB -> unchanged Limits only. The requests stay at 128 MiB (512 for db): the server is one machine shared with the production workloads and already has 80% of its memory reserved, so raising the requests would leave production nothing to schedule into. Raising a limit reserves nothing - it only moves the point at which the kernel kills. Per-instance totals go from 5.875 GiB to 9.5 GiB of limit; reserved memory is unchanged at 1.875 GiB. New numbers only reach instances created afterwards, so this also adds `sudo supaspin upgrade [N]`: it reads back the values Helm stored at install time, replaces the memory block in them and applies the chart again. Pods are replaced service by service; databases, uploads, certificates and .env files are untouched. Without a number it does every instance and reports which ones failed. Also available from the menu. Fixes a rollout that could never finish, found by the first upgrade: Studio binds port 3001 (3002, 3003) on the server itself so it can be reached over an SSH tunnel, and one port can be held by one pod at a time. The default rollout starts the replacement pod before stopping the old one, so the replacement never got the port and stayed Pending for ever. The Studio deployment now stops the old pod first. Postgres was never affected: a StatefulSet already replaces its pod that way. The values file that carries the instance's secrets and the post-renderer arguments are now built by shared helpers used by both create and upgrade. Tests: 113 pass (12 new). Version 1.1.2 -> 1.2.0 in VERSION, the VERSION constant and README. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit was merged in pull request #5.
This commit is contained in:
@@ -1,6 +1,6 @@
|
||||
# supaspin
|
||||
|
||||
Version 1.1.2
|
||||
Version 1.2.0
|
||||
|
||||
Create, list, show and delete throwaway [Supabase](https://supabase.com) instances on a
|
||||
single-node [microk8s](https://microk8s.io) server, from a terminal menu.
|
||||
@@ -17,7 +17,7 @@ endangering them. Nothing in it is specific to any one app.
|
||||
| Postgres | `<LAN IP>:544N` (instance 1 → 5441, 2 → 5442, …), bound on the LAN address only |
|
||||
| Studio (admin UI) | `http://<LAN IP>:300N` (3001, 3002, …), not on the public hostname |
|
||||
| Disk | fixed-size ext4 image files, mounted at boot: **db 2 GB, storage 1 GB, every other volume 256 MB** |
|
||||
| Memory | limits add up to **5.875 GiB** per instance, requests to 1.875 GiB (table below) |
|
||||
| Memory | limits add up to **9.5 GiB** per instance, requests to 1.875 GiB (table below) |
|
||||
| Priority | pods run at priority -100 (`supaspin-low`); when the server runs short of memory they are evicted first and never evict anyone else |
|
||||
| Secrets | generated fresh per instance: JWT secret, anon and service keys, Postgres password, Studio password, and the internal keys |
|
||||
| Uploads | stored in the instance's own storage image (no MinIO); deleted with the instance |
|
||||
@@ -93,6 +93,7 @@ Supabase Helm chart (0.8.0, Postgres 17.6.1.136) once into `/var/lib/supaspin/ch
|
||||
| Show instance | the `.env` (secrets hidden unless you ask), the tunnel command, pod status and recent events |
|
||||
| Delete instance | asks you to type the number; removes everything (see below) |
|
||||
| Copy instance values into another .env file | puts an instance's `.env` values into your app's `.env`, on this server or on another computer (see below) |
|
||||
| Apply current memory caps to running instances | puts this version's memory limits on instances that already exist, replacing their pods one service at a time; databases, uploads and `.env` files are not touched |
|
||||
| Mail settings | view and change the saved default mail settings |
|
||||
| Settings | change any first-run setting |
|
||||
|
||||
@@ -106,9 +107,10 @@ sudo supaspin env N # the full .env (contains secrets)
|
||||
sudo supaspin tunnel N # the tunnel commands, step by step
|
||||
sudo supaspin push-env N TARGET [--port P] [--key FILE] [--dry-run] # see "Copying an instance's values"
|
||||
sudo supaspin delete N [--yes]
|
||||
sudo supaspin upgrade [N] # apply this version's memory caps to running instances
|
||||
sudo supaspin mail # change the saved default mail settings
|
||||
sudo supaspin setup # change settings
|
||||
supaspin --version # prints "supaspin 1.1.2"
|
||||
supaspin --version # prints "supaspin 1.2.0"
|
||||
```
|
||||
|
||||
### Connecting your app
|
||||
@@ -209,22 +211,38 @@ run `supaspin delete N` to clean up.
|
||||
|
||||
## Memory per instance
|
||||
|
||||
| Service | Request | Limit |
|
||||
|---|---|---|
|
||||
| db | 512 MiB | 2048 MiB |
|
||||
| analytics | 128 MiB | 768 MiB |
|
||||
| kong | 128 MiB | 512 MiB |
|
||||
| realtime | 128 MiB | 384 MiB |
|
||||
| vector | 128 MiB | 512 MiB |
|
||||
| auth, functions, imgproxy, meta, rest, storage, studio | 128 MiB each | 256 MiB each |
|
||||
| **Total** | **1.875 GiB** | **5.875 GiB** |
|
||||
| Service | Request | Limit | Highest use measured |
|
||||
|---|---|---|---|
|
||||
| db | 512 MiB | 2048 MiB | 273 MiB |
|
||||
| analytics | 128 MiB | 1536 MiB | 694 MiB |
|
||||
| vector | 128 MiB | 1024 MiB | 290 MiB |
|
||||
| kong | 128 MiB | 768 MiB | 269 MiB |
|
||||
| realtime | 128 MiB | 768 MiB | 300 MiB |
|
||||
| auth, functions, imgproxy, meta, rest, storage, studio | 128 MiB each | 512 MiB each | 203 MiB (studio) |
|
||||
| **Total** | **1.875 GiB** | **9.5 GiB** | |
|
||||
|
||||
Most limits are half of what a production setup would use. Analytics, realtime and vector are higher:
|
||||
at half of production they were killed for running out of memory in a loop (measured on a fresh
|
||||
instance: analytics peaks at 624 MiB, realtime at 280 MiB). Vector is the log collector: it reads the
|
||||
log files of every pod on the server, not only its own instance's, so its memory need follows the
|
||||
server's total log volume; at 256 MiB it was killed in a loop on one instance and used 89% on another. Image resizing (imgproxy) reads the
|
||||
uploads from the storage volume, so it has no image of its own.
|
||||
The request is what the scheduler sets aside for a service and can never be given to anything else.
|
||||
The limit is the ceiling the kernel kills the container at. They do different jobs here.
|
||||
|
||||
The requests are deliberately small: the server is one machine shared with the production
|
||||
workloads, and reserving what three instances really use would leave production nothing to start
|
||||
in. The limits are roughly twice the highest use measured across three running instances on
|
||||
2026-09-22 (right-hand column above).
|
||||
|
||||
The earlier limits sat right on top of normal use and the kernel kept killing the two logging
|
||||
services. Analytics (Logflare, which stores the logs Studio shows) used 612, 694 and 662 MiB of a
|
||||
768 MiB ceiling and was killed 36 times on one instance. Vector, the log collector that reads the
|
||||
log files of every pod on the server and not only its own instance's, was killed 297 times on that
|
||||
same instance, where a backlog it could not deliver kept growing in its memory. The two feed each
|
||||
other: analytics dies, vector cannot deliver, vector fills up and dies, and on restart it replays
|
||||
the backlog into analytics. Realtime at 300 MiB of 384 and studio at 203 MiB of 256 were the next
|
||||
in line. Image resizing (imgproxy) reads the uploads from the storage volume, so it has no image of
|
||||
its own.
|
||||
|
||||
Changing the numbers in this table only affects instances created afterwards. To put them on
|
||||
instances that already run, use `sudo supaspin upgrade`: it reads back the values Helm stored when
|
||||
the instance was installed, replaces the memory block in them and applies the chart again, so the
|
||||
pods are replaced service by service and nothing else about the instance changes.
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
|
||||
Reference in New Issue
Block a user