The two logging services were being killed by the kernel in a loop on instance 1 (supa1.didgibot.com): vector 297 times, analytics 36 times, both exit code 137 (out of memory). The old ceilings sat right on top of what the services normally use, so any backlog crossed them. Measured across the three running instances on 2026-09-22: - analytics 612 / 694 / 662 MiB against a 768 MiB ceiling -> 1536 MiB - vector 287 / 290 MiB against 512 MiB -> 1024 MiB - realtime 269 / 287 / 300 MiB against 384 MiB -> 768 MiB - kong 260 / 269 / 265 MiB against 512 MiB -> 768 MiB - studio 203, storage 187, rest 162 MiB against 256 MiB -> 512 MiB - db 163 / 175 / 273 MiB against 2048 MiB -> unchanged Limits only. The requests stay at 128 MiB (512 for db): the server is one machine shared with the production workloads and already has 80% of its memory reserved, so raising the requests would leave production nothing to schedule into. Raising a limit reserves nothing - it only moves the point at which the kernel kills. Per-instance totals go from 5.875 GiB to 9.5 GiB of limit; reserved memory is unchanged at 1.875 GiB. New numbers only reach instances created afterwards, so this also adds `sudo supaspin upgrade [N]`: it reads back the values Helm stored at install time, replaces the memory block in them and applies the chart again. Pods are replaced service by service; databases, uploads, certificates and .env files are untouched. Without a number it does every instance and reports which ones failed. Also available from the menu. Fixes a rollout that could never finish, found by the first upgrade: Studio binds port 3001 (3002, 3003) on the server itself so it can be reached over an SSH tunnel, and one port can be held by one pod at a time. The default rollout starts the replacement pod before stopping the old one, so the replacement never got the port and stayed Pending for ever. The Studio deployment now stops the old pod first. Postgres was never affected: a StatefulSet already replaces its pod that way. The values file that carries the instance's secrets and the post-renderer arguments are now built by shared helpers used by both create and upgrade. Tests: 113 pass (12 new). Version 1.1.2 -> 1.2.0 in VERSION, the VERSION constant and README. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2 lines
6 B
Plaintext
2 lines
6 B
Plaintext
1.2.0
|