The two logging services were being killed by the kernel in a loop on instance 1
(supa1.didgibot.com): vector 297 times, analytics 36 times, both exit code 137
(out of memory). The old ceilings sat right on top of what the services normally
use, so any backlog crossed them.
Measured across the three running instances on 2026-09-22:
- analytics 612 / 694 / 662 MiB against a 768 MiB ceiling -> 1536 MiB
- vector 287 / 290 MiB against 512 MiB -> 1024 MiB
- realtime 269 / 287 / 300 MiB against 384 MiB -> 768 MiB
- kong 260 / 269 / 265 MiB against 512 MiB -> 768 MiB
- studio 203, storage 187, rest 162 MiB against 256 MiB -> 512 MiB
- db 163 / 175 / 273 MiB against 2048 MiB -> unchanged
Limits only. The requests stay at 128 MiB (512 for db): the server is one machine
shared with the production workloads and already has 80% of its memory reserved,
so raising the requests would leave production nothing to schedule into. Raising a
limit reserves nothing - it only moves the point at which the kernel kills.
Per-instance totals go from 5.875 GiB to 9.5 GiB of limit; reserved memory is
unchanged at 1.875 GiB.
New numbers only reach instances created afterwards, so this also adds
`sudo supaspin upgrade [N]`: it reads back the values Helm stored at install
time, replaces the memory block in them and applies the chart again. Pods are
replaced service by service; databases, uploads, certificates and .env files are
untouched. Without a number it does every instance and reports which ones failed.
Also available from the menu.
Fixes a rollout that could never finish, found by the first upgrade: Studio binds
port 3001 (3002, 3003) on the server itself so it can be reached over an SSH
tunnel, and one port can be held by one pod at a time. The default rollout starts
the replacement pod before stopping the old one, so the replacement never got the
port and stayed Pending for ever. The Studio deployment now stops the old pod
first. Postgres was never affected: a StatefulSet already replaces its pod that
way.
The values file that carries the instance's secrets and the post-renderer
arguments are now built by shared helpers used by both create and upgrade.
Tests: 113 pass (12 new). Version 1.1.2 -> 1.2.0 in VERSION, the VERSION constant
and README.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
- vector reads the log files of every pod on the node, so its memory need follows the node's log volume. At
256 MiB it was killed for running out of memory 122 times on instance 1 and used 89% on instance 2.
The limit is now 512 MiB (request unchanged at 128 MiB) for new instances; the total limit per instance
goes from 5.625 GiB to 5.875 GiB, requests stay at 1.875 GiB.
- README memory table and the test for the totals updated
- version 1.1.1 -> 1.1.2
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
- install.sh said "Installed supaspin supaspin 1.1.0." because --version now includes the name; it now says
"Installed supaspin 1.1.1."
- CLAUDE.md: tells AI agents to read Igor's global rules first, and states the versioning rule (number in VERSION,
code and README together, semantic versioning, major bumps only with Igor's yes, a git tag vX.Y.Z after every merge)
- version 1.1.0 -> 1.1.1
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
- list and tunnel print plain-language tunnel instructions for every instance: keep it open, run it in the
background, close it, with a notice that background/close work on Mac and Linux only, not Windows
- push-env N TARGET puts an instance's .env values into another .env file, on this server or over SSH (Mac, Linux,
Windows): existing keys are replaced, missing keys added, commented-out lines ignored, every other line kept.
A temporary copy of the target is compared after writing and put back on any mismatch. --dry-run shows the
changes without writing. SSH failures are explained with how to set SSH up.
- VERSION file (1.1.0), version line in the README, `--version` prints "supaspin 1.1.0"; a test keeps the file, the
code constant and the README in step
- remove an unused parameter of run() and an unreachable second help branch in main()
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
