Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
e2afacb5fc | ||
|
|
da8f02310a | ||
|
|
ccc49ca7d1 | ||
|
|
4a267c5336 | ||
|
|
d4fedd718f | ||
|
|
2f47ddedf1 | ||
|
|
80b84afb1e |
@@ -1,6 +1,6 @@
|
||||
# supaspin
|
||||
|
||||
Version 1.3.3
|
||||
Version 1.6.4
|
||||
|
||||
Create, list, show and delete throwaway [Supabase](https://supabase.com) instances on a
|
||||
single-node [microk8s](https://microk8s.io) server, from a terminal menu.
|
||||
@@ -13,10 +13,10 @@ endangering them. Nothing in it is specific to any one app.
|
||||
|
||||
| | |
|
||||
|---|---|
|
||||
| Web API | `https://<slot hostname>` (auth, rest, graphql, realtime, storage, functions), with a Let's Encrypt certificate |
|
||||
| Postgres | `<LAN IP>:544N` (instance 1 → 5441, 2 → 5442, …), bound on the LAN address only |
|
||||
| Web API | `https://<slot hostname>` (auth, rest, graphql, realtime, storage, functions); the certificate is from Let's Encrypt when that is enabled, otherwise self-signed |
|
||||
| Postgres | `<LAN IP>:544N` (instance 1 → 5441, 2 → 5442, …); which addresses listen is set by **Port bind** |
|
||||
| Studio (admin UI) | `http://<LAN IP>:300N` (3001, 3002, …), not on the public hostname |
|
||||
| Disk | fixed-size ext4 image files, mounted at boot: **db 2 GB, storage 1 GB, every other volume 256 MB** |
|
||||
| Disk | fixed-size ext4 image files, mounted at boot: **db 6 GB, storage 1 GB, every other volume 256 MB** |
|
||||
| Memory | limits add up to **9 GiB** per instance, requests to 1.875 GiB (table below) |
|
||||
| Priority | pods run at priority -100 (`supaspin-low`); when the server runs short of memory they are evicted first and never evict anyone else |
|
||||
| Secrets | generated fresh per instance: JWT secret, anon and service keys, Postgres password, Studio password, and the internal keys |
|
||||
@@ -32,10 +32,16 @@ of instances at once. Numbers are reused: create picks the lowest free one.
|
||||
## Requirements
|
||||
|
||||
- A Debian/Ubuntu server with **microk8s** (or any Kubernetes with `kubectl` and `helm` on the
|
||||
path), with an **ingress-nginx** ingress class and **cert-manager** installed.
|
||||
- DNS `A` records for every hostname slot pointing at the server, and ports 80 and 443 forwarded to
|
||||
it (Let's Encrypt fetches a test file over port 80 to issue each certificate).
|
||||
path), with an ingress controller installed: **ingress-nginx** or **Traefik** (the microk8s
|
||||
`ingress` addon is either, depending on the version). With **Traefik**, its CRDs must be present
|
||||
on the cluster — they come with the Traefik installation itself; supaspin creates `Middleware`
|
||||
objects and does not install cluster-wide types.
|
||||
- With **Let's Encrypt** enabled: **cert-manager** installed, DNS `A` records for every hostname
|
||||
slot pointing at the server, and ports 80 and 443 forwarded to it (Let's Encrypt fetches a test
|
||||
file over port 80 to issue each certificate). Without it: no cert-manager, no public DNS — each
|
||||
instance gets a self-signed certificate instead.
|
||||
- Root access on that server. `python3` and `python3-yaml` (the installer adds the latter with `apt`).
|
||||
Self-signed certificates additionally need `openssl` on the server.
|
||||
|
||||
## Install
|
||||
|
||||
@@ -70,17 +76,21 @@ sudo supaspin
|
||||
|---|---|
|
||||
| Storage root | folder that holds every instance's images and `.env` (`<root>/<N>/…`) |
|
||||
| Node name | Kubernetes node the volumes are pinned to (detected) |
|
||||
| Server LAN IP | address the Postgres and Studio ports are bound to (detected) |
|
||||
| Let's Encrypt e-mail | for certificate expiry notices |
|
||||
| Server LAN IP | address shown for the Postgres and Studio ports (detected) |
|
||||
| Port bind | `lan` = LAN IP only (default; keeps Postgres and the login-less Studio off a public interface), `all` = every interface incl. localhost (for a box behind NAT running its own apps), `local` = localhost only |
|
||||
| Ingress class | usually `public` on microk8s |
|
||||
| Hostname slots | comma-separated; the first is instance 1, the second instance 2, … |
|
||||
| Ingress controller | `traefik` or `nginx`; every instance's Ingress carries the nginx annotations (50 MiB upload cap, long timeouts) and, in Traefik mode, additionally points at a `Middleware` object with the same 50 MiB cap - so the cap is enforced no matter which of the two actually routes the hostname |
|
||||
| Let's Encrypt | `y` fetches a real certificate per hostname (needs public DNS, ports 80/443, cert-manager); `n` gives every instance a self-signed certificate and creates no cert-manager objects |
|
||||
| Let's Encrypt e-mail | for certificate expiry notices (asked only when Let's Encrypt is on) |
|
||||
| Hostname slots | comma-separated; a name with a number in it keys that slot (`supa4.didgibot.com` = slot 4, so instance 4 gets it with ports 5444/3004), a plain name takes its position in the list |
|
||||
| Public SSH host / port / user | only used to print the tunnel command |
|
||||
| Mail settings | default SMTP settings for the auth service (changeable later) |
|
||||
|
||||
Settings live in `/etc/supaspin/config.yaml` (readable by root only). The first run also creates,
|
||||
only if missing: the `supaspin-local-sc` StorageClass, the `supaspin-letsencrypt` ClusterIssuer,
|
||||
the `supaspin-low` PriorityClass and the `supaspin-certs` namespace, and downloads the pinned
|
||||
Supabase Helm chart (0.8.0, Postgres 17.6.1.136) once into `/var/lib/supaspin/charts/`.
|
||||
only if missing: the `supaspin-local-sc` StorageClass, the `supaspin-low` PriorityClass and the
|
||||
`supaspin-certs` namespace; with Let's Encrypt on it also creates the `supaspin-letsencrypt`
|
||||
ClusterIssuer. It then downloads the pinned Supabase Helm chart (0.8.0, Postgres 17.6.1.136) once
|
||||
into `/var/lib/supaspin/charts/`.
|
||||
|
||||
## Using it
|
||||
|
||||
@@ -107,12 +117,12 @@ sudo supaspin env N # the full .env (contains secrets)
|
||||
sudo supaspin tunnel N # the tunnel commands, step by step
|
||||
sudo supaspin push-env N TARGET [--port P] [--key FILE] [--dry-run] # see "Copying an instance's values"
|
||||
sudo supaspin delete N [--yes]
|
||||
sudo supaspin upgrade [N] # apply this version's memory caps to running instances
|
||||
sudo supaspin upgrade [N] # apply this version's memory caps and log retention to running instances
|
||||
sudo supaspin suspend N # stop an instance, keep its data (see "Suspending an instance")
|
||||
sudo supaspin resume N # start it again
|
||||
sudo supaspin mail # change the saved default mail settings
|
||||
sudo supaspin setup # change settings
|
||||
supaspin --version # prints "supaspin 1.3.1"
|
||||
supaspin --version # prints "supaspin X.Y.Z"
|
||||
```
|
||||
|
||||
### Connecting your app
|
||||
@@ -292,7 +302,9 @@ every delivery with an error, and vector was killed against its 1024 MiB ceiling
|
||||
minutes with the undeliverable backlog. The fix is log retention: every instance gets a pg_cron
|
||||
job (set up on create, upgrade and resume) that deletes analytics logs older than **2 days** every
|
||||
night. pg_cron in this image only runs jobs from the `postgres` database, so the job lives there
|
||||
and reaches the log tables in `_supabase` over a local dblink connection. That bounds both the
|
||||
and reaches the log tables in `_supabase` over a local dblink connection. The dblink extension
|
||||
and the helper function live in the `extensions` schema, where the app's database owner has no
|
||||
rights, so instance code can never call them. That bounds both the
|
||||
db image and vector's backlog, so vector is back to the 512 MiB ceiling the production supabase
|
||||
charts carry.
|
||||
|
||||
@@ -309,6 +321,9 @@ pods are replaced service by service and nothing else about the instance changes
|
||||
- **Certificate stays pending.** The hostname's DNS record must point at the server and port 80 must
|
||||
reach the ingress: `kubectl get certificate,order,challenge -n supaspin-N`.
|
||||
- **"no free hostname slot".** Delete an instance or add a slot in Settings (and its DNS record).
|
||||
- **Create failed** with `no matches for kind "Middleware" in version "traefik.io/v1alpha1"`. The
|
||||
config says Traefik but the cluster has no Traefik CRDs: either install Traefik (its CRDs come
|
||||
with it) or set the ingress controller to `nginx` in Settings.
|
||||
- **Create failed.** `sudo supaspin delete N`, then create again.
|
||||
- **Port already in use.** Postgres and Studio ports are fixed per number (`5440+N`, `3000+N`);
|
||||
another program is using it. Free the port or use another instance number.
|
||||
|
||||
@@ -39,7 +39,7 @@ import tty
|
||||
|
||||
import yaml
|
||||
|
||||
VERSION = "1.3.3"
|
||||
VERSION = "1.6.4"
|
||||
CONFIG_PATH = os.environ.get("SUPASPIN_CONFIG", "/etc/supaspin/config.yaml")
|
||||
STATE_DIR = os.environ.get("SUPASPIN_STATE", "/var/lib/supaspin")
|
||||
CHART_REPO = "https://supabase-community.github.io/supabase-kubernetes"
|
||||
@@ -53,7 +53,10 @@ CERTS_NS = "supaspin-certs"
|
||||
|
||||
# Image cap per volume, in MiB. The chart's separate imgproxy volume is not used: imgproxy has to
|
||||
# read the uploaded files, so it mounts the storage volume instead (see instance_values).
|
||||
VOLUMES = {"db": 2048, "storage": 1024, "functions": 256,
|
||||
# The db image is 6 GB because of the analytics logs: browser-test traffic wrote about 1.2 GB of
|
||||
# logs a day on instance 2 (2026-10-14), the nightly cleanup keeps LOG_RETENTION of them, and the
|
||||
# write-ahead log may take up to 1 GB on top. That peaks near 4.5 GB; 2 GB filled up on instance 4.
|
||||
VOLUMES = {"db": 6144, "storage": 1024, "functions": 256,
|
||||
"snippets": 256, "deno": 256, "pgsodium": 256}
|
||||
# The user Postgres runs as inside supabase/postgres 17.6.1.136 (`id postgres` in the image).
|
||||
# Production's older image used 2000; do not copy that number.
|
||||
@@ -92,34 +95,40 @@ DEFAULT_MEMORY = (128, 512)
|
||||
# which the image pins to `postgres`, while the logs live in `_supabase`. The job therefore runs
|
||||
# in `postgres` and reaches the logs over a local dblink connection, which needs no password: the
|
||||
# image trusts local socket connections for supabase_admin.
|
||||
# dblink and the helper function MUST live in the `extensions` schema, not `public`: the app's
|
||||
# migrations revoke API-role privileges on every routine in public, and `postgres` (the role
|
||||
# Prisma connects as) is no superuser here - revoking someone else's routines fails with
|
||||
# "permission denied for function dblink_connect_u" (42501, maralago 0_init, 2026-10-08).
|
||||
LOG_RETENTION = "2 days"
|
||||
LOG_RETENTION_JOB = "supaspin-log-retention"
|
||||
RETENTION_SQL = f"""
|
||||
create extension if not exists pg_cron;
|
||||
create extension if not exists dblink;
|
||||
create or replace function supaspin_delete_old_logs() returns void
|
||||
create extension if not exists dblink with schema extensions;
|
||||
alter extension dblink set schema extensions;
|
||||
drop function if exists public.supaspin_delete_old_logs();
|
||||
create or replace function extensions.supaspin_delete_old_logs() returns void
|
||||
language plpgsql as $fn$
|
||||
declare
|
||||
conn text := 'supaspin_lr_' || substr(md5(clock_timestamp()::text), 1, 8);
|
||||
t text;
|
||||
begin
|
||||
perform dblink_connect(conn, 'dbname=_supabase user=supabase_admin');
|
||||
perform extensions.dblink_connect(conn, 'dbname=_supabase user=supabase_admin');
|
||||
for t in
|
||||
select tablename from dblink(conn,
|
||||
select tablename from extensions.dblink(conn,
|
||||
'select tablename from pg_tables where schemaname = ''_analytics'' and tablename like ''log_events\\_%''')
|
||||
as log_tables(tablename text)
|
||||
loop
|
||||
perform dblink_exec(conn,
|
||||
perform extensions.dblink_exec(conn,
|
||||
format('delete from _analytics.%I where timestamp < now() - interval ''{LOG_RETENTION}''', t));
|
||||
end loop;
|
||||
perform dblink_disconnect(conn);
|
||||
perform extensions.dblink_disconnect(conn);
|
||||
exception when others then
|
||||
begin perform dblink_disconnect(conn); exception when others then null; end;
|
||||
begin perform extensions.dblink_disconnect(conn); exception when others then null; end;
|
||||
raise;
|
||||
end $fn$;
|
||||
select supaspin_delete_old_logs();
|
||||
select extensions.supaspin_delete_old_logs();
|
||||
delete from cron.job where jobname = '{LOG_RETENTION_JOB}';
|
||||
select cron.schedule('{LOG_RETENTION_JOB}', '17 3 * * *', 'select supaspin_delete_old_logs()');
|
||||
select cron.schedule('{LOG_RETENTION_JOB}', '17 3 * * *', 'select extensions.supaspin_delete_old_logs()');
|
||||
""".strip()
|
||||
|
||||
DEFAULT_MAIL = {"host": "smtp.example.com", "port": 587, "user": "", "password": "",
|
||||
@@ -159,6 +168,54 @@ def tls_secret(n):
|
||||
return f"supa{n}-tls"
|
||||
|
||||
|
||||
_SLOT_NUM = re.compile(r"(\d+)")
|
||||
|
||||
|
||||
def parse_slots(text):
|
||||
"""Slots {number: hostname} from the comma-separated setup answer. A hostname with a
|
||||
number in it keys the slot by that number - supa4.didgibot.com is slot 4 - so the
|
||||
instance number, its ports and the hostname digit stay in step; a name without a
|
||||
number takes its position in the list (the first name is slot 1)."""
|
||||
slots = {}
|
||||
for i, h in enumerate([p.strip() for p in text.split(",") if p.strip()], 1):
|
||||
m = _SLOT_NUM.search(h)
|
||||
n = int(m.group(1)) if m else i
|
||||
if n in slots:
|
||||
raise ValueError(f"'{h}' wants slot {n}, which '{slots[n]}' already takes.")
|
||||
slots[n] = h
|
||||
return slots
|
||||
|
||||
|
||||
def buffering_middleware(n):
|
||||
"""Name of the Traefik Middleware that caps the request body size for instance n."""
|
||||
return f"supa{n}-buffering"
|
||||
|
||||
|
||||
def ingress_annotations(cfg, n):
|
||||
"""Ingress annotations for instance n.
|
||||
Both controllers' annotations are always present: an annotation only takes effect
|
||||
when a controller of that family actually routes the ingress, so on a Traefik
|
||||
cluster the nginx keys sit inert (and vice versa) and nothing needs reconfiguring
|
||||
when the router changes. The upload cap therefore lives in two places that agree:
|
||||
proxy-body-size for nginx, and the Middleware object the traefik annotation points
|
||||
at. The chart's own rewrite-target default must be nullified or every API path
|
||||
breaks. The cert-manager annotation orders a certificate from the
|
||||
supaspin-letsencrypt issuer; without it the TLS secret keeps whatever supaspin put
|
||||
there (a saved or a self-signed certificate)."""
|
||||
anns = {}
|
||||
if cfg.get("lets_encrypt", True):
|
||||
anns["cert-manager.io/cluster-issuer"] = ISSUER
|
||||
# The chart default rewrites every path to "/", which breaks all APIs; null removes it.
|
||||
anns["nginx.ingress.kubernetes.io/rewrite-target"] = None
|
||||
anns["nginx.ingress.kubernetes.io/proxy-body-size"] = "50m"
|
||||
anns["nginx.ingress.kubernetes.io/proxy-read-timeout"] = "3600"
|
||||
anns["nginx.ingress.kubernetes.io/proxy-send-timeout"] = "3600"
|
||||
if cfg.get("ingress_controller", "nginx") == "traefik":
|
||||
anns["traefik.ingress.kubernetes.io/router.middlewares"] = \
|
||||
f"{namespace(n)}-{buffering_middleware(n)}@kubernetescrd"
|
||||
return anns
|
||||
|
||||
|
||||
def lowest_free_number(slots, taken):
|
||||
"""Lowest slot number that is not in `taken`; None if every slot is used."""
|
||||
for n in sorted(int(k) for k in slots):
|
||||
@@ -221,13 +278,15 @@ def instance_values(cfg, n, sec, mail, template_text):
|
||||
subs.update({
|
||||
"PUBLIC_URL": f"https://{host}", "HOST": host,
|
||||
"TLS_SECRET": tls_secret(n), "INGRESS_CLASS": cfg["ingress_class"],
|
||||
"CLUSTER_ISSUER": ISSUER, "PROJECT_NAME": f"supaspin-{n}",
|
||||
"PROJECT_NAME": f"supaspin-{n}",
|
||||
"INGRESS_ANNOTATIONS": ingress_annotations(cfg, n),
|
||||
"SMTP_HOST": mail["host"], "SMTP_PORT": mail["port"], "SMTP_USER": mail["user"],
|
||||
"SMTP_PASS": mail["password"], "SMTP_ADMIN_EMAIL": mail["admin_email"],
|
||||
"SMTP_SENDER_NAME": mail["sender_name"],
|
||||
})
|
||||
# json.dumps gives a quoted YAML-safe string; the port is quoted as well (env values are strings).
|
||||
quoted = {k: json.dumps(str(v)) for k, v in subs.items()}
|
||||
# A dict is dumped as JSON too: JSON flow syntax is valid YAML, so the annotation map drops in.
|
||||
quoted = {k: (json.dumps(v) if isinstance(v, dict) else json.dumps(str(v))) for k, v in subs.items()}
|
||||
text = re.sub(r"\$\{(\w+)\}", lambda m: quoted[m.group(1)], template_text)
|
||||
values = yaml.safe_load(text)
|
||||
apply_memory(values)
|
||||
@@ -478,15 +537,16 @@ def strip_kong_services(text, names):
|
||||
return "".join(out)
|
||||
|
||||
|
||||
def post_render(text, n, lan_ip):
|
||||
def post_render(text, n, cfg):
|
||||
"""Edit the rendered chart: fixed host ports on db and studio, no public Studio/pg-meta route."""
|
||||
bind_ip = host_bind_ip(cfg)
|
||||
docs = [d for d in yaml.safe_load_all(text) if d]
|
||||
for d in docs:
|
||||
kind, name = d.get("kind"), d.get("metadata", {}).get("name", "")
|
||||
if kind == "StatefulSet" and name == f"{fullname(n)}-db":
|
||||
_host_port(d, pg_port(n), lan_ip)
|
||||
_host_port(d, pg_port(n), bind_ip)
|
||||
elif kind == "Deployment" and name == f"{fullname(n)}-studio":
|
||||
_host_port(d, studio_port(n), lan_ip)
|
||||
_host_port(d, studio_port(n), bind_ip)
|
||||
elif kind == "ConfigMap" and "kong.yml" in (d.get("data") or {}):
|
||||
d["data"]["kong.yml"] = strip_kong_services(d["data"]["kong.yml"], ("meta", "dashboard"))
|
||||
for d in docs: # keep multi-line scripts readable in the release manifest
|
||||
@@ -496,10 +556,21 @@ def post_render(text, n, lan_ip):
|
||||
return yaml.safe_dump_all(docs, sort_keys=False, default_flow_style=False)
|
||||
|
||||
|
||||
def host_bind_ip(cfg):
|
||||
"""Address the Postgres/Studio host ports listen on. `lan` (the default) is the LAN IP
|
||||
only; on a machine with a public interface that keeps Postgres and the login-less Studio
|
||||
off the internet. `all` is every interface - LAN and localhost - for a box behind NAT
|
||||
that runs its own apps talking to localhost. `local` is localhost only."""
|
||||
return {"lan": cfg["lan_ip"], "local": "127.0.0.1", "all": None}[cfg.get("port_bind", "lan")]
|
||||
|
||||
|
||||
def _host_port(doc, port, ip):
|
||||
c = doc["spec"]["template"]["spec"]["containers"][0]
|
||||
p = c["ports"][0]
|
||||
p["hostPort"] = port
|
||||
if ip is not None:
|
||||
# None = bind every interface (the `all` mode); omitting hostIP makes the kubelet
|
||||
# listen on 0.0.0.0, which a hostIP cannot express.
|
||||
p["hostIP"] = ip
|
||||
if doc.get("kind") == "Deployment":
|
||||
# A port on the server itself can be held by one pod at a time. The default rollout starts
|
||||
@@ -702,18 +773,21 @@ def ensure_cluster_objects(cfg, log):
|
||||
{"apiVersion": "storage.k8s.io/v1", "kind": "StorageClass",
|
||||
"metadata": {"name": STORAGE_CLASS}, "provisioner": "kubernetes.io/no-provisioner",
|
||||
"reclaimPolicy": "Retain", "volumeBindingMode": "WaitForFirstConsumer"},
|
||||
{"apiVersion": "cert-manager.io/v1", "kind": "ClusterIssuer",
|
||||
"metadata": {"name": ISSUER},
|
||||
"spec": {"acme": {"server": "https://acme-v02.api.letsencrypt.org/directory",
|
||||
"email": cfg["acme_email"],
|
||||
"privateKeySecretRef": {"name": ISSUER + "-account-key"},
|
||||
"solvers": [{"http01": {"ingress": {"class": cfg["ingress_class"]}}}]}}},
|
||||
{"apiVersion": "scheduling.k8s.io/v1", "kind": "PriorityClass",
|
||||
"metadata": {"name": PRIORITY_CLASS}, "value": -100,
|
||||
"preemptionPolicy": "Never", "globalDefault": False,
|
||||
"description": "supaspin test instances: evicted first, never evict others"},
|
||||
{"apiVersion": "v1", "kind": "Namespace", "metadata": {"name": CERTS_NS}},
|
||||
]
|
||||
if cfg.get("lets_encrypt", True):
|
||||
# cert-manager needs this object to know where to order certificates from; without
|
||||
# Let's Encrypt it is dead weight, and cert-manager may not even be installed.
|
||||
docs.insert(1, {"apiVersion": "cert-manager.io/v1", "kind": "ClusterIssuer",
|
||||
"metadata": {"name": ISSUER},
|
||||
"spec": {"acme": {"server": "https://acme-v02.api.letsencrypt.org/directory",
|
||||
"email": cfg["acme_email"],
|
||||
"privateKeySecretRef": {"name": ISSUER + "-account-key"},
|
||||
"solvers": [{"http01": {"ingress": {"class": cfg["ingress_class"]}}}]}}})
|
||||
for d in docs:
|
||||
# Create only what is missing; never overwrite something already there.
|
||||
r = kubectl("get", d["kind"].lower(), d["metadata"]["name"], check=False)
|
||||
@@ -820,11 +894,45 @@ def setup_log_retention(cfg, n, log):
|
||||
scheduling under a fixed job name makes it idempotent."""
|
||||
log(f" log retention: deleting analytics logs older than {LOG_RETENTION}, every night")
|
||||
kubectl("-n", namespace(n), "exec", f"{fullname(n)}-db-0", "-c", "supabase-db", "--",
|
||||
"psql", "-U", "supabase_admin", "-d", "_supabase", "-v", "ON_ERROR_STOP=1",
|
||||
"psql", "-U", "supabase_admin", "-d", "postgres", "-v", "ON_ERROR_STOP=1",
|
||||
"-c", RETENTION_SQL)
|
||||
log(" log retention is set")
|
||||
|
||||
|
||||
def middleware_manifest(n):
|
||||
"""The Traefik request-size cap the ingress annotation of instance n points at. nginx gets
|
||||
the same cap from proxy-body-size; Traefik has no such annotation, the rule lives here."""
|
||||
return {"apiVersion": "traefik.io/v1alpha1", "kind": "Middleware",
|
||||
"metadata": {"name": buffering_middleware(n), "namespace": namespace(n)},
|
||||
"spec": {"buffering": {"maxRequestBodyBytes": 50 * 1024 * 1024}}} # 50 MiB
|
||||
|
||||
|
||||
def ensure_middleware(cfg, n, log):
|
||||
"""Create the Middleware for instance n on Traefik. On nginx the annotation is not
|
||||
emitted, and applying a traefik.io object would fail there - the CRD is not installed."""
|
||||
if cfg.get("ingress_controller", "nginx") != "traefik":
|
||||
return
|
||||
apply(middleware_manifest(n))
|
||||
log(f" created Middleware {buffering_middleware(n)} (50 MiB request cap)")
|
||||
|
||||
|
||||
def ensure_self_signed_cert(cfg, n, host, log):
|
||||
"""A self-signed certificate for the hostname, so https answers without Let's Encrypt.
|
||||
Skipped when a certificate is already in place (restored from supaspin-certs)."""
|
||||
r = kubectl("get", "secret", tls_secret(n), "-n", namespace(n), check=False)
|
||||
if r.returncode == 0:
|
||||
return
|
||||
with tempfile.TemporaryDirectory(prefix="supaspin-cert-") as d:
|
||||
key, crt = f"{d}/tls.key", f"{d}/tls.crt"
|
||||
run(["openssl", "req", "-x509", "-newkey", "rsa:2048", "-sha256", "-nodes",
|
||||
"-keyout", key, "-out", crt, "-days", "3650",
|
||||
"-subj", f"/CN={host}", "-addext", f"subjectAltName=DNS:{host}"])
|
||||
kubectl("create", "secret", "tls", tls_secret(n), "-n", namespace(n),
|
||||
"--cert", crt, "--key", key)
|
||||
log(f" self-signed certificate for {host} (valid 10 years);"
|
||||
" clients must trust it or ignore the warning")
|
||||
|
||||
|
||||
def create_instance(cfg, mail, log, number=None):
|
||||
taken = existing_numbers(cfg)
|
||||
n = number if number is not None else lowest_free_number(cfg["slots"], taken)
|
||||
@@ -842,10 +950,13 @@ def create_instance(cfg, mail, log, number=None):
|
||||
kubectl("create", "namespace", namespace(n))
|
||||
apply([pv_manifest(cfg, n, v) for v in VOLUMES])
|
||||
restore_cert(n, log)
|
||||
if not cfg.get("lets_encrypt", True):
|
||||
ensure_self_signed_cert(cfg, n, host, log)
|
||||
sec = generate_secrets()
|
||||
with open(find_template()) as f:
|
||||
values = instance_values(cfg, n, sec, mail, f.read())
|
||||
vfile = write_values_file(n, values)
|
||||
ensure_middleware(cfg, n, log)
|
||||
log(" installing chart")
|
||||
try:
|
||||
helm("install", release(n), chart_path(cfg), "-n", namespace(n), "-f", vfile,
|
||||
@@ -1762,11 +1873,35 @@ def setup_dialog(cfg, log):
|
||||
cfg["storage_root"] = ask("Storage root (images live here)", cfg.get("storage_root", "/home/igor/k8s-storage/supabase"))
|
||||
cfg["node_name"] = ask("Kubernetes node name", cfg.get("node_name", d["node_name"]))
|
||||
cfg["lan_ip"] = ask("Server LAN IP (Postgres/Studio ports bind here)", cfg.get("lan_ip", d["lan_ip"]))
|
||||
cfg["acme_email"] = ask("Let's Encrypt e-mail", cfg.get("acme_email", ""))
|
||||
bind_default = cfg.get("port_bind", "lan")
|
||||
bind = ask("Port bind: lan (LAN only), all (LAN + localhost), local (localhost only)",
|
||||
bind_default).lower()
|
||||
while bind not in ("lan", "all", "local"):
|
||||
print(f"{RED}Answer 'lan', 'all' or 'local'.{OFF}")
|
||||
bind = ask("Port bind: lan (LAN only), all (LAN + localhost), local (localhost only)",
|
||||
bind_default).lower()
|
||||
cfg["port_bind"] = bind
|
||||
cfg["ingress_class"] = ask("Ingress class", cfg.get("ingress_class", "public"))
|
||||
controller_default = cfg.get("ingress_controller", "nginx")
|
||||
controller = ask("Ingress controller, traefik or nginx", controller_default).lower()
|
||||
while controller not in ("traefik", "nginx"):
|
||||
print(f"{RED}Answer 'traefik' or 'nginx'.{OFF}")
|
||||
controller = ask("Ingress controller, traefik or nginx", controller_default).lower()
|
||||
cfg["ingress_controller"] = controller
|
||||
le_default = "y" if cfg.get("lets_encrypt", True) else "n"
|
||||
le = ask("Fetch certificates from Let's Encrypt? Needs public DNS and ports 80/443 (Y/n)", le_default)
|
||||
cfg["lets_encrypt"] = le.lower().startswith("y")
|
||||
if cfg["lets_encrypt"]:
|
||||
cfg["acme_email"] = ask("Let's Encrypt e-mail", cfg.get("acme_email", ""))
|
||||
cur = ",".join(cfg["slots"][k] for k in sorted(cfg.get("slots", {}))) if cfg.get("slots") else ""
|
||||
hosts = [h.strip() for h in ask("Hostname slots, comma separated (first = instance 1)", cur).split(",") if h.strip()]
|
||||
cfg["slots"] = {i + 1: h for i, h in enumerate(hosts)}
|
||||
while True:
|
||||
try:
|
||||
cfg["slots"] = parse_slots(ask(
|
||||
"Hostname slots, comma separated (a name with a number keys that slot: "
|
||||
"supa4 = slot 4; a plain name takes its position)", cur))
|
||||
break
|
||||
except ValueError as e:
|
||||
print(f"{RED}{e}{OFF}")
|
||||
cfg["ssh_host"] = ask("Public SSH host for tunnels", cfg.get("ssh_host", ""))
|
||||
cfg["ssh_port"] = int(ask("Public SSH port", cfg.get("ssh_port", 22)))
|
||||
cfg["ssh_user"] = ask("SSH user", cfg.get("ssh_user", d["ssh_user"]))
|
||||
@@ -1875,7 +2010,7 @@ def main_menu(cfg, log):
|
||||
def main(argv):
|
||||
if len(argv) >= 2 and argv[1] == "post-render": # called by Helm; no root check, no output but YAML
|
||||
cfg = load_config()
|
||||
sys.stdout.write(post_render(sys.stdin.read(), int(argv[2]), cfg["lan_ip"]))
|
||||
sys.stdout.write(post_render(sys.stdin.read(), int(argv[2]), cfg))
|
||||
return 0
|
||||
if len(argv) > 1 and argv[1] in ("help", "-h", "--help"):
|
||||
print(__doc__)
|
||||
|
||||
@@ -71,13 +71,9 @@ environment:
|
||||
ingress:
|
||||
enabled: true
|
||||
className: ${INGRESS_CLASS}
|
||||
annotations:
|
||||
# The chart default rewrites every path to "/", which breaks all APIs; null removes it.
|
||||
nginx.ingress.kubernetes.io/rewrite-target: null
|
||||
nginx.ingress.kubernetes.io/proxy-body-size: 50m
|
||||
nginx.ingress.kubernetes.io/proxy-read-timeout: "3600"
|
||||
nginx.ingress.kubernetes.io/proxy-send-timeout: "3600"
|
||||
cert-manager.io/cluster-issuer: ${CLUSTER_ISSUER}
|
||||
# The annotation map is built in code (ingress_annotations): nginx gets proxy-* and a
|
||||
# nulled rewrite-target, Traefik gets its Middleware reference, cert-manager its issuer.
|
||||
annotations: ${INGRESS_ANNOTATIONS}
|
||||
tls:
|
||||
- hosts: [${HOST}]
|
||||
secretName: ${TLS_SECRET}
|
||||
|
||||
+165
-7
@@ -51,6 +51,25 @@ class Numbering(unittest.TestCase):
|
||||
def test_no_free_slot(self):
|
||||
self.assertIsNone(sp.lowest_free_number(CFG["slots"], {1, 2, 3}))
|
||||
|
||||
def test_slots_are_keyed_by_the_hostname_number(self):
|
||||
# The digit in the name is the slot number, so instance N, its ports and the
|
||||
# hostname stay in step on every machine.
|
||||
self.assertEqual(sp.parse_slots("supa4.didgibot.com, supa5.didgibot.com"),
|
||||
{4: "supa4.didgibot.com", 5: "supa5.didgibot.com"})
|
||||
|
||||
def test_plain_names_take_their_position(self):
|
||||
self.assertEqual(sp.parse_slots("db.example.com, db2.example.com"),
|
||||
{1: "db.example.com", 2: "db2.example.com"})
|
||||
self.assertEqual(sp.parse_slots("supa4.didgibot.com, spare.example.com"),
|
||||
{4: "supa4.didgibot.com", 2: "spare.example.com"})
|
||||
|
||||
def test_duplicates_are_an_error(self):
|
||||
with self.assertRaises(ValueError):
|
||||
sp.parse_slots("supa4.didgibot.com, supa4.other.com")
|
||||
|
||||
def test_empty_answer_gives_no_slots(self):
|
||||
self.assertEqual(sp.parse_slots(""), {})
|
||||
|
||||
def test_ports_and_names(self):
|
||||
self.assertEqual((sp.pg_port(1), sp.pg_port(3)), (5441, 5443))
|
||||
self.assertEqual((sp.studio_port(1), sp.studio_port(3)), (3001, 3003))
|
||||
@@ -124,7 +143,7 @@ class Values(unittest.TestCase):
|
||||
|
||||
def test_volume_caps(self):
|
||||
p = self.v["persistence"]
|
||||
self.assertEqual(p["db"]["size"], "2048Mi")
|
||||
self.assertEqual(p["db"]["size"], "6144Mi")
|
||||
self.assertEqual(p["storage"]["size"], "1024Mi")
|
||||
for vol in ("functions", "snippets", "deno", "pgsodium"):
|
||||
self.assertEqual(p[vol]["size"], "256Mi")
|
||||
@@ -136,13 +155,21 @@ class Values(unittest.TestCase):
|
||||
# rows older than LOG_RETENTION and is scheduled under a fixed name so re-runs overwrite.
|
||||
# pg_cron in the supabase image only runs jobs from the `postgres` database, so the job
|
||||
# reaches the log tables in `_supabase` over a local dblink connection (trusted there).
|
||||
# dblink and the helper function live in the `extensions` schema: in `public` they belong
|
||||
# to someone else's routines and the app's `REVOKE ALL ON ALL ROUTINES IN SCHEMA public`
|
||||
# fails as the non-superuser postgres role (42501, dblink_connect_u).
|
||||
self.assertIn("schemaname = ''_analytics''", sp.RETENTION_SQL)
|
||||
self.assertIn("''log_events\\_%''", sp.RETENTION_SQL)
|
||||
self.assertIn(f"interval ''{sp.LOG_RETENTION}''", sp.RETENTION_SQL)
|
||||
self.assertIn(f"'{sp.LOG_RETENTION_JOB}'", sp.RETENTION_SQL)
|
||||
self.assertIn("cron.schedule", sp.RETENTION_SQL)
|
||||
self.assertIn("dblink_connect", sp.RETENTION_SQL)
|
||||
self.assertIn("extensions.dblink_connect", sp.RETENTION_SQL)
|
||||
self.assertIn("user=supabase_admin", sp.RETENTION_SQL)
|
||||
self.assertIn("create extension if not exists dblink with schema extensions", sp.RETENTION_SQL)
|
||||
self.assertIn("alter extension dblink set schema extensions", sp.RETENTION_SQL)
|
||||
self.assertIn("extensions.supaspin_delete_old_logs", sp.RETENTION_SQL)
|
||||
self.assertIn("drop function if exists public.supaspin_delete_old_logs", sp.RETENTION_SQL)
|
||||
self.assertNotIn("\ndblink_", sp.RETENTION_SQL) # no unqualified dblink call remains
|
||||
|
||||
def test_imgproxy_reads_the_storage_volume(self):
|
||||
img = self.v["persistence"]["imgproxy"]
|
||||
@@ -171,6 +198,120 @@ class Values(unittest.TestCase):
|
||||
self.assertEqual(v["secret"]["smtp"]["password"], 'p"a$s: #x')
|
||||
|
||||
|
||||
class IngressShaping(unittest.TestCase):
|
||||
"""The ingress annotation map per controller and per Let's Encrypt setting."""
|
||||
def setUp(self):
|
||||
self.sec = sp.generate_secrets()
|
||||
|
||||
def ann(self, cfg=None, n=2):
|
||||
return sp.instance_values(dict(CFG, **(cfg or {})), n, self.sec, MAIL, template()) \
|
||||
["ingress"]["annotations"]
|
||||
|
||||
def test_default_is_nginx_with_lets_encrypt(self):
|
||||
# Configs written before these settings existed must render exactly as before.
|
||||
a = self.ann()
|
||||
self.assertEqual(a["cert-manager.io/cluster-issuer"], "supaspin-letsencrypt")
|
||||
self.assertIsNone(a["nginx.ingress.kubernetes.io/rewrite-target"])
|
||||
self.assertEqual(a["nginx.ingress.kubernetes.io/proxy-body-size"], "50m")
|
||||
self.assertEqual(a["nginx.ingress.kubernetes.io/proxy-read-timeout"], "3600")
|
||||
self.assertNotIn("traefik.ingress.kubernetes.io/router.middlewares", a)
|
||||
|
||||
def test_traefik_also_keeps_nginx_annotations(self):
|
||||
# The nginx keys must be present in traefik mode too: while nginx routes the
|
||||
# ingress they enforce the cap, and they are inert the day Traefik takes over.
|
||||
a = self.ann({"ingress_controller": "traefik"})
|
||||
self.assertEqual(a["traefik.ingress.kubernetes.io/router.middlewares"],
|
||||
"supaspin-2-supa2-buffering@kubernetescrd")
|
||||
self.assertIsNone(a["nginx.ingress.kubernetes.io/rewrite-target"])
|
||||
self.assertEqual(a["nginx.ingress.kubernetes.io/proxy-body-size"], "50m")
|
||||
self.assertEqual(a["nginx.ingress.kubernetes.io/proxy-read-timeout"], "3600")
|
||||
self.assertEqual(a["cert-manager.io/cluster-issuer"], "supaspin-letsencrypt")
|
||||
|
||||
def test_lets_encrypt_off_drops_cert_manager_annotation(self):
|
||||
self.assertNotIn("cert-manager.io/cluster-issuer", self.ann({"lets_encrypt": False}))
|
||||
self.assertNotIn("cert-manager.io/cluster-issuer",
|
||||
self.ann({"lets_encrypt": False, "ingress_controller": "traefik"}))
|
||||
|
||||
def test_middleware_manifest(self):
|
||||
m = sp.middleware_manifest(2)
|
||||
self.assertEqual(m["apiVersion"], "traefik.io/v1alpha1")
|
||||
self.assertEqual(m["kind"], "Middleware")
|
||||
self.assertEqual(m["metadata"], {"name": "supa2-buffering", "namespace": "supaspin-2"})
|
||||
self.assertEqual(m["spec"]["buffering"]["maxRequestBodyBytes"], 50 * 1024 * 1024)
|
||||
|
||||
|
||||
class ClusterObjects(unittest.TestCase):
|
||||
"""ensure_cluster_objects skips the ClusterIssuer when Let's Encrypt is off."""
|
||||
def created_docs(self, cfg):
|
||||
applied = []
|
||||
|
||||
class R:
|
||||
returncode = 1
|
||||
stdout = stderr = ""
|
||||
|
||||
def fake_kubectl(*args, **kw):
|
||||
if args and args[0] == "apply":
|
||||
applied.extend(yaml.safe_load_all(kw["input_"]))
|
||||
return R()
|
||||
|
||||
with tempfile.TemporaryDirectory() as tmp, \
|
||||
mock.patch.object(sp, "STATE_DIR", tmp), \
|
||||
mock.patch.object(sp, "kubectl", side_effect=fake_kubectl), \
|
||||
mock.patch.object(sp, "helm"):
|
||||
sp.ensure_cluster_objects(cfg, lambda *a: None)
|
||||
return applied
|
||||
|
||||
def test_issuer_created_when_lets_encrypt_on(self):
|
||||
self.assertIn("ClusterIssuer", [d["kind"] for d in self.created_docs(CFG)])
|
||||
|
||||
def test_no_issuer_without_lets_encrypt(self):
|
||||
kinds = [d["kind"] for d in self.created_docs(dict(CFG, lets_encrypt=False))]
|
||||
self.assertNotIn("ClusterIssuer", kinds)
|
||||
self.assertIn("StorageClass", kinds)
|
||||
self.assertIn("PriorityClass", kinds)
|
||||
self.assertIn("Namespace", kinds)
|
||||
|
||||
|
||||
class SelfSignedCert(unittest.TestCase):
|
||||
def test_generated_when_missing(self):
|
||||
calls = []
|
||||
|
||||
class R:
|
||||
returncode = 1
|
||||
stdout = stderr = ""
|
||||
|
||||
def fake_kubectl(*args, **kw):
|
||||
calls.append(args)
|
||||
return R()
|
||||
|
||||
def fake_run(args, **kw):
|
||||
calls.append(("run", args))
|
||||
|
||||
class P:
|
||||
returncode = 0
|
||||
return P()
|
||||
|
||||
with mock.patch.object(sp, "kubectl", side_effect=fake_kubectl), \
|
||||
mock.patch.object(sp, "run", side_effect=fake_run):
|
||||
sp.ensure_self_signed_cert(CFG, 2, "supa2.example.com", lambda *a: None)
|
||||
openssl = next(c[1] for c in calls if c[0] == "run" and c[1][0] == "openssl")
|
||||
self.assertIn("subjectAltName=DNS:supa2.example.com", openssl)
|
||||
create = next(c for c in calls if c[:3] == ("create", "secret", "tls"))
|
||||
self.assertEqual(create[3], "supa2-tls")
|
||||
self.assertIn("supaspin-2", create)
|
||||
|
||||
def test_kept_when_present(self):
|
||||
class R:
|
||||
returncode = 0
|
||||
stdout = stderr = ""
|
||||
|
||||
with mock.patch.object(sp, "kubectl", return_value=R()) as k, \
|
||||
mock.patch.object(sp, "run") as r:
|
||||
sp.ensure_self_signed_cert(CFG, 2, "supa2.example.com", lambda *a: None)
|
||||
k.assert_called_once()
|
||||
r.assert_not_called()
|
||||
|
||||
|
||||
class EnvFile(unittest.TestCase):
|
||||
def setUp(self):
|
||||
self.sec = sp.generate_secrets()
|
||||
@@ -270,7 +411,7 @@ data:
|
||||
|
||||
class PostRender(unittest.TestCase):
|
||||
def setUp(self):
|
||||
self.docs = list(yaml.safe_load_all(sp.post_render(RENDERED, 1, "192.168.10.30")))
|
||||
self.docs = list(yaml.safe_load_all(sp.post_render(RENDERED, 1, CFG)))
|
||||
|
||||
def find(self, kind, name):
|
||||
return next(d for d in self.docs if d["kind"] == kind and d["metadata"]["name"] == name)
|
||||
@@ -281,6 +422,23 @@ class PostRender(unittest.TestCase):
|
||||
st = self.find("Deployment", "supa1-supabase-studio")["spec"]["template"]["spec"]["containers"][0]["ports"][0]
|
||||
self.assertEqual((st["hostPort"], st["hostIP"]), (3001, "192.168.10.30"))
|
||||
|
||||
def test_all_binds_localhost_too(self):
|
||||
# `all` omits hostIP: the kubelet then listens on every interface, localhost included.
|
||||
docs = list(yaml.safe_load_all(sp.post_render(RENDERED, 1, dict(CFG, port_bind="all"))))
|
||||
find = lambda k, name: next(d for d in docs if d["kind"] == k and d["metadata"]["name"] == name)
|
||||
db = find("StatefulSet", "supa1-supabase-db")["spec"]["template"]["spec"]["containers"][0]["ports"][0]
|
||||
st = find("Deployment", "supa1-supabase-studio")["spec"]["template"]["spec"]["containers"][0]["ports"][0]
|
||||
self.assertEqual(db["hostPort"], 5441)
|
||||
self.assertNotIn("hostIP", db)
|
||||
self.assertEqual(st["hostPort"], 3001)
|
||||
self.assertNotIn("hostIP", st)
|
||||
|
||||
def test_local_binds_localhost_only(self):
|
||||
docs = list(yaml.safe_load_all(sp.post_render(RENDERED, 1, dict(CFG, port_bind="local"))))
|
||||
find = lambda k, name: next(d for d in docs if d["kind"] == k and d["metadata"]["name"] == name)
|
||||
db = find("StatefulSet", "supa1-supabase-db")["spec"]["template"]["spec"]["containers"][0]["ports"][0]
|
||||
self.assertEqual((db["hostPort"], db["hostIP"]), (5441, "127.0.0.1"))
|
||||
|
||||
def test_other_pods_untouched(self):
|
||||
rest = self.find("Deployment", "supa1-supabase-rest")["spec"]["template"]["spec"]["containers"][0]["ports"][0]
|
||||
self.assertNotIn("hostPort", rest)
|
||||
@@ -310,9 +468,9 @@ class PostRender(unittest.TestCase):
|
||||
self.assertEqual(cm["data"]["kong-entrypoint.sh"], "echo hi\necho there\n")
|
||||
|
||||
def test_instance_number_selects_ports(self):
|
||||
docs = list(yaml.safe_load_all(sp.post_render(RENDERED.replace("supa1", "supa3"), 3, "10.0.0.1")))
|
||||
docs = list(yaml.safe_load_all(sp.post_render(RENDERED.replace("supa1", "supa3"), 3, CFG)))
|
||||
db = next(d for d in docs if d["kind"] == "StatefulSet")["spec"]["template"]["spec"]["containers"][0]["ports"][0]
|
||||
self.assertEqual((db["hostPort"], db["hostIP"]), (5443, "10.0.0.1"))
|
||||
self.assertEqual((db["hostPort"], db["hostIP"]), (5443, "192.168.10.30"))
|
||||
|
||||
|
||||
class Manifests(unittest.TestCase):
|
||||
@@ -320,7 +478,7 @@ class Manifests(unittest.TestCase):
|
||||
pv = sp.pv_manifest(CFG, 2, "db")
|
||||
self.assertEqual(pv["spec"]["claimRef"], {"namespace": "supaspin-2", "name": "supa2-supabase-db"})
|
||||
self.assertEqual(pv["spec"]["local"]["path"], "/data/supabase/2/db/data")
|
||||
self.assertEqual(pv["spec"]["capacity"]["storage"], "2048Mi")
|
||||
self.assertEqual(pv["spec"]["capacity"]["storage"], "6144Mi")
|
||||
self.assertEqual(pv["spec"]["persistentVolumeReclaimPolicy"], "Retain")
|
||||
self.assertEqual(pv["spec"]["nodeAffinity"]["required"]["nodeSelectorTerms"][0]
|
||||
["matchExpressions"][0]["values"], ["node1"])
|
||||
@@ -1103,7 +1261,7 @@ class HostPortRollout(unittest.TestCase):
|
||||
"template": {"spec": {"containers": [
|
||||
{"name": "rest", "ports": [{"containerPort": 3000}]}]}}}},
|
||||
])
|
||||
out = list(yaml.safe_load_all(sp.post_render(text, n, CFG["lan_ip"])))
|
||||
out = list(yaml.safe_load_all(sp.post_render(text, n, CFG)))
|
||||
return {d["metadata"]["name"]: d for d in out}
|
||||
|
||||
def test_studio_stops_the_old_pod_before_starting_the_new_one(self):
|
||||
|
||||
Reference in New Issue
Block a user