From 95fbae8db410a60cd21f9045288976d47e31c353 Mon Sep 17 00:00:00 2001 From: Ryan Westfall Date: Wed, 2 Sep 2026 07:39:03 -0700 Subject: [PATCH] Register college_craft for django deploy (closes #24) (#25) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit ## Summary - Closes [#24](https://git.aimloperations.com/ai_ml_operations/server-infra/issues/24). - Register `college_craft` in `app_catalog` (`type: django`, repo `ai_ml_operations/college_craft`, default branch `master`). - Add `host_apps` prod `:8006` / beta `:8016` on **adama**, **roslin**, **starbuck**, **apollo**, and **ai-server-4080**. - dj-queue **worker singleton on adama** only (`compose_profiles: [worker]`); other hosts web-only. - Document ports, NPM hosts, Postgres DBs, secrets, Nominatim (`10.0.0.128:8089`), and the app-repo `--app client_site` leftover. - Add `college_craft` to `scripts/deploy.sh` `--app` help. ## Out of scope (ops, after merge) - Create Postgres DBs `college_craft` / `college_craft_beta` on `10.0.0.230` and grant `westfarn`. - Write control-node secrets `~/Documents/secrets/college_craft/college_craft_{prod,beta}.env`. - NPM: `collegecraft.com` (+ www) → `:8006`; `college-craft-preview.aimloperations.com` → `:8016`. - Smoke deploy after secrets + DBs exist: ```bash ~/Documents/repos/server-infra/scripts/deploy.sh --app college_craft --env beta --ref master ~/Documents/repos/server-infra/scripts/deploy.sh --app college_craft --env prod --ref master ``` ## Test plan - [ ] Confirm `app_catalog.college_craft` matches other django apps (`compose_file`, `migrate_cmd`). - [ ] Confirm ports 8006/8016 unused elsewhere and match across all five hosts. - [ ] Confirm only adama has `compose_profiles: [worker]` for college_craft. - [ ] After secrets + DBs: deploy beta then prod; `/healthz/` up on both; prod holding page, beta full site. Reviewed-on: https://git.aimloperations.com/ai_ml_operations/server-infra/pulls/25 --- IMPLEMENTATION.md | 33 ++++++++++++++++++++++---- inventory/group_vars/all.yml | 9 +++++++ inventory/host_vars/adama.yml | 2 ++ inventory/host_vars/ai-server-4080.yml | 6 +++-- inventory/host_vars/apollo.yml | 4 +++- inventory/host_vars/roslin.yml | 4 +++- inventory/host_vars/starbuck.yml | 4 +++- roles/app-deploy/tasks/django.yml | 4 ++-- scripts/deploy.sh | 2 +- 9 files changed, 55 insertions(+), 13 deletions(-) diff --git a/IMPLEMENTATION.md b/IMPLEMENTATION.md index 26440bc..9f63384 100644 --- a/IMPLEMENTATION.md +++ b/IMPLEMENTATION.md @@ -216,6 +216,7 @@ After Docker install, re-SSH so the `docker` group membership takes effect. | `chat_backend` | django (docker) | all webservers | beta + prod | active/active behind NPM; Ollama `http://10.0.0.128:11434`; SearxNG `http://10.0.0.128:8088` (`SEARXNG_BASE_URL`) | | `monica_site` | django (docker) | all webservers | beta + prod | active/active behind NPM; no bundled Postgres (like `scha`); dj-queue **worker singleton on adama** only (`compose --profile worker`); Ollama social drafting via `10.0.0.128:11434` | | `url_shortening_service` | django (docker) | all webservers | beta + prod | active/active behind NPM; no bundled Postgres; **no worker**. Two public hosts, same container: short domain (`GET /`, `GET /` 302) and API host (`/api/links/`, Bearer required). | +| `college_craft` | django (docker) | all webservers | beta + prod | active/active behind NPM; no bundled Postgres (like `scha` / `monica_site`); dj-queue **worker singleton on adama** only (`compose --profile worker`); Ollama social drafting via `10.0.0.128:11434`; Nominatim `http://10.0.0.128:8089`; prod `SITE_UNDER_CONSTRUCTION=true` until launch | Django apps use a **shared external Postgres** (via `DATABASE_URL` in each host's env file) so active/active replicas share one database. Beta and prod never share @@ -226,7 +227,7 @@ a DB. - `app_catalog` (`group_vars/all.yml`) — how each app is built (repo, type, compose file, migrate cmd). - `host_apps` (`host_vars/.yml`) — which app+env+port runs on that host. Optional `compose_profiles: [worker]` activates docker compose profiles on that - host only (used for `monica_site` dj-queue singleton on adama). + host only (used for `monica_site` / `college_craft` dj-queue singleton on adama). - Django app = one compose project per env: project name `_`, host port from `host_apps`. Ports match across app hosts so NPM can balance `adama:PORT` + `roslin:PORT` + `starbuck:PORT` + `apollo:PORT`. @@ -244,6 +245,7 @@ future beta replica. | chat_backend | 8013 | 8003 | all webservers | | monica_site | 8014 | 8004 | all webservers | | url_shortening_service | 8015 | 8005 | all webservers | +| college_craft | 8016 | 8006 | all webservers | | dta_webapp (nginx) | 8081 | 8080 | all webservers | | chat_web_app (nginx) | 8083 | 8082 | all webservers | | SearxNG (LAN only) | — | **8088** | ai-server-4080 only (`searxng_stack`); not an NPM upstream | @@ -252,7 +254,8 @@ Host-local services on ai-server-4080 (not balanced by NPM): | Service | Port | Notes | |---------|------|-------| -| Ollama | 11434 | Not Ansible-managed today; GPU host | +| Ollama | 11434 | Not Ansible-managed today; GPU host (`monica_site` / `college_craft` social drafting) | +| Nominatim | 8089 | Not Ansible-managed today; LAN address autocomplete for `college_craft` | | SearxNG | 8088 | `roles/searxng` (#10); JSON API for chat_backend grounded search | | Loki | 3100 | `roles/observability` | | Prometheus | 9090 | `roles/observability` | @@ -278,8 +281,9 @@ chat_backend secrets must use `SEARXNG_BASE_URL=http://10.0.0.128:8088`. to host `{{ apps_env_dir }}` → git checkout at ref → copy `.env` into checkout → `docker compose build` → `up -d` (with `COMPOSE_PROFILES` from optional `host_apps.compose_profiles`) → migrate (run once, shared DB). - **`monica_site` worker:** adama `host_apps` sets `compose_profiles: [worker]` so - deploy starts dj-queue with web. Other hosts omit profiles (web only). + **`monica_site` / `college_craft` worker:** adama `host_apps` sets + `compose_profiles: [worker]` so deploy starts dj-queue with web. Other hosts + omit profiles (web only). - **node-static**: git checkout at ref → `npm ci` → `npm run build:` (writes to the app's `webroot_pattern`, e.g. `/var/www/{env}.realpath.app/html` or `/var/www/{env}.chat.aimloperations/html`). @@ -309,6 +313,7 @@ point each domain at the backend(s): | monica_site | `mkdrealtor.com` (+ www); `monica-preview.aimloperations.com` (beta) | `adama:8004` / `8014` + same on roslin / starbuck / apollo / ai-server-4080 | | url_shortening_service (short) | `aiml.pw` and/or `cidinn.li` (pick when DNS is ready); `short-beta.aimloperations.com` (beta). Proxy `/` + `/[a-z0-9]{4,8}` only — 404 `/api/`, `/admin/`, `/debug/` | `adama:8005` / `8015` + same on roslin / starbuck / apollo / ai-server-4080 | | url_shortening_service (API) | `shortener.aimloperations.com`; `shortener-beta.aimloperations.com` (beta). Proxy `/api/` only — 404 `/admin/` | same ports as short host (one container) | +| college_craft | `collegecraft.com` (+ www); `college-craft-preview.aimloperations.com` (beta) | `adama:8006` / `8016` + same on roslin / starbuck / apollo / ai-server-4080 | ### Required changes IN each app repo (owned separately) @@ -345,6 +350,8 @@ do not). | monica_site | beta | `monica_site_beta` | `postgres://westfarn:@10.0.0.230:5432/monica_site_beta` | | url_shortening_service | prod | `url_shortener` | `postgres://westfarn:@10.0.0.230:5432/url_shortener` | | url_shortening_service | beta | `url_shortener_beta` | `postgres://westfarn:@10.0.0.230:5432/url_shortener_beta` | +| college_craft | prod | `college_craft` | `postgres://westfarn:@10.0.0.230:5432/college_craft` | +| college_craft | beta | `college_craft_beta` | `postgres://westfarn:@10.0.0.230:5432/college_craft_beta` | Server prereqs on 10.0.0.230: create each DB + grant `westfarn`; `listen_addresses` covers LAN; `pg_hba.conf` allows `10.0.0.0/24`; firewall opens @@ -357,6 +364,21 @@ Server prereqs on 10.0.0.230: create each DB + grant `westfarn`; Caller `monica_site` uses `SHORTENER_BASE_URL=https://` and Bearer mint; do not mint via the short hostname. +`college_craft` extra env (control-node secrets, not in git): +`~/Documents/secrets/college_craft/college_craft_prod.env` and +`college_craft_beta.env`. Template: app repo `.env.prod.example`. +`DJANGO_ENV` / `DJANGO_ALLOWED_HOSTS` / `DATABASE_URL` / `WEB_PORT` / +`PUBLIC_SITE_URL` differ per env (prod `collegecraft.com` / `:8006`; +beta `college-craft-preview.aimloperations.com` / `:8016`). +`SITE_UNDER_CONSTRUCTION=true` on prod (holding page until launch), `false` on beta. +`FEATURE_BLOG=true`; other `FEATURE_*` stay false until purchased. Prod also needs +`TIANJI_WEBSITE_ID` + reCAPTCHA keys. If email/SMS or direct mail is turned on later, +add a `college_craft:` entry to `SHORTENER_API_TOKENS` on `url_shortening_service`. + +App-repo companion (`college_craft`, not this repo): deploy workflows still call +`--app client_site` (template leftover). They must become `--app college_craft` +before CI deploy will hit this catalog entry. Default branch is `master` (not `main`). + ### One-time host bootstrap (per target) - [x] Gitea SSH key: the `gitea-key` role (in `site.yml`) generates a key per @@ -432,7 +454,8 @@ Store vault password for CI in a file readable only by the Act runner (e.g. `~/. | 10d | Register + deploy `monica_site` (django, ports 8004/8014) | Done ([#14](https://git.aimloperations.com/ai_ml_operations/server-infra/issues/14)) | | 10e | Auto-start `monica_site` dj-queue worker on adama (`compose_profiles`) | Done ([#17](https://git.aimloperations.com/ai_ml_operations/server-infra/issues/17)) | | 10f | Add starbuck + apollo as app hosts (same workloads as roslin) | Done ([#20](https://git.aimloperations.com/ai_ml_operations/server-infra/issues/20)) | -| 10g | Register + deploy `url_shortening_service` (django, ports 8005/8015) | This PR ([#22](https://git.aimloperations.com/ai_ml_operations/server-infra/issues/22)) | +| 10g | Register + deploy `url_shortening_service` (django, ports 8005/8015) | Done ([#22](https://git.aimloperations.com/ai_ml_operations/server-infra/issues/22)) | +| 10h | Register + deploy `college_craft` (django, ports 8006/8016) | This PR ([#24](https://git.aimloperations.com/ai_ml_operations/server-infra/issues/24)) | | 11 | Gitea container registry (optional) | Future | ## Open Decisions diff --git a/inventory/group_vars/all.yml b/inventory/group_vars/all.yml index d2699d5..a5fa7e6 100644 --- a/inventory/group_vars/all.yml +++ b/inventory/group_vars/all.yml @@ -120,6 +120,15 @@ app_catalog: web_service: web migrate_cmd: "uv run python manage.py migrate --noinput" # No compose worker profile. Two public NPM hosts share one container/port. + college_craft: + type: django + repo: "{{ git_base_url }}/ai_ml_operations/college_craft.git" + default_branch: master + compose_file: docker-compose.prod.yml + web_service: web + migrate_cmd: "uv run python manage.py migrate --noinput" + # dj-queue worker = compose profile `worker` (singleton). Deploy starts it + # via host_apps.compose_profiles on adama only (see host_vars/adama.yml). # Deploy filter vars. CI passes these; manual runs may leave them undefined # to (re)deploy every app listed in the host's host_apps. diff --git a/inventory/host_vars/adama.yml b/inventory/host_vars/adama.yml index 3ad2451..7e935cc 100644 --- a/inventory/host_vars/adama.yml +++ b/inventory/host_vars/adama.yml @@ -22,3 +22,5 @@ host_apps: - { name: monica_site, env: beta, port: 8014, compose_profiles: [worker] } - { name: url_shortening_service, env: prod, port: 8005 } - { name: url_shortening_service, env: beta, port: 8015 } + - { name: college_craft, env: prod, port: 8006, compose_profiles: [worker] } + - { name: college_craft, env: beta, port: 8016, compose_profiles: [worker] } diff --git a/inventory/host_vars/ai-server-4080.yml b/inventory/host_vars/ai-server-4080.yml index 4507ce1..4df3703 100644 --- a/inventory/host_vars/ai-server-4080.yml +++ b/inventory/host_vars/ai-server-4080.yml @@ -34,8 +34,10 @@ host_apps: - { name: chat_web_app, env: beta, port: 8083 } - { name: chat_backend, env: prod, port: 8003 } - { name: chat_backend, env: beta, port: 8013 } - # monica_site worker NOT here — adama host_apps sets compose_profiles: [worker]. + # monica_site / college_craft workers NOT here — adama sets compose_profiles: [worker]. - { name: monica_site, env: prod, port: 8004 } - { name: monica_site, env: beta, port: 8014 } - { name: url_shortening_service, env: prod, port: 8005 } - - { name: url_shortening_service, env: beta, port: 8015 } \ No newline at end of file + - { name: url_shortening_service, env: beta, port: 8015 } + - { name: college_craft, env: prod, port: 8006 } + - { name: college_craft, env: beta, port: 8016 } \ No newline at end of file diff --git a/inventory/host_vars/apollo.yml b/inventory/host_vars/apollo.yml index 1b7dcee..ca76b06 100644 --- a/inventory/host_vars/apollo.yml +++ b/inventory/host_vars/apollo.yml @@ -13,8 +13,10 @@ host_apps: - { name: chat_web_app, env: beta, port: 8083 } - { name: chat_backend, env: prod, port: 8003 } - { name: chat_backend, env: beta, port: 8013 } - # monica_site worker NOT here — adama host_apps sets compose_profiles: [worker]. + # monica_site / college_craft workers NOT here — adama sets compose_profiles: [worker]. - { name: monica_site, env: prod, port: 8004 } - { name: monica_site, env: beta, port: 8014 } - { name: url_shortening_service, env: prod, port: 8005 } - { name: url_shortening_service, env: beta, port: 8015 } + - { name: college_craft, env: prod, port: 8006 } + - { name: college_craft, env: beta, port: 8016 } diff --git a/inventory/host_vars/roslin.yml b/inventory/host_vars/roslin.yml index 345f1f5..9923718 100644 --- a/inventory/host_vars/roslin.yml +++ b/inventory/host_vars/roslin.yml @@ -13,8 +13,10 @@ host_apps: - { name: chat_web_app, env: beta, port: 8083 } - { name: chat_backend, env: prod, port: 8003 } - { name: chat_backend, env: beta, port: 8013 } - # monica_site worker NOT here — adama host_apps sets compose_profiles: [worker]. + # monica_site / college_craft workers NOT here — adama sets compose_profiles: [worker]. - { name: monica_site, env: prod, port: 8004 } - { name: monica_site, env: beta, port: 8014 } - { name: url_shortening_service, env: prod, port: 8005 } - { name: url_shortening_service, env: beta, port: 8015 } + - { name: college_craft, env: prod, port: 8006 } + - { name: college_craft, env: beta, port: 8016 } diff --git a/inventory/host_vars/starbuck.yml b/inventory/host_vars/starbuck.yml index bbb93b8..0389808 100644 --- a/inventory/host_vars/starbuck.yml +++ b/inventory/host_vars/starbuck.yml @@ -13,8 +13,10 @@ host_apps: - { name: chat_web_app, env: beta, port: 8083 } - { name: chat_backend, env: prod, port: 8003 } - { name: chat_backend, env: beta, port: 8013 } - # monica_site worker NOT here — adama host_apps sets compose_profiles: [worker]. + # monica_site / college_craft workers NOT here — adama sets compose_profiles: [worker]. - { name: monica_site, env: prod, port: 8004 } - { name: monica_site, env: beta, port: 8014 } - { name: url_shortening_service, env: prod, port: 8005 } - { name: url_shortening_service, env: beta, port: 8015 } + - { name: college_craft, env: prod, port: 8006 } + - { name: college_craft, env: beta, port: 8016 } diff --git a/roles/app-deploy/tasks/django.yml b/roles/app-deploy/tasks/django.yml index 24b3248..edcba0a 100644 --- a/roles/app-deploy/tasks/django.yml +++ b/roles/app-deploy/tasks/django.yml @@ -104,8 +104,8 @@ - _legacy_systemd_probe.stdout | default('') | length > 0 # Optional host_apps.compose_profiles (e.g. [worker]) activates compose profiles -# on this host only. Used for monica_site dj-queue singleton on adama — without -# it, plain up --remove-orphans can drop a manually started worker. +# on this host only. Used for monica_site / college_craft dj-queue singleton on +# adama — without it, plain up --remove-orphans can drop a manually started worker. - name: "django[{{ _project }}] start containers" ansible.builtin.command: cmd: "docker compose -f {{ _spec.compose_file }} --env-file .env up -d --remove-orphans" diff --git a/scripts/deploy.sh b/scripts/deploy.sh index 9ec24f4..7c408e8 100755 --- a/scripts/deploy.sh +++ b/scripts/deploy.sh @@ -17,7 +17,7 @@ Deploy applications with deploy-apps.yml. HOST Optional. Limit to one host: adama, roslin, starbuck, apollo, or ai-server-4080. Options: - --app NAME App to deploy (company_site, dta_service, dta_webapp, scha, chat_web_app, chat_backend, monica_site, url_shortening_service) + --app NAME App to deploy (company_site, dta_service, dta_webapp, scha, chat_web_app, chat_backend, monica_site, url_shortening_service, college_craft) --env ENV Environment: beta or prod --ref REF Git ref/sha to deploy (default: master) --check Dry run